Automatic Metabase Generation

Any DSL can be describes as a set of concepts, each of which has zero or more attributes which have a cardinality of 0/1..1/n (i.e. are either required or optional and can have a maximum number of 1 or n values). Each attribute has a data type that is either a primitive data type (like a string or an int) or another concept. A valid document in such a language is comprised of one or more instances of one or more of those concepts.

How does that map to a relational structure for storing the metadata? Well, each concept becomes a table, each attribute with a primitive data type becomes a column and each attribute with a “concept” data type describes a join. In fact, for a simple system which doesn’t support conceptual inheritance (where you can describe one concept as extending another in terms of its attributes), the only question left is whether any given join for a concept joined with a 0/1..n cardinality should be modeled as a one-many or a many-many (which would require a joining table) join. For now I’m going to assume a one-many relationship so we just need a simple foreign key in the joined concept table (if an Object has attributes, there will be an ObjectID FK in the Attribute table) and if that breaks down I’ll extend my meta-generator to handle it. I’m also not going to implement conceptual inheritance unless I need it (although if I did, it would be implemented using a “table per concrete class” so it’d be fairly easy to implement).

So, my meta-grammar (the grammar of the grammars) is:

Concept ::= Name [ShortDescription] {Attribute}
Attribute ::= Name DataType [ShortDescription] Required Multiple
Required ::= Required|Optional
Multiple ::= Yes|No

Where concept name, concept short description, attribute name and attribute short descriptions are strings and attribute data type is an enum (list of possible options) consisting of primitive data types and any other concept within the grammar. Required is an enum of “required” and “optional” and Multiple is a boolean.

While it is overkill, I’m just going to use an Active Data Model (metabase) for storing my grammar descriptions as well. In that way I can use the same algorithms for generating metatables that I use for generating regular tables so I don’t have to solve the same problem for two different concrete syntaxes. In practice I’ll add support for importing a custom textual syntax for both meta-grammars and grammars as soon as I find the time to decide on the most elegant approach to parsing textual grammars (Mark Mandell recommended ANTLR, but there are so many tools out there I want to take some time to figure this out!).