The Power of Active Data Models

When was the last time you wrote an e-commerce store that kept the products in a text file? Of course it is possible, and there are use cases where this is the best approach, but I think most developers have now standardized on storing data in relational databases for most of their web based applications. Why? Well, the search, the filtering and the reporting capabilities make it easy to pull the appropriate subset of the data required by a given screen, we have well established patterns for easily accessing and editing records, and the solution scales well to large data sets.

But metadata (configuration information, bean dependencies, object models, workflow rules, etc.) is also data, so would it not make sense to store the metadata in a database as well (an approach often called an “Active Data Model”)? As always, it depends.

If you are only going to reuse the metadata occasionally and are only going to have a few hundred pieces of metadata, a database for storing the metadata is probably overkill. If, however, you want to create a powerful Software Product Line, storing your metadata in a database makes it much easier to re-use and to create feature modeling and configuration tools to deskill the process of associating the appropriate data to a given application you’re trying to build.

There are two major constraints with a metabased approach. The first is that it works best with fairly simple grammars. As your abstract grammars become more complex it becomes much less useful to store your languages in a database. The other is that historically, the evolving of metabased DSLs has been very painful as every time you want to change your syntax you need to edit the metabase schema and the code for accessing the data (as well as the templates and transformations which always have to be changed when you refactor a DSL – irrespective of concrete syntax). This meant in practice that the benefits of a metabased approach were typically limited to mature DSLs which didn’t change.

The problem with this is that as agile development becomes more important, it is becoming (in my mind) essential to practice “agile application generation” with a system designed for the easy refactoring of DSLs over time (in fact I’ve actually submitted an experience report on this for Agile 2007 although I have no idea whether it’ll be selected or not!).

I’ll be posting more shortly on an approach I have developed as a first step towards a “language workbench for active data modelers” which will automate the process of regenerating your metabase schemas and data access code as you modify the abstract grammar of your DSLs using a simple text file. Eventually I’d hope to add quite powerful refactoring support for DSLs, but that raises some interesting issues I haven’t had time to think through yet.