Extreme Versioning

If you only build three or four applications a year and use a framework like Mach-II or Model-Glue with a 6-24 month release cycle, framework versioning is annoying, but not a major pain point. When you plan to generate 1,000+ custom applications a year using a framework with a monthly upgrade cycle, a different approach is required . . .

The first step is that you need to avoid using a general purpose programming language (GPL) wherever possible. Automatically transforming GPL code between versions of an API is a non-trivial problem to solve. Instead, we use a combination of a templating language with a grammar that will not be changing often and a collection of Domain Specific Languages (DSLs). The statements in the DSLs are stored in a database and/or in XML files depending on the use case. Recently I submitted a paper for the DSM forum at OOPSLA this year. With a title of "Automated Transformation of Statements within Evolving Domain Specific Languages", I'll admit it sound (a) dull, and (b) irrelevant to the everyday practice of software engineering. In practice it is anything but (at least for us!).

If an application is built entirely using a fairly static templating language (where the conventions don't change much over time) and a collection of Domain Specific Languages to describe all of the functionality (objects, relationships, properties, validations, transformations, notifications, workflows, actions, steps, views, etc.) then it is possible to write scripts to automatically implement certain classes of transformations. So, if you move from version 2.1 - 2.2 in a framework and it requires different syntax for the XML, depending on the types of transformations, it can be possible to describe the transforms in a way that will automatically transform all statements within a given DSL to the next version (we're working on tooling for automating the transformations for both XML and database concrete syntaxes). This means that for projects with no custom code, they will often be able to be upgraded without human intervention, and for projects with a small amount of custom code, the intervention required to upgrade and test projects will be limited.

Concurrent Versions
There will always be projects where there is sufficient custom code that we won't be able to automate the transformation of the project between versions. Because of this we need the ability to run multiple concurrent versions of the framework with different projects pulling from different framework versions. What I've been thinking about today is the optimal directory structure for such a situation.

At first I wanted to have a base /lightbase directory will a collection of framework and library directories (/framework21, /framework22, etc.) and all of the applications within an applications folder. However, I realized after a bit of thought that it would actually be much cleaner to have a base directory for each version containing its framework code, library and the collection of applications running that version. Firstly it'd make it MUCH easier to see how many applications were running in a given version (and to know when a particular version could safely be removed from a given production server). Secondly it allows for a single project to run on multiple versions of the framework at the same time, allowing for easier testing and a smoother transition period.

Expect to hear a lot more about the automated transformations as the year progresses, but right now I have a bunch of features to add to our framework. Should be an interesting weekend!