An Introduction to Code Generation

This is the resource to read and to point people to if they want to know what code generation is, how they should do it, the different approaches and the benefits of active over passive generators.

The VERY Basics The idea of code generation is that you often have a number of files that are similar either within a project or between projects. Rather than writing them by hand, you look at what varies between the instances (the metadata), create some kind of way of entering that metadata (a wizard, XML file, config file, or my personal favorite - a databased metabase for industrial level reuse of metadata). You then pass that metadata into your generator along with some information about how many files to create, what to name them and where to put them and the generator creates the files so you don't have to.

Active vs. Passive With a passive generator, you enter the metadata, generate the files and the hand edit them as required. This would be fine as long as your clients never changed their minds, you never made a mistake and requirements never changed. For the rest of us a passive generator is better than no generator at all, but it really isn't ideal.

With an active (or "round trip") generator, you can update your metadata and regenerate your templates at any time. There are two tricks to writing an active generator. The first is to do a very good job of specifying your templates and metadata to cover most use cases and the second is to provide a practical extension mechanism for mixing hand coded with generated code. Current best practice in object oriented code generation (used by IronSpeed in the .NET world, Doug Hughes in Reactor and myself amongst others) is using inheritance to inherit generated classes from your custom classes. It works quite nicely, but I'm playing around with using class based mixins instead and I know Doug Hughes mentioned something similar on the Reactor list, so it'll be interesting to see where the state of the art is by years end!

The key thing to note is that the main difference between an active generator and most passive generators is to do with the structure of the metadata and templates - both usually generate the code in the same way. The only other difference is that some really simple passive generators (wizard based ones) don't even keep a copy of the metadata for future reuse which works for their use case but limits their value.

Approaches to Code Generation There are two main approaches to code generation - concatenation and templates. If you like writing dynamic code you can also us dynamic programming (the other branch of Metaprogramming) to do a lot of what a code generator does), but I've already posted on that.

Concatenation This was how I first got into code generation in the late 90's (mainly because I didn't know any better) and I still have a generator that work this way and generates quite acceptable code.

Basically with concatenation, you just manually write bunch of <cfset TemplateString = TemplateString & "your code goes here">. You need to remember to double your double quotes, but it is a very easy way of getting started for generating simple code. You can loop through cfoutputs, use cfifs to decide whether or not to include a particular bit of code, it's really pretty straightforward.

It is however, not very scalable from a complexity standpoint and is extremely brittle. I have tens of thousands of lines of such code still churning out procedural sites, but I wouldn't want to try to refactor that to generate cfcs. It is almost impossible to "see" what the code will look like without just generating and then looking at your generated source code to see if your files look right (and then running them in a test harness to see if they function as expected).

Only thing I would say is if you're generating anything non-trivial, don't use concatenation as it gets really messy, really quick. Also, the first few times you do it, it really messes with your head. Eventually I got pretty good as "seeing" what the generated code might look like by looking at my concatenations, but it takes a while to get to that point and there is really no benefit in using concatenation for anything more than say 20 line templates, and if you want to generate some larger templates, you are much better using the same approach for all of your generation. Also, if you like "pretty" generated code (as I do), it is a pain to keep track of all of your tabchars and newlines in the concatenated code. Add an iff statement to a block of code and now every single line concatenated in that block needs another tabchar added to it. No fun at all.

If you do have a use case for concatenation, typically performance isn't an issue as even an inline generator only generates each script once per change in associated metadata, but if you care I've heard that concatenating by appending to an array is supposed to perform much better and then you turn the array into a string before cffiling the string onto your server hard disk. I'm pretty sure Michael Dinowitz has the lowdown on this if anyone wants to find out more.

Templating The other approach to code generaation is to create templates using a templating language and then to pass in the metadata. The benefit is that the template looks much more like your final scripts and is much easier to refactor. Sometimes you end up with small imperfections in the formatting of the generated code, but it is usually plenty good enough especially as (in theory) nobody should look at the generated code. I just like fairly well formatted generated code so if I'm trying to debug the code I don't just have a complete jumble to look at.

Learning a Language Of course, templating requires a templating language. Currently most people use XML for their metadata and XSLT for their generators. If you are already experienced with XML/XSLT there is nothing wrong with that approach (apart from the possibility of RSI - XSLT can get just a little verbose :->). There are also many other templating languages out there. When I first looked into this in the spring I looked at a number of templating languages and considered writing a parser for them so I wouldn't have to use XSLT.

Then I realized that ColdFusion WAS a templating language and that only a couple of simple conventions were required to allow the full power of ColdFusion for generating ColdFusion code. Thus CF Template was born.

Conclusion If you have lots of repetitive code in your application you should try code generation. If you build lots of applications, you should REALLY consider creating a software product line using code generation and application generation to generate more of your site functionality. Will post plenty more on that going forwards as that is exactly what I do for a living over at SystemsForge.