By Peter Bell

Integrating Custom and Generated Code

I had an interesting conversation last night with Kevin Roche. He does quite a bit of application generation and one of the issues he was trying to decide on was how best to handle the mixing of hand written and generated code. Of course, the problem isn’t so much the mixing of the code but rather supporting regular regeneration of the generated code without losing your custom modifications. This posting looks at why this is important and the most common approaches to solve this problem . . .

As I have mentioned in the past, there are two broad approaches to code generation: passive and active. In brief, passive generators are devil spawned devices designed to flood your application with reams of un-maintainable code and active generators are the one true way to programming salvation :->

The problem with passive generators is that they give your project a starting point, but then it is up to you to maintain the code they generate, so they may help you to get a project started more quickly, but then you have to maintain all of the generated code (which is typically not very DRY). Active generators are designed with some kind of metabase so as your requirements change you can just change your metadata and regenerate the core code base.

This would be fine if you always generated 100% of your code, but in practice most applications need a little bit of hand coded loving to get them to meet your clients needs, so the problem is deciding how to implement an active code generator that allows you to regenerate your code without losing your custom modifications.

Kevin was considering a "super diff" solution which would be smart enough to diff your last generated code with the current code and then to regenerate and re-patch all of your mods in the new generated code. It is an interesting approach, but when you sit down and try to implement such a solution it quickly becomes overwhelming as it is a really hard technical problem to solve (I have a feeling it may be fundamentally insoluble, but I could well be wrong).

A more common approach used by tools like Visual Studio are “custom code delimiters” where you are told not to edit certain parts of the file but only other parts. Any changes to the generated parts of the code will be lost (because you shouldn’t have made them), but any custom code you add between certain character strings will be kept (they are RegEx’d out and then inserted in the same place in the regenerated file). This works and is technically fairly easy to implement, but for most OO generators is now being replaced by inheritance as a code separation mechanism.

With inheritance, you generate two versions of each user extensible class. One contains the generated code and the other extends the generated class and allows you to add any custom code required. In practice for CFC’s I have found this approach to work very well. It does add a bunch of extra files to your project, but I haven’t found that to be a big problem. Also it allows you to very easily extend methods using the super method so you can just extend or replace certain methods. It is NOT a great approach for UI code (I tend to passively generate UI code rather than regenerating it), but for CFCs it works well.

One hint: if you find yourself wanting to make changes WITHIN given generated methods and are repeating a bunch of the generated code, that is probably a sign that your methods are too long. Break down your methods into smaller functions (each of which is a different private method) and you’ll get much better reuse of your generated code.

Comments
I dont have it to hand but the book on how EMF (http://www.eclipse.org/modeling/emf/?project=emf) had a good guide, since it generally overwrites methods you get a bit of JavaDoc that says:
/**
* @Generated:
*/
public void setSomething()

but if you want to override that method you can by adding Gen to the end of it and then it will redirect the re-writing of that method to the setSomethingGen instead of setSomething (for example) or the other way round (cant remember of the top of my head)
so you can change the setSomething method and if you need to still call the setSomethingGen method which has been updated with the new code.
# Posted By Mark Drew | 1/16/07 11:19 AM
Hi Mark,

Interesting approach! There are definitely ways of doing this without inheritance and they have the benefit that you have half as many class files, but I actually like to be able to see ONLY the overloaded methods in my custom business objects so I still tend to prefer the inheritance based approach.

Interesting, though. Will have to play with this (and get the book - thanks for the link :->).
# Posted By Peter Bell | 1/16/07 11:33 AM
The problem with adding another file in there is that that file is all by itself, it is not linked to your generating system. So its a loose node in effect... you need to be able to get it BACK into your system. EMF says that if you create a class, it goes back to your model's definition... anyway...read the book, you will undestand it better than us mere humans :)
# Posted By Mark Drew | 1/16/07 11:46 AM
Hi Mark,

Not really - it's like Reactor - it active generates all of the code, but passive (one time) generates a set of custom classes that inherit from the generated classes so you can tweak them as you want. Don't need to get the tweaks back into your generating system at all unless you are into the whole "automated refactoring of custom code based on changes to metadata and generated code" but that can get pretty complex and is more of a research topic than a practical application at the moment (sure that'll change).

Will definitely check out the book, although I usually struggle with these books as well, so we'll see how it goes :-<
# Posted By Peter Bell | 1/16/07 12:03 PM
Thats true, EMF is used to have a model and a metamodel stored either as UML, Encore files or Java so you can always have things in sync with each other... not sure if the case here applies. But I would think it might do if you re-generate something from a managing application and your changes are not reflected by a pesky CFC that some unscrupulous developer left there.
# Posted By Mark Drew | 1/16/07 12:24 PM
Think may also be that some of the MDA stuff is into true round tripping where if you change the classes it is reflected back up in the model and that isn't something I'm aiming for at the moment.

Will definitely get the book . . .

So much to learn, so little time!
# Posted By Peter Bell | 1/16/07 12:34 PM
Hi Peter,

One java app I built last year, was a cfc generator. It basically let you create(model) cfc methods, arguments and even body content. However, the neat part, was that you could save your progress in a serialized skeleton. So I can continue to slowly model the objects. At generation time, it would generate the cfc's based on a cf template and generate your cf's. Now, it had planned a reverse engineering module, where you would open the same skeleton and just update it by parsing the cfc's. I actually have the parsing working in coldbox. It actually parses your cfc's for your dsp/do/on methods and registers them. That is why you do not register your events in coldbox. Now, this could be interesting to complete or actually build on it. I have been trying to get the time to actually invest more time on it. The original plan, was to create a Lookup Builder. Where you would introspect the database, select tables you want to generate common : list, create, update screens, based on cf templates that were tokenized. The app still has a rudimentary template editor. If you are interested, I can show it to you. I definitely find speed in code generation, but do find the drawbacks of passive and not active generations.
# Posted By Luis Majano | 1/16/07 8:06 PM
Hi Luis,

Sounds interesting. I tend not to use database schemas to drive generation as they aren't rich enough in terms of the metadata available, and to date I haven't played much with true round tripping as I am focusing right now on trying to get everything into the meta model which would allow me to treat the actual ColdFusion as a second class citizen - a generated artifact that I'd only ever write to - not read. That said I'd definitely like to take some time to check out what you've been doing!
# Posted By Peter Bell | 1/16/07 8:13 PM
No problem. When I get home tonight, I'll package it up and send it to you. I have not touched it an about a year. So, not everything works.

Luis
# Posted By Luis Majano | 1/16/07 8:16 PM
Cool - tnx!
# Posted By Peter Bell | 1/16/07 8:19 PM
BlogCFC was created by Raymond Camden. This blog is running version 5.005.