By Peter Bell

Replacing Methods with Metadata

For someone who spends a good part of their time programming, I *really* don't like code. To me every time I add a line to a codebase I see that as a personal failure that I wasn't able to come up with a more elegant way of expressing my intent than actually encoding it into a specific 3rd generation language from which no known mechanism could retrieve my intent automatically (in the general case). I'm currently implementing a generalized approach across my in-house framework for replacing (most) methods with metadata . . .

I've long been a proponent of base classes. When I look at the vast majority of the code I write it could be simply seen as a parameterization of a more fundamental base class. If you're ever written a listNewUsers() method or and updatePrice() method (or any other method that fundamentally related to lists, forms, details, saving, imports, exports, classes of reports, deletes and the like) they you probably know what I mean.

I love the idea of being able to replace the majority of my methods with metadata as it completely divorces the what from the how. The metadata cleanly expresses my intent and I can play with the implementation to my hearts content (whether changing the algorithm used, the language coded in or anything else).

Of course, the problem is that no matter how well you write your base classes (which form an API - a simple form of Domain Specific Language) there are always edge cases that actually need (shudder) hand coding. So what is really needed is a mechanism to do the following.

When you call a method on a class it should see if the method exists (yes, this will be easier in Scorpio with the onMissingMethod(), but it isn't hard now - just a little uglier). If it does, it should call them method. If not, it should see if a "metadata method" exists with the name of that method. If so, it should "call" that, and if not it should handle the error appropriately.

At the moment I've just added a call method to my base class. response:any = call(MethodName:string:required, ArgumentCollection:struct:optional). I've also added a getMethod() which gets the metadata for the "metadata method" from the appropriate metadata bean if it exists.

So, if you want to call getNewUsers on UserService, you just call UserService.call(getNewUsers) and it'll return the appropriately loaded IBO.

Of course, the other approach to all of this is to gen methods based on the metadata. It's an approach, but more and more I'm liking the approach of interpreting rather than compiling (generating code) for my models.

Anyone else playing with using metadata to define methods? Any secrets you're willing to share :->

Comments
I'll play devil's advocate just for a moment - probably not very helpful to you at the moment, but it's where my head is at right now.

Just to hark back to an earlier discussion on code vs data - the CF source itself can be regarded as metadata, and sometimes the most elegant, concise and communicative DSL you can use is the general purpose language itself. CF doesn't have a huge number of verbs, and there are a lot of people who can read (well-written) CF source like reading a newspaper. Your DSL needs to be *very* communicative to improve on that.

A perfect example of abandoning "raw" source code in favour of a less clear abstraction is the habit of wrapping SQL statements in DAO and gateway methods. SQL is a standards-based DSL of unusual clarity and expressive power:

select <field list> from <table> where <condition>

And yet we laboriously hide it behind custom-built DAO and gateway APIs that are at best a 1:1 transliteration of the SQL syntax. I've seen general purpose gateway methods that consist of 1000 lines of CF code and end up producing exactly the line of SQL I wrote above.

Now, I know *why* we do this (playing devil's advocate, remember) - but there are a lot of silly statements made about hiding the SQL behind a nice "clean" abstraction because SQL is somehow ugly or "low level".

So, when introducing an abstraction, it helps to remember that what we are abstracting is usually already an abstraction itself, often a standards-based and widely understood abstraction, and do the cost/benefit analysis accordingly.

</horns and pitchfork>

Keep up the good work, Peter!

Jaime Metcher
# Posted By Jaime Metcher | 5/28/07 5:29 PM
Hi Jaime,

Always need some contrarian views - otherwise I'll never learn anything :->

That said, this time I feel I am on pretty safe ground. Firstly I'll bite on the SQL piece. SQL is indeed a very clean DSL for describing declaritively what you want and not how to get it. That said, I do interpose a data mapper so I can say things like PropertyNameList = Age,FullName,TotalSalary and my mapper is smart enough to replace that with SELECT DateofBirth, FirstName, LastName, (select sum(salary) from tbl_job where tbl_Job.UserID = #UserID#) as TotalSalary. I find the first to be a clearer expression of intent to me than the second, so I find it useful to automate that transformation.

As for the general case, most general purpose programming languages are used to describe how to do something - not to concisely describe what you want done. I'd much rather state name=AdminUserList, DefaultRecordsPerPage=25, PropertyNameList="FirstName, LastName", DefaultOrderBy="LastName, FirstName", ClassActionList="Add", ObjectActionList="Edit,Delete", PossibleRecordsPerPage="10,25,50,100,all" than write all of the code to actually make that work. Of course, I have to write all of the code to either generate or interpret my little DSL, but now if I have 500 projects and I want to change how paging works, I just have to change my interpreter or my generator rather than 500 projects each of which may have 20-30 lists.

CF source code is almost never the most elegant and concise method to express intent. The exception typically being where your intent requires a lot of conditionals that express a unique business intent where you do need to include some kind of mechanism to capture that logic and it is seldom much more concise than the equivalent CF Script, but in the general case you can usually come up with a DSL to more concisely express intent to solve a collection of similar problems when compared to the CF code it replaces.
# Posted By Peter Bell | 5/28/07 6:16 PM
Peter,

OK, wrt a DSL vs CF code I'll give you "concise", and grudgingly concede "elegant" - but how about "communicative"? I don't know your DSL. Even if I did know your DSL, I still don't know Tom's or Fred's. And I don't know what bugs your DSL interpreter has. I *do* know ColdFusion, and I've got a pretty good handle on its warts.

It's intriguing that 3GLs after all these years are still 3GLs. Programming is massively different now to what it was 20 years ago, but the core language constructs are not what has changed. Are we all too stupid to grok higher level languages, or is there something in this? Or are we measuring the wrong dimensions - maybe automatic memory management is actually the fourth generation.

Jaime
# Posted By Jaime Metcher | 5/28/07 10:02 PM
Yep - you have zeroed in on the elephant in the closet with DSLs - you have to document, learn and evolve them and that is non trivial. That said, if they are well designed, it is often worth the effort.

For instance, if you have ever used Fusebox, Model Glue or Mach-II, you are familiar with using DSLs to specify controller functionality. The way they are written still involved a lot of 3GL code, but it is quite possible to conceive of frameworks that need NO 3gl code except for edge cases (that's what I'm building in-house). Equally if you use Transfer or Reactor, you're used to a DSL for describing an ORM layer and many people find them worth learning for the code savings they can provide for well matched use cases.

Another example I'm familiar with is content management systems. In the oldest days people would hard code product data into HTML pages. We saw that this was bad so we created trivial DSLs to describe the properties that make up products, implemented them using the concrete syntax of a set of database tables and wrote code generators that would take the data entered by non-technical users into the db and generate HTML "code" on the fly. In my system you can describe features, objects, properties, relationships, screens, steps, value lists, custom data tpes and a bunch of other things that would usually be the province of a programmer - all using a cms style interface and a set of simple declaritive DSLs.

So, I think we ARE using DSLs, As to why that isn't replacing 3GLs, firstly it is a hard problem to solve (you always have to drop back down for some edge cases), secondly, until you start seeing everything as a DSL, you are focused on "an XML configuration file" or "an API" or "a content management system" rather than seeing you're writing a simple DSL and using proven heuristics to choose the best concrete syntax (db table, xml file, code) and projection (boxes and arrows, tables, trees, free form text in IDEs, etc.) for a specific use case. Thirdly most programmers LIKE their IDEs. I was at a conference on Domain Specific Modeling/Code Generation and while the effort was on deskilling the capturing of specific business requirements, we all wanted to be the ones writing the interpreters or templates - not drawing the diagrams all day. Psychology will be a big barrier to adoption, but we made them move from assembler to c and I think for collections of use cases we're going to make a move to a higher level of abstraction - even if it takes another decade.

Of course, in one way I don't really care - the less people who "get" DSM the more of a competitive edge I have when I go up against them bidding projects. It'll be interesting to see how this all plays out!
# Posted By Peter Bell | 5/29/07 5:08 AM
@General: "communicative"

I think they are more communicative in that they clearly describe what is going on when you read it. There is no mistaking what "select column from table" is doing. Of course, as Peter mentioned you still need to learn - but keeping the docs open and using it for a while makes that simple and many times using it (even while learning) is as fast or faster than using the general purpose language, with the added benefit that you have less code to maintain, and will be even faster once you've learned it all.

@Peter: I really enjoyed reading the previous comment, espectiall the "So, I think we ARE using DSLs," paragraph. That really hit home with my experience.
# Posted By Sam | 5/29/07 7:36 AM
BlogCFC was created by Raymond Camden. This blog is running version 5.005.