By Peter Bell

CFArgument - DAOs are a Waste of Time!

This week Brian and I debate the merits of the Data Access Object (DAO) design pattern. The DAO is a core J2EE design pattern that has been adopted within the ColdFusion community and which you will generally find in object-oriented applications built with ColdFusion. Still, despite the fact that everyone is using them, could we all be wasting our time? . . .

Peter Bell
DAO's and gateways are a waste of time :-)

If I'm moving from procedural coding to object oriented development, I get the vast majority of the benefits from applying good Model-View-Controller separation, creating a service layer as an API to my model and writing rich business objects (IBO's of course!) to encapsulate my business logic.

Creating DAOs means that I have to do a lot of extra typing as I need to create CRUD methods in my service classes that just delegate to CRUD methods in my DAOs. Don't get me started on the whole DAO plus Gateway thing. We really don't need five beans for every business object - whatever Sean might have said in 2004.

I don't see why I should waste the extra time and typing. What do you think?

Brian Rinaldi
I am on the fence about the Gateway; yes, its not a "gateway" as it is defined in J2EE design patterns but I find it a nice way to organize my code. To me, its more a personal preference. That being said, a DAO is necessary because it abstracts potentially fragile database specific code into a class that is specifically designed for that purpose. Without a DAO, your query code ends up in your service or, even worse, your beans/business objects where it is more difficult to separate your business logic from a specific rdbms implementation.

Peter Bell
But who cares?! When was the last time you actually took a live project and changed the dbms? I think that an abstract factory returning different DAOs for different dbms' may make sense for a framework author, but not for most projects. The goal is to encapsulate what varies and in my experience projects just don't change database systems often enough to justify the extra typing required when using a DAO.

Stick your queries in your service classes, write rich business objects and you get most of the benefits of OO.

Brian Rinaldi
True, I have never actually been tasked with changing code to support a new rdbms but you might move some data from an rdbms to some other form of persistent storage for a particular object. Also, as the database changes, adding columns perhaps, it is much easier to isolate those changes as they relate to queries. Lastly, I would argue that since DAOs are pretty generic and easily repeatable, it is very easy to simply generate the basic CRUD if you are so concerned with typing.

Peter Bell
I don't buy the "some other persistent storage" as I don't know about you, but I don't usually have clients come back after I've built a site and ask me if I could persist to XML instead of a database. It happens, but not enough to justify preemtively doing the extra work. That said, there are some benefits that should be considered.

I agree with you that the separation of concerns is nice as it is easier to just go through a DAO to make all of your SQL changes, although you need to be careful about the way you write your DAO API so that you don't end up having to edit your service classes *and* your DAOs when you add a column to your schema.

Interestingly for me, the killer benefit of DAOs is the one that is seldom discussed by CF developers . . . unit testing. Without DAO's you can't mock out (slow) database calls and it is hard to write a good set of unit tests for regression testing your code. However, given the low adoption of unit testing in the CF world to date I'd still argue that many people who use DAOs don't get the benefits they should because they don't need to mock out their db interactions.

Brian Rinaldi
As you note, you probably don't hear much about unit testing because too many people (myself included) don't have any formalized unit testing process (sounds like fodder for another CFArgument). Also, I will agree that if it isn't done right, you can end up making changes to both your service and your DAO when you add a column. Still, you can't deny the benefits of something because many people don't necessarily do it right.

Peter Bell
Well I could - this is the CFArgument not the CFAgreement after all!

To summarize, obviously DAO's provide benefit when used properly. I use them in all my applications and get a lot of benefits from doing so, but I would argue that many CF developers aren't getting a good ROI from using DAOs. If you're just starting out with OO programming I'd recommend focusing on MVC, Service classes and rich IBO's. You can then move to encapsulate your SQL into DAOs when you feel the need - either for separation of concerns or to speed up your regression tests.

Brian Rinaldi
I disagree. DAOs are probably one of the simplest aspects of OO to understand and to write (or generate) and thus aren't a real bottleneck to learning or building OO applications. If we both agree there are more benefits than drawbacks if done correctly, then its just a matter of helping people do it correctly.

Comments
@peter,

these cfarguments posts are outstanding! i'm supporting peter all the way on this one. i think a lot of the "borrowed" j2ee design patterns are crap and just waste our time. seriously, how is a dao and service layer better of a design pattern then say ActiveRecord?
# Posted By tony petruzzi | 9/19/08 1:53 PM
@Tony - I don't think anyone said "better" though I personally am not a fan of activerecord.
# Posted By Brian Rinaldi | 9/19/08 1:58 PM
Before I started using Transfer in my projects, I was toying with the idea of creating beans dynamically, using the cfdbinfo tag to create properties based on the fields of a database table and then exposing the properties via some get() and set() functions Hal Helms shared with me.

In the process, I ended up creating a singleton DAO object that could perform CRUD operations on any of these beans. It would retrieve the table and datasource name from the metadata in the bean, run cfdbinfo to retrieve the names of all of the current fields in the table, and then loop through that table metadata to create and run the necessary SQL commands and interact with the bean properties.

I didn't take it any further once I got familiar with Transfer because Transfer brought so much more to the table (the ability to easily retrieve and update related objects and the cache). But it demonstrated to me that it is possible to build table/bean-agnostic CRUD operations so that that CRUD code doesn't have to be part of every bean/active record.
# Posted By Brian Swartzfager | 9/19/08 3:07 PM
Interesting CFARGUMENT. My approach toward adopting any new technology or pattern is to ask myself "Does it solve a problem for me? Will it?". Too many developers try solving problems that simply do not exist. In the process, making their code and maintenance infinitely more complex.
# Posted By Shlomy Gantz | 9/19/08 3:19 PM
@peter - "when did you last change your DBMS?"

More often than you think, but more importantly how about producing DB neutral applications? You need to give your clients a choice - at least we do
# Posted By richard | 9/19/08 3:31 PM
@richard: A large part of the reason why I originally started working on the tools that ultimately became DataFaucet is because even though Ben Forta had at the time declared it impossible, I wasn't willing to give up half my potential market merely because they standardized on a different database platform. And hence that's why DF started with the objective of abstracting SQL at a lower level and then gradually acquired ORM features rather than the other way around (as is the case with Transfer and TQL).

Part of the reason it may not seem so important to Pete is because of the way he runs his business. He developed his own system for creating OO applications in a RAD way via the use of primarily code synthesis if I'm not mistaken. The end result being that he churns out some 50+ custom applications per year for different clients. My guess would be that these are mostly smaller applications, i.e. he's not replicating eBay or SalesForce.com, but he doesn't need to for that business model. And so because all those apps are custom rather than customizations of an existing app, the subject of portability for different clients doesn't come up.

@Shlomy: I tend to agree with that assessment. But by the same token I have to admit that a lot of developers have had a difficult time understanding the use case or the business case behind some of the tools in my frameworks. I'm still trying to find a way to explain the purpose of the branding features in the onTap framework because that's something that is based on a *major* problem that's affected the health and profitability of nearly every company I've worked for... and yet, a fair number of people read through my explanations or sit through my presentations and find themselves still scratching their heads. Long story short I think part of the reason we tend to make those kinds of assessments about other people's work is simply because of the availability heuristic and actor-observer bias:

http://en.wikipedia.org/wiki/Availability_heuristi...
http://en.wikipedia.org/wiki/Actor-observer_bias
# Posted By ike | 9/19/08 5:13 PM
Here are the two points that really get me most of the time:

"don't change database systems often enough to justify the extra typing required when using a DAO."

"end up having to edit your service classes *and* your DAOs when you add a column to your schema"

I've only changed my dbms once and that was when I moved my personal site from Access to MySQL. All I had were some capitalization issues and reserved column names. It was procedural code and I simply went through my cfqueries and updated them. I can't say that if I had been using a DAO that I would have saved my self any work at all.

And as to adding columns, I don't know that I have ever seen the addition of a column that did not in turn require changes to the business objects, and the services that used them. A DAO would just give me one more place to add it.

That all being said, you might still convince me to use DAO's solely on the grounds of grouping my queries in one place and not duplicating the same query multiple places in the code. In general I don't like to add layers to my code unless there is a distinct advantage. You have to admit, sometimes old procedural code can be the easiest to debug because everything happens right there in one file.
# Posted By Brad Wood | 9/19/08 8:42 PM
@Brad: I was actually not going to post this comment until I noticed it was you that left the last one. :) I really don't understand why people make a big stink about the differences between a "DAO" and an "Active Record". If you read the GoF book, the description of a DAO is actually generic enough to include an "Active Record", amongst a whole ton of other things. And at the end of the day, what we call a DAO and what we call an AR in the ColdFusion world (mostly) accomplish the same goals. Though I don't think AR was described in the GoF book - I think Fowler described it in his more recent patterns book.

Anyway one of the things I find generally frustrating about the code I see other people recommending is how much typing it involves.

In a real world scenario you can never completely eliminate coupling. You can however measure coupling in an application by the number of code-points (places) a particular change requires. More places to change means more coupling (and in other words, BAD OO design). This is something that Joe Rinehart mentioned on his blog a while ago. And yet, I still see people recommending the 5-to-1 object model Pete mentioned. At my last job, for every column I added to the database, there were FOURTEEN places in the code that needed to be changed because of that same design, combined with an insistence on the use of stored procedures for all data access.

I also mentioned this on my blog a while ago in an article about "duck typing" where I was comparing what is common in the ColdFusion community to the way I work. When I add a column to my database, in most cases, ther'es only ONE places I need to change the code outside of the business object that needs the new column. That one place is in the form. :) Working the way other folks ask me to work is frustrating to me in spite of the fact that I'm willing to do it, because there's sooooo much *COUPLING* in most applications.

But you really can do a DAO or an AR type object and you can even have a separate Gateway object if you want, without introducing all that extra coupling. You just have to know the techniques to get you there. And based on the comments I've read on your blog, here and a few other places, I'm betting you'll be pretty happy with the tools you'll find when you finally do get around to installing DataFaucet and the onTap framework to give it a test drive. :)
# Posted By ike | 9/19/08 11:10 PM
@brad - we originally had an app in-house which is now distributed without specific customisation to some 50 or so external organisations - 4 different RDBMS's so far. Oracle and MSSQL are not a perfect match. All I can say is that I'm glad we went with a DAO layer initially. That was a while ago. Now I would still go with this sort of architecture although I would also use an ORM. Some simpler apps we have created have worked seamlessly on Oracle and MSSQL using Transfer without having to change or edit any SQL. However I would still use Transfer via a DAO layer in case we need to use a DB not supported. (Derby has cropped up recently)
# Posted By richard | 9/20/08 2:36 AM
This is a nice stimulating blog post and the whole subject of interactivity between applications and databases tends to be different in the larger enterprises, in my experience. Perhaps I should qualify that a bit more by saying where an enterprise has a strong DBA or DBA's. Typically the database abstraction layer is usually in the form of stored procedures and ColdFusion is locked down so tightly that investigation of the database structure from CF is simply not possible. I actually see that as a much better route in fact and the recent SQL Injections attacks would have been largely ineffective in such an infrastructure.
# Posted By Mike Brunt | 9/20/08 6:50 AM
"When was the last time you actually took a live project and changed the dbms? " - Peter Bell

Of course it doesn't happen right away. I have worked on applications that have considered moving from Oravle to SQL Server or SQL Server to MySQL and have not done so because they felt the one time cost of changing all that code was too high.

We should try out best to write code that will be used and useful in 10 years.
# Posted By Allen | 9/20/08 3:10 PM
Typically the database abstraction layer is usually in the form of stored procedures

Agreed. Do database stuff in database thats what its there for.
# Posted By Bren | 9/22/08 7:14 AM
Changing DBMS? Not sure if you deal with the enterprise, but you have all type of clients of different sizes and when you need to sell a $20 million dollar system. You better offer whatever they ask for or someone else will.
# Posted By Jason Ervin | 9/22/08 9:56 PM
@jason - agreed, and it doesn't need to be $20m system. With CF for example the ability to deliver self contained webapps means there is scope for a different type business model - as you say you should make it available to as wide an audience as possible. Personally I find having a DAO layer clarifies things nicely and gives you a nice comfortable feeling when you are making wholesale changes to the service layer. I think Peter is mainly talking about small bespoke systems where changing DBMS is unlikely. On the other hand I don't think I've ever developed an application that couldn't be used by another client with minimal customisation.......

@bren - stored procedures are certainly efficient, but I'm not sure about going down that route if you want to offer an app that will run on multiple DBMSs. Converting SP's between systems is not fun, and it was not so long ago that mySQL didn't even support them
# Posted By richard | 9/23/08 1:31 AM
I think the main reason of splitting queries from a service layer into dao/gateway objects is separation of concerns. For that, might as well pack everything into a service layer.

I see huge benefits in splitting depending on the intricacies and requirements of the service at hand. I don't see a "YES ALWAYS DO IT" scenario. I have several services that might need 1 query and some services that are better handled by delegating to a data access layer. I see enterprise architecture as a big weave of layers, all with their responsibilities.

So in my opinion, dao/gateway objects can be written to achieve separation and layering, but not 100% of the time. That is when your code smell comes in, finding that happy medium. So I agree and disagree with both. :)
# Posted By Luis Majano | 9/23/08 7:21 PM
@Luis: I agree!

DAO/Gateways could be cool for, say, user and group objects. Makes switching from DB driven, to say, Active Directory, Not Too Hard.

It might not be DB => DB, is what I'm getting at. I think. :-)
# Posted By denny | 10/4/08 11:26 AM
Active Directory ... ouch... now there's a set of semantics that *try* to be difficult to use... last time I worked on an AD project (not long ago actually) AD user info was synced with the db rather than replacing the db user system. Is it really common to just omit the db user system in favor of AD?
# Posted By ike | 10/4/08 9:18 PM
@ike: I don't know how common it is, but using LDAP queries, it wasn't too hard to provide the same interface I'd had for authentication with the DB-DAO, for AD.

Implementing some type of caching might be smart, depending on network/etc..

If you use the samaccountname as your username/ID, and key off of that, you don't really need to pull the data from AD into your DB. But you've got to manage groups and stuff through AD instead of your app, so... eh, depends on the environment, and how portable you want to be.

I'd think syncing up users/groups would be safer-- and your app would work without AD as well. :-)

But my assertion stands (or rather, I'm standing by my assertion:]): the abstraction does make swapping out the bits easier. And it keeps stuff organized. Just swap out the ADUserDAO component with the DBUserDAO, and vice versa, vs. spaghetti code / ifs and elses and whatnot.

And when you need to pop in your WebServerUserDAO instead, hey, you already know the interface to code for.

Good use for inheritance, as they are "is a".

Course, half the sub-classed functions could come back with "unsupported operation"... you can only push the abstraction so far.

In practice, it's a lot harder to code for change, and it might not be worth it.

I don't like absolutes, anyways. Sometimes code for change, sometimes don't... keep things interesting, I say!

=]
# Posted By denny | 10/7/08 8:11 PM
@Denny - Generally agree with the comment about it being a good use case for inheritance if you're going to go with replacing instead of syncing.
# Posted By ike | 10/7/08 8:30 PM
BlogCFC was created by Raymond Camden. This blog is running version 5.005.