The Death of the Gateway?
I was never a big fan of what seemed to me to be an artificial distinction between DAOs and Gateways. Generally the idea was that a DAO would handle CRUD and/or single record operations whereas Gateways would handle aggregate queries. I didn't find that a particularly elegant distinction for two reasons.
Firstly, there is almost nothing to CRUD operations as you're going to abstract them out using either a third party framework like Transfer or Reactor or you're going to roll your own solution - perhaps using a BaseDAO with parameterizable method calls. It never made sense to me to seriously consider hand writing CRUD operations for every single business object - they're just too similar.
Secondly, as I spoke to different people who distinguished between DAO and Gateway, there didn't seem a consistency to the handling of edge cases. If a DAO is for single instance CRUD, where do you put a deleteByFilter() method (e.g. deleteExpiredCarts() or deleteRejectedUsers()) which deletes 0..n-records? Secondly, where do you put multiple instance insert/update methods for adding/updating multiple records? Of course, everyone who had these use cases had an answer, but it wasn't consistent between different developers suggesting (to me) a certain lack of elegance and therefore consistency in the distinction between DAO and Gateway.
Now I do know that some people like the distinction, and I think that as long as you have a consistent set of rules for knowing what goes into each class, if it makes it easier for you to maintain your code, there is nothing wrong with the approach at all (don't be surprised if you have to explain the approach to people not from the ColdFusion world - it isn't an approach I've seen in Java, .NET, Ruby, Python, PHP or any other language), but I'm hoping that it'll start to be seen as a possibility rather than a default approach to OO programming in the CF world.
Here's to some programming perestroika :->
Thoughts?
I know that one of your concerns about many of the ORM frameworks is that they seem to not handle the concept of a single "entity" spread across more than one data table. At least, I feel I remember you asking something about that during a CFUG meeting (to which the presenter didn't seem to follow your thought). In such a case, I am curious as to how you would handle that with an base DAO? Or, is that one of the cases there you would actually create an overriding method since it is more of the exception case than the rule?
And, as far as the filtered delete, I picture these more as convenience methods than anything else. I figure you have SO Much logic (potentially) in you DAO delete method, that you wouldn't want to duplicate that logic in a method just as DeleteRejectedUsers() (think about all the cascading deletes / updates that might occur). As such, I assume that something like DeleteRejectedUsers() would simply call a filtered SEARCH method, then turn around and do individual deletes on the objects for you. This would keep all your delete logic more encapsulated??
If you look at it that way, methods like DeleteRejectedUsers() are more convenience, service-type methods than they are "delete" methods?
Of course I am in like day-two of my OOP learning, so please take my ramblings with a grain of salt.
Great comments. I have never been a fan of the separation between gateways and DAOs (for the same reasons you cite), but I couldn't put forward my thinking as clearly as you did.
Don't want to use a gateway? Don't use one! Like 'em? Use 'em! Who cares? It seems to me we have far more interesting and important OO discussions to have than where people put their db queries.
I think your argument makes sense in the abstract, though in practice, I like separation between the two - especially when dealing with highly relational database architecture. I will say I favor Gateway inheritance of the DAO methods - which I'm sure will be considered heresy by some.
ORM frameworks, as powerful as they may be, will never totally eliminate the need for custom SQL, stored procs, dbms-specific aggregate functions, and the like.
That said, I have no problem with considering it an option - so long as it continues to be recognized as a "best practice" for dealing with unique environmental variables.
It's Peter's blog and he can do what he wants with it of course, so I guess I'm really asking "why again, and why now?" Nothing has changed on this issue since a few months ago other than a trend Peter seems to be seeing on mailing lists.
Honestly, I am becoming more and more convinced that architecture is too fluid that the same answer may not even apply between two separate projects with the same set of developers. Thus, limiting ourselves based on some rigid set of rules seems too constraining to be very useful. While, the "architecture" analogy fits the problem here in the sense that, much like a building, what we build needs to be structurally sound enough to stand on day one, real architects don't have the flexibility to refactor a building as needed once it is constructed (as we do).
I think the key here is to create an architecture flexible enough to adjust as needed during a projects maturation. Whether you stick some query in a DAO or a Gateway is neither here nor there when it comes to the big picture.
I could go on of course, but they are apparently paying me to do something here...just trying to figure out what that is ;)
On a different topic, just because I _have_ to know...tell me you had to look up the correct spelling of "perestroika", Peter...
When you write an ORM, you want to write in support for one table per class, one per concrete class and one per abstract class . . . http://www.pbell.com/index.cfm/2006/10/14/ORM-Appr... It's just a matter of writing the code to automate the joins and to auto-add bit fields for each concrete object depending on which inheritance strategy we use.
@Matt, Gotta disagree. Just because WE remember the last round of discussions doesn't mea everyone does. I come across lots of people at conferences who seem to think a gateway is "the one true way" as it is what they were taught. I think that is a bad thing, When most people I talk to get that it is an option but not a necessity and understand the design decisions and trade offs, then I'll quit talking about it. If my blog postings were only for other people (like yourself) who teach OO design patterns, I'd be blogging very different content.
@Jon, I'm not sure what you mean about gateways inheriting DAO methods (although I'd love to find out!). You're right, depending on use case, and ORM may not get you all the way. I would NOT consider Gateways a best practice at all. I don't think there is anything wrong with considering them as an option, and I think it'd be great to see more postings from people who do use gateways clarifying the use cases they think they are best for. Certainly there is nothing wrong with breaking up your SQL methods into more than one bean, but I don't love the idea of DAOs and Gateways and think there is room for re-evaluation of the distinction and looking for other alternatives.
@Russ, OK. What would you like to see and I'll see if I can oblige!
@Brian, I'll agree - lack of consistency doesn't make the pattern wrong. I also agree that architecture is fluid and architectural patterns should reflect the design forces for a given application. I also know that to do so, developers have to go beyond "gateways are OO" (or "gateways are NOT OO") and beyond "it depends" to discussing the heuristics on which it depends.
@Rob, I could go with that. I'd still like to get away from the DAO/Gateway distinction as the default approach though! And no, I didn't have to look up Perestroika. The benefits of decades of reading the Economist - I guess!
Im for doing things better and faster. Im willing to compromise some level of abstraction for quicker development times as long as it doesnt compromise maintainability or performance. Im not saying one way is better than the other, I just would like to see several different ways of accomplishing the same thing to see what fits my style best. Its no different than frameworks. They all accomplish the same basics things, some just fit you better then others.
Also, what are your thoughts on the actual bean objects knowing how to perform the crud functions on themselves? For instance if I have a User bean that holds some data, instead of calling a save function in my service object and passing in the bean, I could just call User.save(). Havent seem many folks in the CF world doing this but its big in a few of the other languages, especially Ruby and no not just in Rails either.
Let me state my case a bit more succinctly: this isn't a big deal. It just isn't. You either have two objects or one doing db access--if only that was our biggest issue to "solve" in our architectural pursuits.
I guess that personally I'm just tired of the debate because it's so utterly inconsequential. I think people get paralyzed with discussions of such utter minutiae like this, so while I'm probably not one to talk since I haven't done this myself, these discussions are SO much more beneficial if they're presenting with a bit more of an even hand and include code samples. If the people who aren't as well versed in this stuff saw just how irrelevant (in my mind) this issue is then I bet there would be a lot less discussion on this "issue."
Let me do a crude summary that I think actually addresses the issue in its entirety:
"The Gateway Way"
DAO: contains only create/read/update/delete methods--deals with single rows only
Gateway: contains methods for dealing with aggregate data
"The Non-Gateway Way"
DAO (or whatever you want to call it): contains all database access methods including both CRUD and aggregate data operations
Is that really such a big deal? Does it even warrant more than a rather passing thought on the matter?
Gateways aren't good or bad. If you have lots of database access methods, breaking them into two files per business object can help with organization. In the CF world, some people do that by having DAO and Gateway classes.
I personally just have DAOs. They all extend a BaseDAO. Usually you want to favor composition over inheritance, but a UserDAO "IS A" DAO so I have no problems with this. The BaseDAO has parameterizable CRUD methods for insert(), update(), save(), deleteByProperty(), deleteByFilter(), getbyProperty() and getByFIlter() which means I don't have to write the core CRUD methods for each object.
I also have a little bit of magic going on which allows me to use XML to describe most of my additional methods - if I want a deleteExpiredCarts() method or getNewUsers(), I just add a line of XML to my metadata and my framework "pretends" I'd actually written the method so I don't have to write them all and have them filling up my class files. Between that and the base methods, I seldom have very much code in any of my DAOs, although that's also a function of my use case - I build lots of relatively simple apps rather than (say) working for years on one application with hundreds of data access methods per business object.
As for beans and CRUD, as long as it's handled via a composed DAO, it really doesn't matter. I happen to like being able to write User.save() as opposed to UserService.save(User), so I have that method and I use LightWire to inject the UserDAO into the User bean (you can also do this via ColdSpring, of course, although most peole seem to write a factory which is then managed by ColdSpring rather than using singleton=false). I still have a UserService.save() method as I need that should a remote method call (e.g. a flex app) need to save an object, but it just creates a new() User and then User.loadStruct(Input) to give it the data and then calls User.save() so the bean (via the DAO) is actually doing the work. This is more common that you'd think in the CF world. A number of CF people I know use this approach.
I'm not overstating my case. I hear a lot of real confusion when I chat to people at conferences who are learning OO in CF.
You do make a good point though - you're right that Gateways are NOT taught as the one true way. The people teaching this stuff know that they are just an option. However, when the vast majority of people teaching CF OO show a gateway as part of their architecture, one of the templates in the code generators is usually a gateway, one of the patterns described in the lectures is a gateway, it is taken by people who are still learning to be one of the files they need to create.
But it is a good point. I heard from Brian Rinaldi that some people take what I say as an argument that gateways are bad. Truth is, I'm with you. One file, two files - whatever (although I'll err on the side of YAGNI and would suggest the default advice be to have one file until or unless you know you're going to have enough methods to need two). Gateways aren't good or bad.
I still stand by this not being a "bad" posting as there IS confusion out there over this even though we've both looked at the issue and come up with approaches that fit out use cases already.I am hoping we're getting towards the end of the commenting on the posting though!!!
People could have a single db access object called zamboni for all I care as long as they understand what purpose this serves and why it's a good idea to separate things out in this way. Worrying about DAOs vs. Gateways doesn't address the real issue which is isolating db interaction from the rest of your app. What I'm getting at (or trying to) is that the people who are confused to begin with are just as likely to get more confused with this type of discussion rather than less, because they may not even understand why db objects exist in the first place and what their purpose is. Once people understand that basic point, they'll be able to make up their own minds on this issue based on their preference or architectural needs.
One more thing I'd like to say--YAGNI doesn't seem worth applying here and I wouldn't say that one file vs. two should be decided solely based on number of lines of code in either object. YAGNI in my mind is for stuff like "yes, we've been tasked with writing a customer management app, but let's throw wine cellar management in there just for kicks." (An extreme case, but you see my point.) In the case of DAOs and Gateways, the absolute worst case scenario if you create a Gateway and decide later you'd rather just have one object is that you cut and paste methods from your Gateway into your DAO (or vice-versa ... it is just a name after all), change references to the object you're getting rid of, and you're off to the races. If we had refactoring in a CF IDE this would be a trivial task that you wouldn't think twice about doing even on a whim.
Clearly people have to learn how CFCs work on a basic level before they can do any of this, but I'm not sure why you'd try to get people NOT to think of OO while teaching CFCs. If you're talking about just the basics of here's what they are, how they work, here's what cffunction is, how CF looks for them when you instantiate them, then yes, I'd agree. Once you get the hang of that though you're immediately getting into how to actually use the things, at which point I think it would be rather natural to start talking objects and OO.
People learning OO *definitely* shouldn't worry about this. The whole ORM thing is one big pseudo-OO compromise where the precise shape of the compromise depends more on the environmental factors (e.g. language capabilities) than design. Not a good way to learn anything.
@Peter
One way to think of this in a modelling sense is that gateway functions are behaviours of persistence -aware collections. If you think of a type "Set" with usual Set semantics, then a subtype "PersistableSet" that knows how to talk to the persistance store, then allow any object to compose a mixture of persisted and non-persisted collections, the gateway stuff starts to look like an internal implementation detail of a collection type. I stress *internal*. The specific type can then make a call as to whether it wants to do stuff like lazy load and whether to execute searches by in-memory searching algorithms or by querying the persistence store.
The DAO, OTOH, is part of the (hopefully transparent) mechanism for persisting a discrete object. I'd argue that ideally it shouldn't be part of the app at all, but is really a function of the persistence framework.
Bottom line being that for *performance reasons only*, I could create a collection type that will get some speed by directly manipulating the persistance API, but that unless that problem arises my app shouldn't have to deal with gateways or DAOs directly.
Now, obviously this is moot in the CF environment for a whole bunch of reasons, chief among them being that we don't have transparent persistence, and that we don't really have the grunt to deal with large object graphs, and more specifically OO collections. However, it seems to me that in a modelling sense gateways and DAOs really are different things.
Jaime
Matt points out:
"Once people understand that basic point [isolating db interaction from the rest of your app], they'll be able to make up their own minds on this issue based on their preference or architectural needs"
Well, I get that basic point and recognize the importance of abstracting out your db interaction. It's the "preference vs. achitectural needs" issue that I'm stuck on. If it was simply up to my preference, I'd get rid of the Gateway. But as a newbie there's that fear that I'll regret that decision due to an architectural reason somewere down the road that I'm not knowledgeable enough to think of now.
I always just assumed that there was a good reason most people talked about DAOs and Gateways instead of just DAOs. I guess my problem is I haven't really seen (or maybe just not grokked) a good example of an arcitectural decision that would justify using 2 objects over just one. So these discussions are definitely helpful to people like me who are looking for such examples.
1. DAOs are simple, repeatable, and discrete. If they always only do CRUD for a single table, they're easy to generate and leave alone. In many cases you could use code generation to crank out your DAOs and then you never have to open those files or mess with them--they just sit there to be used as needed.
2. Gateways, OTOH, are a lot more abstract. I work with Gateways a lot as I build out applications, adding methods as needed, changing things around, etc. In some applications these objects can get pretty large. Personally I prefer having the DAO as a standard object that just sits there and the Gateway as the one I do a lot of work in because it's more fluid than the DAO.
If these don't make sense to others, then by all means have a single object! In this case I think it would be rather difficult to argue that one way is "better" than the other. At the heart of it all we're talking about is a bunch of cffunction wrappers around cfquery tags (in the most basic terms at any rate), so I just hate seeing people get hung up on this. If you like Gateways, use them. If it feels weird to you to have two separate objects dealing with your DB calls, collapse this stuff into a single object.
What I've learned over the years since is just how quickly people pick up on one piece of advice and hold true to it as "religion". That speaks to the other point that Matt made about people needing to get the big picture and understand that there's no One True Way - as if I haven't said "It Depends" often enough to drive people crazy.
People like simple, clear guidelines. If I'd tried to articulate all the rules of thumb I use when designing applications, people would have either thoughts CFCs / OO was even harder than they do now or been totally confused - or both - and probably thought me some kind of raving lunatic into the bargain.
It's why I've been beating this whole DAO/Gateway horse lately and have also started on Design Patterns (that you don't just pick a set of design patterns and - voila! - instant application... despite some presentations by well-known CFers that lead people to believe that's how you use design patterns!).
Me? I have (data) gateway objects that interact with an ORM for the most part (and no DAO objects). But I don't have one gateway per table either (in general) - I have one gateway per domain concept (usually a closely related group of tables). I have one controller per user interface concept (which may not be the same set of concepts) and as many services as I need for the workflows in the domain model that cannot easily be encapsulated within a single domain object. Would that have been useful advice in 2003? Would anyone have actually understood that? How many people really understand that today?
My architecture/design style evolves over time - I've been doing OO since 1992 (in C++, adding Java in 1997) - so most of my rules of thumb are almost completely internalized now. As folks become more experienced - and, more importantly, as they experiment more - then they'll develop an internalized set of guidelines that balance OO "dogma" with OO "pragmatism".
Here's my take on a DAO that selects and deletes based on any columns you choose and inserts whichever columns you like.
http://www.adrianlynch.co.uk/post.cfm?postID=21
I'd love peoples' opinions on this.
As for the approach, it is conceptually similar to what I do, I just cover some more use cases such as getting by filter (you may want to search where CreatedDate < '1/1/2002' or using LIKE or other operators), I also support automagic joining, so if I as a User for FirstName,LastName,Boss.Title, it'll left outer join on the table the Boss object is in and bring across the users bosses title. I also include the ability to define aggregate fields for has-many relationships so I could also ask for a pseudo property like NumberofAuctionsWon, and allow for deleting by filter (e.g. all old carts) and have versioning so when you edit an object, depending on settings it usually adds a new version of the object.
I also use generic code so I can use the same BaseDAO to do all of the above for any business object so I don't have to generate reams of code, but thats a pretty small implementation decision.
Definitely nice to see another approach!