Configuration vs Programming
Steve Gordon had said in response to an earlier posting:
"Configuring already developed software into a solution is not software development. It is, however, a good way to deliver value, when it is feasible.
Building the software that can be safely configured into solutions is software development. TDD (at both the unit level and system level) is a recommended practice for software development. Consistent quality is especially important if that software is going to be configured into solutions by unknown 3rd parties."
But that lead to the following response by myself. Any thoughts would be much appreciated:
Agreed, but what IS configuration? The more I write software products lines, the less I am able to distinguish between configuration and programming. Let me give an illustrative example . . .
Lets say I want to call a popular framework – say Hibernate for handling persistence methods. If I write classes that call and wrap Hibernate to provide a public API for persisting objects relevant to a particular project, I think it is pretty clear that I am performing software development.
If, at the other extreme, I have a pre-built piece of software for persisting orders and order items within a specific shopping cart and just sell that software to people without making changes, while I performed software development the first time to create the shopping cart persistence code (and therefore should have tests for the core system), I’m clearly not developing new software by just accepting cash and sending a zip file to additional clients.
If I have a general purpose shopping cart with a configurator, so clients go to my website, set some rules about what properties an order item has, how attributes are handled, etc. and get a generated zip file with custom code automatically generated based on their configuration settings I would argue that is probably configuration rather than software development as the code is different for each client, but it is based on bounded configurability and no custom code was written by hand.
But what happens if I provide them with a content management based front end for a DSL for exposing unlimited variability in areas such as the properties of an order or order item and the screens and actions implemented? When they create the specific apps, are they performing software development or not? I’d be tempted to say “not”, but I think an argument could be made either way.
Then what happens if I keep exactly the same DSLs for describing properties, screens and actions but change the concrete syntax to XML – are they programming or not when they customize the application to their requirements? What if I go a step further and again without changing the grammar or constraints of the DSLs, provide them with a little language where they can write text files that are used to generate and/or synthesize code? It’s now starting to look very much like software development with the potential for bugs in their text files, logical errors and a range of other issues that might suggest tests be appropriate, but is it software development or configuration and what criteria do you use to determine whether project specific tests should be developed?
This raises two questions. Firstly, when does configuration become software development? Is it based on the move from bounded configurability (setting values for specific variables) to unbounded configurability (such as using DSLs for extending an application)? Is it a function of the concrete syntax and would you argue that using a content management system is not programming, XML is or isn’t and a little language is?
The second interesting question is when writing project specific tests is appropriate.
I’m guessing the answer to the first question is both subjective and relatively meaningless (although there is great value in agreeing that there isn’t a black and white distinction between configuration and programming). The second question should have an answer that is much more useful and practical as at the end of the day I’m guessing it will be based on whether you can create better, more maintainable applications in less time by creating the tests or by not creating them and that from those kind of first principles the appropriate testing strategy for a given project should emerge.
Thoughts?
Re your first question - agree that it's pretty moot. I've just been reviewing http://www.pbell.com/index.cfm/2007/5/28/Replacing... and http://www.codeodor.com/index.cfm/2007/5/19/Is-XML..., and it most of those points are relevant here as well.
Re your second question - why would one ever not test? "load and eyeball" is the default test methodology, and then the decision as to whether and how much to automate the testing isthe same kind of decision for any "quick hack vs robust framework" decision - how much time can we save by being DRY, and how much repeatability do we need? These dimensions have nothing to do with any distinction between configuration and programming, but rather are to do with
a) the how non-trivial the problem is (and therefore how valuable the solution) and
b) the likely useage, longevity and investment priority of the system in question.
A good example is server monitoring. My LAN server is a standard W23K install, on which I've done nothing but "configure" an AD name and a bunch of users. I can tell whether this system is working or not in about thirty seconds just by trying to log on. The test "fixture" is trivial, but the system priority is so high that I've developed an automated "test suite" (i.e. a set of monitoring rules) that I run every minute.
At the other end of the spectrum is something like an code framework. Unlike my LAN server, where I'm worried about power outages and rogue automatic updates, the circumstances in which my framework will go bad are very limited - basically, one of the approved committers has to let a bug in, and then I have to deploy the upgraded version. Howver, the test fixture is so complex that even though it might only get run every two months, it's very much worth automating.
So in the first example it's the high value of the deployed system that drives the testing decision, in the second it's the high value (i.e. cost) of the testing procedures. Configuration vs programming doesn't come into the picture.
Jaime
Great comments - thanks very much!