Do You Bill for Bugs?
Let's say you write a custom system on either a fixed bid or an hourly basis. After completing the project, the client notices two things:
The first is something that clearly doesn't work correctly as per the agreed, written spec. It was clear in the mock ups that a feature should do x and it just doesn't.
The second is a reasonable edge case that lessens business value - you don't have a validation that the user is over 13 when you save their details and that is a problem (although that problem had never been discussed before).
If you work on an hourly basis, is the additional work also hourly until you're done? What happens if the client is out of budget? Do you usually do the work as goodwill and if so, what's your threshold - 1% of the billable hours? 3%? 5%?
If you provided a fixed bid, I'm guessing that you'd fix the first issue. What about the second?
Input appreciated. I have my own set of answers (which I'll post later), but I'd love to see what everyone else has to say!
I look forward to hearing how others approach this same issue.
That said, fixed bid I always fix bugs for free. Hourly is a little tougher. I often find that I have clients when I'm working hourly who want a "damn the details, get me the features" style of coding. I don't love it, but sometimes I'll just hack something fast because otherwise their requirements and budget would just never meet.
But then what happens if there is a bug? The bug is because I went fast. I went fast because they couldn't afford me to do things right. Even with me charging to fix the bug they're ahead of where they would have been if I'd "done things properly". Why would I eat the cost of fixing such bugs? Right now sometimes I do, but I'm not sure there is any good reason to . . .
To throw it back to you, how do you cover the cost of fixing those bugs (ain't nothing for "free")? Do you charge a higher hourly rate than you otherwise would, throw in a few hours in each estimate to cover bug fixes, work extra hours for free or make less money?!
As for your second scenario, we generally would charge for that at an hourly rate, unless it's something we felt bad about for not thinking of in the first place. Again, it really depends on the client and situation. Some clients we would go to the ends of the earth for. Some we would go to the ends of the earth to get away from :)
However, with fixed pricing all bugs found with a pre-defined period of time would be fixed for free. With edge cases, these would only be fixed for free if there was other work which needed to be done, either extensions of bug fixes.
If the client however, has a few edge cases, we usually just fix price the changes. I must be honest it often depends on our relationship with the client. If the client has been difficult to work with, we usually are reluctant to give any leeway.
In our experience, difficult clients never test their work during the allocated time. Good thing we have a hard-nosed finance guy who gives no quarter :-).
I have not had to exercise this yet, but I like the above comment, it depends on the client - if they have been nice and it is a big job that they have not squeezed the life out of, I would probably fix small issues out of goodwill. On the other side, if the client has been a pain, or the fix time is disproportionally large, they will be told it will cost them.
Somewhat of a subjective criteria, I know, but it's my work and I can make those kinds of rules.
For fixed bid, I would build a few hours in to the contract to fix bugs they may find later.
However, in all cases, I would at least /say/ we're going to charge to fix them. That's because often what may be an edge case to you, the customer may feel it's not - and they may have negative feelings towards you if that happens a lot. (For instance, a lot of these are miscommunication bugs.) If they know up front that they will need to pay for bug fixes, they have different expectations.
That said, if it's something caused by my oversight or incompetence, I generally will fix it without charging them. This is a classic case of underpromising and overdelivering. I'd be loathe to change that, I think.
In any case, my software is typically so bug free the issue just doesn't come up. =)
After that point, I basically do the same as Rob Wilkerson - if I am really embarrassed by the bug I fix it for free (and quickly!).
One thing we do with most clients that help cushion those efforts is install a maintenance contract of fairly small numbers of dollars and hours. We use that as "consulting time" with them to plan growth or build ideas for a next release, or helping with a website update, or what have you; we get enough of those in aggregate that doing bug fixes is then not an immediate loss of revenue.
Its worked out well the last two years. I know many clients will recoil if you sell them a maintenance contract outright, it needs to be value added and your relationship will need to be in very good terms...then its a no-brainer for them and you. Clients love security that their IT guys are around and will be there. Most of them have experienced the code and dash programmers.
As with most others we usually fix bugs on fixed bid for free. I'm trying to do a better job of tracking such changes so we can estimate more accurately - after all nothing is ever REALLY for free. Just gets baked in somewhere else.
We also make a distinction between edge cases and error messages.
The one thing I wish we could do is to do a better job of working with clients to capture edge cases and acceptance tests more thoroughly, but between working with unsophisticated clients, often having to provide fixed bids without a discovery phase and working through resellers and on limited budgets I find that it is a problem for many of those clients as they aren't interested in describing what the app should do - until the site is live and their clients are complaining about it not doing what they had expected!
@Peter
"often having to provide fixed bids without a discovery phase"
Curious that someone of your standing needs to engage on that basis. Isn't that just asking for trouble?
I'm in an interesting position in that my main focus is developing a software product line. The goal of that software product line is to generate custom, but fairly simple and affordable web applications quickly and cost effectively. As such, many of the projects I do are lower end than I would bother with if I wasn't doing them to improve the spl. As such I do a lot of inexpensive projects ($2k-$8k) but can do them very quickly so they're often more profitable than the higher end consulting I also do.
Asking for trouble? Well, yes and no. I have a spl that covers most of the variability I come across and I make it clear that my job is to meet their spec - not their needs, so I can usually spec a solution for (say) a custom cms or e-comm solution in 3-4 hours and build to that and turn a good profit. I've also trained our development partners to do a pretty good job of setting expectations and gathering requirements, so it's working out.
I am starting to move back towards the higher end projects, however. They're just more fun!
We no longer accept hourly projects because of the "do it now, do it quick; tomorrow I'll change my mind" approach. However, when we did such situations, we used to charge for the fix it time.
In essence, unless you're bidding way over to cover that cost, you're effectively escorting people away from doing business how I would think would be the more preferred way - hourly.
In one case, "oops, I messed up, but you don't have to pay for it (regardless of if doing it right the first time would have taken longer." In the other, "oops, I messed up. Too bad. Even though we're closing in on the budget, I'm charging you for it."
It would really suck to be (what I consider to be) the better client in that case. Why would I ever want to pay hourly (again, unless you're charging more for fixed-bid)?
No question that fixed bid is a valid approach. For non-trivial projects we recommend an upfront fixed effort discovery phase. Then we will provide a fixed bid for writing an app that meets the agreed specs (user stories fleshed out with full use cases - screens and actions for primary, alternate and error paths). However, that still doesn't cover how thoroughly the app is developed. When I played around with TDD I found that there were all kinds of edge cases that I usually didn't bother to code for and that wouldn't usually come up in acceptance testing or even in normal functioning. The question then is should I bid twice as much to deliver robust software or keep the cost low for the client knowing they probably wont get an ROI on the better code? And if I admit I'm skipping robustness to deliver them a better price, should I really have to pay for that by throwing in bug fixes for free?!
We historically did do bug fixes for free on fixed price and I'm not saying we won't now, but conceptually it actually doesn't make a lot of sense for the client as all you're doing is either losing money or putting in a pad which some clients will lose on and others will gain on. I see such pads as a tax on agreeable clients and a handout to the picky/annoying ones. Not sure that is quite right!!!
Of course, from an agile perspective, the BENEFIT is that the client can keep changing their mind as it allows you to work with them to build the application they actually need - not the one they thought they needed when they started. To keep the process manageable we recommend only fixed input points so within a scrum there is one round of revisions and the stories can be re-prioritized between scrums (which often last two weeks for us).
How do you price fixed bids? Do you commit to making the client happy or just meeting the spec? How do you bid features with high levels of technology risk (do you just fixed effort bid an additional discovery or spike for the feature or do you have another approach?). How do you manage numbers of revisions and rounds of revisions, and how do you agree "complete" (acceptance testing) and the difference between an un-included edge case and a bug (passing strings into numeric form fields, entering invalid field values to a drop down by hacking the form, browser support, load testing, etc)? Any input appreciated as we still do fixed bid work and are always trying to et better!
We commit to meeting the spec. And as specs change we have clients fill out a change form. Even if this is at "no extra cost" or "no change in schedule" we now have a precedent in place that they have made a change and the next one may not be free. As such, the number of revisions is really zero, although they almost always get the first "Change" at no cost.
With a high level of technology risk, let's assume we're learning a new technology. We'll often bid at a longer schedule, sometimes at a discounted cost to account for our learning curve.
The client signs off on each phase of the project before we start work on the next phase, although acceptance testing is an area where we could improve. When client asks for things and says "this was included." I'll often ask them to point me to the part of the contract spec that specifies this and we'll get it implemented immediately.
I suppose "we fix bugs for free" should be quantified in some manner. If the client is seeing a CF (or Flex) error, then obviously we screwed up and that should be fixed for free. But, if 3 months down the line the client decides that the list should be sorted in ascending order instead of descending order, that's a change request.
Firstly it sounds like we both take a similar approach to "dark matter" (the stuff NOT documented within the spec that comes up during the project). We explicitly gree only to code to spec - not to meet their business needs. The reason for this is we often find that additional needs come up during the development process and many of them just can't be delivered within the origina price. From your comment about show me where in the spec it is documented", it sounds like you take a similar approach to this.
I'm interested in the fix bugs for free bit though. Do you ever have clients who just want a relatively low price to get key functionality live ASAP? What would happen if one of them came back and by doing (say) some form value hacking they managed to get your app to throw an error? Do you just always make your apps very robust for all of the edge cases you can think of and test really thoroughly (thus making your apps more expensive than they would otherwise be)? If not, why would you eat the cost of fixing the error?
To me it's the difference between mil-spec and regular components. You can pay the extra for mil-spec or I'll give you the regular kit, but taking a regular piece of electronics into extreme conditions will void the warranty. Wonder if we should have similar concepts for software?
Thoughts appreciated!
We generally bullet proof form processing pages, which is the most likely cause of errors. Nothing overly complex, but it's along the lines of:
<cfif are all expected form variables defined>
do stuff
<cfelse>
Oops, you got here in error
</cfif>
That will catch a lot of the syntax type errors.
I have just found there are so many ways to break an app based on user input. Passing strings instead of numbers, entering dates like 31st February, entering date of births that are in the future, antering negative numbers (or floats) for integers, using form field hacking to enter a value for a drop down enum field that isn't in the list (and seeing how that affects reporting, etc), and then there are race conditions which can mess with optimistic locking. I'm developing a set of systems that allow me to handle as much of this as possible by just declaritively specifying constraints and custom data types for object properties to catch a lot of this, but I always find more edge cases that my cheaper clients are unwilling to pay me to write an app to test for.
Great question and great discussion!
My answer is that it depends on whether I'm selling time or software. In general, I prefer to sell time. I typically won't sell software unless I'm convinced that the client knows how to manage a custom software order, or has worked with me before on a time project.
I don't fix bugs for free on a time project, but I'm willing to do a ton of goodwill on a software project to make it great.
By the way, since I have the flexibility to do it, I also try to get any cool software under some public license as part of the project - that way, there is far more upside for me to take time away from my family to fix bugs on edge cases (and a by-the-by the way, "edge cases" typically equals "scope creep").
/ejt
The problem we often see, which kind of floats the line on this question, is that the concept of a 'bug' isn't the same from one person to the next. It's never moreso than with the customer. We now require on all quotes.. even if it's 15 minutes of work, a clear specification or statement of work. We won't do a thing without one. In some cases, it almost comes down to a line by line detail of the application, specifications of operation, expected results, etc. We even do this for building report templates for customers. This is a no-brainer to some, but alot of people get themselves into a fix by not doing this.
Generally speaking if we've done something for a customer and there is a resulting application error (meaning thrown exceptions, malformed SQL and the like) that's on us to fix. The problem we have had in the past is learning to think from the customer perspective and tailor our operations to suit. This distinction is often overlooked if most of your operations are made up of tecchy's and not a few suits lingering around.
From the customer mindset, they aren't buying software/code... they are buying a solution. As many of us in the organization came up through the development pool, it has taken us awhile to tune into that. So in that solution if something doesn't work it's considered by the customer a flaw, and thus as they aren't developers and don't understand the boundaries of things, a 'bug'. They will then lean on you heavily to fix their solution because it's flawed.. but to you it's doing everything exactly as they said. Then it comes down to he-said she-said until a resolution eventually comes out... usually with something being done as non-billable work.. enough of that and your projects fall behind, etc.
So when we've completed a project and customer X says "Well when I click dropdown Z, where is the list of wingnuts?" One of three scenarios exist...
1) We go back to the spec... no wingnut list was requested... bill
2) wingnut was referred to and missed ... no bill
3) wingnut was referred to and coded, but it appears the link to the SQL to fill the list was missing an essential WHERE... no bill
So I realize I took an offramp to the topic, but to talk about bugs and customers in the same conversation without bringing up topics like project scope and customer perception/expectations... well you end up with lots of 'bugs' that aren't really bugs and loss of revenue.
I think tools like rspec are a good way of mitigating the problem by working with clients to come up with a set of executable acceptance tests, but the time to do that can price many smaller clients out of getting code developed at all . . .