Estimated vs. Actual - Do you KNOW How Well You're Doing?
One of the many reasons I want to instigate a better project tracking system at SystemsForge is that we don't currently consistently track estimated vs. actuals for tasks, so we're not getting the feedback we need to improve our estimates by seeing how close they are.
We've played with a number of tools, from Excel spreadsheets to Unfuddle to Harvest. but we've never kept up with a given system long enough to get meaningful long term data. Do you track estimated vs. actuals? Do you do retrospectives looking at what caused variances? How do you track actuals in a way that isn't intrusive and/or is actually valuable to the team? We're trying to find that balance between data for measurement and keeping the amount of paperwork and busy work to the minimum. What do you do? How's it working? Would you recommend it? Why?!
I have to admit that while reading it, I was gung ho about attempting to implement such a thing at our company, but never got around to it. While it's very problematic, I will take on Jacobs challenge and risk lying or being delusional ;)
Estimating effort is something that programmers can get better at doing. I'll be the first to admit that I suck at it, and hate doing it. However, I'm better at it now than I was 5 years ago. Just like stepping on a scale after christmas, comparing estimates and actuals is a reality check that will bring home certain truths. Doing this on a regular basis will probably extract tangible patterns which individuals (and management alike) can learn from. No silver bullets, but we all like improvement.
As I said it's not perfect but is better than not traking at all!
biggest hole? missed tasks and task splitting.
If you store a reasonable amount of info about your projects and tasks, then once you get data on some project accumulated, you can start analyzing your project design processes to figure out where you're off and why that might be happening ... you can also start giving more accurate estimates if you've done similar work in the past.
The use of metrics and models is pretty big in software engineering - I took a class on it about three quarters ago. If you develop an estimating and tracking process and tailor it to your environment, it can be powerful. However, as John points out, it won't help you see the future, it'll just give you an idea of what you might expect.
I'm still in the process of setting up the dev environment at my new job. The department is small - me - but I still need to design a good development process. I'll take a look at fogbugz to see if that might be helpful to me.
Good luck!
@Sakri, I also find the more I do this, the better it makes my estimated.
@John, Yep - important to realize your figures won't be perfectly accurate, but ballpark numbers can still tease out useful heuristics for why some estimates are so far off, and soe root cause analysis can often come up with improvements in your processes from those data points.
@Mike, Thanks for the fogbuz recommendation - heard good things about it elsewhere, and of course love reading Joel's stuff.
@Dave, Sounds interesting. Let us know what you end up setting up!