Friday, March 4, 2011

Ignoring long-term planning and estimating

I got on the agile bandwagon fairly late considering people are now celebrating the 10-year anniversary of the agile manifesto. But it's been a good 2 years now and I feel like it's a good time to sum up some of my experiences. And I'll start by writing something about estimating that's recently been on my mind a lot.

I get the impression that a large part of what people use to sell Scrum to managers is the ability to get better estimates and more reliable long-term planning. That tracking velocity and assigning storypoints to your features somehow enables you to determine what exactly you can release half a year from now. I don't think that's entirely honest. You still have no way to reliably predict accidental complexity; and team fluctuation, shifting priorities and delays in external processes that you depend on will still leave you guessing when you'll have that big new release ready.

The constant feedback obviously helps to at least be able to react to some of these obstacles. A looming deadline works wonders to get stakeholders to prioritize the features they want implemented. But I think all of this is looking at agile development from an old-fashioned angle, that is likely to be more frustrating than necessary.

Those big releases and big new major versions of your products just don't make all that much sense if you're not actually burning your software to disks to be put onto shelves. If you have a new little feature that will help your customers have a better experience using your product why wait to release it? If that feature actually doesn't help your customers, why wait to find out about it?

If you're a long-term agilante (agilista? agilero?), that is probably pretty obvious. But personally, I don't think I really grasped the power of that until recently. Really taking the idea of short iterations with incremental changes to its conclusion is not something I've seen much of in the time I've been developing software. You may read about some of the big poster-child internet companies working that way and increasingly, little start-ups seem to have a lot of success with it. But once companies get to a certain size they seem surprisingly eager to hold back on actually releasing their software.


And, with that said, I have decided that I do not care about estimating any more. I do care about the people on the team, I care about splitting items up into fine-grained tasks [edit: 'features' is probably the better word]. I might even care about tracking time spent on tasks to get feedback on whether we've become stuck on something. I care and take pride in the quality of the code we deliver. I care about people using that software and the feedback they can give. I don't care about dates in the future that may as well be arbitrary.

I conveniently ignore any kind of marketing needs or grand c0mpany strategy. Because, really, why shouldn't I? If we can consistently release new features in a short timeframe and the quality is good, why shouldn't the company strategy adapt to this rather than the other way around?

This idea is fairly fresh in my mind and I'm really looking forward to seeing what it develops into. I'm pretty sure I've missed a few things. And I'm curious what other people think about this...

Edit:
And to aptly prove the motto of this blog, Naresh Jain wrote about this two ears ago. Well worth reading.

5 comments:

  1. I think you cannot estimate if you don't have all the necessary information on the task. And seldom do you have them. So when we estimate we ignore (consciously or not) the uncertainties and then we are just too optimistic in our estimations. I think that we need to estimate for the small grained tasks we do, but really we as humans are not able to know enough to estimate with one year ahead for example. A nice model of estimation can be the cone of uncertainty, more information: http://www.codinghorror.com/blog/2006/06/the-mysterious-cone-of-uncertainty.html

    ReplyDelete
  2. Oh, hey Adrian!

    Thanks for the comment. Some of the things described in that link are what set me off writing this rant.

    I have since talked about this with a lot of people smarter than me and there seems to be consensus that estimation is problematic. But I'm not sure there are all that many companies that actually try to change their business around this fact.

    Here is another nice link that just cropped up http://blog.cutter.com/2011/03/08/committing-to-commitment-in-agile-teams/

    ReplyDelete
  3. Hi Daniel!

    I've got at least one good purpose for estimates: Help decide whether to include the feature at all: If it takes too long, i.e. is too costly, it might not be justified to include a certain feature, i.e. some exciter or linear feature (http://en.wikipedia.org/wiki/Kano_model)

    [We usually estimate stories no more than 6 weeks before working on them, not months in advance.]

    Of course this only applies when the developing
    company itself sets the scope. But I guess in fixed-price projects you would need estimates to know the price range...?

    ReplyDelete
  4. Hallo Corinna!

    Thank you for your comment!

    I've since realized that I conflated a couple of issues and I should really get around to writing a follow up post to sort that out for myself.

    I would probably divide the issues into commitment/trust, risk/value assessment and resource and dependency planning. And I think my original problem really was about trust and expected commitment which really doesn't have all that much to do with estimations.

    I don't blame myself too much because mixing up estimation and commitment seems to be a pretty common mistake. ;)

    And another link, just because...
    http://blog.patchspace.co.uk/why-cant-developers-estimate-time

    ReplyDelete
  5. Hi Daniel, sorry I'm late to the party.

    "I conveniently ignore any kind of marketing needs or grand company strategy."

    I believe you've turned your back on the crux of the matter. Continuous Deployment (or deployment as soon as something is Done: defect fix or feature), is a business decision first; It has to fit the company's business model.

    As the manufacturing side of the company, I think our responsibility is to communicate our ability to deliver in multiple ways and what technical and user benefits are associated with each. The ultimate decision to change delivery model is a business decision and generally out of the developer's purview once a company grows past the first handful of employees. But that's no reason to throw in the towel!

    A flavor of continuous deployment that I'm investigating right now is a bit of a compromise. As defects are resolved, push those to the client-accessible product right-away. This may not work for all defects depending on the scope of impact to the system but in my experience the scope of the vast majority of defects is very narrow. This method also has the benefit of freeing up defect resolution from feature development work and puts the tasks on independent tracks.

    ReplyDelete