Using Velocity Charts and Burn-Downs to Drive Development

Posted by Jason Smith - 28/02/10 at 01:02:20 pm

Imagine you’re the driver in a car cruising along the Seven Miles Bridge in the Florida Keys. Sunny day. Clear. There are no other cars, and you’ve been given permission from the Florida Troopers to go as fast as you like – so you’re cooking along at 80-90 miles an hour.

Now imagine you’re doing that with someone in the back seat holding their hands over your eyes and letting you take a 3 second peek every 30 seconds or so.

Sound like fun?

Not if you’re anything like me. I’d want to know if I were getting anywhere off track as soon as it happened. Hundreds of pounds of steel, plastic and simulated walnut trim develop a fair bit of momentum. The only way I’ll keep that car under control at that speed is slight steering adjustments as soon as I notice I’m drifting.

So why do we run projects like that? How often have we sat through projects with a weekly status report where folks give the old “we’re on track” for the first 10 weeks of a 12 week project, followed by a “we’re 90% done” for the next 3 weeks? Or we’ve taken on an 18 month project, and only found in month 16 that it’ll take at least 24 months. How well can a project running under a full head of steam radically change directions without crashing and burning?

Velocity Charts (also called Burn-Downs or Burn-Ups) are the simplest way to get immediate visibility into whether a project is drifting off course (and are incredibly effective at identifying fairly early in the process when a project will really be completed).

The way they work is extremely simple and incredibly accurate.

Estimate Sizes

Given a set of tasks, make an estimate of their sizes (often called “story points” or “task points”). There are a number of rules to get good estimates, but the simplest approach uses these rules:

  1. The estimate is unitless. It’s not hours. Not days, weeks or months. It is just a number. Each task is estimated in size relative to the others.
  2. The values we use are 0, 1, 2, 3, 5, 8, 13 and infinite/too big (each value is the sum of the preceding 2 values). Or you can use a doubling sequence: 0, 1, 2, 4, 8, 16 and infinite/too big. These values are used for the following two reasons:
    • Humans are better at saying a task is “bigger than that other one, but not as big as this” but we have problems identifiying precisely how much bigger. Both series account for that.
    • Humans seem to not be very good at estimating sizes that are more than an order of magnitude difference, so we cap it at 13 (or 16)
  3. If this is the first time you’ve done this, simply identify the smallest task in your set and assign it a value of 1. If this isn’t the first time you’ve done this, compare this to your last project’s items of size 1. Is it the same size as the first? Give it a 1. Bigger? Twice as big? Three times? Give it the appropriate number. Repeat this for each task you’ve identified, comparing their relative sizes to the previously sized tasks.
  4. Any task bigger than a 13 (or 16 if using the doubling sequence) must be broken down into smaller pieces. The likelihood of not understanding the task well enough is pretty high
  5. Record these tasks into a list.

Notice that throughout the exercise above, there is no estimate of time. History demonstrates we software people profoundly suck rocks when estimating how long it takes us to do something (I’m convinced our egos get in the way). This is all about relative size.

Track Progress

This is where time enters the picture – not as a guess or estimate, but as a measure of what’s really happening.

  1. Write your estimates down on a grid (a spreadsheet works beautifully here). First column is each task, second column is the associated size. The rest of the columns will be updated daily.
  2. Regularly (I prefer daily, but you can get away with weekly if you like driving with kids’ sticky hands over your eyes), have a quick standup meeting with the team (shouldn’t last longer than 15 minutes). Ask them what items from the list of tasks are complete. If (and only if) the task is complete, enter a zero in the first empty column for that task, otherwise carry over the size from the previous day.
  3. Sum the columns and generate a line or bar chart from the sums.
  4. Calculate a regression line from the chart (Excel calls it a Series). The negation of the slope of that line is the Velocity for that team for that period. So, if the slope of the line is -1.67 and we are doing this daily, it means the velocity is 1.67 task points per day.
  5. Multiply your Velocity and the sum of remaining sizes, and you have a good idea of the number of periods (days if you’re doing this daily) of time before the project is complete.

Here’s an example chart:

Generated from this data:

Caveats

The velocity’s predictive power improves with more samples, so your first 3 or 4 days won’t give you a very accurate prediction of when the project will be done. Each sample improves things tremendously though.

Different teams have different velocities, based on the size of the task they used as their size 1, as well as the size of the team. As a result of this, DO NOT USE VELOCITY FOR COMPARING THE EFFECTIVENESS OF TEAMS – IF YOU DO, YOU WILL BE COMPARING APPLES TO VOLKSWAGENS AND THE TERRORISTS WILL WIN.

Additional Information

An excellent way to generate even more accurate size estimates can be found here:

http://www.planningpoker.com/detail.html

You can get additional information on Velocity Charts at these locations:

http://www.versionone.com/Resources/Velocity.asp

http://www.controlchaos.com/about/burndown.php

The following is a great way to do burn-downs in the face of changing requirements (that never happens, right?):
http://www.mountaingoatsoftware.com/scrum/alt-releaseburndown

The Bottleneck

Posted by Jason Smith - 20/02/10 at 10:02:47 am

Nearly 70% of all software projects are considered unsuccessful. They are late, over budget and have fewer features than users wanted (and often ones they didn’t want – I’m looking at you, Clippy).

I gotta ask, would this situation exist if we had perfect knowledge of the future?

If we knew the user interface would confuse our users – would we spend as much time as we did implementing it the wrong way? If we knew our design would lead to performance issues, wouldn’t we select an alternate before we were too committed? If we knew the code we just wrote had a bug in it wouldn’t we fix the code before it even consumed a second of QA’s time? If we knew that QA would find a bug in a third party product caused by the way we called the API, would we have altered our design to compensate?

Most of our problems developing software would be solved by knowing, by learning the things we don’t know earlier and faster.

Now consider the Theory of Constraints:

  • Any manageable system is limited in achieving more of its goal by a very small number of constraints.
  • There is always at least one constraint.

In a nutshell, we will see the greatest improvements in our ability to reach our goals by understanding and defeating our greatest constraints.

If that’s so, then software projects are unsuccessful because we haven’t optimized the activity we spend most of our time doing. That activity is not coding, requirements gathering, bug fixing or dropping in an easter egg. The biggest constraint we have is Learning:

  • Learning what the customer wants
  • Learning whether what we wrote is what the customer really wanted
  • Learning how use an API
  • Learning code the guy down the hall wrote
  • Learning when we’re actually going to deliver
  • Learning whether our code actually works
  • Learning why it didn’t in a special case
  • Ad nauseum

Good product developers intuitively understand this. They actively seek out ways to learn faster and earlier.

They learn early that the UI they are doing meets the user’s needs and doesn’t confuse (prototyping and seeking active customer feedback). They understand that it’s orders of magnitude cheaper (in both dollars and effort) to learn about and fix a bug when the bug is introduced (automated unit testing and continuous integration). They understand that the fastest way to help our coworkers learn what we are doing is by collaborating with them (pair programming, daily stand-ups and being colocated). They learn about their risks early by working the risky code early (spike solutions). They know that code that is easy to understand is easier to learn and use (simple design, simple code that expresses intent).

Optimize the process of learning, and we optimize how we develop software.

Exceptions are Easy. Except When They’re Not (Part 2)

Posted by Jason Smith - 13/02/10 at 02:02:28 pm

Nothing is said which has not been said before. ~ Terence (195/185 BC-159 BC) Playwright of the Roman Republic

In a previous post, I posed two questions to my readers (all √(-1) of them): whether a body of code was exception safe, and if there was anything I needed to change to insure the code was consistent should an exception be thrown.

The pertinent code is reproduced here: 

public class Foo
{
    // ...
    private void Load()
    {
        ServiceProxy<IFooConfig> proxy = null;
        try
        {
            proxy = ServiceHelper.CreateProxyFromServiceContractInf<IFooConfig>(
                           InfoSourceUri.ToString());

            IFooConfig config = proxy.ProxyContract;
            this.NumberOfFrobs = config.NumberOfFrobs;
            this.WhereToGetThem = config.FrobSource;
            for (var e in config.MiscellaneousEntries)
            {
                this.Miscellaneous.Add(e);
            }
        }
        catch (Exception ex)
        {

            // Report the error and then ...
            throw;
        }
        finally
        {
            proxy.CleanupProxy(); // Extension method that aborts if proxy is faulted or ignores if null
        }
    }
}

To answer the first question we need to understand what exception safety is. It’s generally broken down into 5 categories:

  • No guarantees — when an exception is thrown, the program crashes — often leaving persistent things in an indeterminant state.
  • Minimal (or “No Leak”) guarantee. The program doesn’t crash, and resources don’t leak, but there are no guarantees about side effects.
  • The basic (or “Weak”) guarantee: the invariants of the component are preserved, and no resources are leaked (called the “weak guarantee”, as the system is left in a safe but altered and unknown state). Recovery is only possible by introspection of the state of the objects involved.
  • The strong guarantee: that the operation has either completed successfully or when an exception is thrown, the state is exactly as it was before the operation started.
  • The no-throw guarantee: that the operation will not throw an exception.

Given those definitions, Foo.Load() provides at least the “minimal/no leak” guarantee for Foo. Since there aren’t any stated invariants about the class’ state, you could also argue it is in a “valid” state, giving the Basic/Weak guarantee.

The second question I asked points to the problem, though. As written, when an exception is thrown in Foo.Load(), the Foo instance will be in an unknown state. Which means it is unusable (or you have to break encapsulation and engage in some introspection to understand its state).

Let’s say Foo is used to configure something else in your system. It got the “NumberOfFrobs” value for that service, but if the call on the line that set “WhereToGetThem” threw an exception, the old value of WhereToGetThem (possibly) remains in that instance. Even worse, we’re not really sure where the situation broke down (what if the exception threw half-way through updating the Miscellaneous array — which portion is valid?).

This problem becomes very interesting when the behavior of an object depends on the state of several members of the object and only half are current.

There’s a very straightforward solution to this mess: Insure code supports the “strong exception guarantee. This means calls either succeed or they don’t change the system. Period. When we catch an exception from a method call, we know our program is in the state it was immediately prior to the call.

If this sounds familiar, it should — it’s just like working with transactional databases (or any other transactional system).

Do the error prone stuff first and then commit your changes in a block of code that can’t throw an exception:

    private void Load()
    {

        int nFrobs = 0;
        Uri where = null;
        Dictionary<string ,object> misc = null;

        ServiceProxy<IFooConfig> proxy = null;
        try
        {

            proxy = ServiceHelper.CreateProxyFromServiceContractInf<IFooConfig>(
               InfoSourceUri.ToString());

            IFooConfig config = proxy.ProxyContract;

            // exception-prone code:
            nFrobs = config.NumberOfFrobs;
            where = new Uri(config.FrobSource);
            misc = new Dictionary<string,object>(config.MiscellaneousEntries);

        }
        catch (Exception ex)
        {
            // Report the error and then ...
            throw;
        }
        finally
        {

            proxy.CleanupProxy(); // Extension method that aborts if proxy is
                                          // faulted or ignores if null
        }

        // NO THROW GUARANTEE required:
        // from this point on, the only valid code is code with
        // no-throw guarantee.  If default property setters can
        // throw, have a version that doesn’t (or set the field
        // directly).

        this.NumberOfFrobs = nFrobs;
        this.WhereToGetThem = where;
        this.Miscellaneous = misc;

    }

Note the comment near the end of the code — to have the strong guarantee, you must have some code that satisfies the no-throw guarantee (e.g., no exceptions are allowed to escape “no-throw” code — a good requirement for the IDisposable.Dispose() method, by the way).

That’s it. If you have a reference to a Foo, outside of Foo.Load(), it’s consistent.

If you’d like to dig further into this subject, David Abrahams originally wrote about this subject several years ago. While his original paper discussed it in the context of C++, the issues apply to .NET based languages as well.

Exceptions are Easy. Except when they’re not.

Posted by Jason Smith - 10/02/10 at 09:02:10 am

Assuming all the code compiles correctly, is Foo.Load() exception safe? Is there anything I need to change in Load’s exception handler to insure an instance of Foo is always in a valid state? 

///Defines a type that associates a URI with the
/// count of frobs, miscellaneous information about those frobs
/// and where to get them.
///
public class Foo
{
    public Foo()
    {
        Miscellaneous = new Dictionary<string , object>();
    }

    private Uri _infoSource;
    public Uri InfoSourceUri
    {
       get { return _infoSource; }
       set
       {
         _infoSource = value;
         Load();
       }
    }

    private void Load()
    {

        ServiceProxy<IFooConfig> proxy = null;
        try
        {
            proxy = ServiceHelper.CreateProxyFromServiceContractInf<IFooConfig>(InfoSourceUri.ToString());
            IFooConfig config = proxy.ProxyContract;

            NumberOfFrobs = config.NumberOfFrobs;
            WhereToGetThem = config.FrobSource;
            for (var e in config.MiscellaneousEntries)
            {
                Miscellaneous.Add(e);
            }
        }
        catch (Exception ex)
        {
            // Report the error
        }
        finally
        {
            proxy.CleanupProxy(); // Extension method that aborts if proxy is
                                  // faulted or ignores if null
        }
    }

    public int NumberOfFrobs { get; set; }
    public Uri WhereWeGetThem { get; set; }
    public IDictionary<string , object> Miscellaneous { get; private set; }
}

Encapsulating Your Way To Better Unit Tests

Posted by Jason Smith - 07/02/10 at 11:02:01 am

I an earlier blog, I discussed the qualities of good unit tests. Primarily, good unit tests are fast, isolate the bugs, repeatable, self-validating and timely.

At first blush, that seems easier said than done. We write distributed software and connect to databases. Some of those databases live in another country.

So, how can an object that sets up connections to a database, prime it with test data, consume that data, manipulate it and then present it in a UI actually be fast, isolating, repeatable and self-validating? If by fast we mean microseconds per test (that’s exactly what we mean), we’re toast as soon as we even talk to the database. How do we isolate our tests when we write to a persistent store, while another test is running in parallel against the same store? What happens when the database fails for some reason, and the data has been partially changed? Is the error in our code, or the database? And self-validating? Come on! Our UI’s need to be poked, prodded and looked at to be tested.

Unit-testing Nirvana sounds like a ton of work.

Actually, it’s the work we already do.

Most of us use object oriented languages because we buy into the idea of encapsulation — or, said another way, we want to reduce couplings to particular implementations. Even if polymorphism and inheritance doesn’t float your boat, old-fashioned structured design pushes this same idea. We simply hide (encapsulate) implementation behind function calls, so we can focus on the work at hand.

For example, assume we have a window where individual elements within it may be visible or invisible based on user preferences. What is it about the algorithm to determine the initial visibility for elements of a window that requires a database or a .config file? Nothing, right? Either will approach work. So will other changes in the underlying program. All the algorithm really cares about is it gets configuration data from somewhere. So we should encapsulate that concept.

We know our window needs to be initialized with some configuration data. The window doesn’t really care where it came from. A candidate method might look something like this:

class MyWindow : System.Windows.Forms.Form
{

     // a list of our window controls that we can enable or disable visibility on.
     private List<Control> _toggleVisibility;

    // ... blah blah
    public void SetInitialWindowVisibility(IDictionary<string , bool> visibilities)
    {
        foreach (var control in _toggleVisibility)
        {
            if (visibilities.TryGetValue(control.Name, out visible))
            {
                control.Visible = visible;
            }
        }
    }
}

We honor the Single Responsibility Principle: No database involved. No System.Configuration.ConfigurationManager. If high cohesion and low coupling are the benchmarks of good design, then this is pretty decent.

Here’s the payoff — this approach gave us a good unit-testable design for free:

  • it’s fast — for our test, we simply pass a preloaded dictionary and verify the window controls’ visibilities match the dictionary state
  • it isolates the error — if the unit-test fails here, it’s because the visibility setting behavior was broken. Period. Not a database, not a parser — just the method under test.
  • It’s repeatable — every time we run it, we get the same results — noone on the outside can poison the test during it’s run,

    • It’s self-validating — our unit test need only iterate over the embedded controls collection comparing the state of the control.Visible value. No popping up the user-interface to hand inspect is necessary.

Sure, that configuration dictionary needs to be initialized somewhere, and that code should be tested as well. But if it connects to a database to get the information, keep the test for that code outside the body of unit-tests (recall, unit tests should be run on the developers’ machines, often. If they aren’t fast, they won’t be run at all). Definitely include database tests in the an integration or smoke test, though — those are run with much lower frequency, so they can afford to take a bit more time to run.

For the adventurous, you can raise the bar even more – what about this algorithm requires that it be implemented in a System.Windows.Forms.Window derivative? Can the implementation be refactored to remove this dependency?

Qualities of Good Unit Tests

Posted by Jason Smith - 02/02/10 at 11:02:08 am

So — you’ve been tasked with insuring your code has unit tests. What does that mean, really?

According to Wikipedia,

unit testing is a software design and development method where the programmer gains confidence that individual units of source code are fit for use. A unit is the smallest testable part of an application. In procedural programming a unit may be an individual program, function, procedure, etc., while in object-oriented programming, the smallest unit is a method, which may belong to a base/super class, abstract class or derived/child class. — http://en.wikipedia.org/wiki/Unit_testing

If you understand the primary beneficiary of unit tests are the developers, then it make sense we should make our unit tests as helpful to developers as we possibly can.

Brett Schuchert and Tim Ottinger, old hands at unit testing, provide some pointers. They describe good unit tests as being “FIRST”:

  • Fast

    If a developer hesitates to run unit tests because they take a long time to run, the tests won’t be run at all. Unit tests need to be run often (as often as you build – the compiler can only check for correct syntax, the tests insure you got the semantics right), so they need to be cheap and easy to run. One test should take a small fraction of a second. A test suite that takes longer than a few seconds to run is painfully far too slow.

  • Isolated

    This actually means two things:

    • Unit tests should be small enough that the bug is obvious as soon as the test is viewed. A unit test tests one feature of one class.
    • Unit tests should never depend on the order the tests are run – if a test breaks because some other test did or did not successfully run before, then the test can no longer be run in isolation
  • Repeatable

    Tests must be able to be run over and over without any intervention. No hand-loading of a database or editing configuration files. They run whether there is a network or not. Unit tests do not test external systems.

  • Self Validating
    It must be obvious whether a test passes or fails. We never have to interpret results in a database or a file somewhere.
  • Timely
    Tests are written at the right time. Many argue the right time is immediately before the code being tested is written (I’m usually one of them). Regardless, unit tests should always be written before the tested code is checked into source control.

Michael Feathers (the author of one of the original CppUnit unit testing frameworks) also describes some features that should never be in a unit test. He states a test is not a unit test if:

  • It talks to the database (not fast, isolated nor repeatable)
  • It communicates across the network (not fast, isolated nor repeatable)
  • It touches the file system (not fast, isolated nor repeatable)
  • It can’t run at the same time as any of your other unit tests (not isolated or repeatable)
  • You have to do special things to your environment (such as editing config files) to run it. (not fast, isolated nor repeatable)

    • He goes on to say:

    Tests that do these things aren’t bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.

    So — are your unit tests FIRST or are they painful?

Powered by WordPress with GimpStyle Theme design by Horacio Bella.
Entries and comments feeds. Valid XHTML and CSS.