Saturday, March 04, 2006
This blogs topic is about software industrialization, which means making software development a predictable and repeatable process.  The only other requirement for software industrialization is that the software created meets the end user's requirements.
Why is software development not a predictable and repeatable process?  I can partially explain this through a small story where after spending 10 years in the software development business, a friend of mine and I opened up our own consulting company in 2001.  Our company was based on one single Microsoft product in which we would offer consulting services for.  That product was BizTalk Server which is a message oriented middleware product for exchanging disparate data and orchestrating business processes across multiple applications.
Over a four year time frame, we custom designed and constructed twenty-five or so integration solutions using every version of BizTalk Server. Even after that many projects, our process for designing and constructing these solutions was still far from being predictable and repeatable.  Sure we got (much) better, but we realized it was the variability that was so difficult to overcome.  I mean variability in everything that is software and the processes used to design and construct it.
For example, there is always large variability in the quantity and quality of software requirements.  A very small percentage of customers know exactly what they want, more still know exactly what they want, but can't articulate it, all the way to the other extreme where customers have no idea what they want, but still want something built.
For every single "discrete" chunkable requirement, there seems to be at least a dozen ways to design it.  For every design, there seems to be almost infinite ways to implement it.  When I say design, I mean a particular design for a particular requirement in which the chunkable output of the design is on the order of 40 hours effort per one developer to complete the requirements, finish a detailed design, code, test the chunk and done.  The culmination of designs to meet all of the requirements is called the software architecture.
Case studies and industry reports point to inadequate and/or always changing requirements as one major contributing factor as to why software development is not a predictable and repeatable process.  Another contributing factor is size and complexity of software development where it is most always underestimated.  I would be the first one to agree with both statements, but I would say that this is more symptomatic then the root cause.
Yet another contributing factor to why software development is not a repeatable and predictable process is programmer productivity.   I have worked with over a hundred software developers in my 15 years in the industry and I can say that programmer variability is just as broad as the other contributing factors as discussed above.  There are several books that quantitatively put programmer productivity variability levels in the range of 20 to 1 and even 100 to 1 between programmers that have been assigned the same code project to, design, construct and test the software.  I have seen the extreme with my own eyes where some developers can't write the code no matter how much time was given, while others can write it in two weeks flat.  Thats off the chart in terms of variability.
One of the reasons for the wide variability in programmers, aside from the skill sets discussed in the previous paragraph, are the tools that are available for programmer use.  The tools themselves are incredibly complex environments and sometimes require people