This blogs topic is about software industrialization, which means making software development a predictable and repeatable process.  The only other requirement for software industrialization is that the software created meets the end user's requirements.

 

Why is software development not a predictable and repeatable process?  I can partially explain this through a small story where after spending 10 years in the software development business, a friend of mine and I opened up our own consulting company in 2001.  Our company was based on one single Microsoft product in which we would offer consulting services for.  That product was BizTalk Server which is a message oriented middleware product for exchanging disparate data and orchestrating business processes across multiple applications.

 

Over a four year time frame, we custom designed and constructed twenty-five or so integration solutions using every version of BizTalk Server. Even after that many projects, our process for designing and constructing these solutions was still far from being predictable and repeatable.  Sure we got (much) better, but we realized it was the variability that was so difficult to overcome.  I mean variability in everything that is software and the processes used to design and construct it.

 

For example, there is always large variability in the quantity and quality of software requirements.  A very small percentage of customers know exactly what they want, more still know exactly what they want, but can't articulate it, all the way to the other extreme where customers have no idea what they want, but still want something built.

 

For every single "discrete" chunkable requirement, there seems to be at least a dozen ways to design it.  For every design, there seems to be almost infinite ways to implement it.  When I say design, I mean a particular design for a particular requirement in which the chunkable output of the design is on the order of 40 hours effort per one developer to complete the requirements, finish a detailed design, code, test the chunk and done.  The culmination of designs to meet all of the requirements is called the software architecture.

 

Case studies and industry reports point to inadequate and/or always changing requirements as one major contributing factor as to why software development is not a predictable and repeatable process.  Another contributing factor is size and complexity of software development where it is most always underestimated.  I would be the first one to agree with both statements, but I would say that this is more symptomatic then the root cause.

 

Yet another contributing factor to why software development is not a repeatable and predictable process is programmer productivity.   I have worked with over a hundred software developers in my 15 years in the industry and I can say that programmer variability is just as broad as the other contributing factors as discussed above.  There are several books that quantitatively put programmer productivity variability levels in the range of 20 to 1 and even 100 to 1 between programmers that have been assigned the same code project to, design, construct and test the software.  I have seen the extreme with my own eyes where some developers can't write the code no matter how much time was given, while others can write it in two weeks flat.  Thats off the chart in terms of variability.

 

One of the reasons for the wide variability in programmers, aside from the skill sets discussed in the previous paragraph, are the tools that are available for programmer use.  The tools themselves are incredibly complex environments and sometimes require people to think in ways that they may not be able to grasp or it is so complicated, no one can figure it out.  I can't grasp C, but I grok Smalltalk from a programming language point of view.  When we asked a printer to print the help file that came with BizTalk Server 2004, he called us to say it is likely going to be 10,000 pages and cost $500.  That's just one product!  And we use a half dozen other products for designing and constructing our integration solutions including, SQL Server, ASP.NET, Visual Studio IDE, Windows 2003 Server, SharePoint Services, C#, FrontPage, .NET Framework, and on it goes.  While some of these products are not the same size and complexity of BizTalk Server, they require deep understanding of just what they heck they do and how all the products fit together in order to provide the tools and framework to design and produce the Customer's solution in any reasonable time frame (read: cost).  Even the .NET Framework Class Library alone has over 5,000 classes to get to know, some very intimately.

 

These are tools and technologies from one vendor!  What about multiple vendors?  Also every vendor seems to be pumping out the latest and greatest tools and technologies every year.  Where does one find the time?  Answer: one does not find the time which results in peoples knowledge of these tools and technologies, plus the specialized skills required and experiences to use them effectively, varies wildly.  This is another major contributing factor as to why software development is not a predictable and repeatable process - the programmer never gets a chance to gain years of experience using one tool or even small set of tools - so everything is (always) new.

 

Even within the Microsoft technologies mentioned above, there are many technologies that do more or less the same job (but the tools are totally different) for one specific area - user interfaces.  There are (at least) five Microsoft technologies for developing user interfaces. To me, it is mind boggling why even within a single vendor, that not only are there five different technologies to develop user interfaces (actually 7 if you count InfoPath and SharePoint Designer), there are multiple tools for each technology.  For example, ASP.NET - there is Visual Studio and FrontPage.  Both have very deep features, but the tools are completely different.

 

Some would say introducing standards would alleviate this problem.  While I concur and it has proved to help industrialize other industries (e.g. electronics), it is still early game in the software world and the technology advances far outpace the speed at which standards can be ratified.  Also, believe or not, Microsoft's latest technologies are all (mostly) standards complaint and all with public specifications. So what's the value of standards?  What our industry needs is innovation.  What would be truly innovative from Microsoft (and other vendors) is simply one technology and tool that produces any type of user interface you want.  From a developers perspective, this means being able to focus on "one" tool or technology to do a specific task, like designing and constructing any type of user interface.  With one tool and language (for user interfaces), then we might have a hope of industrializing software development.

 

Let me put it another way, how many people do you know that are fluent in six foreign languages or more?  How many of those people are fluent in both the spoken and written word?  Have you ever tried learning a foreign language so you are just as fluent in it as your native language?  Learning and becoming fluent in any foreign language is no easy task.  But for software programmers, we must learn multiple foreign languages to design and construct software.  It may even be tougher than learning a traditional foreign language as our programming languages regularly change including the introduction of brand new ones (e.g. XAML). This gives some insight as to one of the major reasons why software development is not a predictable and repeatable process even for the software programmers.