There is an excellent article written in 1994 called, Softwares Chronic Crisis whose conclusion is, Despite 50 years of progress, the software industry remains years-perhaps decades-short of the mature engineering discipline needed to meet the demands of an information-age society.  That same year, while I was taking Software Engineering classes taught by Karl from Motorola University, I was told the same thing.  Motorola shared their own software metrics to validate what the industry was saying.

 

Again, that same year, (what was it about 1994?), a CHAOS report came out that said the US spends $250 billion per year on software development for roughly 175,000 projects ranging from $140K to $2.3 million per project.  Out of those projects:

Flash forward 10 years later and the Standish Group has this update about our software development track record:

So what does this mean?  It means we as the software development industry (still) have major problems in developing software on time and budget.  Shhh, dont tell our customers.  If you read the reports, there are a variety of reasons why we have this record, and not all of it on to ourselves.  However, after 15 years in the software development biz, including running my own software company, I can safely say that these metrics are quite real.  In my experience, the only way to deliver on time and budget is to give up features.  Thats the state of the art today.  Even the largest and most successful software company in the world has the same problem on making software development a predictable and repeatable process, Vista 2007, Fire the Leadership Now!"

 

Softwares Chronic Crisis is right on the money back in 1994 and still is today in my opinion.  The root cause is that our software development industry has yet to be industrialized.  This is in comparison to other industrialized industries like electronics, mechanics, etc.  These industries have matured from one-off custom development to mass customization (i.e. industrialization).  Mass customization has the following characteristics:

Our software world currently does not yet have these characteristics of industrialization.  Our software development products are designed and constructed one source code line at a time.

 

So what does this have to do with IT is a commodity?  Some people think IT is a commodity.  While I think the hardware side of IT, including some basic firmware applications, maybe a commodity, the software side (bought s/w products or custom dev) is still far, far away from being a commodity.  Look at our track record.  Look at your own products/projects.  How many bugs (do you know about) are in it?  According to a recent NIST Study on the impact of software bugs, conducted in 2002, "Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product. More than half of the costs are borne by software users, and the remainder by software developers/vendors."

 

If you could visibly see the thousands of bugs in most commercial software products, would you still buy it?  We do it all the time.  I like the fact that companies sell "packaged" enterprise application software and then suggest to you that you pay for services to install and configure the software and it will cost less than a custom build.  Maybe, maybe not.  I personally know of several configurations of ERP systems in the Oil and Gas industry in Western Canada that have cost 10 to +100 times the cost of the product itself.  The product cost is $1 million.  If I were a Customer of such a service, I would also want my own personal jet (and full-time pilot) for that kinda of money. Cmon!

 

How is IT (i.e. software development) a commodity?  Software development is still an incredibly skilled and massively labor intensive process.  Our development tools lags platform technology.  What do I mean by that?  While we have platforms that provide the technology to do (most) everything we want, our development tools are so low-level that we are (still!) handcrafting every solution as a custom one-off, one source code line at a time.

 

How do we industrialize our software development industry?  Next post we will look at how other industries have done it, and ironically, using computer technology as an enabler to industrialize their own business domain.

Some years ago I had the opportunity to participate in a software technology bake-off. Technology selections or bake-offs go something like this, Dear Vendor, you have been selected to show your wares/skills at our esteemed company.  No, you dont get paid for this privilege, but you have one week to complete the tasks in this envelope that you will perform on-site.  And if you win, we are not sure what we are going to do anyway. So, dance vendor, dance!

 

OK, while I am being a bit tongue in cheek, it was exactly like this.  I view bake-offs as the easy way out for CXOs that dont know what they are doing or dont want to do their homework themselves.  In my experiences, it has been the former.  Unfortunately, for the vendor, it is a very risky proposition to participate if the org does not understand what they are doing or getting involved in.  At the time of this particular bake-off, it was all about Enterprise Application Integration (EAI) even though Service Oriented Architecture (SOA) principles applied, this buzzword had yet been invented.  In other words, this org was a very early adopter of SOA and did not know it.  This meant that Geoffrey Moores Technology Adoption Curve is in full play here.

 

While I am going to name the technologies that were used in the bake-off, I do want to make clear that years later, I came to the conclusion that it did not really matter what technology was used.  The reality is that any of the middleware products would have worked for this org.  As an analogy, they were looking for a