There is an excellent article written in 1994 called, Softwares Chronic Crisis whose conclusion is, Despite 50 years of progress, the software industry remains years-perhaps decades-short of the mature engineering discipline needed to meet the demands of an information-age society.
That same year, while I was taking Software Engineering classes taught by Karl from Motorola University
, I was told the same thing. Motorola shared their own software metrics to validate what the industry was saying.
Again, that same year, (what was it about 1994?), a CHAOS report
came out that said the US spends $250 billion per year on software development for roughly 175,000 projects ranging from $140K to $2.3 million per project. Out of those projects:
- 16% are on time and budget but deliver less than planned (avg 42%)
- 53% overrun (avg 189%)
- 31% are canceled, losing $140B/yr
- 29% succeeded
- 53% challenged
- 18% failed
So what does this mean? It means we as the software development industry (still) have major problems in developing software on time and budget. Shhh, dont tell our customers. If you read the reports, there are a variety of reasons why we have this record, and not all of it on to ourselves. However, after 15 years in the software development biz, including running my own software company, I can safely say that these metrics are quite real. In my experience, the only way to deliver on time and budget is to give up features. Thats the state of the art today. Even the largest and most successful software company in the world has the same problem on making software development a predictable and repeatable process, Vista 2007, Fire the Leadership Now!
Softwares Chronic Crisis is right on the money back in 1994 and still is today in my opinion. The root cause is that our software development industry has yet to be industrialized. This is in comparison to other industrialized industries like electronics, mechanics, etc. These industries have matured from one-off custom development to mass customization (i.e. industrialization). Mass customization
has the following characteristics:
- Configure, adapt and assemble components to produce variants
- Standardize, integrate and automate production processes
- Develop and configure extensible tools for rote or menial tasks
Our software world currently does not yet have these characteristics of industrialization. Our software development products are designed and constructed one source code line at a time.
So what does this have to do with IT is a commodity? Some people think IT is a commodity
. While I think the hardware side of IT, including some basic firmware applications, maybe a commodity, the software side (bought s/w products or custom dev) is still far, far away from being a commodity. Look at our track record. Look at your own products/projects. How many bugs (do you know about) are in it? According to a recent NIST Study
on the impact of software bugs, conducted in 2002, "Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product. More than half of the costs are borne by software users, and the remainder by software developers/vendors."
If you could visibly see the thousands of bugs in most commercial software products, would you still buy it? We do it all the time. I like the fact that companies sell "packaged" enterprise application software and then suggest to you that you pay for services to install and configure the software and it will cost less than a custom build. Maybe, maybe not. I personally know of several configurations of ERP systems in the Oil and Gas industry in Western Canada that have cost 10 to +100 times the cost of the product itself. The product cost is $1 million. If I were a Customer of such a service, I would also want my own personal jet (and full-time pilot) for that kinda of money. Cmon!
How is IT (i.e. software development) a commodity? Software development is still an incredibly skilled and massively labor intensive process. Our development tools lags platform technology. What do I mean by that? While we have platforms that provide the technology to do (most) everything we want, our development tools are so low-level that we are (still!) handcrafting every solution as a custom one-off, one source code line at a time.
How do we industrialize our software development industry? Next post we will look at how other industries have done it, and ironically, using computer technology as an enabler to industrialize their own business domain.