"I did not get what I envisioned" from the project, the senior official acknowledged. But he said the F.B.I. today had a better understanding of its computer needs and limitations as a result of the effort." The lesson we have learned from this $170 million is invaluable," he said.

This quote is from a $170 million failed software project in the US called the Virtual Case File or VCF by the FBI.  The system was designed to give the bureau's nearly 12,000 agents around the country instant access to F.B.I. databases, allowing speedier investigations and better integration of information both within the bureau and with other intelligence agencies that must coordinate national security matters.

 

While there were several software failures for 2005, I chose this one to highlight a point.  The very first sentence, I did not get what I envisioned is a telling statement that demonstrates a fundamental reason why software projects fail.  There is an incredible amount of information written about why and how projects fail that has been well documented for dozens of years. The following reasons for software failure are a summary: poor user requirements, poor project management, no change control, too much complexity, unskilled staff, political issues, poor software development processes, and on it goes.  So if these reasons are well documented over the last 20 years, why do we still have failed software projects?

 

I would say we have a more fundamental problem than all of these reasons combined and that is we have no way of visualizing the finished software product before it is built.

 

In other industrialized engineering disciplines, we have many ways of visualizing the end product without actually building it first.  For example, models are used extensively to visualize cars, buildings, cell phones, iPODs, bridges, airplanes, trains, you name it and somewhere there is a model that visualizes what the finished product looks like.  The model may be an artists rendering or a precision built to scale model of an airplane.  The fact remains that no matter what the product is, there is a visualization of some sort that people can look at as a concrete vision of what the end product looks like before it is built.

 

What do we have in the software world to visualize what the finished software is going to look like?  Not much.  Some people would call prototyping or screen shots or using Visio as a way of visualizing software.  Unfortunately, none of these represent what the actual finished software is going to look like.

 

As a software professional, whats most embarrassing about the VCF failure and other failures is that 95% of all business software applications are nothing more than retrieving data out of a database, presenting it to a human, who may manipulate it and saving it back to the database. Thats it.  We do it over and over again. No matter if you are SAP or salesforce.com or a custom application in the business world, all that is being done is retrieving and saving data to a database, yet billions are spent each year doing this and if you believe statistics, over 50% of these projects will fail.

 

It would seem to me that the software industry is ready for some industrialization lessons learned in other engineering disciplines, with the first lesson being lets visualize what the end product is going to look like before we build it.  Here is an excellent example of a visual model that was created 100 years ago to visualize an immense and complicated structure.  To me, this model puts into perspective how little we have evolved in the software industry, with all of our technology we still cant build software right.  Maybe next year