I remember in the mid 90s when software reuse was a big deal.  Several books came out that discussed the need for software reuse and how to set up a reuse library in your organization.  My favorite is, , by Will Tracz.  One of the very few textbooks I have read that was as entertaining as it was useful.

 

10 years later I dont see much about software reuse at all.  I dont find it as a topic of discussion on any programming related forums.  Is it because we as computer scientists, software engineers, or programmers have institutionalized it as standard operating procedure or is it because the fascination we have over programming languages, software reuse is a forgotten practice?

 

Looking up software reuse on Wikiepedia gets redirected to code reuse.  This was a real surprise to me.  I use Wikipedia quite a bit for software related information and it is usually is up to date and accurate.  However, to equate software reuse with code reuse is missing the point of software reuse in general.  I would define software reuse as the ability to leverage any existing software artifact that makes solving any software problem easier.   Whether it is a project management schedule template to a fully blown domain specific language for automatically generating ecommerce solutions would be considered software reuse in my book.

 

I know there are several libraries, SDKs, commercial components, etc., that come with your favorite programming language.  Even K&R C from 1978 had this and probably even earlier before my time entering the computing world.   The mid 90s also had the explosion of design patterns, the Gang of Four being the most familiar to software designers and programmers.  Today there is an incredible wealth of design patterns.  I am wondering if we ever use them?  I must admit that while I have used some design patterns in a few projects and probably some unknowingly, I dont go out of my way to decompose my software designs into patterns, cause quite frankly, in my world of professional software development services, we never have time for such activities.  My software development world is governed by the clock get it done as soon as possible.

 

So how do we implement reuse in our organization?  In a nutshell, we Google it.  Here is how it works.  Our customer needs a custom web application built with certain functionality (what that functionality is does not really matter, because it has usually been done before).  Our projects typically take on two distinct phases.  Phase 1 is to write up a requirements specification, usually in the form of use cases and storyboards, all of which we have a library of templates which we reuse, including the statement of work, project schedules, etc.  The customer signs-off and we are on to Phase 2.

 

Phase 2 begins with, what do we have from a previous project that we can reuse?  That means any ASP.NET web pages, controls, components, custom assemblies that we have written before that we can leverage.  For the pieces we dont have, we start our search with Google and other tools, looking in the usual suspect places where we know we can find example/sample designs and code that we can reuse in our overall solution.  Since we are a Microsoft shop specializing in custom ASP.NET, SharePoint and just plain .NET applications, we know where to look.  In fact, we have several links in our SharePoint library that points to our regular spots.

 

Once we have gathered all of the pieces, we look at what actual custom code is left to do.  Surprisingly (or not) not much is left to do other than glue the pieces together and apply the paint.  Now I make it sound easier than it is, but I would say on average that well over 50% of any custom development work we do is reusing pieces (on top of the .NET Framework Class Library) and in some cases as high as 80% software reuse.  Just to be clear, we are a .NET shop only and we are not implementing or deploying packaged products, except in the case of SharePoint and then we only get involved in custom development using the APIs and/or page layout.

 

We have our test harnesses for unit testing, system testing, FxCop design and code coverage tests, performance and stress testing, bug tracking templates, etc.  These are also reusable tool and template assets that we have accumulated and customized over the years.  Some of the custom applications we have designed and developed handle over 1 million transactions per day that I think are good pieces of work.  And others, well, you know, right?  Since we have yet to reach a stage of industrialization in our software world, some projects are better than others.

 

So when does software reuse not work well?  In our case it is the adoption of new technologies such as Microsofts SharePoint 2007 which just reached Beta 2.  The problem here is that the architecture is so new that very little in the way of samples, examples or how tos has been developed.  Here we are on our own, with little documentation, massively changed APIs and of course a Beta product.

 

What does this have to do with asking the question if we as the software community are reusing software?  I guess thats the point.  We seem to be doing it but when we are hiring new candidates (including seasoned professionals) only a few of them can point to reusing software and it usually is code reuse and even fewer have experience with design patterns.  I cant believe we are the only ones doing it.  But the friends I have in other software organizations also point to a very small percentage of software reuse in their own orgs.  Why is that?

 

I dont think I have the answer to that question.  I see that there is an international conference on software reuse this year.  Not sure who attends these.  I wish I could go to Italy   Spending more time on Google does not help as most of the links to the subject matter in question are relatively old.  It seems no longer a hot topic.

 

I would be interested to hear other peoples stories of software reuse.  If you have a moment and would like to share, let me know.  Thanks.

 

There is an excellent article written in 1994 called, Softwares Chronic Crisis whose conclusion is, Despite 50 years of progress, the software industry remains years-perhaps decades-short of the mature engineering discipline needed to meet the demands of an information-age society.  That same year, while I was taking Software Engineering classes taught by Karl from Motorola University, I was told the same thing.  Motorola shared their own software metrics to validate what the industry was saying.

 

Again, that same year, (what was it about 1994?), a CHAOS report came out that said the US spends $250 billion per year on software development for roughly 175,000 projects ranging from $140K to $2.3 million per project.  Out of those projects:

Flash forward 10 years later and the Standish Group has this update about our software development track record:

So what does this mean?  It means we as the software development industry (still) have major problems in developing software on time and budget.  Shhh, dont tell our customers.  If you read the reports, there are a variety of reasons why we have this record, and not all of it on to ourselves.  However, after 15 years in the software development biz, including running my own software company, I can safely say that these metrics are quite real.  In my experience, the only way to deliver on time and budget is to give up features.  Thats the state of the art today.  Even the largest and most successful software company in the world has the same problem on making software development a predictable and repeatable process, Vista 2007, Fire the Leadership Now!"

 

Softwares Chronic Crisis is right on the money back in 1994 and still is today in my opinion.  The root cause is that our software development industry has yet to be industrialized.  This is in comparison to other industrialized industries like electronics, mechanics, etc.  These industries have matured from one-off custom development to mass customization (i.e. industrialization).  Mass customization has the following characteristics:

Our software world currently does not yet have these characteristics of industrialization.  Our software development products are designed and constructed one source code line at a time.

 

So what does this have to do with IT is a commodity?  Some people think IT is a commodity.  While I think the hardware side of IT, including some basic firmware applications, maybe a commodity, the software side (bought s/w products or custom dev) is still far, far away from being a commodity.  Look at our track record.  Look at your own products/projects.  How many bugs (do you know about) are in it?  According to a recent NIST Study on the impact of software bugs, conducted in 2002, "Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product. More than half of the costs are borne by software users, and the remainder by software developers/vendors."

 

If you could visibly see the thousands of bugs in most commercial software products, would you still buy it?  We do it all the time.  I like the fact that companies sell "packaged" enterprise application software and then suggest to you that you pay for services to install and configure the software and it will cost less than a custom build.  Maybe, maybe not.  I personally know of several configurations of ERP systems in the Oil and Gas industry in Western Canada that have cost 10 to +100 times the cost of the product itself.  The product cost is $1 million.  If I were a Customer of such a service, I would also want my own personal jet (and full-time pilot) for that kinda of money. Cmon!

 

How is IT (i.e. software development) a commodity?  Software development is still an incredibly skilled and massively labor intensive process.  Our development tools lags platform technology.  What do I mean by that?  While we have platforms that provide the technology to do (most) everything we want, our development tools are so low-level that we are (still!) handcrafting every solution as a custom one-off, one source code line at a time.

 

How do we industrialize our software development industry?  Next post we will look at how other industries have done it, and ironically, using computer technology as an enabler to industrialize their own business domain.

The software industry has a long history of recreating incompatible solutions to problems that are already solved 

 

In 2001, my good friend Barry Varga  and I co-founded this company for a variety of reasons.  One of those reasons was to make decisions for our own company as previously we had worked for a variety of other companies where we watched the people in charge make, well, lets just say questionable decisions.  We felt that we could make better decisions or at least, we would be accountable for our own decisions we thought how, could we do worse?  I am sure this is how MiniMicrosoft kinda feels.

 

One of those decisions directly affected our programming team, which was, dont write code at least initially.  It sounds strange as most programmers (read: all) love to write code including yours truly.  The issue is that designs precede code and requirements precede design.  Pretty fundamental one would think.  However, as programmers, we are all guilty (and still are) of not conforming to this basic tenet.  Oh to be sure, there are a whole pile of reasons behind this we are good at rationalizing that we dont have any requirements or market intelligence to tell us what to code.  We also have great imaginations and think we know what the customer wants.  We also believe that regardless of size, complexity and/or change, by coding first we can figure it out

 

Since we were writing code on the MSFT platform, and have been since VB1, we knew, as old school programmers that it all has been done before, you just have to find an example.  The World Wide Wasteland has many examples of whatever you are coding.  In fact, since some 50% of the 8 million programmers out there code on the MSFT platform, you are bound to find an example or ten (or hundreds!) that is pretty close to what you need for your component/project/product.  Of course, the cynical programmer and/or perfectionist programmer type (arent we all?) will want to write their own code because what they write will be better than whatever can be found on the internet- surely.  Sometimes this is true, but in most cases not, or a least in my 15 years experience in the field.  This is also sometimes called the Not Invented Here (NIH) syndrome.  This is one of the major barriers to reusing perfectly good code that already exists. Oh, did I mention that coding is fun?

 

We also got our teams to think of different ways to solve various problems (once we had some semblance of requirements to iterate on).  In other words, come up with a design first.  Then think of 3 ways to solve the design, including searching the internet to see how other people have solved similar problems.  Then present those findings in an objective manner.  Once people thought this way, we were able to leverage and reuse several pieces of existing code (and design patterns) to solve a much bigger puzzle.  This meant that we could solve the problem in a much more timely and realistic manner.

 

The key word being realistic.  Anyone that has been writing code for any number of years knows how hard it is to write code from scratch to solve problems of any real size or complexity.  Not only is it really hard work, but the amount of effort required sometimes seems insurmountable.  We do have best practice processes and tools that raise the level of abstraction, but it is still at such a low level it still sometime feels like assembly language programming (to me anyways).  Also, when you the programmer, are the owner of your own software shop, you get a bit concerned as to how much time and effort is required to solve various problems particularly when you know they have been solved before (many times!).  Here is where refactoring really helps find any way to get the problem solved, like right now (hopefully by reusing some already existing code) and we will come back (given time) to refactor .

 

Why am I saying this?  I see programmers everyday still re-inventing the wheel or NIH or doing everything else but trying to reuse and leverage some chunk of code, component, design that has already been designed, tested and reused by many to solve the same problem.  I ponder the reasons to why that it is while I have mentioned a few in this article, it still amazes me.  Even seasoned pros fall into the trap.  Maybe it is just too easy?  Maybe it is a combination of a whole pile of things.  Maybe it is simply that our software industry still has not gone through the industrialization (i.e. maturity) process like other similar engineering industriesWhatever it is, next time you open your favorite code editor stop!  Ask yourself this question did I look on the internet to see that this problem has been solved before?  If not, ask yourself why maybe you have the answer.