Software Industrialization - Software Development Gone Insane

Wednesday, April 08, 2009

I may be getting old, but I don’t think I am out of touch. My sanity of being a software development professional seems to be tested daily by our industries predilection for Silver Bullets. The latest it seems is Scrumban. My wife can’t stop laughing when I say it to her.

I have not read the book, so I have no opinion on it. I have read some of the author’s blog material. Seriously, I mean no disrespect to Corey Ladas at all, but when I read the material, I can’t help to think that this is written by someone with a MBA. When I read this article, I think I get what he is saying, but I swear it is written in a different language than the software engineering world I live in.

Regardless of what marketing terms are used, the reality is that software development is always: understanding the requirements, creating a design, implementing the design, and testing the design and implementation to ensure it meets the requirements. Requirements, Design, Code and Test. Always have, always will, no matter what other fancy (marketing) words are used to describe it.

With respect to very large software projects, I understand the label Feature Crews, but to me, this is nothing more than classic functional decomposition at work, but with a new (fancy marketing) label.

Questions that the Agile, Scrum, Lean, give it a name, practitioners seem to never answer is the two most asked questions by people that pay for software development projects, which are, “how much is it going to cost and how long is it going to take?” It seems there is no best practice in any of these methodologies to answer these fundamental questions as these methodologies are focused on very limited scopes of the project as opposed to the entire project. And quite frankly, that is my main beef with these Silver Bullet’s, because as Fred Brooks postulated 23 years ago, there is no such thing as a Silver Bullet in software development. In my opinion and professional experience, he is right.

Software development is an incredibly complex, massively manual labor intensive effort, whose primary work product is lines of source code. For people that write code day in and day out, know this to be the truth. There is no hiding from it. We grind it out as we know how and love it. So when I am asked how much the software development project is going to cost and how long it is going to take, I apply a tried and trued approach to answer what is a very, very difficult question. This is why you see software estimation models like the Constructive Cost Model or CoCoMo for short. You can read the gory details of the model here. PDF alert! No surprise to see that counting lines of source code (or equivalent lines of source code) is the way to answer the tough question of how long and how much. Dr. Barry Boehm had it figured out years ago. When I took Software Engineering as a post graduate course at the University of Calgary, this is what we were taught and has been consistent with what I have found in the field. So yes, I use CoCoMo to answer the ugly, gnarly questions. And there are several automated tools that implement CoCoMo’s model, even ones online.

So what has happened to our software development industry that we need to keep reinventing the wheel in the name of continuous improvement? I think it is indicative of anything that is really tough to do. Everyone is looking for an easy answer or the next big thing. But as most things in life, the answer is already figured out. You just need to look in the right place, listen to wisdom and apply some common sense. Congratulations, you just made pro software engineer!

What’s my point? My point is two fold. Our software development industry is being run by marketing people and has gone insane ;-) Ok, I half jest. I know over time the pendulum will swing back to the basic fundamentals of what software development really is. My other point is to the young aspiring software engineers. Kids, look at some of the real software engineering techniques that hold up to the test of time. These are the gems. This is what is real in our industry. CoCoMo is real, it works, and is based on facts and historical proof. If you were to only read one book on software engineering, read the Mythical Man Month as it embodies what is really happening in our software development world, regardless of when the book was written. There is a reason why Fred Brooks earned the Turing Award. Your assignment is to understand why.

Industrialization

Wednesday, April 08, 2009 11:43:44 PM (Pacific Daylight Time, UTC-07:00)

Comments [8]

Thursday, April 09, 2009 3:51:44 AM (Pacific Daylight Time, UTC-07:00)

Well said. Software development is a fashion business and in quite regular intervals we put that old wine into new bottles. I only partly agree however with your reduction of lines of code. While I like CoCoMo for its repeatability and accuracy (erring regularly on the high side) it doesn't really answer the question.
"I need a system to track XYZ, how much is it" doesn't translate into "Oh that is 34,000 lines of code" which you can feed into the CoCoMo model. CoCoMo also neglects visual tasks: how much lines of code is the creation of a database schema in an ER modeler tool? Or the "painting" of a report in your favoured BI tool? Or assembling widgets to deliver the expected result? This is the question in the gap: How do I arrive at reliable LOC estimates I can feed into CoCoMo? Another dis-tractor is coding style, since you can say it in 5 or 500 lines. I call that the "ornamental level" of code.
All the agile methodologies are designed to stay close to the target which has proven to always move faster than expected. So they fulfil important functions, unfortunately estimation of a whole projects isn't one of them.
:-) stw

Stephan H. Wissel

Thursday, April 09, 2009 8:15:06 AM (Pacific Daylight Time, UTC-07:00)

Spoken like someone who is worried that process improvement will make them irrelevant. If we look at one componant of your argument, quoted from your post, "Software development is an incredibly complex, massively manual labor intensive effort..." and transform the line to another industry "Automobile manufacturing is an incredibly complex, massively manual labor intensive effort". You sound like the hounds did in America mid last century when Toyota was revolutionizing manufacturing with kanban, an American concept that was originally rejected in its country of origin.
The idea behind process improvement is that yes, what we are doing WORKS, but is it the most efficient and most effective way to perform the task. When we multiply lines of code by millions this becomes a very important question. We shouldn't be reinventing the wheel when we code, but looking for ways to get the most out our time. And by dismissing new ideas as "written by an MBA" or calling it "marketing speak" you make me believe that you haven't truly taken the time to understand the concepts and apply them to your trade. You sound like someone who is closed minded, believing that only a coder could understand your process. You begin to sound like a fossil from ages past when coders and management wrestled for control in a mindless tailspin. At the highest level, process is process, my friend.
When and only when we reach 5 9's effectiveness at hitting our time estimates, and 5 9's hitting our code quality measures, should we consider our search complete. Until then embrace change.

MBA Coder

Thursday, April 09, 2009 8:26:12 AM (Pacific Daylight Time, UTC-07:00)

Hi Stephan, thanks for your reply. With respect to CoCoMo, it has been my experience that once you calibrate the model, based on actual results of your project, that it does indeed become quite accurate. However, it does take time and patience as it means tracking each and every of your projects and getting the data entered.
Our company does a lot of SharePoint (UI work) and BizTalk (mapping XSL, orchestrations, etc,) development, which uses tools that are not text editors for entering lines of source code. In CoCoMo, there are calculations and considerations for this very aspect of non source lines of code. Have a look at sections 5.1 thru 5.3 and 6.1 in the CoCoMo model to see what I mean.

Mitch

Thursday, April 09, 2009 8:55:06 AM (Pacific Daylight Time, UTC-07:00)

Aloha “MBA Coder” Hmmm your comment IP address says University of Chicago. So maybe you are a student or a prof, wait, let me guess, taking or teaching a course on what, Agile? It is sweet irony that you posted your comment on a blog all about Software Industrialization.

Mitch

Friday, April 10, 2009 2:59:51 PM (Pacific Daylight Time, UTC-07:00)

Mythical Man Month is indeed a useful book, and many of its core concepts do remain true.

But for the 'young developer' it is just as possible they might find it a bit irrelevant given it wears its 1975 heritage on its sleeve. I would still encourage them to read it, but its not the the complete reference for all things software engineering.

There are other books that build on its foundations, such as a couple of the Steve McConnell books, which cover CoCoMo as well as a number of other estimation techniques.

Basically, the best process is the one that works for your team or your company. An average process which has the support of all works better than a great process improperly understood by some and adopted by the few.

You are right in saying some of methodologies seem to have as much marketing behind them as science, and a good portion of the agile stuff seems to be supported by 'the force' and a feeling rather than hard facts, but the only correct answer to 'what is the right methodology/estimation technique to use' is 'it depends...'

Steven Herod

Friday, April 10, 2009 4:14:33 PM (Pacific Daylight Time, UTC-07:00)

Hey Mitch

Interesting post. As it happens, we had friends over for dinner last night and the topic of Scrum came up in conversation. The general position was that Scrum helps define the "business" side of things, but is pretty limited from a practical day-to-day point of view. Some of the practices make sense, but have to be added to to form a useful software development process. This is why I don't say our team uses "Scrum", but adopts a set of practices and protocols appropriate to our projects - borrowing from Scrum, XP and a range of other methodologies (including so-called traditional approaches).

Agile is something of the little brother of Lean, so it's not surprising to see someone try to merge the two - though the term "Scrumban" does make me wince. I'd rather see us drop the 'agile' moniker altogether and accept that what we are trying to do is produce software in a 'lean' manner. The chief difference between the two approaches is a that agile is a set of positions, or assumptions, backed up by a philosophy - whereas lean is backed up by both a philosophy and scientific rigour.

Both approaches have their place, as sometimes the cost of scientific rigour is difficult to justify in a "it builds, ship it now - they need it urgently" environment. But it's important to know what you are trading off. Sometimes it's just a matter of business priorities, sometimes it's a way to satisfy an innate desire to run wild.

CoCoMo is arguably a scientific approach that has its place in a lean model of software development. Agile could probably benefit as well, as a means for providing the important feedback loop that methodologies like XP value so highly. For agile methodologies to work they have to have the right mix of practices and processes for the job at hand. My chief concern with CoCoMo is that it assumes the Big Design Up Front model that's just not appropriate in many projects.

As an aside, Ohloh ostensibly uses CoCoMo to derive its metrics on open source projects. Looking at the cost of projects I've been involved with or followed over time, the estimates often seem way too high - sometimes by an order of magnitude. But then, as a software developer I have to acknowledge the tendency to underestimate how long a project will take anyway. :) As an investigative tool, CoCoMo may have some merit in examining how much a project *should* have cost - as a post-implementation tool to uncover inefficiencies in a Scrum project.

This is rapidly becoming a blog entry in itself, so I may copy this comment and extend it on my own blog. Thanks for a great article. It's given me a lot to think about.

ferrisoxide

Friday, April 10, 2009 4:33:39 PM (Pacific Daylight Time, UTC-07:00)

Hey Steven, thanks for your comment, totally agreed! To be sure, I am not saying Mythical Man Month is the only book on software engineering or that CoCoMo is the only estimation model available. I am pointing out that these, among other “proven” methodologies and estimating techniques are the foundations of software engineering to be built off of. My point is that we seem to have moved far far away from these proven software engineering “best practices” or at least not have built off of them. And for sure, best tool/process/technique for the job – totally agreed, but I am taking the position that the tools, processes and techniques in the toolkit are “industry proven” and have successful track records, not some made up concoction of who the heck knows what – that’s way too high risk for me and software development is already risky business.

Mitch

Saturday, April 11, 2009 10:40:07 AM (Pacific Daylight Time, UTC-07:00)

When I read this, I feel like I am reading something I talked to you about personally. Check out the stuff I have said in this regard on my blog - you, my friend, are on the ball.

It's funny; watch "Flip This House" and think about all the excuses for why software development is a special animal when it comes to PM / process. Half of those reasons fall away, and you see scrum happening in the living room instead of the dev or conference room. Developing anything at all involves context, details, people, and so many variables that the temptation for a silver bullet is overwhelming (cha-ching!), but I think that old fashioned experience and work might be that bullet.

Thanks for this...

Best,

Josh Milane
MIT Technical Consultants

josh milane

Comments are closed.

On this page....

Categories