# Thursday, 04 December 2008

I love to write code.  I am 49 years old and have been programming off and on from 1977 – 1990 and as a fulltime professional since 1991.  I hesitate to even guess how many programming languages I have used over that time frame.  Since I love to program, not only was I using several different programming languages at various jobs, I was also experimenting with several others after work.

I don’t do as much “on the job” programming as I my role has been a “Software Architect” for several years now, but I still do a fair share and even more so in my spare time.  For example, I released an open source project that I have been working for the last two years called Global System Builder.  It was supposed to be fun, but that is the crux of the issue I am having – it was mostly a lot of really hard work.  Not that it was technically difficult, but seemingly something very simple turned out extremely hard to do.

Let me digress a moment to illustrate a point.  As the domain name of this blog indicates, I was progressing in programming languages as the level of abstraction was being raised over time.  Meaning in modern day times, not worrying about memory management, ala garbage collection in languages like C#, enjoying the REPL feel of dynamic programming languages and marvelling at the power of functional languages and then...

I came to the realization that as time marches on, rather than programming becoming easier (read: simpler) it was actually becoming more complex, to the point where today even to write anything simple seems to take a monumental effort, and seemingly more configuration effort than programming, and with so many moving parts, fraught with errors that are not compile time related.  I felt this not only in my professional software development world that I live in, but even on the open source project I was working on in my spare time for fun.  And it was supposed to be fun, but instead it was a design exercise at every turn figuring out which of the programming languages, frameworks, and widgets I could use that were the lesser evil since none of them did what I wanted them to do.  As you will see this is the irony.

Sure we use all sorts of frameworks today that supposedly make our programming lives easier.  The one I am most familiar with is the .NET framework.  At +11,000 types and +100,000 members, I am overwhelmed by the sheer size and complexity of the framework.  I can barely wrap my head around 7 +- 2 items let alone several orders of magnitude larger in size and complexity.  I spend more time looking at MSDN documentation trying to figure out how some Type works and the members I can use rather than actually writing code.

The argument is that we become more productive cause “it is taken care of in the framework” My experience and others would tend to disagree from a practical perspective and let alone the simple math truth.  Our brain is not designed to juggle thousands of Types and so we spend a great deal of time, searching, looking up docs, figuring out what to use from a design perspective, looking at the samples from an implementation perspective, looking at how others have used it – only to find that while it is close to what I want, it does not really meet my requirements.  Fine, close enough and with a few overrides, no problem.  But when you get into low level design and implementation, you run into hard stop limitations and then you a) try and find ways around the limitations or b) go back to the drawing board or c) think about not programming anymore.

And so ends the digression.  The point being, as told by Charles’s Petzold, “Does Visual Studio Rot the Mind” under the subtitle, “The Pure Pleasures of Pure Coding”, where he states, “...but there’s no APIs, there’s no classes, there’s no properties, there’s no forms, there’s no controls, there’s no event handlers, and there’s definitely no Visual Studio. It’s just me and the code, and for awhile, I feel like a real programmer again.

At last we arrive at why I am possibly deciding not to code anymore.  There is so much in the way and there is so much “of it”, that doing anything simple has become an incredibly complex task and doing anything complex takes teams of software folks to deliver due to the sheer size and complexity of our own programming environments, let alone trying to solve the problem domain we have been given.

Here comes the “I remember at time when...”   Love it or hate it, Visual Basic 6 (or the Delphi equivalent at the time) was the most productive programming language and toolset I have ever worked with (Digitalk Smalltalk takes 2nd place) for building business applications in the last 17 years I have been doing software development.  Why?

I used to do storyboards right in the VB6 IDE asking the business guys what they wanted, “so you need a form, with a list box and what should be in the list box, and when you clicked ok, now what... “And on it went.  In a couple of days to a week, we basically had the front end of the application prototyped.  Since it was back in the “two tier” days all we had to do was hook it up to the database.  And if we got really “fancy”, (or had the luxury),  we added a third tier of business logic written out as VB6 classes, not in the stored procs, and after a bunch of testing, report writing, etc., boom!  off it goes into production.

There were a lot of happy people back then.  The business users were happy because they got to sit in and basically design the user interface with me while trying to figure out the app.  And then weeks later it was delivered and did exactly what they wanted it to do.  I was happy, along with my team as we got a buzz on from being so productive and delivering what the users wanted.  And the tools, language and database was simple enough back then to not to get in anyone’s way.  Everyone happy!  So what happened?

One part of it is web applications happened.  Even in 2008, Visual Studio ASP.NET cannot give you a WYSIWYG view of your web application.  Yet, VB1 was able to give us WISYWYG back in 1991!  What the heck?  Not to just pick on Microsoft, I would say the majority of vendors’ web development tools are like a decade or so behind their “traditional” development tools for building desktop applications.  Further the vendors push more and more features, more and more “layers”, which mean more moving parts, which means more complexity, which means making it harder and harder for the programmer to use.   It reminds me of the infamous air-conditioning “duct” scene in the movie Brazil.

Our tools and applications have become so large, many and complex, it is literally making us less productive.  And to me, productivity is everything.  Productivity.  It is as simple as that.

Updated: A Programmers Dilemma - Productivity Lost - Part II

Thursday, 04 December 2008 22:24:03 (Pacific Standard Time, UTC-08:00)  #    Comments [14]
Friday, 05 December 2008 09:30:23 (Pacific Standard Time, UTC-08:00)
Sounds like a lot of whining to me.

I couldn't disagree with you more. It's probably time for you to just retire.

Nobody is forcing you to use any of the classes in the .NET framework. You could limit yourself to the core of the language itself and notepad if you feel you can be more productive this way.

Obviously if you try this, you'll find out pretty quick that these classes do offer value and you that you will be less productive trying to roll your own.
James Foster
Friday, 05 December 2008 09:49:38 (Pacific Standard Time, UTC-08:00)
Awesome article, and I enjoyed the Brazil reference. I felt largely the same way coming from the J2EE world. But rather than just going from simple ---> complex as time passes, I think I've rediscovered your VB 6 experience of 1991. When I got into iPhone programming this year, and got the hang of using Interface Builder to rapidly prototype things, I really got a feeling that, this is how simple things should be in other frameworks, and have rediscovered my love of programming again. I recently read an excellent blog post on hacker news, can't recall the source, than basic thesis is that the web in it's current form is broken, any web framework for building apps leaves much to be desired.
Friday, 05 December 2008 10:15:04 (Pacific Standard Time, UTC-08:00)
You should try ruby. Or python.

Both languages are fine (I am on the ruby trek and I only laugh about the constant and pointless debate about RoR)

I am 100% sure that scripting languages that are GOOD FOR ONE'S HUMAN BRAINS will easily beat huge frameworks that grow in complexity.

I do however have a simple general rule - if something starts to become too complex, it is time to simplify as much as possible. I am restricting myself to use a subset of ruby for example, and still I can do pretty everything I have to.

I think if a language allows this then it clearly has a lot of brain power behind it.
mark
Friday, 05 December 2008 10:27:25 (Pacific Standard Time, UTC-08:00)
You raise some good points, but the conclusion is weird:

"to me, productivity is everything. Productivity. It is as simple as that."

To me, delivering something of value to users is everything. I could write desktop software if I wanted, but that provides significantly less value for my users. They'd be tied to a specific operating system. They wouldn't be instantly upgraded when I released a new version. They couldn't try it out without "installing" it (what's that even mean?) first. If the text is too small, they can hit one keyboard shortcut and make it bigger. If something goes wrong, instead of crashing and risking their data, I have a log of what happened. There's arguments to be made against some of these, but this is what my users *want*; there's no point in complaining.

So the architecture seems backwards from desktop apps, but it's not because it's not productive. It's just productive at different things. You could have written this post on a proprietary system, after all, but then I would not have even been able to read it.

Besides, you can write web apps without a framework. My last project, I did just that, and it was very well received. You should only use a framework if you need what it provides.
Roe
Friday, 05 December 2008 11:08:38 (Pacific Standard Time, UTC-08:00)
It's not just you. I don't know what's going on with these other guys but I feel what you're saying just as strongly as you do and I'm only 27.

We're reaching levels of complexity where what we've been doing just doesn't work anymore. Especially with the web. I'm not quite sure what to do either, but I'm pretty certain learning another language or framework isn't going to fix it. There's just too much going on for us to keep track of it all. And it isn't fun anymore.

I think we would benefit from reducing the layers of abstraction. Mashing them down into fewer, thinner layers somehow. A lot of the web frameworks are trying to do this by making decisions for us or dictating to us how to use some of our resources, but it isn't enough.

I wish I had the answer. If you come up with something, I hope you'll let me know.
Friday, 05 December 2008 14:39:07 (Pacific Standard Time, UTC-08:00)
You are not a software engineer, you are a hobbyist, and it shows through clearly in your article. You had clients because there was a time when incompetent hobbyists could fool themselves and honest businesspeople into thinking they were making a productive system with inadequate tools.
I'm sorry you don't like C# and the .Net framework. They make perfect sense to me, and I love working with them. I found VB6 incredibly limiting and felt it made doing things far too complex if you were doing anything other than simple hobbyist apps. And fond rememberances of the good old two-tier days? Oh, for goodness sake. Open a flowershop and hope I don't ever interview you.

Michael Robinson
Michael Robinson
Friday, 05 December 2008 15:07:23 (Pacific Standard Time, UTC-08:00)
This is true of all fields. A mechanic working on cars 30 years ago will find far more complexity today.

Quite simply expectations have risen. You didn't have to provide online help in the past, or printing, or multiple locales, or networking or security. The increased complexity is because you have to do all that now and the list keeps getting longer. That is also why there are so many more classes.

As a developer you have to keep getting better tools, and get better at using those tools. As a trivial example using dual monitors lets you write code on one screen while using the other to filter through documentation on 100,000 classes. Source code systems have improved from being a version store to integrating bug trackers, wikis, continuous integration, offline access, lots of branching, testing etc.

If you try to use the old tools to do new things then you will be at a serious productivity disadvantage, just as a 20 year old email reader would be almost useless today or a mechanic only using a screwdriver can't do much to your engine.

When I job interview developers, one of my questions is about their preferred editor. I don't care which one it is, but instead am looking to find out if the candidate cares about their own productivity and actively seeks out new tools and techniques.
Roger
Friday, 05 December 2008 16:27:20 (Pacific Standard Time, UTC-08:00)
I've got to jump in and disagree too I'm afraid

We know that code re-use is a wonderful thing, and its just a real surprise to hear someone argue otherwise. The .NET libraries and the J2SE libraries are both a joy to use for me. Both have areas of weaknesses (e.g. the aneamic .NET Collections), some areas of awfulness (Java Date), but also areas of great stuff (java.nio, java.concurrent.*)

I agree that this does mean that programming can often feel like its been reduced to plumbing good abstractions together to get stuff done, but in many ways this was the holy grail in the first place right?

If you miss a lower level or detail oriented flavour of programming, you can still find this is the embedded sector, or even just in plain library design/implementation - you think Doug Lea didn't have an awful lot of fun coding up the java.concurrent apis? ;)

Once you have a nose able to smell/guess where the API's are likely to be and what the class names are likely to be, I think arguing that you're less productive would be something of a challenge :)

Lastly, I'm not sure quite why you've got some of the highly passionate responses above, for sure its an opinion that differs from most people I've met, but you're not too far of death threat territory in some cases :P
Andrew
Friday, 05 December 2008 17:57:21 (Pacific Standard Time, UTC-08:00)
I fully agree with your lament. I invite you to try Zoho Creator creator.zoho.com which tries to provide on the web the ease of use you get in a VB6 environment. Please give it a spin and tell us if we have succeeded!

Thanks,
Sridhar Vembu
CEO
Zoho
Friday, 05 December 2008 18:06:03 (Pacific Standard Time, UTC-08:00)
I completely agree with you.

At age 17 I sit in programming class trying to find my path through the maze of the .NET framework, thinking to myself 'okay, so I can do it . . . but how does it work?' Even when these frameworks are more efficient when you know how to use them, I believe they are leading to poor programming practices.

I see professional programmers using 'Math.Pow(2, 2)' when they could go '2*2'. Please, tell me what's more efficient! Maybe you could add some strings together with the '+' operator. Ha..ha.
Alecks
Saturday, 06 December 2008 05:06:22 (Pacific Standard Time, UTC-08:00)
I am sorry to disagree with you (even after reading part 2 of this post). I believe you need to think outside the .Net world. I was Windows programmer at some point and used to think of a very limited set of tools in my ecosystem. Trust me, there are more expressive languages and frameworks out there. Python and Ruby are good examples of languages where types do not matter (if you don't care about them). Haskell is an example of beauty of terse expressiveness.

If prototyping VB forms used to be a joy, I enjoy prototyping in my browser using Firebug. I can customise the application to pixel level in a web application instead of just playing around with a few form/control attributes in a property window. My users don't have to be skeptical about downloading a zip file that they don't trust. The application doesn't have to uninstalled and reinstalled for each version.

I can go on and on. What's new is not always complex. The productivity in modern languages have actually improved dramatically. People release new features all the time. But then, you need to stop using the old tools.
Saturday, 06 December 2008 05:22:34 (Pacific Standard Time, UTC-08:00)
You are so RIGHT, really, I appreciate someone finally feels the way I do about all that silly evolution. Don't worry to be too old, I'm aged 26, I don't think I'm ready for retirement yet, still, I know what you mean. Complexity is exponential and doesn't neccessarily add value. I'm missing the pureness of programming, architecture and design, things that seperate me from the kiddies, the users of frameworks, I don't like to spend years learning and becoming dependend (the main reason behind that efforts) on any architecture some major contributor came up with. It's not that I'm not able to keep on track, I'm simply not willing to follow, I'm not willing to lose my independence (e.g. I used to hate ASP.NET that much that I made my own damn framework including a control compiler and it was a hundred times faster, even if it took a hundred times more time to develop it). The same way I currently feel about Workflows and all that would-be-standards. Let me stress the point that I don't cry about progress catching up and making my former skills obsolete, but about becoming a worker bee puzzling around with kids building blocks. I dislike the role that is assigned to one nowadays. That's not what used to be the spirit and I somehow lost my independence when MS caught me with C#.
Yes, I read the comments. I know I don't have to stay with MS, and I guess I will switch to Python very soon. I predict that this will be the problem what will break the neck of the .net framework. Really really sad, but I don't want me to hear saying > "At last we arrive at why I am possibly deciding not to code anymore.".
And the other intrinsic point in these statements is that there soon is no place left for a programmer but to become a small part in a stupid business cycle, _as_long_as_ the so-called featuritis remains in place, i.e. there is nothing really new to make, it's just extending existing things by more kitsch on the base of faster hardware. It really is time to come up with something new, something that breaks the neck of the established crap. I dislike the word "web 3.0", but maybe you get my point.
frevd
Saturday, 06 December 2008 17:37:47 (Pacific Standard Time, UTC-08:00)
Couldn't agree more. I think it should be as easy to build apps for the web as it is for the desktop. That's something we're constantly working on at Springbase -- getting everything that's not essential out of the way so you can just get things done.
Saturday, 06 December 2008 22:08:34 (Pacific Standard Time, UTC-08:00)
A bad craftsman blames the tool.
PJ
Name
E-mail
Home page

Comment (Some html is allowed: a@href@title, b, strike, u) where the @ means "attribute." For example, you can use or
.  

Enter the code shown (prevents robots):

Live Comment Preview
© Copyright 2008 Mitch Barnett - Software Industrialization is the computerization of software design and function.

newtelligence dasBlog 2.2.8279.16125  Theme design by Bryan Bell
Feed your aggregator (RSS 2.0)   | Page rendered at Monday, 08 December 2008 17:47:37 (Pacific Standard Time, UTC-08:00)