I have made some progress developing a Smalltalk like System Browser for reflecting .NET Types with an IronPython embedded console interpreter to manipulate the reflected Types.  Not as much progress as I would like, but with a few hours a week, what can you do.

 

You can reuse code to increase productivity.  Here is something that seems a relatively simple task to do, I want to add an icon (bitmap actually) to each item in my list boxes that represent assemblies, types, properties, methods and events.  It turned out not to be so simple as a .NET listbox does not have this capability out of the box.  Now what?  As I have suggested in the past, the WWW is an excellent (unstructured) catalog of reusable software.  I make the assumption that each one of the components I am looking for to build this application has already been built by someone.

 

Searching away brought up several specialized listbox implementations, but this one I liked the best is called. .NET Color Listbox.  Alex has done a great job here as his implementation provided not only the ability to add the bitmaps to each item in the listbox, but also uses DoubleBuffering to get rid of the flickers when scrolling looks nice and smooth.

 

Where to get the bitmaps that represents assemblies, classes, methods, etc.?  If you have Visual Studio 2005 installed, you will find them at: C:\Program Files\Microsoft Visual Studio 8\Common7\VS2005ImageLibrary as VS2005ImageLibrary.zip.  So far so good, no reinventing the wheel yet.

 

While I still have more work to do on the System Browser part in the way of trimming namespaces in the listboxes, hiding properties that have public assessors, making the constructor bitmap different than the method bitmap, etc., I am going to move onto code comments.

 

So when I am reflecting an assembly and its Types, I want to include the code comments, if any are supplied, as well.  While I know about the XML Document Comments in C# and have used them for some time, you can use Ndoc for, C# and XML Source Code Documentation. In other words, API documentation ala MSDN style.

 

However, it turns out that the C# comments you put in your source files never gets compiled into the assembly.  I thought maybe the XML document was an embedded resource as indicated by this example of, Working with Embedded Data :

 

Using System.Windows.Forms as myAssembly

string[] names = myAssembly.GetManifestResourceNames();
foreach(string name in names)
{
   Console.WriteLine(name);
}

Nothing in there other than bitmaps, icons and cursors.  While you can embed the XML document as a resource, it is not done at compile time nor included in the .NET FCL.

I am a bit surprised by this.  Having the assembly and the documentation for the assembly in two separate files violates the principle of encapsulation in my mind.  The first thing to get lost when moving an assembly around is the XML documentation file.

 

I wonder why two separate files?  Yes, I know that the XML doc is also used for IntelliSense comments, but what if it was in the assembly and you reflected on it from the assembly?  Is it a size issue?  Looking in C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727 I see that the largest XML file is mscorlib.xml at approximately 7 meg in size whereas the actual mscorlib.dll is just over 4 megs in size.  However, the majority of other XML files are in the 100Kbyte range.  So maybe memory could be an issue, but at the very least, I would like to have the option of embedding my comments in the DLL so that if other people are reflecting on my Class library, they can also read the comments. 

 

Ok, well, I can still read the comments from the XML document files and load them into the System Browser.  Sounds like more code reuse as I am sure someone has written this before.  And as it turns out, yes in a couple of ways.  One is "XML Comments" which loads the XML documentation based on what you are reflecting on.  I have this implemented in the System Browser with one line of code.  The other code sample is called, Documenting .NET Assemblies which generates code comments directly from an assembly, even if there is no source comment XML dcoumentation.  Looks promising.

 

I know this post does not have much to do with IronPython (yet), but is the setup for embedding the IronPython interactive console interpreter into the System Browser.  The point about using IronPython to dynamically build apps from the .Net FCL and other assemblies requires a System Browser so that we can inspect the Types, grok the meaning and start dynamically building apps all from the same window.

A couple of weeks ago I suggested a way of dynamic programming with IronPython.  Essentially it is a combination of using a Smalltalk like System Browser to browse types using .NET reflection and embedding an IronPython interactive interpreter into the System Browser to manipulate the reflected types.

 

Why would I want to do this?  While I really like Visual Studio, which I use for my day job, sometimes it takes forever to build a simple prototype or experiment with a new class library.  For me, I want to see an app up and running as soon as possible, but at the same time leveraging class libraries and third party components.  And I want to do this dynamically.

 

I am also one of those people that learn best by doing.  While I can read the framework class library documentation till I fall asleep, (.NET 2.0 has 29,000 types in it!), I like to play with the types live and through some experimentation, find out how they work.  Sure I can use Visual Studios Object Browser or Lutzs Reflector (even as an add-in to VS), but it is not suited for how I work.

 

Maybe I am old school, but I like the Smalltalk System Browser approach to software development.  In .NET it is a little harder because the Smalltalk world was really simple, you had classes, instances and methods.  Thats it.  In .NET it is more complicated, but still what I am interested is inspecting an assembly for its types and for each type, what are its public properties, methods and events in other words I am using the System Browser as a way to work with the types, not just browse them.  Having an IronPython console window embedded in the System Browser lets me play with those types dynamically.  To me it is the best of both worlds, leveraging pre-built types to dynamically build my prototype or application.

 

As a side note, this blogs focus is on software industrialization.  I have explained what that means in other posts, but one aspect of software industrialization I would like to point out is simplicity.  The ability to browse libraries using the Smalltalk System Browser paradigm and then use those libraries interactively in the same window is simple.

 

The System Browser is initially built as a C# application, just to get parts of the code design down of what I want and in a language I already know.  I am also mirroring the development of the application as a Python Windows application, which is what the final product will be.  While I have run into a few problems, mostly my own, I have had excellent support from the folks at the IronPython forum.  Of course, I have already made a fool of myself on the forum by repeatedly submitting a chunk of Python code that had a (read: my) mistake in it, which I missed on a couple of occasions.  Maybe if I slow down a bit, I will see it before I submit it   Thanks to Dino, Vagmi and J. M. for being patient.

 

I should also point out that for Windows Forms programmers looking to get a tutorial on creating Windows Forms dynamically using IronPython, should look at Michael Foords excellent tutorials. 

Back to the System Browser - maybe I will have something implemented in a couple of weeks or months.  Stay tuned.

As I mentioned in a previous post, I am really excited about IronPython on the .NET platform.  The main reason is being able to program dynamically.  What I mean is that I can import .NET classes and start using them interactively with the interpreter.

 

For example, I can interactively build a simple WPF (formerly Avalon) application in just a half dozen lines of code.  I dont mean running a script either; I mean entering in one line at a time and immediately seeing the results on the screen.

 

For example, the following code imports the WPF library (and a small script to allow interactivity between the interpreter and application being constructed), creates a new window instance, displays the window, sets the title and some text in the form, sizes the text and finally sizes the form to the text.

 

from avalon import *
w = Window()
w.Show()
w.Title = Simple WPF App
w.Content = TextBlock()
w.Content.Text = Aloha IronPython!
w.Content.FontSize = 50
w.SizeToContent = SizeToContent.WidthAndHeight

 

Alright you say, no big deal.  Well, on one hand you are right on the other, I think the point is that it satisfies a major requirement that I have as a programmer which is to interact with building my application in real time.  This goes beyond just the compile, link, deploy cycle of compiled languages.  This allows me to see what I am doing as I am doing it.  Some would say this is hacking with no design documentation and just winging it.  Others would use the analogy of a painter and the canvas where you are seeing tangible results in real-time as you are painting just like in the real world and not in our make belief software world.  Or even if I had a design document and was following Agile methods, this to me gives immediate verifiable results as you are seeing it on the screen, again in real-time.  This whole notion of dynamic programming and the semantic meaning I have for it will be a future topic.

 

I could take the sample above much further and in fact, if you download IronPython you can follow the tutorial which this snippet is from, plus load up XAML, build event handlers and hook them all up to produce a real working live application with a handful more lines of code.  As tutorial states, IronPython provides a very easy way to develop live application in a dynamic and exploratory way.  That is what has me very excited about this dynamic programming language on the .NET platform.

 

.NET 2.0 Framework Class Library has roughly 29,000 types in it.  Plus, as I have written elsewhere in this site on software industrialization, there are tens of thousands more COM dlls and other .NET dlls that have been written.  My point is that though IronPython any of these types can be imported and used dynamically in any application you may be building, particularly if you are predisposed to performing programming in a dynamic and exploratory way.

 

How to so this?  In my post on Software Industrialization using .NET Reflector and IronPython, I suggested building a Smalltalk like System Browser with and IronPython interpreter built into it.  While Lutz Roeders .NET Reflector product is excellent and highly recommended, for me, it is difficult to use from a tree view perspective.  I much prefer Smalltalks System Browser approach plus I can work with types in fewer mouse clicks.  Finally, I can embed the IronPython interpreter in the code window so I can work with the types that I am exploring and later as we will see, add my own types on the fly that are subclasses or extensions or glue code to all of the other types that I have loaded.

 

First things first.  How do I get type information?  There are several excellent articles on .NET Reflection and the best one summarizes it here.  The first thing is to write a wee bit of IronPython code to find the types in any given assembly.  Here we are going to get the types out of mapack.dll

 

IronPython 1.0.2280
Copyright (c) Microsoft Corporation. All rights reserved.
>>> from System.Reflection import *
>>> a = Assembly.LoadFrom("mapack.dll")
>>> Types = a.GetTypes()
>>> for Type in Types:
...       print Type
...
Mapack.CholeskyDecomposition
Mapack.EigenvalueDecomposition
Mapack.LuDecomposition
Mapack.Matrix
Mapack.QrDecomposition
Mapack.SingularValueDecomposition
>>>

 

Easy enough.  Next post in this series is to build a Windows App (in IronPython of course) that starts to take on the shape of a Smalltalk System Browser so we can start reflecting types for possible inclusion for the apps we are building.

I remember in the mid 90s when software reuse was a big deal.  Several books came out that discussed the need for software reuse and how to set up a reuse library in your organization.  My favorite is, Confessions of a Used Program Salesman:  Institutionalizing Software Reuse, by Will Tracz.  One of the very few textbooks I have read that was as entertaining as it was useful.

 

10 years later I dont see much about software reuse at all.  I dont find it as a topic of discussion on any programming related forums.  Is it because we as computer scientists, software engineers, or programmers have institutionalized it as standard operating procedure or is it because the fascination we have over programming languages, software reuse is a forgotten practice?

 

Looking up software reuse on Wikiepedia gets redirected to code reuse.  This was a real surprise to me.  I use Wikipedia quite a bit for software related information and it is usually is up to date and accurate.  However, to equate software reuse with code reuse is missing the point of software reuse in general.  I would define software reuse as the ability to leverage any existing software artifact that makes solving any software problem easier.   Whether it is a project management schedule template to a fully blown domain specific language for automatically generating ecommerce solutions would be considered software reuse in my book.

 

I know there are several libraries, SDKs, commercial components, etc., that come with your favorite programming language.  Even K&R C from 1978 had this and probably even earlier before my time entering the computing world.   The mid 90s also had the explosion of design patterns, the Gang of Four being the most familiar to software designers and programmers.  Today there is an incredible wealth of design patterns.  I am wondering if we ever use them?  I must admit that while I have used some design patterns in a few projects and probably some unknowingly, I dont go out of my way to decompose my software designs into patterns, cause quite frankly, in my world of professional software development services, we never have time for such activities.  My software development world is governed by the clock get it done as soon as possible.

 

So how do we implement reuse in our organization?  In a nutshell, we Google it.  Here is how it works.  Our customer needs a custom web application built with certain functionality (what that functionality is does not really matter, because it has usually been done before).  Our projects typically take on two distinct phases.  Phase 1 is to write up a requirements specification, usually in the form of use cases and storyboards, all of which we have a library of templates which we reuse, including the statement of work, project schedules, etc.  The customer signs-off and we are on to Phase 2.

 

Phase 2 begins with, what do we have from a previous project that we can reuse?  That means any ASP.NET web pages, controls, components, custom assemblies that we have written before that we can leverage.  For the pieces we dont have, we start our search with Google and other tools, looking in the usual suspect places where we know we can find example/sample designs and code that we can reuse in our overall solution.  Since we are a Microsoft shop specializing in custom ASP.NET, SharePoint and just plain .NET applications, we know where to look.  In fact, we have several links in our SharePoint library that points to our regular spots.

 

Once we have gathered all of the pieces, we look at what actual custom code is left to do.  Surprisingly (or not) not much is left to do other than glue the pieces together and apply the paint.  Now I make it sound easier than it is, but I would say on average that well over 50% of any custom development work we do is reusing pieces (on top of the .NET Framework Class Library) and in some cases as high as 80% software reuse.  Just to be clear, we are a .NET shop only and we are not implementing or deploying packaged products, except in the case of SharePoint and then we only get involved in custom development using the APIs and/or page layout.

 

We have our test harnesses for unit testing, system testing, FxCop design and code coverage tests, performance and stress testing, bug tracking templates, etc.  These are also reusable tool and template assets that we have accumulated and customized over the years.  Some of the custom applications we have designed and developed handle over 1 million transactions per day that I think are good pieces of work.  And others, well, you know, right?  Since we have yet to reach a stage of industrialization in our software world, some projects are better than others.

 

So when does software reuse not work well?  In our case it is the adoption of new technologies such as Microsofts SharePoint 2007 which just reached Beta 2.  The problem here is that the architecture is so new that very little in the way of samples, examples or how tos has been developed.  Here we are on our own, with little documentation, massively changed APIs and of course a Beta product.

 

What does this have to do with asking the question if we as the software community are reusing software?  I guess thats the point.  We seem to be doing it but when we are hiring new candidates (including seasoned professionals) only a few of them can point to reusing software and it usually is code reuse and even fewer have experience with design patterns.  I cant believe we are the only ones doing it.  But the friends I have in other software organizations also point to a very small percentage of software reuse in their own orgs.  Why is that?

 

I dont think I have the answer to that question.  I see that there is an international conference on software reuse this year.  Not sure who attends these.  I wish I could go to Italy   Spending more time on Google does not help as most of the links to the subject matter in question are relatively old.  It seems no longer a hot topic.

 

I would be interested to hear other peoples stories of software reuse.  If you have a moment and would like to share, let me know.  Thanks.

 

You and a million people other people can use the same chunk of knowledge without diminishing it.  In fact, the greater the number of people who use it, the greater the likelihood that someone will generate more knowledge with it.

 

- Alvin and Heidi Toffler from the book Revolutionary Wealth

 

The Tofflers are back with more futurism continuing their work from the Third Wave which was written in 1980.   I remember at the age of 12 reading Future Shock and all about information overload.  Coming from a small town in the Great White North of Canada, I had no idea what this meant.   Now I do.

 

But I digress, back to the quote above.  What does this have to do with software industrialization?  Everything actually.  In fact, thats the point of reusable software a chunk of software that is built upon another chunk of software that can be used by more people.  While I have accused our software industry of always writing software from first principles, commenters on this blog have pointed out that most always we write code on top of reusable frameworks or libraries.  While that is true, we still do write some of the same software over and over again (and one source code line at a time).  How many login dialogs have been written over and over again?  Customer classes?  Order classes?  You know what I mean, we are kinda stuck at a certain level of reusability where anything above framework or library code, we are back to writing code from first principles again.

 

I think the Tofflers quote also parallels Brad Coxs Software IC vision and specifically in his book, Superdistribution Objects As Property On The Electronic Frontier where we as programmers manufacture objects made of bits, but we still have not found a way to buy, sell and own them.  Brad has a solution which in many ways parallel what the Tofflers are talking about, yet, this industrialization of software has yet to mature to a point where it is a reality.  Aside from the distribution problem, which I will discuss in another post, I am interested in the reuse problem at the moment.

 

In my last post I discussed an idea around a design tool that reuses software components, or assemblies or classes, depending on how you see it.  Now I am going to discuss these ideas in the context of .NET as that is what I know the best, even though I have programmed in several languages (I do miss Smalltalk), I make a living as a .NET architect so this is the context I will use.

 

Lutz Roeder has a fantastic tool called .NET Reflector which is a "class browser, explorer, analyzer and documentation viewer for .NET. Reflector allows to easily view, navigate, search, decompile and analyze .NET assemblies in C#, Visual Basic and IL."  In some respects it reminds me a lot of a Smalltalk System Browser or Objective-C Browser.  The point being is that I can load any .NET assembly and effectively discern its design specification.  Why would I want to do that?  Well, if I want to reuse a piece of software, then I want to know what it has to offer so I can discern whether I want to reuse it or not.  Moreover, if I decide to reuse it, I need to know what methods, properties are available to me.

 

Where am I going to find reusable software?  Well, aside from the few catalogs that are available, Google (search) has the biggest reusable software catalog in the world.  So lets say I find some components that I want to reuse.  For example, the ubiquitous customer class or assembly.   I can add a reference to it in VS2005 and start using it, lets say in C#.  But then it is only one component (or class in this case) and I then I am back to writing single lines of code again.  No, what I am talking about is collecting as many reusable classes and/or assemblies that can solve a business problem and then composing my solution out of these assemblies.  In fact the design tool I am thinking of can assemble these assemblies dynamically.  Thats the other half of the design tool I am thinking of.  One half is disassembling components for their design specification and the other half assembling the components into a finished application.

 

This is why I am really excited about IronPython, a dynamic programming language that seamlessly integrates with the .NET Framework Class Library (and any other .NET assembly and COM object).  This means my design tool can glue classes together, subclass and essentially dynamically bind objects on the fly.  Now if I could that, then all of a sudden I got a real software design tool.

 

Maybe the design tool will look like a Smalltalk System Browser where a product and/or functionality like .NET Reflector can be used to fill the browser with classes and methods from the various imported assemblies (including the .NET framework class library).  Then using IronPython as the embedded interpreter, I can add code to glue the assemblies together to produce the final solution.

 

Whats the point of this?  Well, I think that in order for software industrialization occur, we need to be able to reuse the vast amount of software components (classes, assemblies, whatever) that have already been written.  In fact, I am fairly sure that whatever your software problem is, a solution already exists (or at least the majority of components), you just have to search for it on Google.  Thats a rather sobering thought, the software you have been tasked to write, in all likelihood, has mostly been already written.   Or at the very least close enough for you to subclass or add methods to an existing class (or assembly) where you are composing applications rather than coding them from scratch.  While Quartz Composer is a GUI Visual Programming Designer that assembles GUIs with little to no coding, the same principles apply.  Software Industrialization is just a matter of time.

 

Next post, it may take a while, but I would like to prove this idea out.  First I need to learn IronPython

 

- Russell Ackoff, Ackoffs Fables.

 

 

- George Bernard Shaw, Mrs. Warrens Profession.

 

A couple of quotes to start off this post as I was lamenting the fact that we don't have any real software design tools as described in my last two posts.  Time now to do something about it.

 

The .NET 2.0 Framework Class Library has approximately 29,000 types in it.  Many more times that has been built in the COM world, plus the .NET world over the last 10 years.  Even using Google to narrow the scope of locating a specific Customer.DLL turned up 275 hits.  Of course, not all of these are .NET or even COM components, but my point is that an incredible amount of software has already been designed at the component level, but there is no easy way to reuse or leverage these assets in the way that I want from an application design perspective.  In other words, why build a new customer class when a couple hundred have already been designed and built.

 

The design tool I am thinking of needs to be able to leverage these reusable assets.  When I speak of design time, I mean the ability for a tool to dynamically access pre-built types, subclass them, add methods or properties, tie other types together and collectively compose applications based on reusable parts with some level of coding to glue it all together.  This design tool is actually an application that anyone could use to compose their own applications using whatever pre-built types or components they wish to use in an easy to use fashion.  The design tool would be targeted at business analysts and programmers and even power users who need to build a relatively simply application where Excel is not powerful enough, but the Visual Studio IDE and C# are too complex and time consuming to learn. 

 

As a person that gets paid to architect and code in the Microsoft world, I have been following a nifty dynamic language called IronPython.  In fact, I am really excited about IronPython as the .NET implementation is incredibly reflective, exactly what I need for the type of design tool I am talking about.  IronPython is the code name of the new implementation running on .NET of the Python programming language. It supports an interactive interpreter with fully dynamic compilation. It is well integrated with the rest of the framework and makes all .NET libraries easily available to Python programmers.  Also, is happens to be open source from Microsoft.

 

To get an idea of how powerful this dynamic language is, have a look at Jim Huginins 15 minute demo on the MSDN site.   I recommend downloading the videos to watch locally on your computer to avoid popup hassles.  One interesting part is embedding IronPython in a WPF XAML application, something that I am considering for my design tool.

 

Now I have no intention of declaring IronPython better than any other programming languages or the foibles of duck typing.  What I am suggesting for my purpose in building a design tool for interface extensibility for .NET types, it could not get much better for me.  One of IronPython's key advantages is in its function as an extensibility layer to application frameworks written in a .NET language. It is relatively simple to integrate an IronPython interpreter into an existing .NET application framework. Once in place, downstream developers can use scripts written in IronPython that interact with .NET objects in the framework, thereby extending the functionality in the framework's interface, without having to change any of the framework's code base.

 

IronPython makes extensive use of reflection. When passed in a reference to a .NET object, it will automatically import the types and methods available to that object. This results in a highly intuitive experience when working with .NET objects from within an IronPython script.

 

Perfect for my design tool scenario. Next post, what this design tool might look like.

 

I came from the R&D electronics engineering world in the 80s and got into software development in 1991.  After 15 years in the software dev biz, the only conclusion I can draw is compared to the industrialized electronics world I came from, the software development world is decidedly unindustrialized.  In addition, we are going backwards instead of forwards in our progress towards industrialization.  I am genuinely concerned about the future of our software industry.

 

Nonsense you say?  Let me put some context and perspective around my statements.  In the 80s when I was still in the electronics engineering world, software computer programs (i.e. CAD/CAM) were on their way to the top of the Technology Adoption Curve,  particularly with the release of AutoCAD in 1982, which soon brought a level of design automation to the electronics engineering world that was literally revolutionary.  I lived it.  Here is a concrete example of what I experienced.  Instead of handcrafting printed circuit board layouts by hand, within a couple of years, we were operating several sophisticated computer (i.e. CAD) programs that could assist with the design of the layout of the board cheaper, better, and faster than I could do it manually.  What the CAD programs didnt have was my experience so in the hands of a skilled and experienced operator meant the ability to put cheaper goods, with much more quality in the hands of consumers, faster.  It also meant some serious electronics industrialization has happened over the last twenty-five years - almost feels exponential in my opinion.

 

I used the term unindustrialized in the first paragraph, what do I mean be that?  I mean that in our software development world, we do not have any (real!) predictable or repeatable processes for the specification (i.e. design) of software.  And please dont tell me about yet another great process methodology**.  Worse yet, when we do design and construct a software artifact, it invariably is the wrong thang.  We also cant scale in anyway to design large scale architectures.  Finally, our software development world today is based upon designing software from first principles, rarely do we leverage other peoples excellent work, including CAD systems and/or approaches that worked in other, now industrialized industries (e.g. electronics).  I see the latter being the biggest hurdle to overcome as it is a cultural change instead of a technical one.

 

** Note In 1995 I worked at Eastman Kodak in Rochester, NY.  Kodak Equipment Commercialization Process (KECP) was a new product introduction (NPI) process methodology that has evolved for decades we have nothing this evolved in our software world.  Further, some would say that in order to make a process predictable and repeatable, you need to be using the same tools over and over again or at least for a period of time to establish a baseline and then a feedback loop.  Our software tools keep changing every six months (or earlier), so we cant even establish a baseline let alone a feedback loop  Kodak knows a thing or two about industrialization and we in the software industry could learn from that.

 

What do I mean by going backwards?  I feel that we are going backwards in two areas, programming languages and more importantly, design environments that can produce design specifications, and no I do not mean CASE tools.  We are woefully lacking Computer Aided Design tools that are now best practices (and have been for years) in other industrialized industries like electronics, manufacturing, building construction to name a few.  Do you think it is unreasonable of me to expect better software design tools when I was handed exponentially more powerful tools in the electronics industry years earlier?  Thats my gripe, still hand crafting single lines of code after 15 years in the industry.  Something is wrong, man!  Maybe I am real naive or born in the wrong era, but you would think that we in the software industry, after 15 years (try 30!) could come up with better design tools than a glorified text editor, now mockingly called an Integrated Development Environment.  No?

 

So, programming languages, how can I say we are going backwards?  Preposterous!  Well, whatever happened to Smalltalk?  Yes, every programming language is a foreign language, some easier to learn than others and thats my point.  Have a quick read of this short article from 1991 to see what I mean.  From my industrialization perspective look at how incredibly simple and efficient Smalltalk is!  The language consists of 5 keywords (Java and C++ have +50) and the entire language specification can be expressed on a single sheet of paper!  Isnt this the pinnacle? The state of the art?  Its language roots are +30 years old and seem to be one of the most mature programming languages available.  What happened?  Not only was the language simple, but the development environment was equally as simple along with the most seamless source control I have ever worked with.  Again, what the heck happened?  Whats wrong with Smalltalk?  No, I dont mean strongly typed versus dynamic or VM or GC or anything technical.  I mean is it because our industry is so immature we cant see a good thing when we see it?  Is it because our industry is dominated by a handful of companies which dictates the software programming languages market?

 

The multiple programming languages I use today and the IDE I am in are far more complex by at least an order of magnitude.  Further, rather than optimizing a language that is simple to learn and effective, we proliferate choices for the sake of choice, nothing technical here.  I know of one vendor that supplies 7 different graphical user interface technologies, each complete with its own framework class library and even more IDEs.  7 UI technologies from one vendor?  How is this making it easier (for anyone)?  How do I know which one to choose?  How do I become expert in any of them, as I still need use my primary language (C#) and another incredibly large framework class library, .NET 2.0.  If anything, it is getting harder and harder to develop custom applications, not easier.

 

XAML is a brand spanking new declarative programming language from Microsoft.  This raises the level abstraction from an impetrative programming model (e.g. C#) to a declarative programming model.  So if I need a day job and live in the MS world, I would take a long hard look at XAML.  It goes far beyond then just another user interface mark-up language (using Windows Presentation Foundation).  Sure the view is separated from the model, yay, but that was a Smalltalk invention (i.e. MVC).  However, the crucial invention here is that XAML is a language that enables developers to specify a hierarchy of common language runtime (CLR) objects with a set of properties and logic.  Meaning specifying a hierarchy of any .NET objects using a declarative language.  Hmmm there is a CAD program for designing software in there somewhere.  Key point to get - just like Objective-C provided an interface to any objectified clib to be plugged together like electronic integrated circuits.  Remember Brad Coxs Software ICs?

 

When I was working at a software company called the AND Group in the early 90s, an extremely sharp programmer named Tim Wasko (look at you now Timbo!), showed me Brads invention on a NEXTSTEP computer.  I was blown away and so was Tim.  Brads Interface Designer and Integrated Circuit (IC) library approach to supplying and assembling software components using a common interface (Objective-C) rang true to me then as it does now, 15 years later.  It is an excellent analogy to the tools and approach we took in the electronics engineering world.  Not only did it work, but software programs like AutoCAD had a major impact on the entire product development process for the electronics industry (and others), meeting the requirements of industrialization.  Finally, a piece of software that can really claim automation of processes and offers scalability that no human could do.  AutoCAD (and similar) revolutionized the electronics engineering through the 80s.  Note that the tool enabled the process of industrialization and in fact, you hear little about the product development process, cause it is well known.  However, in our industry, because we have no standard way of designing software, all we have is process methodologies with dumb a$$ names.

 

Brads Objective-C System Building Environment is interesting as most people classify his invention as yet another programming language.  Unfortunate, as the crucial invention here was that the language (i.e. building environment) allowed systems to be easily designed based upon assembling composite objects using an Interface Designer and IC Library.

 

This approach is very similar to the electronics world where our designers are various CAD programs that would specify what we wanted in terms of electronic schematic circuit diagrams, circuit board layouts, mechanical drafting design, etc., which all represented the specifications (i.e. design) of what was to be implemented (built, constructed, made real, realized, whatever you want to call production).  We also had our IC libraries called electronic parts catalogs that we could order parts that met our specifications and then could be assembled onto our now made printed circuit board whose layout was based upon a circuit diagram specification, all of which we specified (i.e. designed) in some sort of CAD tool.

 

Why do I think this comparison to the CAD world of electronics and our own?  I believe this is the path to software industrialization.  I recently talked to an old mate (hi Darren!) I used to work with in the early 80s when we were part of an electronics R&D shop.  He went on to get his Masters in Electrical Engineering and has been working in Nortels R&D department as a RF Designer for the last 8 years.  I asked him what some of the advances were in the way of specification (i.e. design) tools.  He laughed and said that they have a directory tree full of CAD tools used to specify what they want built.  Most if not all of the tools are visual design tools with a few (visual) simulation tools.  In other words, they are domain specialized tools, with their own languages for each aspect of the specification that needs to be designed.  In fact, if you were to classify it to our software world, one could say that each one of these CAD tools encapsulated their own Domain Specific Language (DSL).

 

I would also say that Brads System Building environment, from 15 years ago, was also a DSL that objectified clibs using a standard interface (i.e. pluggable palettes and a common design surface).  In other words, the visual design raised the level of abstraction to a point where you were assembling prebuilt C libraries that plugged into a standard interface.  That meant any supplier of clibs could objectify their library and have it plug into the design environment.  One analogy is like Visio where you buy a set of 3rd party symbols (like electronic symbols for example) that plug into Visio as a standard palette in which now you can interact with on the drawing surfaces and design circuits.

 

In fact, this is the point of the analogy.  In the electronics world, the electronics circuit schematic diagram with its symbols and lines represent a design specification.  Every other artifact and/or transformation is derived from the design specification.  Note that everyone uses the same (domain) specific language to design electronic circuits.  This language consists of numerous symbols.  Each symbol is an abstraction of the real thing.  There are symbols for resistors, capacitors, transistors, integrated circuits, etc.  But note that the symbols are simply interfaces.  Each interface has a definition, like a resistor symbol typically has two connectors and between those connectors is where the specification of the resistor is specified, in which there are a million combinational values. Or another way to look at it is someone elses spec sheet is the resistor value.  In fact for every symbol or interface on the design surface is where you indeed plug in a spec sheet that came from a catalog or that you have to custom build yourself or a third-party builds it for you.

 

Of course, in the electronics world, we use CAD programs to draw our schematic diagrams using these standard symbols.  We still dont have these types of specifications tools in our software design world.  In fact, we still dont get what software design is.  Have a look at this 1992 article on, What is Software Design.  Software design is still the 1000s of lines of code - in other words, the actual program listing is the only real design specification.  This program listing was hand-crafted one source code line at a time.  The closest activity I can think of that is similar to writing software is like writing a novel, but in a foreign language next blog topic.

 

So what does this all mean?  It means that we in the software industry have built all sorts of CAD, CAM, etc., tools for virtually every industry but our own.  How ironic.  In my opinion we are still using hammers and chisels to sculpt one-off master pieces.  How many e-commerce applications in the world have been hand-crafted? Millions of them. Functionally, they are all virtually identical and still today we keep hand crafting them, one source code line at a time.  This is what really makes me mad about our industry, the all so mighty not invented here syndrome or must build from first principles or whatever it is to avoid using someone elses perfectly fine programming language, development environment, etc.  So every year we just re-invent the same thing over again and give it a new acronym.  If I have to learn one more programming language or one more class library I am going to quit this nonsense and grow oranges, or since I am in BC, grow some BC b$d.

 

Until we as the software development industry get serious about producing some real CAD tools for designing software, we are going to hand-craft our profession into extinction.  I can tell you that the future of software industrialization wont look like and that is what it is today building the software world one source code line at a time.

I have a great deal of respect for Brad Cox.  In fact, consider this essay a tribute to Brad. Ten years before he wrote, Planning the Software Industrial Revolution, I was living in the integrated circuit world of R&D electronics, where it was already an industrialized industry.  I could order chip, card and rack level components from catalogs and plug them together though a set of standardized interfaces that solved a set of problems that I had specified in an electronic schematic diagram.  During that time, I was also a participant in industrializing printed circuit board layouts that initially took 6 to 8 weeks of hard manual labor on a light board to, ironically, using a computer program to read an electronic schematic diagram (i.e. specification) and producing the printed circuit layout in a matter of minutes.  In other words, I experienced first hand what industrialization was and meant in the electronics industry.

 

When I first got into the software industry in 1990, I thought it was going to be similar to the industrialized electronics industry I was used to.  I could order software components (like integrated circuits) from catalogs and using a schematic diagram, CAD like software program of some sort, assemble those components to solve a set of problems.  Nothing could be further from the truth.  Then I read Brad Coxs article and realized what I took for granted in the electronics engineering world may not even happen in my lifetime in the software world.  It was quite a shock to me.  I had no idea the software world was so far behind, and in fact, completely unindustrialized.

 

Flash forward 15 years and I have performed just about every role one could imagine in the software industry, including running my own software company.  I feel like I have gone nowhere in this industry and in fact, in some respects gone backwards.  I still write the same source level code I did 15 years ago and build most everything from first principles.  Sure technologies have changed, but we (meaning everyone in this industry) are still building our software world, one source code line at a time.  As Brad says, grains of sand where bricks are needed.  Whats wrong with our software industry?  Why dont we learn from the successes of other industrialized industries?  Are we that arrogant that we have to do everything from first principles over and over again?

 

In Brads article he discusses the importance of Blanchards pattern lathe and that the real crucial discovery made was, that implementation tools were insufficient unless supplanted by specifications tools capable of determining whether parts complied to specification within tolerance.  As Brad points out, this discovery has not yet occurred in software.  In my opinion, 15 years later, it still remains undiscovered.  Making software tangible and observable, rather than intangible and speculative, is the first step to making software engineering and computer science a reality.  Brads sentence still applies today.

 

Ironically, computers and computer programs have helped other industries become fully industrialized.  In 1982 AutoCAD came to the world and revolutionized the industrial design and manufacturing industries, including my printed circuit board story above.  The craft of drafting has not gone away, but the tools used have raised the level of abstraction to the point today where those specifications are now items that I can order out of a catalog.  For example, if I want to design a house, I dont have to invent the drafting symbols that represent my house, they already exist and in fact, can be ordered from a catalog.  Further, I can order the entire house specification (funny enough called an architectural blueprint from a catalog and customize it (using standard symbols) to what I want.  That specification is then used to implement (i.e. build) the house.  Some people would say in the software world we have that through UML. When I showed a business person a UML use case diagram, he said, Stickmen?  Thats all you have is stickmen? And then he proceeded to laugh uproariously and just walked away.  At the time I felt angry and thought another bozo that does not get it.  But as years went by, I realized he was totally right.  How embarrassing for our software world where the best we can do on the specification side is stickmen.

 

So we need better specification tools, thats for sure what else do we need?  As Brad writes, the programmer shortage can be solved as the telephone-operator shortage was solved by making every computer user a programmer.  I wholeheartedly agree.  This would dramatically increase the chances of industrializing our software world.  At the very least it would help us with the most frustrating part of our world and that is human machine interface design.  Generally speaking, software engineers and engineers simply dont get it, for whatever reason.  Lets look at several examples of what I mean and tell me I am wrong.

 

Take your average DVD player. Did you ever happen to notice that the hardware (DVD player) does not control the software (DVD disc)?  One of my commenters (thanks Michael K) on my blog said, I want a DVD player that works like a VCR.  When I press fast forward on a VCR - the recording goes forward. With a DVD - it is up to the disk whether I can fast forward it or not - total lunacy!  Its my player it should do what I tell it, now!  And just how many DVD players are there in the world?

 

How about automated phone attendants? A sure way to increase the blood pressure of even the calmest West Coaster, particularly the new and improved attendants that respond to your voice commands instead of key commands. Insert your favorite voice command here   The fact companies purport that they are customer driven by using automated attendants is a joke or is it that they dont get it? (ha!).  When I speak to a company, I want to hear a human being answer the phone and one that knows the inner workings of the company to provide me with a layer of abstraction between the companys goofy organizational structure/behavior and myself.  S/he is the public interface to the company that hides the inner workings from me.  The thing about it is that we used to have it, so what happened?

 

What about ATMs?  Well, it is a matter of convenience.  The fact that I can quickly get money, usually without waiting in line, is what I want from a users perspective.  So that part of the abstraction is good, but the human machine interface could do with another tweak.  That is, if I have only one account, then dont give me account choices.  I get asked for checking, savings, and other, every time I use the machine, no matter where I go in the world and I have only one account.  How hard can it be?

 

Even the firmware in my wifes German precision engineered washing machine is koo koo.  It has lots of intelligence built-in, but the machine does not automatically shut-off by itself.  It will beep beep, beep at you until you come and manually turn it off.  Going through the 27 page manual, I found a way to at least turn-off the beeping.  However, months later it came back and I forgot the magic key combination to turn it off and I refuse the read the frickin manual again.  How can this be?  How many millions of washing machines have rolled off the production line in the last 25 years?  And still cant get down to the one button interface called wash clothes, and dont bother me again.

 

Or how about, Windows Vista, 5 million lines of code, 5 years later and Hasta La Vista Baby!  As you will see they have also opted for aesthetics over functionality.  What does the customer really want? An extremely fast, low memory, transparent to the user and completely reliable operating system.  Thats it!   Again, how hard can it be?  Yet, wait till you see it and by the way, you will need to buy a new computer to run it.  That from the largest and most successful software development company in the world today.

 

Finally, while not really software based, is an excellent example of poor human to machine interface design.  The building that I work in is having its washrooms renovated and you guessed it, upgraded to automated faucets, soap dispensers, urinals and toilets.  Yes, the toilet is fully automatic, with no manual flush.  Of course, the renovations are occurring floor by floor meaning that for the people on the floor that is being renovated will need to go up or down a floor, which means those washrooms are rather busy.  So what if the automatic toilet does not flush?  And there are people waiting?  I can tell you that one of the toilets, (3rd floor, right stall), is batting a .500 average.

 

The first lesson for anyone becoming a software programmer is to become a designer first.  There is an excellent book called. "The Design of Everyday Things", written around the same time as Brads essay.  This book should be required reading for every designer, software, hardware, industrial, art or otherwise.  One story discusses the design of doors and the visual clues it presents to the user to determine does this door push forward or draw backward and if I need to turn a knob or push on a bar or what exactly to make it work.  Yes, something as simple as a door, we still cant get right.  Aesthetics win over functionality, almost every time cause thats what the consumer perceives s/he wants.  Me?  I just want the door to open like in old school Star Trek, complete with the cool sound.  Oh yeah, you will recognize the book as it has a picture of a red tea pot on the cover with the spout and the handle on the same side.  Nice design!

 

So what does this have to do with industrializing software?  Everything actually.  Until consumers demand better software, cheaper and faster, we will continue to handcraft solutions one source code line at a time that cant be measured for conformity to any users specification.  Eventually consumers will revolt or go elsewhere where someone else has figured out a better way to do it, just like when consumers got fed up with American cars, they went to Japan.

 

I am somewhat encouraged by the fact that Domain Specific Languages (DSLs) are making a resurgence as I believe this is the right path towards industrializing software development and meets Brad Coxs vision.  DSLs raise the level of abstraction to a point where a visual model specification is used to specify the implementation and can verify its correctness.  AutoCAD can be classified as a DSL as it allows non-draftsperson or non-engineer to still produce a drawing that is accurate enough for someone to implement it.  For example, from a catalog, I can order a drawing of a piston, put it in a CAD program and modify a dimension, save the electronic file output, take it to a milling shop and get my pistons manufactured, and yet I know nothing about the industry.  Now thats industrialization.

 

A Challenge to My Software Developer Peers

 

I challenge my software developer peers to help industrialize our software development industry.  Brad Cox has made a major contribution not only with the article he has written, which I reference here, but also with Objective-C System Building Environment that he invented.  My very small contribution to software industrialization is co-inventor of a DSL called Bridgewerx that allows a Business Analyst (not a programmer) to specify (AutoCAD like) an application integration specification that is then automatically implemented, without any programming knowledge required.  And before you crucify me for plugging a product I no longer have any financial interest in, I did it because I am personally sick and tired of writing the same source level code over and over again for the last 15 years.  I want higher level abstractions in better tools.  What are you doing to raise the level of abstraction in our software industry?

 

I leave you with the last paragraph that Brad Cox wrote 15 years ago in his article.  It is just as applicable today as it was then and probably still applicable when I am no longer around.

 

I only wish that I were as confident that the changes will come quickly or that we, the current software development community, will be the ones who make it happen.  Or will we stay busy at our terminals, filing away at software like gunsmiths at iron bars, and leave it to our consumers to find a solution that leaves us sitting there?