When I use the term software engineering, the definition I am referring to is:

“Software Engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software.” 

So what then is a Software Abstraction?  Take this paragraph from the awesome book, :

“Software is built on abstractions.  Pick the right ones, and programming will flow naturally from design, modules will have small and simple interfaces, and new functionality will more than likely fit in without extensive reorganization.  Pick the wrong ones, and programming will be a series of nasty surprises: interfaces will become baroque and clumsy as they are forced to accommodate unanticipated interactions, and even the simplest of changes will be hard to make.  No amount of refactoring, bar starting again from scratch, can rescue a system built on flawed concepts”.

Brilliant!  It’s the first paragraph of the 1^st chapter into a world of real software design - i.e. formal specification, including the ability to model and automatically analyze your design:

“Unlike theorem proving, this analysis is not “complete”: it examines only a finite space of cases.  But because of recent advances in constraint – solving technology, the space of cases examined is usually huge – billions of cases or more – and therefore offers a degree of coverage unattainable in testing.”

“Moreover, unlike testing, this analysis requires no test cases.  The user instead provides a property to be checked, which can usually be expressed succinctly as a single test case.  A kind of exploration therefore becomes possible that combines the incrementally and immediacy of extreme programming with the depth and clarity of formal specification.”

Check out Alloy Analyzer and the Community.

It’s slow going for me, but I am getting through the book.  Until you read this and run Alloy Analyzer, you cannot imagine the number of “cases” Alloy Analyzer can run, like the text says, a billion or more.  It picks apart your design like you have never seen before.  Imagine what the number of cases would be when you combine dozens of models to formulate a software application. 

This is a leap forward in software engineering design in my opinion.  Being able to analyze your design against a billion cases without writing a single line of production or test code is unheard of in mainstream software development industry today.  Using this design approach will produce software design's that will be more robust, perhaps by orders of magnitude, than any manual method we apply today.  The software design itself, is a complete formal specification used by the “end” software programming language to transform the complete design into a runtime solution.

What about the software engineering definition comment in the opening?  I am pointing out that we are following the definition of software engineering by using Alloy for designing software; it is one way to apply a systematic, disciplined and quantifiable approach to software design.  In my opinion, we as an industry, and as professionals, should be applying more effort towards software design.  Why?  Read the “Software is built on abstractions…” paragraph again.

Daniel Jackson and his team’s work on Alloy Analyzer, to me, is the state of the art in software design today.  When will it become mainstream?  Good question, but you can help.  As a Software Designer, have a look at Alloy.  Maybe apply it to a hobby project.  With enough practice, perhaps introduce formal design specification to the workplace on one small specific issue or algorithm you are working on.  That's my approach.

I know change is hard and everyone resists, but look at it as an experiment to try out, see how it works, and if it makes sense to you.  As a software engineering practitioner, I am trying to become a better software designer by learning new concepts and techniques.  I believe Daniel Jackson’s Alloy makes me a better software designer.

 

Goes to zoomii

 

 

In my last post, I asked the question, “Show Me a Good User Interface Design.”  I asked the question simply because of the length of time I have been in the software development business, sometimes you get so close to your work that you can’t see the forest for the trees.  And I was certainly feeling that way.  It is further exacerbated by the more you look at user interface designs the less objective you become – I needed a fresh perspective.

 

Thankfully, Breton commented on my blog about zoomii and as soon as I saw it, I knew I was back in business.  As I commented to Breton, zoomii reminded me of a user interface design I was involved with back in 1994 working for a company that developed banking applications using the Smalltalk programming language (oh, how I miss that programming language and environment, but that is another story).  We developed a visual (read: physical) representation of a banker’s desk (by banker’s role) where there were forms and files in desk drawers that could be pulled out, a calculator on the desk, calendar, etc., you get the idea.

 

What I remember the most about the user interface design was when we rolled it out to the bank employees and had a person sit down in front of the interface, she just started using it with no training!  The bank person said, “oh, I get it, it’s just like my desk”, and proceeded to pull out on of the drawers and flip through the file folders and clicked on a form to be filled out.  We were initially astonished by this as we had developed a training course to use the software, but the bank personnel did not need any training!  It was amazing, pretty soon everyone wanted to try it out and was using it immediately, with no help from us.

 

Now the paradigm is back and even better. And what is that paradigm?  I am sure there is a formal way of saying it that is probably in Jeff Raskin’s book, The Humane Interface, but for me it is dead simple, it is a model of the physical world designed and implemented in software as a zooming user interface.

 

Chris Thiessen, the one and only developer for zoomii, states on his web site, “…spending afternoons wandering the shelves. Happening across great books I didn't even know existed. But it's an experience I never found online.”  It is so true.  I used to frequent a computer bookstore in Calgary Alberta that carried only computer books on its shelves.  I would spend 2 to 4 hours on Saturday afternoons just browsing the shelves, picking up books and browsing through them and leaving the stores with a pile of books.  Over a few years, I had accumulated over 80 books this way.  Unfortunately, the bookstore succumbed to competition by the big box book store chains that moved into Calgary and is no longer.  I sure miss it!

 

But it is back in zoomii!  Let’s look at some of the user interface design elements.  First of all, if you click on Categories, you get this view:

 

 

Note that as you hover the mouse cursor over the left hand categories, the right hand representation of category on the book shelves is highlighted (white in this case).  Brilliant!  Very quickly, I could easily discern where the Computer book "shelves" were, so when I went back to the shelves, I could easily navigate (pan by holding the left mouse button down and zooming in and out using the mouse wheel) to the computer books and get the view below in a matter of seconds and minimal mouse clicks/movements.

 

 

Then I can zoom into a shelf that is sorted by the author’s last name:

 

 

Notice the little left and right arrows on either side of the Computers name category.  If you click on one of these, you navigate to the next shelf on the left or right.

 

When you find the book you want, you can click on it and get a detailed description of the book, plus the ability to look at what reviewers of the book had to say about it.  Essentially, for the reviews, it takes you to Amazon.com and if you read zoomii's frequently asked questions, Chris tells us that he uses Amazon’s Associates Web Services to interact with Amazon’s data and refers the sales to them and gets a cut using .  Brilliant business model.

 

 

Chris also states that the way he stocks his shelves is, by the top ranked 25,000 books.  He does state that if he used subcategories that if could to 100,000 or more. Chris is considering an optional view that will shelve Amazon’s entire book inventory.  I hope he does as I would be very interested in that – more books to browse!

 

I love the simple navigation of the site:

 

 

 

The basic navigation fits into the classic 7 +- 2 paradigm that our short term memory can handle, a far cry from the +60 clickable items navigation you get with "Office 2007 and the Killer Ribbon".

 

Clicking on the home button, produces this view:

 

 

Creating an account is dead simple, with no confusing user name versus email address.  Simple, yet complete – something we truly lack in our software industry.

 

 

For the techies, if you right click and view source in your browser, you are presented with a minimalistic file that contains mostly CSS and a bit of JavaScript and the barest of HTML.  If you look closely, you can see the JavaScript file Chris uses, and with a little know how, you can download his zoomii.js and pop it open in your favorite code editor to take a gander.  It is some 10,000 lines of JavaScript code, but Chris appears to use no 3^rd party AJAX frameworks, or Flex or Flash or Silverlight plug-ins, it is all his own work.  Truly amazing!

 

Another feature is the unlimited undo.  Clicking on the back button in the browser basically retraces all of the steps (zooms, clicks, everything!) that you performed from the time you entered the site.

 

The search capability is my only small quibble on the site.  It is not as exact as I would like, but it is not Chris’ design, but rather the Amazon web service returning too many books that seemingly don’t match the search criteria.  For example, when I type in, “Expert F#”, I get returned 29 matches, when there are only really 2 matches that should have shown up (ideally it should have been one match).

 

 

 

I am sure I am missing other features, and certainly will peruse Chris’ zommi.js more to understand how he put it all together, but for now I am enjoying myself more by simply using the interface then almost any other web based interface I have used or designed on my own.

 

As someone who develops ecommerce applications at work, I can see this user interface design making shopping at any “brick and mortar” store online a lot of fun and more so, exactly what the general public would want and enjoy, IMO.  I can just image browsing through the “shelves” at Bestbuy or Costco or FutureShop or…  I could see how it would be applied in a B2B supply chain relationship scenario as well for electronic parts or manufacturing for example.  It would seem the applicability is universal.

 

Kudos to you Chris for designing a top notch user interface that brings the joy back for someone that used to browse computer books for fun many years ago.  I am even more impressed that you did this all by yourself! 

 

Which really begs the question for our software development industry, are we all doing it wrong and Chris is doing it right?

“The required techniques of effective reasoning are pretty formal, but as long as programming is done by people that don’t master them, the software crisis will remain with us and will be considered an incurable disease.  And you know what incurable diseases do: they invite the quacks and charlatans in, who in this case take the form of Software Engineering Gurus.”

EWD1305. Answers to questions from students of Software Engineering, Edsger W. Dijkstra, 2000.

A very insightful, but somewhat harsh observation by Professor Doctor Dijkstra.  Also consider:

“No, I’m afraid that Computing Science has suffered from the popularity of the Internet.  It has attracted an increasing – not to say: overwhelming! – number of students with very little scientific inclination and in research has only strengthened the prevailing (and somewhat vulgar) obsession with speed and capacity.”  Again from EWD1305.

As an aside, Dijkstra, and as some of you may have heard of Dijkstra’s algorithm, made a number of fundamental contributions to the area of programming languages and received the Turing Award in 1972.  You can read his notes diary on line which makes for fascinating reading.

Why did I quote Dijkstra?  Well, I tend to agree with his view and as described in my previous post, I don’t think we, as Software Engineers, know how to perform Software Engineering.  In fact, I don’t even think Software Engineering really exists in our world today – and in some respects, we seem to be moving farther away from it instead of getting closer.  That is to say our predilection for programming languages blinds us to what our real focus as Software Engineers should be.

Let me be more succinct.  When I say Software Engineering, I am picking specifically on that huge black hole called software design – i.e. we don’t know how to “design” software.  We sure know how to program the heck out of it, with our humungous list of programming languages, but what techniques and tools do we have for “designing” the software to be programmed?  How do we model our design?  How do we prove (i.e. verify) our design is correct?  How do we simulate our design without coding it?  Ponder.

Designing software is all about designing “abstractions.”  Software is built on abstractions.  Programming languages are all about implementing abstractions.  But where do those abstractions come from and how do we describe or model (i.e. design) those abstractions?

Let’s look at some approaches to designing software “proper.”  Formal methods is an approach to software design.  From Wikipedia, “In computer science and software engineering, formal methods are mathematically-based techniques for the specification, development and verification of software and hardware systems.  The use of formal methods for software and hardware design is motivated by the expectation that, as in other engineering disciplines, performing appropriate mathematical analyses can contribute to the reliability and robustness of a design.  However, the high cost of using formal methods means that they are usually only used in the development of high-integrity systems, where safety or security is important.”

“Hoare logic (also known as Floyd–Hoare logic) is a formal system developed by the British computer scientist C. A. R. Hoare, and subsequently refined by Hoare and other researchers. It was published in Hoare's 1969 paper "An axiomatic basis for computer programming". The purpose of the system is to provide a set of logical rules in order to reason about the correctness of computer programs with the rigor of mathematical logic."  

Another approach is program derivation, “In computer science, program derivation is the derivation of a program from its specification, by mathematical means.”

“Model checking is the process of checking whether a given structure is a model of a given logical formula. The concept is general and applies to all kinds of logics and suitable structures. A simple model-checking problem is testing whether a given formula in the propositional logic is satisfied by a given structure.”  As an aside, you will note that the Unified Modeling Language (UML) is not on the list of model checkers.  Interestingly enough, any UML diagram cannot be checked (i.e. verified) for correctness, so what good is UML?  OK, I am being a bit facetious, but for the most part, I have found this similar to Dilbertisms – they are satirical truths.

One approach that is familiar to me is, “Design by Contract, DbC or Programming by Contract is an approach to designing computer software. It prescribes that software designers should define precise verifiable interface specifications for software components based upon the theory of abstract data types and the conceptual metaphor of a business contract."  The metaphor comes from business life, where a "client" and a "supplier" agree on a "contract”

However, while a lot of companies talk about Design by Contract, very few, at least in my experience, actually perform this and particularly at the level required for it to be beneficial.  Further, while it is a “clear metaphor to guide the design process” that in of itself makes it hard to simulate the design of a software system or “prove” (i.e. verify) the correctness of a software system, before writing any code.

So how do we design software that meets the criteria of modeling the software and simulate the running of the software to verify its correctness?  After much research, I came across one approach that seems to make the most sense to me, in which why I will explain shortly.   First, an introduction to Alloy and Alloy Analyzer from the Software Design Group at MIT. 

“Alloy is a structural modelling language based on first-order logic, for expressing complex structural constraints and behaviour. The Alloy Analyzer is a constraint solver that provides fully automatic simulation and checking.   Our philosophy is to couple lightweight design and specification notations with powerful tools.”

There is a book called, “Software Abstractions” by Daniel Jackson on Alloy and Alloy Analyzer.  You can also download a few chapters of the book.  In the preface chapter, I was particularly impressed with these two paragraphs,

 

“The experience of exploring a software model with an automatic analyzer is at once thrilling and humiliating. Most modellers have had the benefit of review by colleagues; it’s a sure way to find flaws and catch omissions. Few modellers, however, have had the experience of subjecting their models to continual, automatic review. Building a model incrementally with an analyzer, simulating and checking as you go along, is a very different experience from using pencil and paper alone. The first reaction tends to be amazement: modeling is much more fun when you get instant, visual feedback. When you simulate a partial model, you see examples immediately that suggest new constraints to be added.

 

Then the sense of humiliation sets in, as you discover that there’s almost nothing you can do right. What you write down doesn’t mean exactly what you think it means. And when it does, it doesn’t have the consequences you expected.  Automatic analysis tools are far more ruthless than human reviewers.  I now cringe at the thought of all the models I wrote (and even published) that were never analyzed, as I know how error- ridden they must be. Slowly but surely the tool teaches you to make fewer and fewer errors. Your sense of confidence in your modeling ability (and in your models!) grows.”

 

Let me step back in time for a moment to illustrate to you why these two paragraphs are of particular interest to me.  Back in the 80's, in the electronics field, I went through this type of “industrialization” first hand with electronics circuit design.  At first we drafted our circuit designs on real blueprint paper, we then built some sketchy prototypes (anyone remember breadboards?), designed our tests and implemented test harnesses (i.e. scopes, analyzers, generators, etc.) and tested the design “after” it was implemented.  Note that it may take a number of manual iterations or cycles to get the design right as well. 

 

I would say this pretty much sums up the same approach we use to design software today.  I.e. “sketch-up" some designs, start coding right away, spend an inordinate amount of time “refactoring” and developing test after test after test and then in the end, in some cases, figuring out that, guess what? all of the tests simply validate and verify that the software design is so flawed from the original specification (read: sketch-up) that we have to start over again.  Whether the process is called TDD, Agile, iterative, waterfall, whatever, in the end it really does not matter as the process itself is flawed because it completely misunderstands the role of software design and therefore the result can only be the sorry state of software in our industry.  But I digress.

 

Then the electronics design world was revolutionized when SPICE (Simulation Program with Integrated Circuit Emphasis) came along.  It ran on a computer that not only allowed you to design your electronic circuits, but also simulated the implemented design (i.e. an instance) where you hooked up your “virtual” test instruments, in software, and completely emulated the circuited deign without ever having to breadboard the circuit. I personally lived that era and is one of the main reasons, after spending 17 years designing and programming software since, have come to the humble realization, there must be a better way.

 

The industrialization part of SPICE meant that we were moving from a completely manual process, in which we could only verify the circuit design after the fact, to dramatically reducing the cycle time to design and verify the design of an electronic circuit without ever expending any tooling or material costs whatsoever.  Further, you could do numerous design iterations basically for free, plus apply thousands (billions!) of test cases to the simulated design that we would never be able to achieve manually.  This was modern day industrialization in real practice. We certainly can’t do design iterations in our software world for free today.

 

From Software Abstractions, I love the lines, “then the sense of humiliation sets in, as you discover that there’s almost nothing you can do right. What you write down doesn’t mean exactly what you think it means. And when it does, it doesn’t have the consequences you expected.  Automatic analysis tools are far more ruthless than human reviewers.”  

 

This is exactly what happened to us electronic circuit designers when we first started using SPICE.  We thought we really could design analog circuits and as it turned out, even some of our basic design assumptions were completely flawed.  We struggled at first to understand what the heck we were doing was so wrong.  Then after careful analysis, tuning and testing of the design models, we started seeing the errors or our ways and our designs became more exact, precise and tolerant of numerous error conditions.  It was truly a humbling experience.

 

And that’s the point of this post.  In my opinion, we in the software design community could use some humbling.  I am pretty sure that most of our designs, regardless of programming language used, are majorly flawed.  We, meaning so called Software Engineers, find this out after being in the field for several years, hacking (or is that tooling?) away in our favourite programming languages without any real verifiable proof that our design is the right one or even correct or would even work before we started coding. 

 

Maybe we are all in denial.  Maybe I am over generalizing, but having been designing and programming software for many years, I feel I need to reset and look at software design from a much more formal perspective, hence this intro to Alloy and Alloy Analyzer.  Both the technique and tools embody what I personally experienced in another industry that underwent nothing short of a revolution in the way how electronic circuits are designed (i.e. industrialization).  One could say today that the electronics design industry has been fully industrialized.

 

Software industrialization will occur one day; history will repeat itself, in the same way that it happened in the electronics design world (and other engineering disciplines) will also happen to the software design world.  In fact, it already is happening today.  Software design techniques and tools like Alloy and Alloy Analyzer are making it possible to design software and verify the design of the software before the software is actually implemented (i.e. coded).  And that is what I call the industrialization of software.

 

I ordered the Software Abstractions book, downloaded the tools and tutorials and will report back my findings sometime in the future.  Needless to say, this is the first time, in a long time, I have become excited again about being in the software development industry.