 Thursday, 28 June 2007
This is an open source software project that is highly experimental. In fact, it might be just plain crazy. Our industry is a buzz about Web 2.0, SOA, distributed computing, etc. Yet, what tools do we have to support distributed programming?
What I mean by distributed programming is the ability to program multiple remote computers (i.e. servers) from within one IDE instance, or as I call it, a DPE – Distributed Programming Environment:
This ASP.NET AJAX DPE consists of the following parts labeled in red:
1) An Object or Class Bowser, similar to Lutz Roeder's outstanding Reflector, except mine is laid out in the old school Smalltalk System Brower format. Where are the assemblies IronMath and System.Windows.Forms coming from?
2) This is a Remote Desktop Connection (RDC) that allows connections to multiple remote computers. One tab per connection.
3) This is a Windows Forms application (or service if you will) that runs on the remote computer(s) that allows you to add assemblies to be reflected. It uses Windows Communication Foundation (WCF) to communicate with the web server hosting the DPE, which then displays the reflected assembly information in the browser window GridViews. In addition, this Windows Forms application hosts IronPython (and soon the DLR) which also communicates with the web server via WCF so that IronPython code can be executed on the remote server(s).
4) An IronPython source code editor which provides syntax highlighting, handling multiple source code files, etc. Again, using WCF to communicate with the IronPython DLL that is running on the remote server(s).
5) Is the result of running the IronPython code that is displayed in the editor window.
6) To find remote computers, I use Virtual Earth’s map control to display remote computer locations. Optionally, a MapPoint Web Service can be used to upload custom locations and attributes.
7) Oh yeah, an interactive command line console is a must have.
The context of which remote computer I am programming is activated by which tab has been selected. Meaning that when I click on any tab, any related context, like other dependant tabs and grids are switched too. I add distributed computers by adding tabs for each RDC session, a source code editor, and an interactive interpreter (i.e. console) per remote computer.
Think of the web server as a middle-tier to the WCF enabled remote services that host’s the code for reflecting assemblies and executing Python code.
As I mentioned earlier, I will be embedding the DLR which will add support for more dynamic languages. Also note that the web-based console window is generic in the sense that it can be a console to a remote cmd window or PowerShell window or… Same goes for the source code editor, its syntax highlighting also supports multiple languages.
Like I say, it will either be useful or just plain crazy. That’s why I call it Global System Builder 
January 30, 2008 Update - Global System Builder is available for download at: http://www.codeplex.com/gsb/
Official web site: http://globalsystembuilder.com
Saturday, 23 June 2007
Previously, I discussed a web-based source code editor for IronPython, which is part of a larger application that I am working on. Most of the editor is complete, just working out the details on handling multiple source code files. Today we will drive IronPythons’ interactive interpreter, using a web-based console, which is also part of my application.
Using the console, we can whip up a simple Windows forms application in seconds:
And here is the output:
The console is based on two components, one is Sam Stephenson’s Prototype JavaScript framework and the console GUI is brought to you by Jeff Lindsay’s Joshua, which I modified to work with IronPython.
Running a web console over the internet presents a few challenges. Everywhere you read,
AJAX is the sh!t. I use a ton of it in the app I am building. A stands for Asynchronous and therefore, as some have stated, synchronous over HTTP is bad form mate. Well, in the case of a console application, I think a synchronous call is one (only?) way to make it “really” work:
//synchronous call to the server passing the IP string command and returning the response
function getFile(url, passData)
{
if (window.XMLHttpRequest)
{
AJAX = new XMLHttpRequest();
}
else
{
AJAX = new ActiveXObject("Microsoft.XMLHTTP");
}
if (AJAX)
{
AJAX.open("POST",url,false);
AJAX.setRequestHeader("Content-type", "text/xml");
AJAX.send(passData);
return AJAX.responseText;
}
else
{
return false;
}
}
One nit pick with the ASP.NET AJAX framework is that you can’t do synchronous. I would hope that the framework would support this option as a) other frameworks provide it and b) I want it man! The point of a framework is to provide options so that a variety of similar, but different business requirements can be met with “one” AJAX framework. I am already using 3 AJAX frameworks as each one brings a unique piece of functionality that I require. Who knows, maybe it will be more by the time I am finished.
If you look at the console closely, you will see one small GUI issue. When parsing interactive = true with Python, visually, the next line of Python code should reside beside the “…” and not on the next line beside the command prompt (i.e. >>>). Also note that the 4 spaces indent after the def statement does not work at the moment.
Aside from a couple of GUI issues, and a missing blinking cursor, there is one last feature to be implemented and then the console will be complete.
The astute reader may notice that the Windows application is running in a Terminal Services session, which is being hosted in my web browser. Did I remote into the web server? No. On the client computer? No. Where is the Windows application running?
January 30, 2008 Update - Global System Builder is available for download at: http://www.codeplex.com/gsb/
Official web site: http://globalsystembuilder.com
Friday, 15 June 2007
Why web-based? Well, the IronPython source code editor I am working on is one piece of a larger web-based, open source application that I am building and will become clear in future posts as to the reasons why it needs to be web based.
In the meantime, I thought I would write about some of the available components that provide real-time syntax highlighting and other features one might want in a web-based source code editor. After extensive research, there are 3 web-based source code editors that I would recommend.
The screen shot above comes from a portion of my web-application that I am building. I am using Christophe Dolivet fantastic EditArea. EditArea is written in JavaScript and really is quite outstanding given the number of features it has, particularly that it is free (released under LGPL license).
One of my main requirements is that it can work with ASP.NET
AJAX (a free framework) and can be embedded in a TabControl as you see in the screen shot. I will discuss the web-based IronPython interactive console in another post.
I am really impressed with what Christophe has done, not only is EditArea fully featured, but the JavaScript source code is extremely well written using prototypes. Further, the documentation is excellent – you would think this is commercial based source code editor. EditArea supports multiple languages, both the software itself and the syntax highlighting it supports, for example, in my case English and Python.
Another real-time syntax highlighting source code editor I have come across is CodePress by Fernando M.A.d.S. It is also written in JavaScript and covered by LGPL. I really liked this editor as well. While it is not as full featured as EditArea, it does a very good job. It also supports many languages, but not Python at the moment. I see it is on the todo list. While I could get it to run in a bare bones ASP.NET AJAX application, I could not get it to run embedded in a tab control. However, I think the limitation may be my JavaScript coding skills (I am a C# programmer hooked on Intellisense) and not necessarily CodePress itself.
Syntax Highlighter by Marijn Haverbeke is another web based, real time syntax highlighter. It is released under a BSD like license. It works quite well, but not as fully featured as the other two editors. However, it is quite fast and seamless. Dos not support Python syntax highlighting, but as in the case of CodePress, would be fairly easy to implement. What I found really interesting was Marijn’s story on how it was designed. Fascinating Captain!
Thanks to Christophe, Fernando and Marijn for creating and developing these real-time syntax highlighting source code editors. It has saved me a lot of time not only from building these from scratch, but also proves a theory I have about software industrialization. That is, it has been done before and I don’t need to reinvent the wheel. Some people call this mashups, others reusable software or open source, or… No matter what you call it, I am thankful that so many people are willing to share their hard work.
When will it be released? Not sure as there still is lots of work to be done and can only work on it part time. I wish it was my full-time job!
January 30, 2008 Update - Global System Builder is available for download at: http://www.codeplex.com/gsb/
Official web site: http://globalsystembuilder.com
Saturday, 09 June 2007
Years ago I lived in a Smalltalk world and never knew how good I had it. As Larry O’Brien says, “Smalltalk has enough proponents so you're probably at least aware that it's browser and persistent workspace are life-altering (if you aren't, check out James Robertson's series of screencasts).
Unfortunately, in the Great White North, not too many Smalltalk jobs were available and in fact it really boiled down to programming Windows or Unix. And somehow I found myself in the Windows world using Visual Studio and Visual Basic. In some respects, VB was almost Smalltalk like – I loved the interpreter of stepping through code, finding a mistake, fixing it on the spot, setting next statement to run a few lines back and stepping through and keep on going. I found myself enjoying the rhythm of it and how productive I was. That was the thrill for me.
Then came C# which is a statically typed language, but what really got me was the rote of write code, compile it, fix compile errors, then compile it again, fix runtime errors, compile it again, step through, note the error, stop debugging, edit code, compile it again, etc. As you can imagine from the VB world, the most common error I got was, “are you missing a cast?” Also, my productivity just was nowhere near as fast.
A year or so ago, I came across IronPython. What I loved immediately was the interactive console that interpreted code on the fly. I type in my line of code and execute it with immediate results. For me, I work best like this. It allows me to experiment with CLR types and once I have it figured, I can then cut n paste into a code file and build up my application.
What is really exciting to me is that the DLR is intended to support a variety of dynamic languages, such as IronRuby by John Lam and others… yes, even Smalltalk. Way to go Peter Fisk!
Wednesday, 28 March 2007
In 1980 I worked for an advanced R&D electronics company that produced datasets and switches from initial paper napkin design, to engineering prototypes, to full on production manufacturing. One of my roles was to design the printed circuit board layout, commonly referred to in the biz as the artwork. I did this by laying down various decals on a transparent Mylar sheet that represented where the electronic components would go and then I would lay down traces (red and blue tape for a double-sided board) which represents the electrical connection between components.
I used an electronic circuit schematic diagram to read the electrical connections and mark off each trace that I put down on the layout design (i.e. artwork). The scale was 4 times the size of the physical printed circuit board that the design would eventually become copper traces and the physical components would get plugged into the board and soldered.
From a process perspective, I had to shoot a negative (or positive) of the finished Mylar printed circuit board layout design that was reduced by 4 times. This positive was then physically laid on a flat solid layer copper printed circuit board template that was photosensitive, so that under a UV light, the board was exposed and then the board had the photographic imprint of the design on it. The exposed board is submerged into a caustic chemical soup that would etch away the copper on the board except where it was exposed by the positive. I end up with a printed circuit board as a replica of what was on the positive design, Mylar layout design and ultimately the circuit diagram, with no loss in translation. Next hand-drill the holes in the printed circuit board and load it up with all of the electronic components, which I soldered in. Finally, mounting the board in its case and voila, a finished prototype ready for testing.
The printed circuit board layouts I designed had some 200 electronic components including a microprocessor, many ICs and several analog components plus hundreds of traces. It took about 8 weeks of effort times 8 hour days to actually layout a circuit design of this size and complexity. Then it took a few days to get the positive shot, another week or so to get the board etched and another week or so before you had the board masked, drilled, parts loaded, soldered and it was in the test rack in the lab.
In 1980, this was a manual labor intensive process, working at a light table for almost 2 months straight. Add another month to finish prototype. Kinda reminds me of the process of developing modern day software prototypes.
And then in 1982, a specialized computer arrived at my desk with a Computer Aided Design (CAD) program for designing printed circuit board layouts. It did not replace my job. Instead I became a computer operator that required specialized domain knowledge (i.e. printed circuit board layout design). It took a while to figure it all out, but it did reduce my manual effort from 8 weeks down to a couple of days. This was a significant productivity increase that I gladly welcomed.
You programmed the CAD application by entering in your electronic components (mported from a Bill of Materials list with some metadata) and then drawing the traces (actually using a light pen to touch one component lead to another on a graphical display) and the computer would figure out the width of the trace based on how much current was required (if you loaded Bill of Materials metadata) and auto-routed the trace in the most efficient way by trying hundreds of combinations and rearranging other traces, each time a trace sequence was inputted
What does this mean? It means that the level of abstraction for solving this particular problem was raised significantly with the introduction of a highly specialized CAD program. Measuring my involvement with this technology using the Technology Adoption Curve, I would be classified as an early adopter of this innovative and disruptive technology according to Geoffrey Moore. This technology approach radically changed the way printed circuit board layouts were designed thereafter.
Today, some 25 years later, printed circuit boards are still laid out requiring humans to develop the design, but the level of abstraction and automation is unbelievable. One only has to investigate the ASIC world to see how industrialized this industry has truly become. We have abstracted this specialized process so much that major chunks of designs are reused and assembled from massive libraries built upon standards. This is how mature this industry is. Back to the printed circuit board design, the standards based CAD output file is directly interpreted by a Computer Aided Manufacturing (CAM) device that can output a completely etched, masked, drilled printed circuit board with parts placed by robotics and wave soldered, all in a matter of minutes.
At that time in the 80s I saw various parts of the electronics engineering design and manufacturing world go through a small industrial revolution with these highly specialized CAD CAM technologies that have now evolved to the point where DVD players are $30.00. Who would have guessed? It was not that long ago that an HP-41c calculator was $600.
Following opportunities in the electronics design industry, I got into the software development world in 1988. I was exposed to several programming languages. One that always made the most sense to me from an efficiency point of view was Smalltalk. Its entire syntax could be captured in a one page document. Whereas C just made my head hurt, as did assembler I just dont think that way. One Smalltalk app was developed for employees at banking institutions where the user interface modeled the physical objects on and in a bankers desk. Aside from calculators on the desk, the pull-out drawers that contained various folders and forms cold be selected by the mouse, drag the drawer open, select a folder and pull up a form ready to be filled in on your computer screen. The graphics were not that slick in those days, but it really looked like a physical replica of the bankers desk and all of the items it contained.
What totally got me was how the user interface design was received when we first tried it on a real banker. I will never forget, the person sat down and just started to use the software because it really did model his desk, where all the right documents were in the right folders, etc. He thought it was the coolest thing in the world. I thought I could not believe that this person started using the software with no training he just sat down and went to work. Really says something about how user interfaces should be presented for people doesnt it. Hint: think domain specific.
I got into regular Windows programming for both the Mac and Windows (starting with VB1). Man, this was much harder than the Smalltalk programming I did. As a side note, when researching Smalltalk years later, I found out that it was a programming language designed for kids, I guess thats why I really liked it  Back to our regular programming, and did I say how hard it was? I would spend hours trying to figure out how to make something, anything, work. It was a maximally manual labor intensive process, almost felt like I was back in my electronics design world hand laying out printed circuit board designs, trace by trace. I was stepping back in time from an industrial evolution point of view.
Flash forward 15 years, now I am a so called Architect in the professional software development world. Over 15 years, I have worked virtually every position one could have in the software development industry, both in product companies and professional services, including being the President of my own software development company for 4 years with a staff of up to 25 people.
While I think the level of abstraction has risen with tools and techniques in the software design/development world over the last 15 years, it sure has not made the productivity leap from 8 weeks down to 2 days that I experienced with the introduction of a specialized CAD tool in the electronics engineering world. Specialized being the key word.
I am not going to get into analogy arguments of comparing the detailed process and risks in designing software versus designing electronics, but I would suggest that natural evolution of tools and techniques for designing software will follow a similar path to what has already occurred (25 years ago) in the electronics engineering design world. Over the last 15 years, I have seen steady progress in raising the level of abstraction in our software technologies (e.g. framework class libraries and runtime execution environments), tools (e.g. sophisticated designers and programming IDEs) and processes ( e.g. Agile.).
Consider this, that all of the tools and applications discussed above could be considered Domain Specific Languages in our software development world of techno-speak. The key phrase being domain specific or specialized. I know there is some history with CASE tools of not having much success, but one probable downfall was that they were not domain specific (or specialized) enough. CASE tools tried to cover the entire SLDC. In the electronics worlds, I used a separate circuit diagram CAD tool that had a very specific palette of electronic circuit symbol objects and the only thing the tool was designed for was drawing electronic circuits. I used a different domain specific tool to layout the printed circuit board, but note that through standards, the file format from the circuit diagram could be read, along with a bill of materials, into the printed circuit diagram layout design tool, and then again input to a CAM tool that automated the production of the finished product.
I see a similar pattern occurring in the software design world with the introduction of Software Factories. This is why this is very encouraging news to me. Jack Greenfield and crew are advancing the state of the art in software development techniques and tools akin to what John Walker introduced with AutoCAD in 1982 to the engineering design and manufacturing world.
I believe we are at the beginning of an industrial revolution in the software design/development industry. No, I am not talking about robots coding software. I am talking about humans using domain specific, visual design tools to design software in a specialized CAD environments (remember my electronics example above) and have its standard output interpreted by code generators to produce the runtime executable. Code generation is similar to the automated manufacturing process, except we call it an automated build process. Note that the manufacturing process in building software costs as little as selecting the build command from the menu in Visual Studio. All of the effort is in the design, i.e. hand writing source code.
As mentioned above, we already have that today in other industrialized industries. Computer Aided Design software, allows designers to draw and specify in full detail (usually a model that is 100% complete) the design, lets use a real example, of a piston for a car. The drawing output is saved to a standard file format (i.e. DXF, DWG), which can be interpreted by many devices to produce the physical output of the design. For example, a Computer Numeric Control (CNC) milling machine can interpret the design file as a loaded program of thousands of sequential Cartesian coordinates, and execute the program on a solid block of aluminum. The output? An aluminum piston milled to one thousands of an inch tolerance or better that exactly matches the design, with no loss in translation. Fully industrialized and, ironically, driven by software.
I mean ironically in the way that software has industrialized other industries (e.g. electronics design and manufacturing) but our industry in designing the software has not yet been industrialized. We still hand craft solutions by hand, in source code editors, one line at a time and have not seemingly benefited from the level of automation our software provides other industries, such as Product Line Engineering in the electronics design world.
However, we are closing the gap with recent innovations in the art of software design with the introduction of Software Factories. When you think of a Software Factory, think of the end to end process, from initial design to code generating the solution. Most people think the Factory analogy applies to just manufacturing, but completely forget about the design side, which is one of the reasons, that I believe, most of our software design tools are still code (i.e. text) editors
In Microsoft techno-speak, a Software Factory embodies the AutoCAD process of using a specialized CAD like tool to fully design (i.e. model) the software problem domain in 100% fidelity, meaning nothing lost in translation. The standard file format the design is saved in, like the DXF mentioned above, can be exchanged with many other software programs that can interpret the file format. One example is to reconstruct the model in another design time tool with no loss of fidelity. Or, have an interpreter that can read the file format to assemble and configure a multi-project Visual Studio solution in which your interpreter can code generate and compile the solution into a runtime executable.
In the Microsoft speak, the AutoCAD like tool is called a Domain Specific Language (DSL) and in the Visual Studio SDK there is a toolkit (DSL Toolkit) provided for designing DSLs. The toolkit is built around the Visual Studio SDK environment that allows you to design and build your own visual domain specific language. Once designed, you use the supplied code generation framework to produce a runtime version of your specialized Application Designer. This generated Application Designer (hosted in the Visual Studio shell) is designed for a targeted user (i.e. a Business or Systems Analyst) to model and code generate an application that is specific to the problem domain the DSL was designed for.
The Business Analyst uses the runtime Application Designer to draw the solution using the domain specific language, and through the use of property sheets, configure all of the drawn objects for a particular solution problem domain (e.g. a CRM application). The output is saved in a known file format which populates source and component library artifacts of a multi-project solution (using the Visual Studio shell the designer is running in) and builds the output as a runtime executable of the (domain specific) designed application. This entire scenario is called a Software Factory. Need to make changes to the runtime design? The BA changes the drawing, regenerate and done. Over and over again. Need to make changes to the domain specific language itself? Same process, except the programmer changes the DSL drawing design that describes the end-user designer application and recompiles the design tool. Over and over again. Note the capabilities for predictability and repeatability, which are the cornerstones of an industrialized process.
Software Factories are at the innovation stage of the Technology Adoption Curve with 100,000 early adopters. As mentioned before, I am really excited by this as this is the type of industrialization I saw in the electronics engineering design world starting some 25 years ago. I wonder if history is about to repeat itself?
Barry Varga and I could be called innovators when we developed a Software Factory for BizTalk Server, called Bridgewerx. It works using the same principles and concepts that are described in the Software Factories book written in 2004 and behaves much in the same way as the recently downloadable 4 software factories do. The exception is that we code generate the entire Visual Studio solution as our visual integration designer is a highly specialized DSL for completely describing application to application integration solutions built on top of a middleware product (i.e. BizTalk).
If you are someone just getting involved in the software development world or a seasoned pro, you owe to yourself, at the very least from an educational perspective, to evaluate what Software Factories means for you and the future of our industry.
Monday, 01 January 2007
After a 6 month hiatus from the blogosphere, I am back. Well, almost 6 months. On July 17th, 2006 I proclaimed that after 80 posts on the topic of software industrialization, I did not have much more to say about it. Today I do.
Before I dive into the post about what I think the best new programming language released in 2006 was, (the language isnt new, but the implementation is), I want to take a slight detour. I have a confession to make.
The confession is that I fell into the pervading consciousness of some in our software development industry where cynicism, negativity and ego rule. One reader of my blog, (bless you Martin Danner) said that my recent posts conveyed a sense of frustration, cynicism and downright hopelessness. Martin told me to cheer up, all is not lost and in fact the seeds of change have sprouted. Thank you Martin, you were right. I was spending way too much time reading posts from: Mini-Microsoft, Joel Spolsky, Steve Yegge, Jeff Atwood, Rands in Repose, TheDailyWTF, etc. Regular readers of these authors know what I mean.
Now dont get me wrong here. I have a lot of respect for these very smart people and mostly what they write is well written, entertaining, and even thought provoking. However, there are times where each of them have also succumbed (in some cases, more often than not) to the cynical world of Dilbert software development where everything is a rant or complaint or WTF! You would think you are reading Dennis Miller Rants.
I will be the first to admit I have done my fair share of that in the past and on this blog. For those of you that knew me when I ran my own software company, I could be the darkest, most cynical you-know-what in the world. So whats changed? I realized that after 15 years in this biz I could just be yet another cynical puke or I could do something positive for myself and the software industry I work in - like contribute an open source project of some sort that would make the process of designing software more predictable and repeatable than it is today (i.e. the industrialization of software). I started that quest over 6 months ago and may take a year or two to finish. But thats another post and not my point.
The point of this digression is that I am a bit disappointed in the Joel Spolskys, Steve Yegges and the others I mention above, of the world. I am disappointed in that they could really influence the positive direction of our software development industry, rather than b*tching and stitching, in a well written manner about how everything sucks. I know some of it is not to be taken seriously, but at the same time people look up to these industry thought leaders. How is this constant barrage of negativity going to help our industry?
Do you understand what I mean? I try not to take myself or my work too seriously, but I do. I cant help it. And neither can most of the programmer types I have worked with over my software career. Its simply who we are and it is interesting to find other like minded people have chosen this profession.
So what does that have to do with the best new programming language released in 2006? A lot actually. I find it an uplifting story where Jim Hugunin, 3 years ago, started to think about ranting on why the .NET CLR would be a terrible platform for dynamic languages. Three years later on September 5th, 2006 an open source programming language from Microsoft called IronPython was released. Having played with IronPython for six months, it gets my vote for easily being the best new programming language released in 2006.
Why? My criteria is very simple. What is the simplest way, or to phrase it differently, the most productive way to write a windows program? (e.g. ASP.NET web form, WinForms, or XAML WPF). Here is an example:
Note while the least number of code lines count to a certain degree, it is the simplicity that matters the most. What could be simpler than opening an interactive console window and typing in, line by line, almost English sentences while getting immediate feedback after executing each line. Either the line you executed succeeded or failed. I love when decisions are binary. In fact, by line 3 in the example above, you already have a real live active WPF window being displayed on the screen, so now it becomes mostly a visual experience. Who needs testing? The testing is built right in, in real time either the window displayed or it did not. I know I am being simplistic, but perhaps our programming world could do with more simplicity. I have over 10 years invested in learning the Visual Studio IDE and even with James Averys most excellent book, Visual Studio Hacks, I still only know maybe 60% of the complex features of VS2005.
The reason I said most productive is because I believe that the code, build, run, and debug loop of most statically typed programming languages are a real barrier to productivity. I love C#. I use it in my day job. It also frustrates me because I am locked into the code, build, run, and debug cycle. I just dont work that way. I like the exploratory iterative approach with immediate feedback as a way to design software.
James Schementi has written a nice article on IronPython called, CLR Inside Out. Jeff Cogswell has written another nice article called, "A First Look at IronPython: Where Python meets .NET. Both of these authors do a much better job than I explaining the productivity virtues of IronPython.
IronPython is free. The source code is also free and both the object and source can be freely distributed in your own application, which is exactly what I am planning to do with my own open source software design tool. Stay tuned.
Monday, 17 July 2006
Well, to answer the question, I wrote 80 mini articles in just over a year that comprises this blog. However, if you want to read just one article on the subject, may I suggest, Planning the Software Industrial Revolution 15 Years Later.
My blog has been a year long journey of discovery which has led me to a conclusion in the field of software industrialization. This conclusion is now my main pursuit.
Therefore, no more blogging and frankly, after 80 posts on the subject area, I dont have anything more to say, or at least for a while anyway.
The one thing I do have to say is that I wish my blogging software had the blog titles in the Archives list instead of the month/year date labels, including the number of posts beside each month. Given the subject matter, I think this rather ironic. To overcome this, I manually cut n pasted the titles into this post (newest first, in descending order) as a human readable index to the articles.
So long, and thanks for all the fish.
Dynamic Programming using IronPython Part 5
Dynamic Programming using IronPython Part 4
Dynamic Programming using IronPython Part 3
Dynamic Programming using IronPython Part 2
Dynamic Programming using IronPython Part 1
Software Reuse: Enigma or SOP?
Software Industrialization using .NET Reflector and IronPython
A Software Design Tool idea using IronPython
Got any CAD Tools for Designing Software?
Planning the Software Industrial Revolution 15 Years Later
Is IT a commodity or are we still in a Software Crisis?
The Joy of Software or Why dont they get it? Part 3
Our Software Industry is a Sham(bles)
The Joy of Software or Why dont they get it? Part 2
Mini-Microsofts blog explodes on Vista delay
The Bloody Edge of being a Beta Tester
The Joy of Software or Why dont they get it? Part 1
Vista 5308, WPF amd XAML way cool but got the Cider and Sparkle blues
Software Developments nemesis variability
The people you meet in the software biz
Whats new in software development for 2006? Part 10 Finish
Whats new in software development for 2006? Part 9
Whats new in software development for 2006? Part 8
Whats new in software development for 2006? Part 7
Whats new in software development for 2006? Part 6
Why Software Industrialization? Part III
Whats new in software development for 2006? Part 5
Whats new in software development for 2006? Part 4
Whats new in software development for 2006? Part 3
Whats new in software development for 2006? Part 2
Whats new in software development for 2006? Part 1
Top Software Failure for 2005 or The Future of Software
The Evolution of Software Industrialization Part 7 Conclusion
The Evolution of Software Industrialization Part 6
The Evolution of Software Industrialization Part 5
The Evolution of Software Industrialization Part 4
The Evolution of Software Industrialization Part 3
The Evolution of Software Industrialization Part 2
The Evolution of Software Industrialization Part 1 Introduction
Marketecture or why software vendors are not helping industrialize our industry
Hey Software Programmer Stop Coding!
Raising the level of abstraction Part II Programming with Models
Book review Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools
SOA details Contract-First Development
Service Oriented Architecture (SOA) the Truth
Code is Model Damn Right!
BRIDEGWERX A Software Factory approach for generating Application Integrations
lesscode and more design
lesscode using Domain-Specific Languages (DSL)
Leadership in the Software World
Future Directions in Modeling Tools
SOA this, SOA that, SOA what!!
Why Software Industrialization? Part II
Dynamic Interrogation SOA magic!
An on-demand application generator for producing application integrations
On-Demand Application Generation exposed
Size does matter
Generating On-Demand Workflow Applications in SharePoint
Sideways or Groundhog Day or both?
What is a Software Factory?
More on code generation
What is code generation?
Bringing a software invention into the business world
A standard for application integration has been invented
Raising the level of abstraction
Software Industrialization Arrives
Stupid Computer Tricks got your secret decoder ring?
Size and complexity does matter
The sorry state of our software industry is there hope?
What is an IT Architect anyway?
So you wanna be a certified IT Architect do ya?
Doesnt ANYONE want to be a programmer?
BRIDGEWERX a full fidelity software modeling tool ala AutoCAD style
AutoCAD for software development? Introducing model driven development
The Dynamics of Software Development
What weve got here is a failure to communicate
Software Engineering, Not Computer Science Part 2
Why Software Industrialization?
Software Engineering, Not Computer Science
Tuesday, 04 July 2006
Have you seen the IronPython integration project in the June VSSDK? Aaron Marten and Team have done an outstanding job integrating IronPython into the Visual Studio IDE. I have lived in Visual Studio since 1997, and can say that the IronPython integration is totally pro.
In the VSSDK path C:\Program Files\Visual Studio 2005 SDK\2006.06\VisualStudioIntegration\Samples\IronPythonIntegration you can find the IronPython.sln solution file for the IronPython Project System which features include:
Defines a VSPackage that controls the interaction between Visual Studio and the IronPython compiler.
Participates in building, deployment, and source code control.
Includes MSBuild targets and templates.
Offers debugging support, but no expression evaluator.
Displays and manages project items in the Solution Explorer.
Participates in the Open Project and New Project dialog boxes.
Exposes project and configuration properties, both generic and specific to IronPython, such as the "main" program file.
References web services, databases, and other resources, but not other IronPython assemblies.
Supports automation.
Connects the Windows Form Designer to the IronPython CodeDom.
Press F5 to build the sample, register it in the experimental hive, and start Visual Studio from the experimental hive.
The last statement is interesting. For anyone that has used the pioneering DSL Toolkit knows that running this solution will launch another instance of Visual Studio, and in this case, complete with the integrated IronPython project system now running. If you click on New Project, the following dialog appears:
And sure enough, the PythonPoject is fully integrated into this instance of Visual Studio 2005. It works just like a C# Project Type, for example. In other words it is seamless and consistent within the Visual Studio IDE paradigm. A real testament to Visual Studios pluggable language architecture.
I developed a small IronPython Windows app to try it out. I wanted to see how the Win Forms Designer generated the IronPython code and what it looked like compared to C# Windows Forms apps. So I dragged a ListBox onto the Design surface. Yes, all of the WinForm Designer and toolbox are there. Then in the Form code, I called System.Reflection to get the Types from Mapack.dll and then add the Types to listBox1. Currently there is no support for Partial classes so the Designer code and your Form code are in the same file. One way around this is intelligent use of VS #regions to package code bundles in the same source file.
Btw, the Mapack.dll is an excellent example of the type of reusable Assembly I am talking about with respect to software industrialization. Here is an encapsulated package of Math functions that you can just plug-in to your application. A good abstraction and packaged at the right level in my mind.
My only complaint is that the Mapack.xml code documentation file did not come with the DLL. I wish the VS IDE came with a compile option to include source code documentation in the Assembly. As I have mentioned elsewhere, I think having the source code documentation in a separate external file breaks an encapsulation principle in my opinion. And the first thing to get lost, yes? Plus if I want to program Assemblies instead of source code files, then it would be nice to be able to retrieve the documentation through System.Reflection as it seems you can do everything else with this namespace.
Note the IronPython Visual Studio project is the bottom most Application on the screen. When that is run, another instance of Visual Studio is created with the IronPython project integrated into the running instance. Then I opened up a IronPython windows application project called, Browser1. Running that project launches the small windows app with listBox1 populated with the Mapack.dll Types. You can see the IronPython code in the Browser1 project to the left of the Windows app and on the far right, you can see the 5 projects that make up the IronPython VS integration project.
Another cool integration point is the IronPython interactive console that is directly integrated into the Visual Studio environment on one of the tabs:
The console window is a tool window that creates a TextBuffer object and hosts a TextView control that is initialized to use the text buffer. The interaction between the console window and the IronPython engine is handled by the implementation of the IConsole interface and the definition of the streams used by the engine as standard output and standard error. The history is implemented using a class called HistoryBuffer that stores the list of commands executed inside the console, sorted by execution order. Statement completion and colorization are implemented using the same IronPython language service that is used inside the standard editor when editing a py file. The different behavior of the language service in the editor window and console window is implemented using one functionality exposed by the IronPython language service through its version of the Source class. This object, defined in the MPF, is an abstraction on top of the text buffer and text view.
Wow, when I read that I was expecting a super deluxe console window and it is. Now thats an interactive console window!
As you browse the source files in the PythonProject you will also notice several XML Overview files, Class Diagrams and test suite metrics for each project. Nice.
If you are a Visual Studio code jockey, do yourself a favor and have a look at the Visual Studio IronPython integration project, its as pro as it gets in our software development world. Kudos to Aaron and team!
If you want to tune in to the IronPython world, I would recommend that you visit CodePlex to get IronPython and sign up to the Listserver. Dino and crew are extremely helpful in answering any questions that are posted. The IronPython community is vibrant one with daily action.
Finally, if you are a statically typed programmer feeling a bit shackled by a universe that the compiler creates, unshackle yourself and enter a universe that is still dynamic after compile time. You owe it to yourself to check out the IronPython integration project in the VSSDK.
Monday, 26 June 2006
There are approximately 28,600 Types in the .NET 2.0 Framework Class Library (FCL). I am not sure how many more are in WinFX, now called .NET 3.0. Even 28,000 Types is a staggering number to learn a FCL. There is a very good article that explains the 9 most common Types to get started. As the article summary states,
The .NET Framework Class Library and the common language runtime (CLR) serve as the foundation for all .NET-based applications. But how much do you know about any of the thousands of CLR classes in the library and where do you begin to learn about them?
In this article, the author uses the number of relationships with other types to determine which types are going to affect your programming most frequently and how often you'll encounter them. He programmatically surveys the CLR library using reflection to make that determination. Following that, an in-depth examination of the nine most important CLR types is provided.
I like this article as it shows a pragmatic way of understanding the .NET FCL. In addition, using Lutz Roeders Reflector, you can literally inspect, disassemble and decompile any Assembly, (.NET FCL or other assemblies), to your favorite .NET language. Do you believe that source code = design? Even though the article is 14 years old, it is one of the most decisive articles on software design I have ever seen. It also most closely resembles my belief the source code listings are the design documents.
Since the FCL and runtime (application execution) environment are packaged together, you basically have a programmable runtime environment. Using a dynamic language, like say, IronPython, and through an interactive console, I can program the FCL and have it execute as I am programming line for line.
As each line is entered in the IronPython interactive console it is being dynamically executed by the IronPython engine (interpreter). In some lines of code, I have built a trivial Windows Presentation Foundation application borrowed from the tutorial that comes with the IronPython download.
As trivial as it is, there is an important concept being demonstrated here which is with IronPython embedded in your .NET application or in Visual Studio (i.e. using the VSSDK) or using the standalone interactive console, you can dynamically program the execution environment so to speak. A programmable runtime environment.
I have always been a fan of software reuse, mainly because I am lazy. Also, I know it works. I make a living at it. In my world of programming business applications, the reality is that most if not all of it has already been coded several times over. As seasoned programmer once said to me, its all been done before. When was the word processor invented? How many word processors have we had since? Whats different? Probably the same thought goes for spreadsheets, workflow - paper or otherwise, invoicing, ecommerce, etc. Most of these business problems have already been solved with software many (or a million) times over. My question is, how can I reuse this incredible wealth of software that has already been designed and built? Thats my thesis to prove.
I can reuse software much like the way I am practicing today, finding components of various sizes and shapes that loosely or exactly match a functionally decomposed problem I need to solve. For example, my System Browser needed a way to parse an Assemblys XML document file to retrieve code comments on Types and Members when reflecting on an Assembly. Surely that has been written before is what I am thinking. And with a little bit of searching, yes it has, to the point where after I have included the example class (or Assembly) in my System Browser application, I can integrate it with one line of code.
This means I did not have to modify the code at all, I could just use the public interface and problem solved. Next component. And that is how I am building my System Browser application. The 80/20 rule is in effect. 80% reuse and 20% net new code, distributed between coding new functionality, modifications to existing components, and glue code to stitch all of the pieces together.
So what am I saying? I am saying that there is a lot of code out there in the world to reuse. I and others have been doing it for years. Sometimes known in our practice as clip-board inheritance. But, how can I pragmatically reuse pre-built code? While having the source is nice, it does not have to be a requirement. In fact, using System.Reflection will let you look at practically anything inside the Assemblies black box. And I can load any black box (i.e. Assembly, COM Object) and not only inspect it, but use it dynamically in my application using an embedded IronPython engine. Thats what the System Browser is all about. A tool to dynamically program the runtime environment.
The System Browser application is designed to be an emulation of Smalltalks programming environment. It has a dynamic/interpreted programming language integrated with a large class library and an application execution environment. Essentially a virtual machine.
Rather than Smalltalk, and the Smalltalks class library, I am using IronPython and the .NET 2.0 FCL. The CLR on a Windows OS takes care of the application execution environment. Through a similar GUI like Smalltalks System Browser, I can use System.Reflection to get access to the Type system. Using IronPython I can dynamically create new classes and subclass the FCL at will.
Not only can I browse and use in real-time the FCL, I can do that with any other .NET assembly or COM object. I wonder how many .NET assemblies are compiled everyday world-wide? How many COM objects over the years have been compiled? I must have produced over 100 DLLs by now after 15 years at it. I am one of approximately 12 million programmers worldwide. According to IDC, the number of professional developers worldwide will increase to 13.3 million by 2006 from 7.8 million in 2001.
Consider that there are 200 billion lines of COBOL in use today with billions of lines being written each year. What does all of this code do? Oh yeah, there is NETCobol that runs on the CLR.
Theoretically speaking, even if each developer compiled the equivalent of 1 DLL a year over the last 10 years, times 10 million developers, thats tens of millions of DLLs that have been produced. Even if it is a magnitude more or less, its still an incredible amount of potentially reusable DLLs. Again, how to use them?
I modeled my System Browser after Smalltalks System Browser. Why? Smalltalk and its environment was best suited for me where I can browse classes, instantiate them, inspect other instances, manipulate them, all in a simple to use dynamic environment. At least for me, that was the pinnacle of an OO development environment, along with Brad Coxs Objective C System Browser. Now I can have the same thing but different. Maybe it is just the way I feel most comfortable navigating and manipulating a FCL.
Another reason for this approach is assisting in the industrialization of software development. I want to program at a higher level of abstraction. To me, that means heavy reuse of pre-designed and pre-built, components is critical to raising this level of programming abstraction. Why write source when the compiled unit is already there?
In the many pages of this blog, you will have read statements like, from a software development point of view, given whatever software application or component you have been tasked to build, chances are that it has already been done before. Look it up on the internet. In fact, this System Browser application is being constructed using this reuse pattern.
Once I have found an existing Assembly or COM component or source code (that will be compiled into an Assembly) that meets 80% (or best choice) of my requirement, I download the assembly and import it into my image or System Browser which means it just became part of the programmable runtime environment.
Now I can reflect on the assembly and all of its Public Types and Members. If there is an associated XML code documentation file, then that is parsed and displayed each time I click on a Type or Member. If there isnt a corresponding document XML file, then I code generate on the fly an XML code documentation file, using a standard template, so that a sentence is formed for each Type and Member as basic (but editable) code documentation. It is already generated and reflected back into the System Browser before you can click on anything.
Using the embedded IronPython inside the System Browser, as I am entering code in the console window (interactive mode) and/or code editor (file mode), both are being recorded then compiled and added to System Browser on the fly. This is done in real-time and totally seamless. At least that is one of the goals of my System Browser.
I would not take too much of this too seriously. This is more of an experiment for me to learn from. It is for my research on software industrialization. However, the System Browser will be freely available just like the IronPython interpreter embedded in it. You will be able to reference assemblies and COM objects, browse their public Types, Members, code documentation to learn from and also be able to dynamically use those Types in your applications. Or at the very least experiment with the Types. I just like programming at a higher level and leveraging perfectly good code that was built before me. Based on my research and own experiences, there sure is a lot of pre-built code out in the world to be reused in your applications, or at least in the business applications I have been tasked to develop at my day job.
This last post was my design rationale or thesis on its applicability to assisting software industrialization by promoting Assembly reuse. A higher level programming abstraction. Something I think our software industry could use.
Wednesday, 14 June 2006
I have made some progress developing a Smalltalk like System Browser for reflecting .NET Types with an IronPython embedded console interpreter to manipulate the reflected Types. Not as much progress as I would like, but with a few hours a week, what can you do.
You can reuse code to increase productivity. Here is something that seems a relatively simple task to do, I want to add an icon (bitmap actually) to each item in my list boxes that represent assemblies, types, properties, methods and events. It turned out not to be so simple as a .NET listbox does not have this capability out of the box. Now what? As I have suggested in the past, the WWW is an excellent (unstructured) catalog of reusable software. I make the assumption that each one of the components I am looking for to build this application has already been built by someone.
Searching away brought up several specialized listbox implementations, but this one I liked the best is called. .NET Color Listbox. Alex has done a great job here as his implementation provided not only the ability to add the bitmaps to each item in the listbox, but also uses DoubleBuffering to get rid of the flickers when scrolling looks nice and smooth.
Where to get the bitmaps that represents assemblies, classes, methods, etc.? If you have Visual Studio 2005 installed, you will find them at: C:\Program Files\Microsoft Visual Studio 8\Common7\VS2005ImageLibrary as VS2005ImageLibrary.zip. So far so good, no reinventing the wheel yet.
While I still have more work to do on the System Browser part in the way of trimming namespaces in the listboxes, hiding properties that have public assessors, making the constructor bitmap different than the method bitmap, etc., I am going to move onto code comments.
So when I am reflecting an assembly and its Types, I want to include the code comments, if any are supplied, as well. While I know about the XML Document Comments in C# and have used them for some time, you can use Ndoc for, C# and XML Source Code Documentation. In other words, API documentation ala MSDN style.
However, it turns out that the C# comments you put in your source files never gets compiled into the assembly. I thought maybe the XML document was an embedded resource as indicated by this example of, Working with Embedded Data :
Using System.Windows.Forms as myAssembly
string[] names = myAssembly.GetManifestResourceNames();
foreach(string name in names)
{
Console.WriteLine(name);
}
Nothing in there other than bitmaps, icons and cursors. While you can embed the XML document as a resource, it is not done at compile time nor included in the .NET FCL.
I am a bit surprised by this. Having the assembly and the documentation for the assembly in two separate files violates the principle of encapsulation in my mind. The first thing to get lost when moving an assembly around is the XML documentation file.
I wonder why two separate files? Yes, I know that the XML doc is also used for IntelliSense comments, but what if it was in the assembly and you reflected on it from the assembly? Is it a size issue? Looking in C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727 I see that the largest XML file is mscorlib.xml at approximately 7 meg in size whereas the actual mscorlib.dll is just over 4 megs in size. However, the majority of other XML files are in the 100Kbyte range. So maybe memory could be an issue, but at the very least, I would like to have the option of embedding my comments in the DLL so that if other people are reflecting on my Class library, they can also read the comments.
Ok, well, I can still read the comments from the XML document files and load them into the System Browser. Sounds like more code reuse as I am sure someone has written this before. And as it turns out, yes in a couple of ways. One is "XML Comments" which loads the XML documentation based on what you are reflecting on. I have this implemented in the System Browser with one line of code. The other code sample is called, Documenting .NET Assemblies which generates code comments directly from an assembly, even if there is no source comment XML dcoumentation. Looks promising.
I know this post does not have much to do with IronPython (yet), but is the setup for embedding the IronPython interactive console interpreter into the System Browser. The point about using IronPython to dynamically build apps from the .Net FCL and other assemblies requires a System Browser so that we can inspect the Types, grok the meaning and start dynamically building apps all from the same window.
© Copyright 2009 Mitch Barnett - Software Industrialization is the computerization of software design and function.
newtelligence dasBlog 2.2.8279.16125 Theme design by Bryan Bell
 | Page rendered at Monday, 23 February 2009 17:07:13 (Pacific Standard Time, UTC-08:00)
|
On this page....
Search
Categories
Navigation
Sign In
|