 Tuesday, July 22, 2008
I was reading an interesting post at Ted Leung’s blog called, “IDE’s and Dynamic Languages”. It is interesting to me for a number of reasons. One is how a Text Editor can be considered an IDE, even though Ted does say that automatic syntax verification and code completion is certainly beyond a Text Editor.
One thing that did surprise me was no discussion on debuggers as part of an IDE. How can people code using a text editor without a debugger? I guess I have been (totally) spoiled back in the VB6 days (yes, I will admit it) that I could step though code and when I bumped into an error, I could back up the debugger a few statesments, make my correction and keep on stepping through. I have never been as productive since! Know what I mean? Yes, I know this says nothing about design, but an IDE is a tool for using a programming language, yes? So how come we (as in developers) have so few tools or is that choice of tools?
According to a recent analyst report, 97% of developers on the .NET framework use Visual Studio and over 70% use Eclipse or Eclipse based IDE’s for Java. As much as I love Visual Studio, and being a .NET developer, I have no control as to what I want in an IDE. Worse yet, with the emerging new dynamic languages, IronRuby, IronPython and Managed Jscript, have almost no tool support at all in Visual Studio. While there have been some announcements, and articles and some tooling, its like bolt-ons to Visual Studio and still yet to come.
As a .NET developer, specifically a .NET web developer, I would like to use something other than Visual Studio to develop web applications using a dynamic language. My wish list is for something lightweight and web-based so that I can explore using, an interactive interpreter and a simple code editor just using a web browser. Maybe something like this:
And this:
This was my first version of a web-based “very” lightweight IDE that used a JavaScript based interactive console and a JavaScript based code editor. You can download the code from Codeplex
Several months have passed since I released this and I have been working on a new version. In that time, other folks have started to develop similar web-based interactive consoles and code editors. One example is Jim Hugunin’s DLR (interactive) Console which you can download from the Silverlight Dynamic Languages SDK. I should point out that this is a modified version. Also note that it uses Silverlight:
As an old ASP.NET/AJAX/JavaScript/HTML/CSS developer, I am quite excited about Silverlight for a number of reasons. I am hoping the adoption rate and the tooling for this technology also increases over time. Silverlight development also suffers from not being fully integrated in Visual Studio, but I should be fair as it is also Beta. But trying to run a Silverlight “web” application on a web server and interacting with other lanaguages is tough at best. But the rich UI experience is really quite nice compared to old school ASP.NET forms apps. So even after 6 months of releasing my lightweight dynamic languages development environment, I am still torn between “tried and trued” JavaScript and the new Silvelight.
I thought about how I would implement intellisense in a JavaScript console and cringed at the thought of actually how to implement this. I could see using a few of the controls from the AJAX Control Toolkit to implement this, but it would be quite the effort. Not only does the DLR console support intellisense, but so does this alternative interactive console called Silvershell:
Wow, I am impressed!
Here is another web based shell (that you cannot download) from Oleg Tkachenko:
Also Michael Foord has a Silverlight based Python in the Browser:
Jimmy Schementi has an IronRuby console:
Of course there are other implementations, but they are not web-based. Nonetheless, Joe Sox’s IronTextBox works very well:
And the DLR Pad:
And Ben Hall has just released his IronEditor:
So what’s my point? I think all of these projects are great and kudos to the people that built them. It takes a lot of time and effort above and beyond just regular work hours. I have been there myself, my hats off to you folks! But, there are 8 versions of the interactive console and a few versions of a basic code editor. I know it may be a dream, but it would be great to collaborate with these people and write out a simple set of requirements for what a great DLR console and code editor would be. And then as a virtual team, implement it.
After all, to a large degree, it will be how well supported the language is from a tools perspective that will really determine the rate of adoption. And right now, the tools (or IDE) experience for Dynamic Languages on .NET is severely lacking to the point of having several people independently developing their own tooling. In this post I only pointed out a handful of these tools and I know there are others, but I was really targeting web-based IDE’s. Maybe that is an opportunity? Or is it a pipe dream?
I am also wondering how, in a code editor, one could hook up a debugger to actually step through the code, regardless what Dynamic Language (or Static or JavaScript or Silverlight) is being used? Hopefully I won’t have to wait until PDC 2008 to see what the next step is from MSFT. Who knows maybe there is enough interest to develop a Web-based IDE for Dynamic Languages on .NET.
Wednesday, January 30, 2008 Friday, August 03, 2007
In part 1, I introduced a distributed programming environment called Global System Builder (GSB). The IDE is an ASP.NET web application. Another part of GSB is a Winforms application that runs on a remote computer that hosts an IronPython engine and communicates back to the web server using Windows Communication Foundation (WCF). Hosted in the web browser is a source code editor with syntax highlighting and an interactive console – both of which communicate with the IronPython engine on the remote computer to execute Python code (and soon to be IronRuby using the DLR).
Also hosted in the web browser is a Virtual Earth control for locating remote computers, a Remote Desktop control to host a user session to the remote computer and a Smalltalk-like System or Class Browser for browsing classes (i.e. Types), member info, etc., that were loaded in the Winform app (i.e. GSB Service) on the remote computer.
One of the features I am implementing is the capability to extract code comments from any assembly that was loaded into the System Browser via the remote service (i.e. the Winforms application on the remote computer). The idea is that as assemblies are loaded into the remote service, WCF communicates with the ASP.NET web server by passing up reflected assembly information (i.e. TypeInfo, MemberInfo, etc., using System.Reflection) that is then bound to a custom GridView control in the web browser, including code comments.
I implemented an ASP.NET AJAX hover menu that will display context sensitive menu items depending in which grid you are hovering over. These menu items are helper functions (almost like macros) that will add the necessary IronPython (or IronRuby) code snippet to the console or code editor window, for example, to add an assembly references, stub out methods, insert DSL, etc. For this post, I am discussing a way to view code comments that are part of an assembly, which when clicked in the hover menu, will display a panel with the code comment (plus parameter info, etc.) for the item hovered over, depending whether it is a type, method, prop, etc. Think of it as a web-based version of Lutz Roeder’s cool Reflector, but in a Smalltalk like System Browser.
As an aside, I have one (major) design issue with the way assemblies are packaged out of Visual Studio as there is no way to include the code comments that you meticulously put in your source code files into the actual assembly itself. It is packaged as a separate XML file. In my mind, packaging the code comments in a separate file simply breaks the cardinal rule of encapsulation. I wish Visual Studio at least gave you the option to embed your code comments directly into the assembly itself.
OK, so how to get at the code comments? Searching through my vast libraries of reusable open source code on the internet, I came across this post and a nicely packaged class called XMLComments. Only takes one line of code to implement and retrieve code comments. Perfect. Thanks Stephen Toub!
I used System.Windows.Forms.dll as the guinea pig as it is a whopping 5 megs in size and so is the companion XML code documentation file. Everything seems to work ok, but a few observations. It seems even in Microsoft’s own code documentation, only 50% of the +2200 Types in the assembly have a code comment. Also, what about assemblies that don’t have the companion XML file and therefore no code comments at all?
There are a couple of choices, one is to extend the XMLComments class to derive some form of comments. Ultimately, I would like to be able to build comments into the assembly (separate XML file or assembly) by using GSB’s source code editor and the necessary comments syntax to inject or attach comments to existing assemblies that don’t have code comments (loaded into the System Browser).
[Update August 14, 2007 I recently came across, “Documenting .NET Assemblies” by Robert Chartier (Thanks Robert!) that does a really nice and fast job of documenting assemblies that… have no documentation. Exactly what I was looking for. Robert has done a really nice job here, and has given me a few ideas to rework the System.Reflection code I already use to pull Assembly Names, Types, Member Info, etc.]
How important is this feature to the project? Admittedly it is not a core feature, but at the same time, the point of the System Browser is to browse assemblies for reuse, or more importantly, for programming at a higher level of abstraction, so having code comments that came with the assembly or being able to attach your own code comments to the existing assembly, would be a good thing. In my previous life in the electronics R&D industry, getting a component, like an IC, without the accompanying documentation with the package renders the component useless. Looking at the Types list in the screenshot above, what do you think AmbientValueAttribute does? Not sure what to make of our software development industry where source code documentation is either non-existent or in a form or place that is makes it virtually unusable.
Reusing assemblies to program at a higher level of abstraction is another topic and several posts in of itself. There seems to be a lot of issues (angst?) in our industry about reusing other people’s source code or assemblies. I do it all the time, both professionally (aka my day job) and in my little hobby here. In fact the majority of GSB reuses several source code libraries and assemblies. Without the many people that have taken the trouble to build the open source components that make up this application (aside from my own code), I would never be in the position I am today with a working application.
Anyway, I digress. Displaying code comments is one feature of a growing list of features I collected after presenting GSB to a few potential users (i.e. dynamic programmers). Based on the growing feature list, I still have a long way to go – not the 90% complete I alluded to in my previous post. Also, I have decided to release GSB as open source when the DLR (with IronPython, IronRuby, Dynamic VB and Managed JScript) reaches RTM. This is going to take a little bit of architectural refactoring (mostly the hosting namespace) to plug in the DLR engine and provide multi dynamic language support much like seen in this excellent video with John Lam and Jim Hugunin. All good though as I have nothing but time, yes 
January 30, 2008 Update - Global System Builder is available for download at: http://www.codeplex.com/gsb/Release/ProjectReleases.aspx?ReleaseId=10314
Official web site: http://globalsystembuilder.com
Thursday, June 28, 2007
This is an open source software project that is highly experimental. In fact, it might be just plain crazy. Our industry is a buzz about Web 2.0, SOA, distributed computing, etc. Yet, what tools do we have to support distributed programming?
What I mean by distributed programming is the ability to program multiple remote computers (i.e. servers) from within one IDE instance, or as I call it, a DPE – Distributed Programming Environment:
This ASP.NET AJAX DPE consists of the following parts labeled in red:
1) An Object or Class Bowser, similar to Lutz Roeder's outstanding Reflector, except mine is laid out in the old school Smalltalk System Brower format. Where are the assemblies IronMath and System.Windows.Forms coming from?
2) This is a Remote Desktop Connection (RDC) that allows connections to multiple remote computers. One tab per connection.
3) This is a Windows Forms application (or service if you will) that runs on the remote computer(s) that allows you to add assemblies to be reflected. It uses Windows Communication Foundation (WCF) to communicate with the web server hosting the DPE, which then displays the reflected assembly information in the browser window GridViews. In addition, this Windows Forms application hosts IronPython (and soon the DLR) which also communicates with the web server via WCF so that IronPython code can be executed on the remote server(s).
4) An IronPython source code editor which provides syntax highlighting, handling multiple source code files, etc. Again, using WCF to communicate with the IronPython DLL that is running on the remote server(s).
5) Is the result of running the IronPython code that is displayed in the editor window.
6) To find remote computers, I use Virtual Earth’s map control to display remote computer locations. Optionally, a MapPoint Web Service can be used to upload custom locations and attributes.
7) Oh yeah, an interactive command line console is a must have.
The context of which remote computer I am programming is activated by which tab has been selected. Meaning that when I click on any tab, any related context, like other dependant tabs and grids are switched too. I add distributed computers by adding tabs for each RDC session, a source code editor, and an interactive interpreter (i.e. console) per remote computer.
Think of the web server as a middle-tier to the WCF enabled remote services that host’s the code for reflecting assemblies and executing Python code.
As I mentioned earlier, I will be embedding the DLR which will add support for more dynamic languages. Also note that the web-based console window is generic in the sense that it can be a console to a remote cmd window or PowerShell window or… Same goes for the source code editor, its syntax highlighting also supports multiple languages.
Like I say, it will either be useful or just plain crazy. That’s why I call it Global System Builder
January 30, 2008 Update - Global System Builder is available for download at: http://www.codeplex.com/gsb/Release/ProjectReleases.aspx?ReleaseId=10314
Official web site: http://globalsystembuilder.com
Friday, June 22, 2007
Previously, I discussed a web-based source code editor for IronPython, which is part of a larger application that I am working on. Most of the editor is complete, just working out the details on handling multiple source code files. Today we will drive IronPythons’ interactive interpreter, using a web-based console, which is also part of my application.
Using the console, we can whip up a simple Windows forms application in seconds:
And here is the output:
The console is based on two components, one is Sam Stephenson’s Prototype JavaScript framework and the console GUI is brought to you by Jeff Lindsay’s Joshua, which I modified to work with IronPython.
Running a web console over the internet presents a few challenges. Everywhere you read, AJAX is the sh!t. I use a ton of it in the app I am building. A stands for Asynchronous and therefore, as some have stated, synchronous over HTTP is bad form mate. Well, in the case of a console application, I think a synchronous call is one (only?) way to make it “really” work:
//synchronous call to the server passing the IP string command and returning the response
function getFile(url, passData)
{
if (window.XMLHttpRequest)
{
AJAX = new XMLHttpRequest();
}
else
{
AJAX = new ActiveXObject("Microsoft.XMLHTTP");
}
if (AJAX)
{
AJAX.open("POST",url,false);
AJAX.setRequestHeader("Content-type", "text/xml");
AJAX.send(passData);
return AJAX.responseText;
}
else
{
return false;
}
}
One nit pick with the ASP.NET AJAX framework is that you can’t do synchronous. I would hope that the framework would support this option as a) other frameworks provide it and b) I want it man! The point of a framework is to provide options so that a variety of similar, but different business requirements can be met with “one” AJAX framework. I am already using 3 AJAX frameworks as each one brings a unique piece of functionality that I require. Who knows, maybe it will be more by the time I am finished.
If you look at the console closely, you will see one small GUI issue. When parsing interactive = true with Python, visually, the next line of Python code should reside beside the “…” and not on the next line beside the command prompt (i.e. >>>). Also note that the 4 spaces indent after the def statement does not work at the moment.
Aside from a couple of GUI issues, and a missing blinking cursor, there is one last feature to be implemented and then the console will be complete.
The astute reader may notice that the Windows application is running in a Terminal Services session, which is being hosted in my web browser. Did I remote into the web server? No. On the client computer? No. Where is the Windows application running?
January 30, 2008 Update - Global System Builder is available for download at: http://www.codeplex.com/gsb/Release/ProjectReleases.aspx?ReleaseId=10314
Official web site: http://globalsystembuilder.com
Friday, June 15, 2007
Why web-based? Well, the IronPython source code editor I am working on is one piece of a larger web-based, open source application that I am building and will become clear in future posts as to the reasons why it needs to be web based.
In the meantime, I thought I would write about some of the available components that provide real-time syntax highlighting and other features one might want in a web-based source code editor. After extensive research, there are 3 web-based source code editors that I would recommend.
The screen shot above comes from a portion of my web-application that I am building. I am using Christophe Dolivet fantastic EditArea. EditArea is written in JavaScript and really is quite outstanding given the number of features it has, particularly that it is free (released under LGPL license).
One of my main requirements is that it can work with ASP.NET AJAX (a free framework) and can be embedded in a TabControl as you see in the screen shot. I will discuss the web-based IronPython interactive console in another post.
I am really impressed with what Christophe has done, not only is EditArea fully featured, but the JavaScript source code is extremely well written using prototypes. Further, the documentation is excellent – you would think this is commercial based source code editor. EditArea supports multiple languages, both the software itself and the syntax highlighting it supports, for example, in my case English and Python.
Another real-time syntax highlighting source code editor I have come across is CodePress by Fernando M.A.d.S. It is also written in JavaScript and covered by LGPL. I really liked this editor as well. While it is not as full featured as EditArea, it does a very good job. It also supports many languages, but not Python at the moment. I see it is on the todo list. While I could get it to run in a bare bones ASP.NET AJAX application, I could not get it to run embedded in a tab control. However, I think the limitation may be my JavaScript coding skills (I am a C# programmer hooked on Intellisense) and not necessarily CodePress itself.
Syntax Highlighter by Marijn Haverbeke is another web based, real time syntax highlighter. It is released under a BSD like license. It works quite well, but not as fully featured as the other two editors. However, it is quite fast and seamless. Dos not support Python syntax highlighting, but as in the case of CodePress, would be fairly easy to implement. What I found really interesting was Marijn’s story on how it was designed. Fascinating Captain!
Thanks to Christophe, Fernando and Marijn for creating and developing these real-time syntax highlighting source code editors. It has saved me a lot of time not only from building these from scratch, but also proves a theory I have about software industrialization. That is, it has been done before and I don’t need to reinvent the wheel. Some people call this mashups, others reusable software or open source, or… No matter what you call it, I am thankful that so many people are willing to share their hard work.
When will it be released? Not sure as there still is lots of work to be done and can only work on it part time. I wish it was my full-time job!
January 30, 2008 Update - Global System Builder is available for download at: http://www.codeplex.com/gsb/Release/ProjectReleases.aspx?ReleaseId=10314
Official web site: http://globalsystembuilder.com
Monday, January 01, 2007
After a 6 month hiatus from the blogosphere, I am back. Well, almost 6 months. On July 17th, 2006 I proclaimed that after 80 posts on the topic of software industrialization, I did not have much more to say about it. Today I do.
Before I dive into the post about what I think the best new programming language released in 2006 was, (the language isnt new, but the implementation is), I want to take a slight detour. I have a confession to make.
The confession is that I fell into the pervading consciousness of some in our software development industry where cynicism, negativity and ego rule. One reader of my blog, (bless you Martin Danner) said that my recent posts conveyed a sense of frustration, cynicism and downright hopelessness. Martin told me to cheer up, all is not lost and in fact the seeds of change have sprouted. Thank you Martin, you were right. I was spending way too much time reading posts from: Mini-Microsoft, Joel Spolsky, Steve Yegge, Jeff Atwood, Rands in Repose, TheDailyWTF, etc. Regular readers of these authors know what I mean.
Now dont get me wrong here. I have a lot of respect for these very smart people and mostly what they write is well written, entertaining, and even thought provoking. However, there are times where each of them have also succumbed (in some cases, more often than not) to the cynical world of Dilbert software development where everything is a rant or complaint or WTF! You would think you are reading Dennis Miller Rants.
I will be the first to admit I have done my fair share of that in the past and on this blog. For those of you that knew me when I ran my own software company, I could be the darkest, most cynical you-know-what in the world. So whats changed? I realized that after 15 years in this biz I could just be yet another cynical puke or I could do something positive for myself and the software industry I work in - like contribute an open source project of some sort that would make the process of designing software more predictable and repeatable than it is today (i.e. the industrialization of software). I started that quest over 6 months ago and may take a year or two to finish. But thats another post and not my point.
The point of this digression is that I am a bit disappointed in the Joel Spolskys, Steve Yegges and the others I mention above, of the world. I am disappointed in that they could really influence the positive direction of our software development industry, rather than b*tching and stitching, in a well written manner about how everything sucks. I know some of it is not to be taken seriously, but at the same time people look up to these industry thought leaders. How is this constant barrage of negativity going to help our industry?
Do you understand what I mean? I try not to take myself or my work too seriously, but I do. I cant help it. And neither can most of the programmer types I have worked with over my software career. Its simply who we are and it is interesting to find other like minded people have chosen this profession.
So what does that have to do with the best new programming language released in 2006? A lot actually. I find it an uplifting story where Jim Hugunin, 3 years ago, started to think about ranting on why the .NET CLR would be a terrible platform for dynamic languages. Three years later on September 5th, 2006 an open source programming language from Microsoft called IronPython was released. Having played with IronPython for six months, it gets my vote for easily being the best new programming language released in 2006.
Why? My criteria is very simple. What is the simplest way, or to phrase it differently, the most productive way to write a windows program? (e.g. ASP.NET web form, WinForms, or XAML WPF). Here is an example:
Note while the least number of code lines count to a certain degree, it is the simplicity that matters the most. What could be simpler than opening an interactive console window and typing in, line by line, almost English sentences while getting immediate feedback after executing each line. Either the line you executed succeeded or failed. I love when decisions are binary. In fact, by line 3 in the example above, you already have a real live active WPF window being displayed on the screen, so now it becomes mostly a visual experience. Who needs testing? The testing is built right in, in real time either the window displayed or it did not. I know I am being simplistic, but perhaps our programming world could do with more simplicity. I have over 10 years invested in learning the Visual Studio IDE and even with James Averys most excellent book, Visual Studio Hacks, I still only know maybe 60% of the complex features of VS2005.
The reason I said most productive is because I believe that the code, build, run, and debug loop of most statically typed programming languages are a real barrier to productivity. I love C#. I use it in my day job. It also frustrates me because I am locked into the code, build, run, and debug cycle. I just dont work that way. I like the exploratory iterative approach with immediate feedback as a way to design software.
IronPython is free. The source code is also free and both the object and source can be freely distributed in your own application, which is exactly what I am planning to do with my own open source software design tool. Stay tuned.
Tuesday, July 04, 2006
Have you seen the IronPython integration project in the June VSSDK? Aaron Marten and Team have done an outstanding job integrating IronPython into the Visual Studio IDE. I have lived in Visual Studio since 1997, and can say that the IronPython integration is totally pro.
In the VSSDK path C:\Program Files\Visual Studio 2005 SDK\2006.06\VisualStudioIntegration\Samples\IronPythonIntegration you can find the IronPython.sln solution file for the IronPython Project System which features include:
Defines a VSPackage that controls the interaction between Visual Studio and the IronPython compiler.
Participates in building, deployment, and source code control.
Includes MSBuild targets and templates.
Offers debugging support, but no expression evaluator.
Displays and manages project items in the Solution Explorer.
Participates in the Open Project and New Project dialog boxes.
Exposes project and configuration properties, both generic and specific to IronPython, such as the "main" program file.
References web services, databases, and other resources, but not other IronPython assemblies.
Supports automation.
Connects the Windows Form Designer to the IronPython CodeDom.
Press F5 to build the sample, register it in the experimental hive, and start Visual Studio from the experimental hive.
The last statement is interesting. For anyone that has used the pioneering DSL Toolkit knows that running this solution will launch another instance of Visual Studio, and in this case, complete with the integrated IronPython project system now running. If you click on New Project, the following dialog appears:
And sure enough, the PythonPoject is fully integrated into this instance of Visual Studio 2005. It works just like a C# Project Type, for example. In other words it is seamless and consistent within the Visual Studio IDE paradigm. A real testament to Visual Studios pluggable language architecture.
I developed a small IronPython Windows app to try it out. I wanted to see how the Win Forms Designer generated the IronPython code and what it looked like compared to C# Windows Forms apps. So I dragged a ListBox onto the Design surface. Yes, all of the WinForm Designer and toolbox are there. Then in the Form code, I called System.Reflection to get the Types from Mapack.dll and then add the Types to listBox1. Currently there is no support for Partial classes so the Designer code and your Form code are in the same file. One way around this is intelligent use of VS #regions to package code bundles in the same source file.
Btw, the Mapack.dll is an excellent example of the type of reusable Assembly I am talking about with respect to software industrialization. Here is an encapsulated package of Math functions that you can just plug-in to your application. A good abstraction and packaged at the right level in my mind.
My only complaint is that the Mapack.xml code documentation file did not come with the DLL. I wish the VS IDE came with a compile option to include source code documentation in the Assembly. As I have mentioned elsewhere, I think having the source code documentation in a separate external file breaks an encapsulation principle in my opinion. And the first thing to get lost, yes? Plus if I want to program Assemblies instead of source code files, then it would be nice to be able to retrieve the documentation through System.Reflection as it seems you can do everything else with this namespace.
Note the IronPython Visual Studio project is the bottom most Application on the screen. When that is run, another instance of Visual Studio is created with the IronPython project integrated into the running instance. Then I opened up a IronPython windows application project called, Browser1. Running that project launches the small windows app with listBox1 populated with the Mapack.dll Types. You can see the IronPython code in the Browser1 project to the left of the Windows app and on the far right, you can see the 5 projects that make up the IronPython VS integration project.
Another cool integration point is the IronPython interactive console that is directly integrated into the Visual Studio environment on one of the tabs:
The console window is a tool window that creates a TextBuffer object and hosts a TextView control that is initialized to use the text buffer. The interaction between the console window and the IronPython engine is handled by the implementation of the IConsole interface and the definition of the streams used by the engine as standard output and standard error. The history is implemented using a class called HistoryBuffer that stores the list of commands executed inside the console, sorted by execution order. Statement completion and colorization are implemented using the same IronPython language service that is used inside the standard editor when editing a py file. The different behavior of the language service in the editor window and console window is implemented using one functionality exposed by the IronPython language service through its version of the Source class. This object, defined in the MPF, is an abstraction on top of the text buffer and text view.
Wow, when I read that I was expecting a super deluxe console window and it is. Now thats an interactive console window!
As you browse the source files in the PythonProject you will also notice several XML Overview files, Class Diagrams and test suite metrics for each project. Nice.
If you are a Visual Studio code jockey, do yourself a favor and have a look at the Visual Studio IronPython integration project, its as pro as it gets in our software development world. Kudos to Aaron and team!
If you want to tune in to the IronPython world, I would recommend that you visit CodePlex to get IronPython and sign up to the Listserver. Dino and crew are extremely helpful in answering any questions that are posted. The IronPython community is vibrant one with daily action.
Finally, if you are a statically typed programmer feeling a bit shackled by a universe that the compiler creates, unshackle yourself and enter a universe that is still dynamic after compile time. You owe it to yourself to check out the IronPython integration project in the VSSDK.
Monday, June 26, 2006
The .NET Framework Class Library and the common language runtime (CLR) serve as the foundation for all .NET-based applications. But how much do you know about any of the thousands of CLR classes in the library and where do you begin to learn about them?
In this article, the author uses the number of relationships with other types to determine which types are going to affect your programming most frequently and how often you'll encounter them. He programmatically surveys the CLR library using reflection to make that determination. Following that, an in-depth examination of the nine most important CLR types is provided.
I like this article as it shows a pragmatic way of understanding the .NET FCL. In addition, using Lutz Roeders Reflector, you can literally inspect, disassemble and decompile any Assembly, (.NET FCL or other assemblies), to your favorite .NET language. Do you believe that source code = design? Even though the article is 14 years old, it is one of the most decisive articles on software design I have ever seen. It also most closely resembles my belief the source code listings are the design documents.
Since the FCL and runtime (application execution) environment are packaged together, you basically have a programmable runtime environment. Using a dynamic language, like say, IronPython, and through an interactive console, I can program the FCL and have it execute as I am programming line for line. As each line is entered in the IronPython interactive console it is being dynamically executed by the IronPython engine (interpreter). In some lines of code, I have built a trivial Windows Presentation Foundation application borrowed from the tutorial that comes with the IronPython download.
As trivial as it is, there is an important concept being demonstrated here which is with IronPython embedded in your .NET application or in Visual Studio (i.e. using the VSSDK) or using the standalone interactive console, you can dynamically program the execution environment so to speak. A programmable runtime environment.
I have always been a fan of software reuse, mainly because I am lazy. Also, I know it works. I make a living at it. In my world of programming business applications, the reality is that most if not all of it has already been coded several times over. As seasoned programmer once said to me, its all been done before. When was the word processor invented? How many word processors have we had since? Whats different? Probably the same thought goes for spreadsheets, workflow - paper or otherwise, invoicing, ecommerce, etc. Most of these business problems have already been solved with software many (or a million) times over. My question is, how can I reuse this incredible wealth of software that has already been designed and built? Thats my thesis to prove.
I can reuse software much like the way I am practicing today, finding components of various sizes and shapes that loosely or exactly match a functionally decomposed problem I need to solve. For example, my System Browser needed a way to parse an Assemblys XML document file to retrieve code comments on Types and Members when reflecting on an Assembly. Surely that has been written before is what I am thinking. And with a little bit of searching, yes it has, to the point where after I have included the example class (or Assembly) in my System Browser application, I can integrate it with one line of code.
This means I did not have to modify the code at all, I could just use the public interface and problem solved. Next component. And that is how I am building my System Browser application. The 80/20 rule is in effect. 80% reuse and 20% net new code, distributed between coding new functionality, modifications to existing components, and glue code to stitch all of the pieces together.
So what am I saying? I am saying that there is a lot of code out there in the world to reuse. I and others have been doing it for years. Sometimes known in our practice as clip-board inheritance. But, how can I pragmatically reuse pre-built code? While having the source is nice, it does not have to be a requirement. In fact, using System.Reflection will let you look at practically anything inside the Assemblies black box. And I can load any black box (i.e. Assembly, COM Object) and not only inspect it, but use it dynamically in my application using an embedded IronPython engine. Thats what the System Browser is all about. A tool to dynamically program the runtime environment.
The System Browser application is designed to be an emulation of Smalltalks programming environment. It has a dynamic/interpreted programming language integrated with a large class library and an application execution environment. Essentially a virtual machine.
Rather than Smalltalk, and the Smalltalks class library, I am using IronPython and the .NET 2.0 FCL. The CLR on a Windows OS takes care of the application execution environment. Through a similar GUI like Smalltalks System Browser, I can use System.Reflection to get access to the Type system. Using IronPython I can dynamically create new classes and subclass the FCL at will.
Not only can I browse and use in real-time the FCL, I can do that with any other .NET assembly or COM object. I wonder how many .NET assemblies are compiled everyday world-wide? How many COM objects over the years have been compiled? I must have produced over 100 DLLs by now after 15 years at it. I am one of approximately 12 million programmers worldwide. According to IDC, the number of professional developers worldwide will increase to 13.3 million by 2006 from 7.8 million in 2001.
Theoretically speaking, even if each developer compiled the equivalent of 1 DLL a year over the last 10 years, times 10 million developers, thats tens of millions of DLLs that have been produced. Even if it is a magnitude more or less, its still an incredible amount of potentially reusable DLLs. Again, how to use them?
I modeled my System Browser after Smalltalks System Browser. Why? Smalltalk and its environment was best suited for me where I can browse classes, instantiate them, inspect other instances, manipulate them, all in a simple to use dynamic environment. At least for me, that was the pinnacle of an OO development environment, along with Brad Coxs Objective C System Browser. Now I can have the same thing but different. Maybe it is just the way I feel most comfortable navigating and manipulating a FCL.
Another reason for this approach is assisting in the industrialization of software development. I want to program at a higher level of abstraction. To me, that means heavy reuse of pre-designed and pre-built, components is critical to raising this level of programming abstraction. Why write source when the compiled unit is already there?
In the many pages of this blog, you will have read statements like, from a software development point of view, given whatever software application or component you have been tasked to build, chances are that it has already been done before. Look it up on the internet. In fact, this System Browser application is being constructed using this reuse pattern.
Once I have found an existing Assembly or COM component or source code (that will be compiled into an Assembly) that meets 80% (or best choice) of my requirement, I download the assembly and import it into my image or System Browser which means it just became part of the programmable runtime environment.
Now I can reflect on the assembly and all of its Public Types and Members. If there is an associated XML code documentation file, then that is parsed and displayed each time I click on a Type or Member. If there isnt a corresponding document XML file, then I code generate on the fly an XML code documentation file, using a standard template, so that a sentence is formed for each Type and Member as basic (but editable) code documentation. It is already generated and reflected back into the System Browser before you can click on anything.
Using the embedded IronPython inside the System Browser, as I am entering code in the console window (interactive mode) and/or code editor (file mode), both are being recorded then compiled and added to System Browser on the fly. This is done in real-time and totally seamless. At least that is one of the goals of my System Browser.
I would not take too much of this too seriously. This is more of an experiment for me to learn from. It is for my research on software industrialization. However, the System Browser will be freely available just like the IronPython interpreter embedded in it. You will be able to reference assemblies and COM objects, browse their public Types, Members, code documentation to learn from and also be able to dynamically use those Types in your applications. Or at the very least experiment with the Types. I just like programming at a higher level and leveraging perfectly good code that was built before me. Based on my research and own experiences, there sure is a lot of pre-built code out in the world to be reused in your applications, or at least in the business applications I have been tasked to develop at my day job.
This last post was my design rationale or thesis on its applicability to assisting software industrialization by promoting Assembly reuse. A higher level programming abstraction. Something I think our software industry could use.
Wednesday, June 14, 2006
I have made some progress developing a Smalltalk like System Browser for reflecting .NET Types with an IronPython embedded console interpreter to manipulate the reflected Types. Not as much progress as I would like, but with a few hours a week, what can you do.
You can reuse code to increase productivity. Here is something that seems a relatively simple task to do, I want to add an icon (bitmap actually) to each item in my list boxes that represent assemblies, types, properties, methods and events. It turned out not to be so simple as a .NET listbox does not have this capability out of the box. Now what? As I have suggested in the past, the WWW is an excellent (unstructured) catalog of reusable software. I make the assumption that each one of the components I am looking for to build this application has already been built by someone.
Searching away brought up several specialized listbox implementations, but this one I liked the best is called. . NET Color Listbox. Alex has done a great job here as his implementation provided not only the ability to add the bitmaps to each item in the listbox, but also uses DoubleBuffering to get rid of the flickers when scrolling looks nice and smooth.
Where to get the bitmaps that represents assemblies, classes, methods, etc.? If you have Visual Studio 2005 installed, you will find them at: C:\Program Files\Microsoft Visual Studio 8\Common7\VS2005ImageLibrary as VS2005ImageLibrary.zip. So far so good, no reinventing the wheel yet.
While I still have more work to do on the System Browser part in the way of trimming namespaces in the listboxes, hiding properties that have public assessors, making the constructor bitmap different than the method bitmap, etc., I am going to move onto code comments.
However, it turns out that the C# comments you put in your source files never gets compiled into the assembly. I thought maybe the XML document was an embedded resource as indicated by this example of, Working with Embedded Data :
Using System.Windows.Forms as myAssembly
string[] names = myAssembly.GetManifestResourceNames();
foreach(string name in names)
{
Console.WriteLine(name);
}
Nothing in there other than bitmaps, icons and cursors. While you can embed the XML document as a resource, it is not done at compile time nor included in the .NET FCL.
I am a bit surprised by this. Having the assembly and the documentation for the assembly in two separate files violates the principle of encapsulation in my mind. The first thing to get lost when moving an assembly around is the XML documentation file.
I wonder why two separate files? Yes, I know that the XML doc is also used for IntelliSense comments, but what if it was in the assembly and you reflected on it from the assembly? Is it a size issue? Looking in C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727 I see that the largest XML file is mscorlib.xml at approximately 7 meg in size whereas the actual mscorlib.dll is just over 4 megs in size. However, the majority of other XML files are in the 100Kbyte range. So maybe memory could be an issue, but at the very least, I would like to have the option of embedding my comments in the DLL so that if other people are reflecting on my Class library, they can also read the comments.
Ok, well, I can still read the comments from the XML document files and load them into the System Browser. Sounds like more code reuse as I am sure someone has written this before. And as it turns out, yes in a couple of ways. One is " XML Comments" which loads the XML documentation based on what you are reflecting on. I have this implemented in the System Browser with one line of code. The other code sample is called, Documenting .NET Assemblies which generates code comments directly from an assembly, even if there is no source comment XML dcoumentation. Looks promising.
I know this post does not have much to do with IronPython (yet), but is the setup for embedding the IronPython interactive console interpreter into the System Browser. The point about using IronPython to dynamically build apps from the .Net FCL and other assemblies requires a System Browser so that we can inspect the Types, grok the meaning and start dynamically building apps all from the same window.
Tuesday, June 06, 2006
A couple of weeks ago I suggested a way of dynamic programming with IronPython. Essentially it is a combination of using a Smalltalk like System Browser to browse types using .NET reflection and embedding an IronPython interactive interpreter into the System Browser to manipulate the reflected types.
Why would I want to do this? While I really like Visual Studio, which I use for my day job, sometimes it takes forever to build a simple prototype or experiment with a new class library. For me, I want to see an app up and running as soon as possible, but at the same time leveraging class libraries and third party components. And I want to do this dynamically.
I am also one of those people that learn best by doing.& |