Telescopes and views

Jon Udell's essay "The wrong end of the telescope ?" raises some interesting issues and summarizes some thoughts from several sources.

They all go to illustrate how difficult it is to put a simple application into a browser page. A part of the problem is that the browser/http/web server combination was never designed to support the rich applications we are trying to shoe into it. Therefore the issue is not really different ends of the telescope but rather different telescopes. Programming the browser is very different from programming the server and they do not connect very well.

The problem lies in the marked impedance between the browser model and the server programming model. Server side programming is relatively easy - the tools are there and it is up to the practitioner's to use them correctly.

To match the server side interfaces with the client is hard when you go beyond a simple form. Entering the world of multi-page applications becomes challenging to say the least. We have invented all sorts of interesting work arounds/add-ons - cookies, frames, iframes, DHTML etc to try and make it possible to deliver rich applications. Some people have created neat frameworks to help such as this from cross-browser.com but in the end the complexity remains in connecting the back to the front.

A good indication of the overall complexity is the number of attempts to build easier to use frameworks - everyone has made an attempt at creating either a new scripting language, a new template framework, a new application stack, application builder and now rich clients. The
proliferation of these tools is a symptom of the problem- sophisticated web applications are too hard to build. The reason is, I believe, there is a impedance mismatch between the browser and the server - HTTP is not the issue.

Sometimes it feels like we are all using different telescopes to hunt for the most elusive thing of then all - the easy model of building, debugging and maintaining sophisticated and usable internet applications.

EDI - Simple enough?

A comment by Mark Baker around EDI got me thinking about the current amount of infrastructure we are building on top of the transport. Today millions of business messages are flowing formatted as EDI messages and thousands of business processes are built using the MEP defined by EDI documents.

So what is wrong with just using EDI. Probably a couple of things, at one point the cost of the initial transport hardware and communications infrastructure but this has been solved by the draft AS2 standard which is essentially EDI over http(s) which has reduced the costs dramatically. So much so Walmart has mandated it for all communications.

The amount of work that has gone into the MEP and the definition of the document elements is huge and extremely valuable IP. The only remaining piece is the actual format itself which is very “concise” as it was designed when bandwidth was expensive. The format itself is hard to work with compared with XML due mainly to the lack of tools. If every language had an open source EDI parser (or two) and a transformation tool like XSLT would everyone be using EDI today?

The mechanics of EDI at the MEP level provide a fairly complete set of business interactions but getting into the details of the message and extending the messages is very complex and requires very specific knowledge that is only applicable to EDI - no one has every used EDI as a format for config files or build scripts. XML on the other hand has become the universal data language, because it is so easy to manipulate and mold.

In some ways the discipline that EDI imposed has resulted in a loss of simplicity as instead of a set of well defined MEPs we now have a large number of standards that try and do the same but in fact do not focus on the MEP but rather on making the communication more complex. Where is the WS-850 rather than the generic WS-Metadata. Are we missing the point because it is to easy to avoid it?

Information overload: volume or change

While there has been a dramatic increase in information volume the actual problem for me is the rate of information change. Or maybe I am unable to manage the fundamentals of the information flow, how do pick out the key pieces of information and relate back to my existing knowledge framework.

When the information sources and formats change so rapidly we are unable to categorize them before we move onto the next piece. The whole idea of refereed technical papers was to try and place the information in a broader context and relate it to other similar information. Today's hyperlinks may be technically better but the information schema is not well designed so they do not perform the necessary context creation, or at best it is ad hoc.

Of course Blogs (like this one) add to the problem as we publish more a stream of consciousness (speaking only for myself) rather than a well consider view with information to back it up. Blogs are thoughts in progress and do contain many nuggets but they needed to be forged into coherent thoughts and placed in a larger context.

Is the next great Google type company not something that finds the atoms but something that creates the contextual fabric of knowledge?

Open Source to Open Services

Open source has shown that simplicity can be created in the infrastructure but it still misses its potential as code still has to be designed to be installed and maintained on various platforms that are changing at uneven rates. It should rather be offered as an open service that can be combined with other services to create new services that live in the network. I do not want software I want solutions to problems. (I still like software and looking at code but i really do not need it - figuring out how Google Maps works is fun and interesting but I do not need to know how it works to use it. To integrate with it I should need to know very little but integration is still too hard.

Integration is a problem because most software is not designed to be integrated. Just as till the advent of open source most enterprise software was not designed to be easily maintained, built and deployed. A huge effort went into (and still goes into) designing software to be deployed on a large number of target platforms. This effort provides zero value to the consumer of the solution.

If the same effort by smart people went into creating simple open service interfaces as they have done to create simple maintenance and deployment for open source we would have a very rich ecology of shared services and a very different economic model. Open source is trying (and in many cases succeeding) in moving the economic model to a maintenance model - well take the next logical step and move to a complete utility model that measures economic value by ease of integration and SLA (Service Level Agreement).

The economics are on the side of shared infrastructure and software as a service. Moving to open services would dramatically increase the impact of the open source community and move the value to the SLA of the service not the portability of the service.

More simplicity

After my last post on simplicity I received an email from an old Teknekron/Tibco colleague Steve Wilhem who made a couple of great points that should be shared. I have para-phrased his email so any errors in interpretation are mine.

The first was around refactoring and how to achieve simplicity (specifically in a framework) the need to constantly refactor, do not expect to get it right the first time but keep evolving it. The other was more subtle and I thought most important. Do not try and solve all possible problems - leave the system open enough such that the last ten percent or so can be done by the user of the system. I think this is key as this is typically where we add complexity - trying to solve all the weird edge conditions we can possibly dream up. Let's assume our users are smart and give them the hooks to solve the problem in the way they see fit. Moves any complexity to specific applications rather than the infrastructure where it does not belong. Thanks Steve great suggestions.