Wednesday, March 31, 2004

Programming and visual metaphors

Programming well is a complex task and requires deep understanding of the concepts being manipulated. By way of Sean McGrath and Eric Newcomer comes this thread that is well worth discussing Programming ain't pictures - yet .

The big gap is between those who believe that programming can be done in the whole through a visual metaphor with minimal understanding of the underlying mechanics and those who believe that it is a tool that can assist experienced developers to be more productive. I am more in the later camp, for two reasons, 1) havng spent a lot of time with craftsmen who use tools to work metal and wood I know that the result depends on the craftsman not the tool. (no matter how good the tool I still have two left thumbs) 2) As Sean makes the point everything today is based on imperative and proceedural logic, until computers are based on a new set of concepts, visual programming is always going to assist experienced developers rather than replace them.

Thursday, March 25, 2004

Embedding Visualization into your own page

One of the interesting attributes of creating well defined services is how easy it is to distribute the display code and have it completely separate from the business logic. For the blog mapping software the HTML code below will create a new display widget. The blog displayed on startup in the URL in bold. This example is for Jon Udell's weblog, it does take a little time to load as he has a lot of links but it is fun to be able to naviage all his links in a visual manner.

This is an example of the HTML code for a page to create a blog map

<title>DreamFactory 6.34 Project: Viewer</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<BODY bgcolor="#FFFFFF" topmargin="0" leftmargin="0" marginwidth="0" marginheight="0">

Anyone can add this code to their blog and provide a new metaphor for navigating their neighbourhood. As I add more features they will be delivered on demand i.e. no need to update software everything is delivered out of the network.

Tuesday, March 23, 2004

Update to visualization tool

With help from Bill Appleton at DreamFactory I have upgraded the visualization tool. If is now full screen and much more useful. If you want to play with the source click on "Author". There are still some initialization problems that I am working through.

To use just click on display and you will get a graph of the blogs I link to. Then double click on any block and the home page will appear in a new browser window. The URL will also be inserted into the URL field. If you now click on display a view of the blogs around that blog will be displayed.

The next steps are to do some more clean to the initialization code. This will allow anyone to insert this widget on their blog and specify their blog as the root node - cool everyone can have their own personal mapping server courtesy of DreamFactory with no server code. Demonstrates the simple value of software as a service.

Monday, March 22, 2004

Rich Clients and Services

I have been messing around with display graphs of blogs again. This time I have designed the system to be a service (the blog crawler) that can be accessed at"blog url". This will return an XML file of all the related nodes this blog refers to. To see a visual representation I have created a simple DreamFactory client here (windows and mac plugin only). If you see the source code on the page you can embedded this in any page of your own and have a graphical view of your local blog neighborhood.

The ability to "loosely couple" the service and the display is a key advantage of the new world of network clients. DreamFactory is a great example as it does not require a server to be deployed. Therefore the service and the display are truly loosely coupled. All the network client and service know is the interface. As I evolve both they will be able to move independently.

Compared to a classical web approach with significant amounts of display logic embedded in the server this is a dramatic leap forward. We are still at the early stage of this shift but have been through everything from character screens to the web I see this as a truly tectonic shift.

Note: software still buggy and the UI is not very elegant

Friday, March 12, 2004

Why Integration as a service?

Companies must become more agile and work together to remove costs from the value chain to reduce friction. This will improve the overall efficiency the overall value chain that can either increase margins or reduce prices to the consumer. This is a growth strategy rather than a cost reduction strategy. Industry leaders such as FedEx, Wal-Mart, Dell and Cisco have demonstrated the effectiveness of an integrating their value chain to expand their businesses. Integration is a strategic need in the industry and it is going to happen, the question is what is the fastest and most cost effective approach to cross-organizational integration.

The value chain is composed of heterogeneous systems that need to be integrated. The value chain is and will remain heterogeneous. This is due to three powerful forces at work. One of the key arguments for the value chain becoming homogeneous is that standards will, at some point in time, deliver a homogenous world. This will only happen if you believe that the objective of standards is to create a single standard. Rather the goal of standards is to create a number of interoperable standards those information workers can use as their toolbox to solve problems. The goal is to create a well-defined set of tools not a single tool to solve every problem. As a simple example take file transfer, today I can use SMTP, HTTP and FTP to transfer files between two systems. These protocols are all standards and are useful for solving different problems. Standards will not cause two of the three to disappear or any of the other host of means of transferring files.

The number of platforms used to serve enterprise applications is continually changing as new platforms arise, e.g. Amazon, eBay and SalesForce are becoming increasingly important platforms for enterprises while PeopleSoft and JDEdwards have merged. This evolution will continue to happen and as such the range of connections required will continue to change and evolve.

The final argument for a heterogeneous world is the evolution of companies from small companies with limited IT resources and sophistication to large companies with sophisticated systems. In the real world they are all part of the value chain and companies do not select customers and partners solely by evaluating the sophistication of their IT infrastructure. While company size may be a factor in deciding marketing strategy it does not in the final days of a quarter determine sales.

These arguments clearly demonstrate that the value chain is and will remain heterogeneous. It may become a little more homogeneous as we improve standards and reduce the technology gap between the spectrum of large and small organizations but it will remain heterogeneous. Therefore to deliver on the vision of an agile value chain that reduces costs for everyone we need to consider how to bridge the technology gap between heterogeneous systems.

There are three possible approaches to bridging the gap:

  1. Enforce homogeneity by dictating that all parties in the value chain must use the same software, messaging protocols, security standards and levels of service.

  2. Support heterogeneity at the edge by having each party connect point to point and implement a broad set of technologies that are required by all their partners.

  3. Utilize shared infrastructure to broker the conversations and allow everyone to leverage costs that the shared infrastructure. Use the loosely coupled infrastructure to mediate the differences in connections, security and levels of service.

Enforcing homogeneity only works for large organizations where they are the “gorilla” in their value chain. This approach does not improve the cost structure of the company at the other end of the dictate. Unless the gorilla is their only customer they probably need to support multiple systems and the range of interactions the gorilla imposes on them is probably a limited set of the overall interactions they have with other organizations. The costs and the limited range of interactions delivered by this approach make it short term fix to gain market dominance by a gorilla at the expense of its suppliers and other companies that have similar value chains.

The approach of leaving it up to every organization to implement all the necessary infrastructure is probably the most cost prohibitive and approach that delivers the least amount of agility to the enterprise. The amount of costs inserted into the value chain is significantly. As a simple example consider the case of supporting MIME and DIME. These are two attachment formats used to attach non-XML data to SOAP messages. DIME is supported only by .Net while the majority of other toolkits supports MIME. For any company to support attachments in their external web service infrastructure they need to have a means to support both formats and also determine which partners are using which and ensuring that everything works seamlessly. This seams like a small task and it is the specifications, design, coding, management, testing and deployment should not account for more than 6 man months. This translates roughly to $50K, if there are 50 partners/customers in the value chain this equates to $2.5 million in direct costs added to the value chain just for this small feature. There is no resulting business value except that conversations can now happen seamlessly over SOAP. What happens when a partner insists on SOAP 1.2 support or REST rather than SOAP, or FTP? The costs of supporting this and all the other technologies quickly become a significant cost that is inserted into the value chain and provides no economic value in return.

The solution to this problem and that of forcing homogeneity on the value chain is to leverage shared infrastructure. In the example above the MIME – DIME conversion is done in the shared infrastructure as a service available to all participants. This means the cost of supporting MIME-DIME is fixed at $50K no matter how many participants there are using the shared infrastructure. This is true for all services provided, mediation is provided this allows every participant to connect using there own technologies and the infrastructure takes care of the mediation between parties. As the mediation is provided as a shared service it is extremely cost effective for all participants and reduces costs in the value chain dramatically compared to other approaches.

History has shown that the solution to many-many problems is shared infrastructure. As the value chain is a many to many problem and is going to stay that way there is a need for shared infrastructure for enabling integration to be as simple as plugging in a phone jack.