Technology Musings
by John McDowall


Friday, September 20, 2002  

Information should be location Independent - Part II Enterprise



I had been thinking over this issue and a headline on Slashdot brought a lot of focus to my thoughts: Should Open Source Content Management Inter-operate?. This is an oxymoron similar to should network computers inter-operate, I would go further and drop the "Open Source" from the headline. This illustrates how badly we have screwed up the whole information model when this question even needs to be asked!

We have developed the compunction to store everything away in transactional silos, I have been to a couple of conferences in the last week where the issues around enterprise integration were discussed. Everyone, myself included, got caught up in how to integrate the silos we did not ask why they were there in the first place. We are turning them into an institution by not questioning why they even exist - we are taking the attitude, that silos are natural and the way to solve for them is to develop complex integration software rather than removing the need.

Nearly all data should be secure and stored reliably and not subject to corruption, but does it need full ACID level integrity - my bank account sure, purchase orders and other accounting information, but not too much else. Lets think of a model where everything is stored reliably, securely and with user level locking, more ACID level functions should be properties of the data not a fundamentally new way of storing information.

One of the major innovations of XML is schema (including namespaces) that separate the description of the data from the data. This allows the data to be stored efficiently and multiple data models to be built separately from the data, if necessary rather than the current relational model where the schema is an integral part of the data design and must be done before the data can be stored. Today search engines can index almost a billion pages, with minimal human intervention, why do I need to design a schema every time I want to use a database. The biggest database of them all is the web and is was definitely not designed but finding information is fairly efficient, in a local intranet it is very efficient as all information can be indexed, whereas I think we are still trying to index the whole web.

We appear to be digging ourselves into a bigger and bigger hole by making the technology more complex and assuming that silos are natural - it is time for information to be pervasive and accessible!

posted by John McDowall | 10:58 PM


Friday, September 06, 2002  

Information should be location Independent - Part I Desktop


Or the death of C:\ and its sibling /usr/home


Once a user gets above a few hundred files on their desktop (unless they are very organized), finding files again becomes a major chore. I believe that the problem is that the physical storage of the information is needlessly exposed to the user. We do not think about our information as having a particular location but rather containing a particular set of information. Given the strangle hold on the industry by a few major players, who have no real vested interest in changing status quo we are faced with a problem.

OpenOffice and similar projects attempt to unseat the reigning monopoly by emulating the features of the leading desktop office product. You do not catch up with a leader by playing by their rules - you need to change the game and the rules to successfully unseat the leader. Office products have not changed in 10 years - more bells and whistles but they are still the same painful products to use they were 10 years ago and they do not enhance the abilities of the user. What has this to do with location independent information: an opportunity to change the rules and deliver a new way of working with information that will deliver real value to users and differentiate open source office products from the stagnant monopolies of the past.

OpenOffice has only a few steps to go to provide the necessary infra-structure for the change - it already saves files as XML. From XML to RDF is a small step. The two more significant steps are to save information through an engine similar to a search engine that will extract contextual information and then what is the interface to ask for information? An interesting example is Kartoo while a little immature is a good example of what a new way of looking at information could be. Several years ago there was a company called Perspecta which had a similar idea, I always thought they had something, I think I now see what...

Extending this idea the engine becomes a personal research tool, it knows what you want and can be directed to external resources to find related information, either within the enterprise or outside to other public or private resources. Am I describing the Semantic Web, a new Office suite, or Mozilla with a rich set of plugins; only time will tell but I know this will come...




posted by John McDowall | 10:13 PM
 

Information should be location independent


We have coupled location too tightly with information, i.e. if saving a file requires saving it to a location not to a context. The context is how we think about the information not about some abstract location imposed by an operating system. Currently we are confusing how information is saved and its context. What is missing is the layer of abstraction that separates the physical storage from the contextual information about the file.

As information has been tightly coupled to its location so user interfaces have propagated this error. Some of the problem has been lack of processing power to allow the abstraction to be implemented. This is only a partial reason as Apple showed with the Macintosh. At that time creating a graphical user interface used a substantial part of the available computing resources, but the result was such a major leap in usability people were willing to have less processing power available.

One the first companies to talk about the idea of location independent information was Tibco, one of the basic concepts for the information bus was that users of the bus just requested information, they were not required to know its location just its context, similarly for putting information onto the bus. This simplicity has been lost, mainly ((IMHO) because we are assuming that exposing the file system is a fundamental building block rather than a mistake.

The web and now the semantic web is a sincere attempt to transform this problem and create a new way of looking at information. The major step that needs to be taken is when the interface into information becomes similar to the concept of the information bus, i.e. I save it to the bus and request it from the not by its location but by the information context. A part of the context should be how transactional interactions should be and the level of security (also perhaps the certified identity of the creator - the value of the information is in many ways directly proportional to the level of trust and respect the consumer has for the producer.).

I have started two additional notes on what and how to attack this problem on both the desktop and the enterprise - stay tuned

posted by John McDowall | 9:54 PM


Monday, September 02, 2002  

Creating value and is EAI a dead end...


Our industry is rife with stories of major technology investments gone wrong, and most of us have first hand experience with them. There is usually enough blame and recriminations to go round, but are we perhaps just expecting too much out of people rather than blaming the software, implementation etc. In the past several years there has been a push to develop software in "internet time", this was a concept I fought against (and in most cases lost...) as developing good software takes time. What the internet provided was a faster delivery mechanism and feedback system, i.e. you could deliver poor software faster to customers and hear about how bad it was in real-time - not a great model.

I believe that tools are only as good as the people that use them, a great IDE does not make a poor developer a good developer. Similarly deploying complex enterprise software and then expecting the users to suddenly understand their business better and hence deliver value is almost equivalent to giving an average person a complete set of power tools and expect them to create a high quality custom home. We need to move away from mentality that clever software will fix all problems, software is a tool and for it to be really effective it needs to be used by well trained people. (Of course poorly designed software helps no one and this is not meant to be an excuse for poorly designed software.)

Currently so much effort goes into the design of the software and systems and so little into the design of the human systems required to make it successful. This is not to suggest that we should be designing software with very simplistic interfaces, but rather we should match the software with the people using it and view them as a complete system to be optimized and designed to deliver business value.

Deploying enterprise software and in fact any software should be considered as deploying a competitive weapon and as such the better people are trained to use it the more effective they will be. The time a team takes to become facile in any new tool is dependant on the amount of change from existing approaches, the learning capacity of the team, and the ability of the team to wok together. The bigger the distance to be crossed by the team the greater the possibility of failure or perhaps even worse; ending up at a different destination. This is what I think makes traditional EAI projects such high risk and why EAI is doomed to fail as a technology category. Enterprises will integrate systems and use this as a competitive weapon but only if they approach it a step at a time. To succeed requires tremendous focus and commitment to the team that is going to use the technology to achieve a new set of business goals.

What is lacking from traditional EAI approaches is the focus on the business team and an incremental approach to developing success, both the software vendors and the system integrators are not orientated to helping companies achieve incremental successes as they are looking for large projects and as such tend to over promise and under deliver. The reason for under delivering is that an organization can only absorb so much new technology, processes and information at a time. Incremental delivery allows the organization to change and adapt to the new information and processes and use these new tools to succeed.

Until we start putting organizational scalability before systems scalability we are not going to deliver on the potential of enterprise software.

posted by John McDowall | 4:22 PM


Wednesday, August 28, 2002  

Are we done, is this all....


Looking around at our industry I see very little new innovation, (maybe this is a sign of advancing years ;-)). This to me is not a sign of that we have invented everything and we should just go into maintenance mode and put our collective feet on the desk. It is not akin to the head of the US Patent Office who at the end of the 19th Century proposed closing the office as everything worthwhile had already been invented.

I think this is more of a pause where we collect our creative energies for the next burst of innovation. I strongly believe that this will happen, the pressures are building and the groundwork is being laid. The pressures I see are that the information revolution has become an accepted fact but it is being held back by the innate complexity of todays architecture. This is both true in the consumer and enterprise space. The major innovation of the desktop is creating an open source office product, in the enterprise we are creating web services which are just an open systems approach to existing integration technologies. In both arenas it is becoming too hard to go beyond where we are today. This is the pressure for change, when there is an opportunity to dramatically change the landscape no amount of monopoly power or inertia can hold it back.

I discussed this with a friend recently and he commented that Scotty in StarTrek never had to mount/search disk drives or deal with arcane information retrieval systems to get to information. So where do we go from here.....

posted by John McDowall | 8:43 PM


Friday, August 16, 2002  

When are you a Guru ?


So I just got one of those annoying headhunter calls offering me two outstanding web services "gurus". The first question that leapt to my mind was how can anyone be a guru in a technology that is less than two years old? Do I need to address engineers I work with as guru now as most have been working on web services since mid to late 2000? (The answers I got fell into two catagories, unprintable and "does that mean I get free beer?"). I actually think I like my gurus well rounded (like the Budda) and have them be deep technologists with the ability to learn new technologies quickly, like web services.

So if any of you fast talking recuiters read this - do not send me web services gurus (or whatever the next next cool thing is), send me bright, motivated and articulated people want to build the future.....

posted by John McDowall | 1:42 PM


Thursday, August 15, 2002  

From Blogs to subscription software..


So I was using Radio to publish this Blog but was pretty unhappy with it for several reasons

  • Hard to use - configuration was a major pain
  • Complex user interface
  • Can only publish from one location i.e. cannot publish from any computer

So I have now moved to Blogger and it is way cool - simple to use and configure. Ability to log into the blog interface from any location and
publish away - you would think the people at Userland forgot what the web was for.

In many ways the differences between the two programs illustrate one of the problems with our whole industry - feature creep!
Is it necessary to continually add features to protect and preserve a the revenue stream. I am more likely to pay for a new version of word
that removes features and makes it easier to use. In fact I am moving to Linux for most of my work including desktop applications as I
am really tired of paying Microsoft for new versions of Office. I am not sure if I really want to calculate how often I have paid for Office over the past 15 years, at teh same time I think I have used the same feature set. Are you a better communicator if you use transition effects in PowerPoint or format your word document in innovative ways?

Subscription softwares day has arrived - I am going to pay Blogger a yearly fee to keep using the service to publish this blog and I will be very happy (until they make the interface to complex ...). I pay an annual subscription to keep my Linux machines up to date and I am very happy to do so. On the enterprise front I think SalesForce.com is going to make companies such as
Siebel obsolete in the long term. This is happening because there is a well defined set of features that are required in a Sales application and if you capture them and make it easy to use and deploy why would you do anything else.

So is the next wave of applications updates going to boast of fewer features and ease of use - unfortunately not but the team at OpenOffice and Xandros might think about this as a differentiator....

posted by John McDowall | 5:52 PM


Wednesday, August 14, 2002  

Looking at problems in a
new way.


Project Oxygen at MIT is looking at human
centered computing, this is a good place to start
but I would ask the question should we not think
about throwing away all existing paradims?  I
am currently working through (slowly) Stephen
Wolfram's "A New Kind of Science" one of the most
interesting things so far is that he has approoch
the problem of understanding complex systems from
a completely fresh point of view. I cannot tell if
he is right or wrong, I will leave that to time
and people smarter than I. the major contribution
I see so far is that he has challenged
preconveived notions of how things should be.


So is Project Oxygen looking at things new or
just re-packaging what we have, while I admire
their goals, I cannot help but think that they are
starting with a broken platform and they actually
need to go deeper into the fabric of how we design
computer systems.



Take Seti@Home as an example, they have been able to use millions
of hours of shared computer time, but
unfortunately very few problems are ameanable to
this type of analysis. There are several grid
computing and peer to peer systems today that try
and attack the shared computing issue but they are
struggling, because, IMHO they are constrained by
the fundamental architecture we have built.


I think this is where Stephen Wolfram has got
(at least the approach right), instead of beating
your head against a wall trying to brute force a
solution (I used to work in fluid dynamics so I
understand a little of the issues surrounding the
complexity of modelling), take several steps back
and try and find another approach.


So what would be a new apporach to the
computing problem, and where will it come
from?

posted by John McDowall | 10:31 PM
 

The other side of the
problem....


If we change the information model, moving from
a centralized database model to a more distributed
model how do we handle persistant and recovery.
Very few systems are set up to have the same
degree of backup and recovery as relational
databases, the flip side is the need to centralize
systems due to management complexity.


How to change the information model and yet
still provide a high degree to reliability and
accessibility?

posted by John McDowall | 10:31 PM
 

Why are we
limited....


To only what we can access directly, the web
opens up access to all sorts of information, but
it is still a fetch model rather than a push
model. Subscribing to RSS channels is better but
the channel list and information is still manually
selected. There have been several attempts to
create agents that live on the desktop to learn
your interests and then go get information for you
but they have been limited for several reasons,



  • limited to one machine - I want my agent to
    move with me (Radio has this problem - I want my
    Blog editor to be mobile)
  • Unable to differentiate between various
    aspects of life, personal, hobbies, and work
  • Need to interact with me and get guidance

So how do we turn our information models into
push rather than pull?


Information does not want to be free it just
needs to be understood!

posted by John McDowall | 10:30 PM
 

Information
models.....


If the problem with computers is that the
information model is wrong, by this a mean is that
it is too complex to access and use information. A
good example may be the current problems the FBI
and CIA have had relating information, a lot
existed but all the relative information was never
put together or even a fraction of the relevant
information was assembled.


Is one of the problems we are trying to ensure
a perfect solution rather than get it close enough
to get the job done. Designing information models
to be totally predicatable i.e. put them in a
static store, typically a relational database
takes too long and by the time it is done the
information is out of date. There was an article
on Slashdot, refering to a new search engine
Kartoo.
While several other efforts have tried this
approach and I am not sure this one will be
successful it does point the direction to a new
way of finding and navigating information. What
kinds of user interfaces could be buiilt on this
model?

posted by John McDowall | 10:30 PM
 

How to get
there..


Like any major change in technology it will
probably require a fundamental shift and a
dramatic new idea that will be simple and widely
available. There was an interesting article in the
Economist this week regarding innovation in a
capitalistic system. One of the key points is that
in a free market economy, the benifits of
innovation are widely dispersed, so it is not only
the originator of the new technology that benifits
but society as a whole.


It is intesting to consider what pressures
cause that spark and if we are there yet, 
for the economic benefit to reach deep into
society, of the information age, it must be easily
accessible to everyone. Today hardware and
software is uniformly hard to use, the browser has
helped a lot as it has actually forced designers
to create simple software that does not require a
great deal of knowledge to use - this does not
solve the problem as most of use using Radio (with
great respect to the people at UserLand)
configuring it is not user friendly.


The problem goes much deeper than the user
interface, it does to the fundamental information
model our software is based on. In turn this may
be influenced by the current computer
architecture.


What is the nature of the information model
that makes software so hard?

posted by John McDowall | 10:29 PM
 

If I could just access the resources when I
needed them...


I would just be able to access the computing
resources I needed, without having the need to
manage and maintain a computer. How would the
economics of this work, is it even possible.
Similar to network storage models the economics
are not possible as the equivalent of Moore's Law
for storage costs makes storage cheaper faster
than the capital costs can be recovered in usage
fees.


What would this model look
like....

posted by John McDowall | 10:29 PM
 

If I did not have a
computer....


but had access to computing resources the same
way as I have access to the phone network and the
power grid, would life be better? Not sure there
are many arguments that can be made that the power
grid is too centralized, I think the power grid
will start to evolve when cheap fuel cells (or at
least some form of cheap and efficient energy
storage). Computers on the other hand are too
distributed and hence the costs of management are
too expensive and complicated. This may sound like
a rant for the return of thin computing but it is
not - if I start a rant I will clearly mark it
with <rant></rant> tags :-).


Very much like the ASP market some good ideas
were masked by many bad implementations. Taking
off the shelf enterprise software and turning it
into an ASP offering IMHO was very dumb, it was
not designed to do that and the only cost savings
that could be acheived was install and
maintenance, additional costs were added by the
need to have secure connections to a remote
facility, which also added latency and bandwidth
costs. ASP's like SalesForce on the other hand
developed applications that were tailored to the
market and are becoming quite successful.


So back to computers - thin client computing
was not successful for several reasons, I think
mainly it was an idea not only before its time but
one that lacked the appropriate software
infrastructure to support it.

posted by John McDowall | 10:28 PM
 

Are Computers too hard to use...


Almost a redundant question, we would all
probably say yes but for many different
reasons.  It has been really eye opening to
me seeing my various family members struggle to
connect to the internet and exchange information.
I have started questioning all the activities I
was doing to maintain and run my own
computers.


Should we start not as engineers but rather as
niave users - for most of us this is very hard, I
wrote my first program in Algol in 1975, since
then I have programmed almost in every language
and operating system, ( I missed Cobol and
mainframes....). Living through the problems of
smart but technologically unsophisticated users
brings home the problems of getting connected. Are
we trying to do too much, very like the VCR - too
many functions, with a user interface that
requires continued reference to the manual. As
engineers we are driven by adding more bells and
whistles rather than driving to simplicity. Why
was the Palm so successful - not the breadth of
features but for its simplicity. Unfortunately the
handheld market is heading in the same direction,
people are staring to compete in terms of features
not in terms of simplity.


So should we be thinking of a new design
approach to defining and delivering
solutions....

posted by John McDowall | 10:27 PM
links
archives