A world without the Wintel Hegemony.

Today, while chatting with a friend online, as is our wont, about the future of computing, my friend said something very interesting, very deep and, in my opinion, very wrong:

me: I say NRAM is way out there because any memory system that a) is mechanical and b) relies on the Van der Waals force for operation is just... wrong.
friend: I kind of like the idea
me: Oh, I like it too. It's just... wrong. It's a MECHANICAL SWITCH! At nano scales.
friend: combine that with some of the new nano-scale optical switching stuff and you've got potential for some radically different tech than today ... but it'll have to support an x86 instruction set :-)

It's this last point I disagree with him on.

I think the future is not a continuation of this current world of the Wintel Hegemony. I think the reason we have a Wintel Hegemony is largely one of history but that there are several forces which are now impacting us that will end said hegemony. Thankfully my friend was too busy working on really crappy Java code to be able to continue the discussion at that point, so I've been given a chance to put all my thoughts into order.

Here are my reasons for thinking we're seeing the beginning of the end of the Hegemony.

First, there is the issue of security. As computers become ever more ubiquitous and interconnected, we are reaping what we sowed in building a computing ecosphere with little to no diversity.

In a biological ecosystem, diversity is the key to survival because, while, say, disease or invading species or the like can wipe out a few species, it's unlikely that it could occupy every niche taken by every species in a proper, natural ecosphere (Australia notwithstanding). One of the concerns that ecologists have with the continued spread and development of human habitats is the reduction of biodiversity. Complex ecospheres are, in effect, being turned into monocultures and, as a result, being rendered highly fragile.

Computing outside of the embedded world is such a monoculture and is, as a result, incredibly fragile. Because everything out there, for all practical purposes, is Wintel, attackers who break Wintel through any of a myriad of ways have, in effect, 90% of the world's computing horsepower at their disposal. These computers, too, are increasingly interconnected, so when threats are released into the wild they can cause huge amounts of damage that impact all networked computers in the world even if they don't happen to be in the set of "species" under threat. This monoculture has to go, and more and more people are recognizing this. As a result I think the inevitable trend will be toward a larger "biodiversity" of computing.

(Incidentally, before smug Linux or Mac types feel too good about themselves: the same thing would happen—I'm aware of the arguments to the contrary and laugh at them—were the monoculture Linux or MacOS or even z/OS. No system is impregnable and if you have a huge monoculture you have all the attacks going on that single point.)

Now, of course, there's a reason why this monoculture developed and why it has continued for so long in the face of an ever-increasing security threat. This has to do with the harsh realities of commercial software development. The fact is that software is hard to write, hard to maintain and hard to test. And by "hard", of course, I mean "expensive". It was pretty much inevitable that a monoculture would develop. All it would take is for one platform to attain a lead and a monoculture is almost guaranteed as the final outcome.

The reason for this is simple. Let's say we have a culture where Wintel is 60% of the market and everybody else is spread evenly over the remaining 40%. You make, say, a statistical analysis package. 60% of your hypothetical userbase is going to be Windows-on-Intel. Obviously you will develop, package and test for that platform. Now let's say 20% is MacOS-on-PowerPC. The cost of developing, packaging and testing for that platform will be roughly equal to the cost of the Wintel platform. For one third the customers. You face two choices at this point: you can charge your Mac customers more to keep reasonable profit margins or you can take lower profits (and possibly even losses) to support the platform. The first makes you unpopular with that platform's users (I've heard lots of Mac people whining about how the "same" application is more expensive on Macs!); the second makes you very unpopular with your shareholders.

Inevitably, once you have a single, clear dominant platform, the nature of commercial software development leads straight down the slope to a hegemony. This can't be avoided from basic economic necessity.

The world is changing, however, both for good and for bad, with the rise of Free/Open Source Software; and with that change comes a huge change in the economics of software development. Since the source is open, we no longer rely upon the business decisions of a company that jealously guards its source code for support. Instead minority platforms can do the necessary steps to port software by themselves, thus permitting more brains to think on the problem and more platforms, as a result, getting supported.

Comparing two comparable products – Windows NT and Linux – gives a good taste of the radically different nature of open source development. When Windows NT was first developed and released, it supported i860, x86, Alpha and MIPS platforms. It also quickly developed PowerPC and even Sparc and Clipper support. (Not all of these were commercially available.) As new versions came out, however, supported platforms were pared back until today the NT kernel (XP and Vista) supports x86 or x86-64 platforms only. It was just not commercially viable to support so many platforms.

Linux went in exactly the opposite direction. When it first came out it was for the i386/486 only. Indeed, if memory serves, Linus Torvalds even claimed that it was unlikely that it could be ported to other architectures. Yet today Linux has been ported to such a bewildering variety of architectures that I'm not even going to try to list them all. It can be found on tiny embedded devices and it can be found on massive IBM z-series mainframes and on everything in between, practically.

This increasing trend toward open source is one piece of the puzzle that spells the end of the Wintel Hegemony, but it is not the final word. We still have the issue that porting software from one platform to another is a pain and requires, often, quite significant (and frequently absent) technical know-how. This is where my final piece of the puzzle comes into play: virtual machines.

I first stumbled over the idea of a virtual machine (without knowing it was called that) with the old p-Systems and, if you squint right, with Forth and other such threaded interpreted languages. I saw its potential, however, first when I stumbled over the extremely obscure Taos operating system.

(That's right: virtual machines were not invented by Sun for Java. They're actually very old technologies that have just foud a renaissance!)

The idea of a VM is the oft-mocked mantra from Java's early days: Write Once, Run Everywhere. (The obvious rejoinder very quickly became "Write Once, Test Everywhere" because of some ... issues.) The JVM is not the right vehicle, however, for the future because it's too tied to Java. Microsoft's CLR is an improvement, but still has issues of its own in terms of language support, not to mention its close ties with an entity not noted for playing well with others. Still the day of the virtual machine has arrived and there's lots of research (like Parrot or LLVM) happening in this vibrant field.

VMs have a lot of things going for them once implemented properly, especially in combination with things like open source development. A well-designed VM can allow applications to be "ported" almost trivially – ideally with no work at all – from platform to platform. Low level systems programming will always have to be there, of course, to support devices and, of course, to implement and optimize whichever VM(s) are being used. Since, however, the overwhelming majority of applications shouldn't be concerned with low level details at all, this is not as big a barrier as is trying to port, say, C++ code across disparate architectures. Performance is not really an issue since modern JIT compilers can optimize almost as good as native code compilers (and, indeed, projects like LLVM show promise in exceeding native compilers in optimization across platforms!). As the technology improves and expands, I forsee the day when VMs are the norm for applications, not the exception. And with that day comes the day where the underlying platform of hardware and OS becomes increasingly irrelevant.

So this is it. In a nutshell there are three forces conspiring to force an end to the Wintel Hegemony:

1. security needs demand that we diversify our computing ecosystem before a true disaster strikes computing dead;
2. the rapid expansion of the open source development model is allowing us to move away from an economic system that favours monoculture to one that supports diversity;
3. the increased use of virtual machines, combined with the huge strides being made in rendering them plausible delivery platforms for real-world software, renders the underlying combination of operating sytem and hardware increasingly moot even for closed source software (but also suited to open source software).

I expect to see much of this change happening in my lifetime.

RWEB 0.2.0

RWEB has been updated to v0.2.0. The internals have been completely reorganized, but nothing has changed on the user interface front yet. Newly added is XHTML support with automatic detection of the syntax library for transparent syntax highlighting of woven code. The project home page has more detail including documentation and file downloads.

I've been advised by the developer of the syntax library that he has ceased work on it. He recommended a different library for use—one he says is more stable and complete—so the syntax library requirement may change in the near future.

RWEB 0.1.0

RWEB is a literate programming environment for Ruby. (Much of it is language-agnostic, but its focus is Ruby.) It is patterned somewhat after Donald Knuth's WEB literate programming tool, as well as its descendants, but has a couple of added twists in planning. When it is complete it will feature:

• a utility, rtangle, which "tangles" an RWEB document to produce executable code in the format expected by the Ruby interpreter;
• a second utility, rweave, which "weaves" an RWEB document into human-readable text in a variety of back-end formats (plain text and XHTML planned for the moment, but other formats are likely as well as a user plug-in system for end-user customization);
• a facility to allow RWEB documents to be directly executed by the Ruby interpreter as if they were native Ruby code.

As of version 0.1.0's release, all three features are in place in "proof-of-concept" condition. The self-executing RWEB feature is working and unlikely to change. The tangling code, too, is complete and unlikely to visibly change. The weaving code is in place for plain text formatting only with an XHTML interface in place but incomplete and stubbed out. The back-end plug-in system is not even vaguely in place yet.

RWEB 0.1.0 is available for download from RubyForge and the documentation too.

For a brief taste of what RWEB can do at the moment, consider this simple, but complete, RWEB document, broken down for discussion.

#! /usr/bin/ruby -w
require 'rubygems'
require 'rweb'
eval RWEB.tangle(DATA)
__END__

This header of boilerplate is the trick that allows self-executing RWEB documents. It's a cheap trick, really, that just requires the RWEB library and then evaluates the string returned by tangling the special variable DATA. Since DATA is an IO object that contains everything after the __END__ directive, it means that it tangles and executes everything after __END__.

{style => Plain}
{title => RWEB Demo}

This portion is a set of directives. We're telling the weaver that the document style is plain text (the only one currently supported, although XHTML is syntactically supported) and the title of the document. Everything thereafter is the actual RWEB document.

This file contains a demonstration of the RWEB literate programming system for
Ruby. It is not held up as an example of good style, but only as an example of
the system's capabilities.

First we open up the mainline code. Note how the tag for opening the mainline
code has no name. Every RWEB document must have such a mainline "chunk". We
begin with just some requires.

<< {
require 'rubygems'
require 'rio'
}>>

Now this program uses the RIO library for no good reason. Let's define a
function that looks for RWEB documents and tangles them into Ruby scripts if
they haven't already been tangled. (We'll ignore dates for now.) It will also
weave documents for these.

<< utility functions {

{<<get rweb documents>>}

{<<get ruby scripts>>}

{<<find the differences>>}

}>>

Note that the contents of this chunk doesn't look at all like Ruby code. It is,
in fact, Ruby code, but only if the chunk references (contained in the {<<>>}
tags) are expanded. The blocks in question have not yet been defined, but this
isn't a problem. Chunks can be defined in any order. Only circularity is grounds
for complaint.

The functions for getting the rweb documents and ruby scripts are simple:

<< get rweb documents {
def rweb_documents
  rio('.').files['*.rweb'].collect{|r| r.basename}
end
}>>

<< get ruby scripts {
def ruby_scripts
  rio('.').files['*.rb'].collect{|r| r.basename}
end
}>>

The name of these chunks is inserted between the "<<" and the "{" tokens. Before
use the names have any leading and trailing whitespace truncated. This is for
purposes of readability.

Now let's take a look at comparing the two arrays:

<< find the differences {
def find_differences
  rweb_documents - ruby_scripts
end
}>>

Now here we're going to re-open the mainline chunk. All this does is append any
lines in the re-opening to the alread-existing lines of the chunk. This re-
opening capability, combined with forward references, gives us lots of leeway
for structuring code and documentation for maximal comprehension.

<< {
{<<utility functions>>}

find_differences().each do |file|
  system "rtangle #{file}.rweb #{file}.rb"
  system "rweave #{file}.rweb #{file}.txt"
end
}>>

The rtangle output on this RWEB document looks like this:

# RWEB Demo
# =========

require 'rubygems'
require 'rio'
# utility functions
# -----------------

# get rweb documents
# ------------------
def rweb_documents
  rio('.').files['*.rweb'].collect{|r| r.basename}
end

# get ruby scripts
# ----------------
def ruby_scripts
  rio('.').files['*.rb'].collect{|r| r.basename}
end

# find the differences
# --------------------
def find_differences
  rweb_documents - ruby_scripts
end


find_differences().each do |file|
  system "rtangle #{file}.rweb #{file}.rb"
  system "rweave #{file}.rweb #{file}.txt"
end

The output of rweave looks like this:

RWEB Demo
=========
This file contains a demonstration of the RWEB literate programming system for
Ruby. It is not held up as an example of good style, but only as an example of
the system's capabilities.

First we open up the mainline code. Note how the tag for opening the mainline
code has no name. Every RWEB document must have such a mainline "chunk". We
begin with just some requires.

RWEB Mainline
-------------
> require 'rubygems'
> require 'rio'

Now this program uses the RIO library for no good reason. Let's define a
function that looks for RWEB documents and tangles them into Ruby scripts if
they haven't already been tangled. (We'll ignore dates for now.) It will also
weave documents for these.

utility functions
-----------------
> 
> {<<get rweb documents>>}
> 
> {<<get ruby scripts>>}
> 
> {<<find the differences>>}
> 

Note that the contents of this chunk doesn't look at all like Ruby code. It is,
in fact, Ruby code, but only if the chunk references (contained in the {<<>>}
tags) are expanded. The blocks in question have not yet been defined, but this
isn't a problem. Chunks can be defined in any order. Only circularity is grounds
for complaint.

The functions for getting the rweb documents and ruby scripts are simple:

get rweb documents
------------------
> def rweb_documents
>   rio('.').files['*.rweb'].collect{|r| r.basename}
> end

get ruby scripts
----------------
> def ruby_scripts
>   rio('.').files['*.rb'].collect{|r| r.basename}
> end

The name of these chunks is inserted between the "<<" and the "{" tokens. Before
use the names have any leading and trailing whitespace truncated. This is for
purposes of readability.

Now let's take a look at comparing the two arrays:

find the differences
--------------------
> def find_differences
>   rweb_documents - ruby_scripts
> end

Now here we're going to re-open the mainline chunk. All this does is append any
lines in the re-opening to the alread-existing lines of the chunk. This re-
opening capability, combined with forward references, gives us lots of leeway
for structuring code and documentation for maximal comprehension.

RWEB Mainline
-------------
> {<<utility functions>>}
> 
> find_differences().each do |file|
>   system "rtangle #{file}.rweb #{file}.rb"
>   system "rweave #{file}.rweb #{file}.txt"
> end

This plain text version isn't perfect -- especially for, say, blogging, but it is a whole lot better than raw code ever will be. Too, as back ends get plugged in (XHTML is next on the queue), the documentation capabilities will be far superior to any other tool for documentation including even things like RDOC (or JavaDoc/Haddock/Doxygen) which use code comments to cook up documentation.