Programming, creativity and open source
There are people who think software development is devoid of creativity. Of course, anyone who has even a passing interest in development, or, say, has ever found him- or herself having a late night chat in a disreputable Sydney pub with a Drupal/Node.js developer, knows that this is untrue.
It's not just the programming process itself — breaking down a problem into its component parts and figuring out how to solve it in the most efficient (or least dangerous!) way possible — that involves creativity
Software development is also about understanding the kinds of tools and processes that people need — even if they don't know they need them — and designing a product that can meet them.
The wonderful thing about open source is that, thanks to the efforts of thousands of individuals over the decades since Richard Stallman founded the GNU movement, free tools (in both the beer and freedom senses) are available for developers to unleash their creativity.
I recently had the pleasure of interviewing Kaj de Vos, the lead developer of Syllable — an open source operating system that is based on neither Linux nor one of the BSDs. And the interview gives you a strong sense of what's possible when vision meets open source tools meets the blood, sweat and tears of people who contribute to projects like Syllable.
Below is Kaj's answer to a question about where the developers of Syllable feel that many modern operating systems have gone wrong. Expect to see the rest of the interview early next week. And, as you can guess from the answer below, expect it to be quite lengthy! But it is well worth reading.
As always, feel free to drop me a line at rohan_pearce at idg.com.au if you have a great project you're working on that you feel the world should know about. You can also find me as @rohan_p on Twitter.
Techworld: Just following on from that, do you think that there are areas where other current operating systems have gone off track?
Kaj de Vos: Don't get us started! In our view, it has been mostly downhill from [Amiga OS]. It is well known that the best technology usually does not win in the market.
The Amiga was a rare case where it did for some time because it was a pioneer market, and it probably gave many technologists false hope.
Due to organisational causes, large companies have huge difficulties developing new operating systems and other substantial innovations, which can be seen in the technology news every day.
Innovative and integrated designs like the Amiga and Syllable need to come from very small, very focused groups — especially software systems, because they are mental exercises.
These incubation groups are by definition not equipped to market the result. The Amiga was developed by such a small technology group that was saved from bankruptcy by the then-largest computer manufacturer in the world, who ran it in the ground in later years by squandering the technological advantage.
This scenario has played out again and again in similar cases. Only a few companies have found a balance between these conflicting requirements, and only for limited periods of time.
All we have are compromises, and I will freely admit that Syllable is one of them. For someone with deep technical understanding it is easy to point out what is wrong with current systems. It is just extremely difficult to do it better.
For a company, it is natural to shift the compromise towards better marketability. What we are trying to do is to shift the compromise towards better technology.
We think this can be made viable because the internet and open source has changed the relationships between companies and technologists. As an open source project on the internet, we can exist without the substantial support that used to be needed.
Companies can help, but we do not need them, so we can survive a company that would go off track trying to sell Syllable. The internet is also a marketing tool, so if our proposition of easy access to innovative technology is not enough to interest companies, we can still try to market it ourselves.
As for what is wrong with mainstream systems, many things can be listed, but in essence, most of them come down to excessive complexity and artificial limitations of capabilities.
Early systems used to be structured around functional capabilities, such as the tradition of modular command-line programs on Unix-like systems (which Syllable also supports).
More recent systems are structured around marketable units. As an example, you take a picture. To import it in your computer, you often need programs supplied by the manufacturer of your camera.
If you are lucky, you can circumvent these, but why should this be necessary? The software is not even available for any but the most mainstream systems.
On indoor pictures, you want to remove the 'red eye effect' caused by the flash. On outdoor pictures, you notice the horizon isn't straight and you would like to correct that.
These are common, but technically complicated manipulations on pictures. The correction of red eyes may be offered by multiple applications on your system. The straightening of horizons may require you to buy yet another image manipulation application.
Why can't you plug in the camera, have its icon appear on your desktop without extra software and click on it, then click on a picture and be offered one option to correct red eyes and one option to straighten a horizon?
Because commercial interests like to sell you these basic abilities over and over again in different, overly complex collections called applications. We are fine with companies charging for innovations, but the current situation impedes progress.
The Amiga ecosystem developed a solution for this more than two decades ago.
Similar to modular Unix command-line programs, many graphical Amiga programs had standardised interfaces that opened up their capabilities as building blocks that could be mixed and matched.
Besides greatly increasing flexibility and potential, such frameworks greatly reduce complexity by standardising on shared infrastructure.
Sadly, even though open source arose from the tradition of Unix building blocks, it has largely failed to develop this ability on user-oriented graphical systems.
It does exist to some extent in network systems, often because it was already there by building them on the modular Unix system.
This modular design arose in the small technology lab that incubated the Unix system four decades ago. It has never been modernised or succeeded by an encompassing new modular system after Unix left the research lab and was further developed by multiple companies and the open source community.
Graphical open source programs tend to mimic the commercial programs they are trying to replace, and have grown equally complex.
Above the basic, well designed Unix system layer, open source systems have fragmented into a kaleidoscope of competing and experimental graphical subsystems and application platforms. Their mantra is that choice and competition are good.
That all sounds well and good, but our problem with it is that none of these competing choices have even matched the quality of the integrated systems that we admire.
This lack of integration creates artificial borders between software and actually blocks proper competition between software functionality. We continuously end up with two annoyingly different programs that can remove red eyes and zero programs that can straighten horizons, and never the twain shall meet.
Something gives us hope, though. We cannot fix closed commercial systems, because we cannot modify them. Open source, however, thrives on alterations.
We design our own modular frameworks and modify existing open source software to plug into them as building blocks.
It is still a compromise, because we invite complexity by not building everything ourselves. But that would be too perfect, because it would take too long.
However it does mean that we are fighting a war on two fronts, if you will. Commercial interests, once established, resist standardisation and modularisation because they level the playing field with competitors.
In our experience, volunteer open source developers often really do the same thing, though out of 'Not Invented Here' syndrome instead of commercial considerations.