20 years on, open source hasn’t changed the world as promised
- 12 January, 2018 22:00
Open source has officially been a thing for 20 years now. Did anyone notice?
No, really. For something as revolutionary as open source, you’d think it would have changed the way all software is developed, sold, and distributed. Unfortunately for those party planners looking to celebrate the 20-year anniversary of open source, it hasn’t—changed software, that is. For most developers, most of the time, software remains stubbornly proprietary.
Open source has won infrastructure but not software
Back in 1999 Eric Raymond argued that 95 percent of software is written for use, not sale, and therefore could and should be open source. But it’s not; nearly all of that code remains closed today.
Ten years after the term “open source” was officially coined by the Open Source Initiative, an organization on whose board I used to sit, not much had changed, as Red Hat CEO Jim Whitehurst noted at the 2008 Red Hat Summit, decrying the waste of enterprise software:
The vast majority of software written today is written in enterprise and not for resale. And the vast majority of that is never actually used. The waste in IT software development is extraordinary. … Ultimately, for open source to provide value to all of our customers worldwide, we need to get our customers not only as users of open source products but truly engaged in open source and taking part in the development community.
Some observers saw more progress. According to a European Commission Flossmetrics study in 2009, 35 percent of all code (for sale or not) was open source. That’s a very generous estimate.
And, as Cloudera cofounder Mike Olson argued in 2013, open source had come to dominate enterprise infrastructure:
There’s been a stunning and irreversible trend in enterprise infrastructure. If you’re operating a datacenter, you’re almost certainly using an open source operating system, database, middleware, and other plumbing. No dominant platform-level software infrastructure has emerged in the last ten years in closed-source, proprietary form.
Olson, of course, is right: Much of the innovation in enterprise infrastructure is increasingly governed by an open source license. Although we’re still a ways from saturation, the container revolution is powered by Docker and Kubernetes, both open source. Big data? Hadoop, Kafka, and more open source technology sit under the hood. And new-school machine learning and AI? That’s driven by open source TensorFlow, MXNet, and more.
So, our platforms are increasingly open source even if our applications are stubbornly closed and proprietary. How can it be simultaneously true that much of our future depends on open source code even as the vast majority of code continues to be locked up in proprietary licensing?
If an increasing percentage of the best code is open, why wouldn’t much more of it go open, faster? As ARM’s John Mark Walker told me, “All the major innovations happening right now are with open source platforms,” and yet “there are still a lot of people … reinventing wheels.”
Enterprises don’t put their money where their open source mouth is
Geir Magnusson, an early Apache Software Foundation director, and CTO of Sourcepoint, answers this way:
The impact [of open source] has been huge for things that are nondifferentiating or infrastructure. But in that “95 percent of software” [that Eric Raymond calls out] is a lot of uninteresting glop that is purpose-built for (real or perceived) private/specific needs.
In other words, there’s a lot of code that remains closed, and we should be grateful we don’t have to see it, because it’s somewhat useless code beyond the enterprise where it’s written. Could it be open source? Yes. Should it? Well, …
It’s also true that there’s a very real cost associated with open-sourcing code, as Red Hat strategist Dave Neary highlights. “As a sole user” of that code, he argues, “the benefits are low.” Building on this, Apache Software Foundation director (and former senior director at Capital One) Jim Jagielski posits, “Companies say they want to embrace open source, but balk at the resources and investment required to adequately do so, so they fail.” This, in turn, “causes a ripple effect,” leading them to “blame open source, not themselves.”
In short, the reason most software remains locked up within the four walls of enterprise firewalls is that it’s too costly with too small of an ROI to justify open-sourcing it. At least, that’s the perception. Such a perception is impossible to break without walking the open source path, which companies are unwilling to walk without upfront proof. See the problem?
There’s hope for more open source in the next 20 years
This chicken-and-egg conundrum is starting to resolve itself, thanks to the forward-looking efforts of Google, Facebook, Amazon, and other web giants that are demonstrating the value of open-sourcing code. Although it’s unlikely that a State Farm or Chevron will ever participate in the same way as a Microsoft, we are starting to see companies like Bloomberg and Capital One get involved in open source in ways they never would have considered back when the term “open source” was coined in 1997, much less in 2007.
It’s a start.
Let’s also not forget that although we have seen companies use more open source code over the past 20 years, the biggest win for open source since its inception is how it has changed the narrative of how innovation happens in software. We’re starting to believe, and for good reason, that the best, most innovative software is open source.
Not for all software, of course. As Apache Software Foundation director and Adobe principal scientist Bertrand Delacretaz asserts, “Open source works best for infrastructure software.” It’s less likely to take over application software because, as he notes, “as you go up the layers [of the software stack] it’s harder to agree on things.” It’s also true that the population of developers with interest and aptitude in a given piece of software will shrink the higher up the stack you go.
But for that foundational software, the narrative is now that open source drives innovation. To the extent that enterprises are “reinventing infrastructure software wheels,” to paraphrase Walker, we’ll almost certainly see this cease over the next 20 years, with ever-rising levels of participation in open source communities.
This is what open source has taken 20 years to give us, and it’s a fantastic start for the next 20 years.