Amazon vs Elasticsearch and the end of open source

Original note from: MuyLinux
To read the original in Spanish by J Pomeyrol, click here

For years we have talked about the new challenges for open source that the cloud poses, a term that for the common user can mean different things, but that for the company is summarized in services. And it is that the economic benefits generated by open source software are not comparable to those obtained when the same software is offered through services, beyond - but including - support.

This differential fact has been causing friction between developers and providers for some time and there are those who are even anticipating the end of the open source development model as we know it. Who is right? Is the situation so bad? The matter is complex, despite the fact that only the voice of one party is collected: that of the developers.

In their presentations, representatives of companies and open source projects that are very popular in the business world, explain the alleged damage caused by the use that large providers of cloud services make of the software they develop, and how some have considered and applied a more closed approach to their products in order to avoid what they call ‘looting’. There are statements that deserve to be rescued to provide context to the discussion:

“The role that open source plays in creating business opportunities has changed. I would put it in a very forceful way: for many years we were fools and we allowed them to take what we had developed and make tons of money from it. Abby Kearns, CEO of the Cloud Foundry Foundation.

“Amazon Web Services has made hundreds of millions of dollars offering Redis to its customers without contributing as much to the open source community that builds and maintains that project. It is impossible to know exactly how much money we are talking about, but it is true that AWS and other cloud providers benefit from the work of open source developers that they do not employ. Ofer Bengal, CEO of Redis Labs.

“There is a widely held myth in the open source world that projects are powered by a community of contributors, but in reality, paid developers contribute most of the code in most modern open source projects” . Luke Kanies, founder of Puppet.

“99% percent of contributions to Redis were made by our company, Redis Labs.” Dev Ittycheria, CEO of MongoDB.

Open source and cloud services

In short, all these voices complain about two things: the huge benefits that cloud service providers get from their software without paying them accordingly, and the lack of collaboration keeping the products that profit. However, make no mistake, the crux of the matter is primarily money. The most lapidary phrase in this regard was that of Salil Deshpande, managing director of Bain Capital Ventures and investor of Redis Labs:

“Our view is that open source software was never intended for cloud service companies to take and sell.”

All these statements date back to 2018, although the trauma came from before. But it was then that several of these companies rose up on the warpath, threatening changes in their licenses that would put an end to the abuse they were denouncing. The reflection I made at that time was very basic: «whether you are right or not, which remains to be seen, how do you think the giant you threaten to turn off the tap of open source software on which it bases its business will react? ? »

On the other hand, «if it is possible to bifucate a free project that is closing, wouldn’t it have been better to collaborate with it earlier and have avoided closing? If that which gives benefits is not invested and maintained in health, it may end up disappearing, ”the article concluded.

Already in 2019, the effects of this struggle began to be felt, and the best example to illustrate this is that of MongoDB, whose new license was rejected outright not only by the main cloud service providers: also by the large solution distributors. software, read Red Hat and others. For a simple reason: open source has to meet certain requirements to be considered as such.

That same year, two of the large providers took positions: Google signed a collaboration agreement with seven open source companies, including Elastic, Redis Labs or MongoDB, with the intention of integrating their software into their services, "deriving the support and distributing the profits, it is understood ». Amazon, for its part, stood before what it understood as a kind of blackmail, warning that “open source must remain open, otherwise it loses all confidence.”

It is not the first, nor will it surely be the last time, that the leader in Internet services reacts like this: it did so with Java a while before and it is also worth remembering the case of Android, a system that governs several of its commercial devices. Following the code of the Android Open Source Project and bypassing Google’s hoop and the requirements it imposes on manufacturers of “certified” devices, Amazon offers “its own” Android.

Amazon vs Elasticsearch is not David vs Goliath

But if there is a recent struggle that serves to shed some light on this issue, it is the one between Amazon and Elastic (Elasticsearch BV), developer of the powerful Elasticsearch search engine, used by many large companies such as Amazon itself. In collaboration with some of them, the giant launched Open Distro for Elasticsearch, a fork of the original based on the latest code available with a commitment to keep it as an open project.

The interpretations regarding this initiative range from the dissuasive measure to the insurance, in the event that Elastic materializes the Elasticsearch license change; but both are complementary. Everything went ahead, in addition: Elasticsearch changed licenses and Amazon forked it definitively, giving rise to a curious paradox: from now on, Elasticsearch will be able to feed on the code of the solution sponsored by Amazon, but the same will not happen the other way around. Elasticsearch is no longer open source.

Seen this way, we could close the case with divine justice: the “little” open source developer has gotten rid of the vampire, will be able to recover the dedicated effort of the new code that it generates and will also be able to charge what it deems appropriate to its clients , be these end or intermediaries. However, if you noticed from the beginning that the matter is more complex than it seems, it is because it is so.

This is the case for several reasons: the first and most obvious is that Elastic has ‘gotten away with it’ by ditching the development model that drove its business; The second, not so obvious, is that although compared to Amazon Elastic it is a small company, it should be qualified: Elastic is a company that has a turnover of around 500 million dollars a year, but whose income has not stopped increasing in recent years and whose stock market valuation is around 10 billion dollars.

Another object of reflection is the Amazon-led fork of Elasticsearch: how is it possible that a company that simply takes advantage of someone else’s work could have put together an alternative in record time, further reinforced with the commitment to keep it open and in active development? Simply by dropping bills and hiring developers? If so, they could have already invested that money in the original project, not to mention that experienced Elasticsearch developers are not easy to find either.

The truth is that Amazon has a team of engineers with years of experience in Elasticsearch not only fine-tuning the product for their customers, but also contributing directly to the development of both Elasticsearch and Apache Lucene, the true original project from which Elasticsearch derives…. and from which you can also continue to feed, precisely because your license allows it. Ergo, what prevails in this conflict is nothing more and nothing less than the pure economic benefit.

The phrase that “open source software was never intended for cloud service companies to take and sell is worth taking up now.” What was open source intended for, then? There is no open source or free software license recognized by the Free Software Foundation or the Open Source Initiative that prohibits doing business with the software. What they prohibit is discrimination in the capacity and scope of its use depending on the party, whether it is an individual or the largest imaginable multinational.

You might also wonder if they would have gotten where companies and products like Elastic and Elasticsearch have gotten had they chosen a closed development model and not inherently employed on service platforms like AWS. Or if they would have done it if there were not previously open source projects like Apache Lucene and many others that they rely on and with which they will not always contribute, because you cannot contribute everything.

What is the solution to such a massive mess? The only thing that is clear is that it is not a question of black and white and the considerations are too many to continue delving into: companies that are listed on the stock market want more money from other companies - which are also listed on the stock market and for much more - than in addition to software They put the infrastructure on which they distribute their offers and that they have the ability to clone your product in the blink of an eye, not only because they have the capital, but because they have the necessary experience after contributing technically, but also in many cases, economically. , for a long time.

Despite this, this situation is altering the current paradigm, in which the open source development model has been imposed as a driver of innovation in the business sector, and there are already those who speak that we are approaching the end, or the beginning from the end of the Open Source Era, whose preponderance would be sentenced by the cloud revolution, ultimately the greatest stimulus that open source has had to date.

The future, then, would go through Shared Source Software, under which different companies with aligned interests would collaborate in the development of specific projects, but limiting their commercial exploitation to themselves. We are not there yet, however, and it does not seem that the relief will be given anytime soon. If it happens, it will be very striking: the death of open source by a misunderstood success.