all that jazz

james' blog about scala and all that jazz

A sad day for open source

This month is a month of mixed emotions for me.

It is my 10 year anniversary of working at Lightbend, then Typesafe. I’ve loved working for Lightbend, I certainly would not still be working here if I didn’t. I almost passed up the opportunity. At the time Typesafe contacted me I was living in Germany, but I was about to return to Australia. I had taken one years leave without pay from Atlassian when I moved to Germany, and I was really looking forward to going back to Atlassian, it was and still is a great company.

Out of the blue I got an email from Typesafe asking if I’d like to work full time on Play Framework. I had been contributing to Play a lot in my spare time, and I ran the Berlin Play User Group. I was flattered at the offer, but I did a quick lookup of where Typesafe was based - San Francisco - and I really didn’t want to move to the US, so I politely declined, saying I was about to move back to Australia. The response was “we’re all remote, you can work in Australian if you want.”

Initially I thought damn, now I have to come up with a proper excuse of why I didn’t want to work there. But then I remembered something. Open source had been a passion of mine, ever since I was in university. I had dreamed of having a job where I could contribute to open source full time. This job offer was actually for my dream job. So, I took the job, and it didn’t disappoint, working full time in open source was everything I had hoped it would be and more.

All this is to say that open source is something that I’m passionate about, I love the open source model, I love working on it, I love using it, I love being involved with communities of people from all around the world focussed on solving technical problems. Which is why this month is also a very sad month for me. Lightbend is no longer what I would call an open source company. With the relicensing of Akka to the Business Source License (BSL), the donation of Play Framework to the Open Collective, and our smaller contributions to Scala over the years, I can’t say I work for an open source company in good faith.

Of course, the BSL isn’t a terrible license. It reverts to Apache 2 after 3 years. It allows companies with less than US$25 million in annual revenue to still use Akka for free. And there are exclusions to ensure it can be still used as part of Play Framework to ensure Play users aren’t impact by the relicensing. But it’s not in the spirit of open source, as I dreamed of working many years ago.

Also, I don’t think Lightbend is making the wrong choice. We’re not the first company to adopt this style of license. Akka is not just another library that people use, it, and the architectural approach that it allows, forms a core platform on which scalable and resilient systems are built. Akka puts incredibly powerful and complex distributed systems tools into the hands of every day programmers who don’t have PhD’s in distributed computing, allowing them to build scalable and resilent systems easily. Developing and maintaining Akka requires some of the best distributed systems experts our industry has. The open source model simply isn’t working when it comes to maintaining that, and hence, in order to continue to offer and develop this incredibly unique but powerful tool, Lightbend needs to look at an alternative model. I think BSL is a good fit for this.

Nevertheless, I don’t feel good about it at all. It’s not what I signed up for. It’s not the dream that I was hoping to live. But I don’t think this problem is unique to Lightbend. I think the dream of the open source business model, across the industry, is proving impossible. The main open source projects that succeed are ones that are side effects of another companies business, and hence, the project itself doesn’t get the prime focus by its developers, but that company’s other business interests. Which is ok, but also isn’t really what open source is about. So, what I’m sad about is not Ligtbends move, but rather what open source has become in general.

Don’t get me wrong, open source is not going away. The world will continue to run on open source software, forever. But the days of teams of open source developers whose primary focus was on serving their open source communities, funded by businesses whose business interests were well aligned with the interests of their open source communities, those days are over. Projects like Kubernetes will be run by businesses who are not very aligned with their communities in terms of goals and motivations, but will manage to produce just enough to keep the project useful to the masses. Projects like Akka will be developed mostly in a proprietary manner, with decisions made based on what customers will pay for. If I’m being honest, Akka has been developed that way for a long time now, even if it was technically open source.

As for my future, I’m not leaving Lightbend anytime soon. I do still enjoy my job, it’s different, I’m working with production systems now, as the architect of Kalix. One complaint of many open source developers is that when they spend too long working on open source, they forget what it’s like to actually run software in production. So it’s good to be back getting direct exposure to what it’s like running the software I’ve been developing all these years.

Open Source - it's not about free as in beer or speech

I’ve been thinking a bit about open source software, and why I’m so attracted to it. Often when people talk about the thing that makes open source so great, they use the phrase “it’s free as in speech”. Of course it’s also free as in beer, that is, it’s free as in you don’t have to pay for it, and that certainly is an advantage of open source software, but the point that they’re making, in contrast to free as in beer, is that the real value of it is that you are free to do with the software as you choose.

But when I think about what attracts me to open source software, like free as in beer, free as in speech is only an advantage, a nice to have, it’s not the primary attraction.

So firstly, why isn’t free as in beer the primary attraction to me? The answer is it’s about risk. Dijkstra famously stated that we should count lines of code as “lines spent” rather than “lines produced”. Every line of code in your system is a line of code that you must understand and maintain for the life of that system. The more lines of code, the more complexity, the harder the system is to understand, the more expensive it is to maintain.

This doesn’t just apply to the code that you write - it applies to the third party libraries that you’re using. Every library that you use is another layer of code, another part of the system that you have to maintain, and so counts towards the number of lines spent. It may be a commercial library that you’re paying someone else to maintain - but it’s still your responsibility to ensure that it is maintained since it’s your system that the library is running in. When you pay someone to maintain a library for you, you are only delegating that responibility, not transferring it.

So, let’s say there was a library that I wanted to use, and it was free, as in beer, but there was no source code available for it. I could take the binaries, call them my own, use them in my own system, they are free, but I could never get the source code, I had to rely on the maintainer. Would I use such a library? Never. The risk is simply too high, since that library is part of my system, it’s my responsibility to maintain it. But I can’t maintain it myself, since I don’t have the source code. And because I’m not paying the maintainer for it, I can’t delegate the responsibility to them, because they have no obligation to me to fulfil that responsibility. Hence, I cannot use the library since I cannot be held responsibible for it. So, while free as in beer is great, it is not the primary advantage of open source, there are more important things than free as in beer.

So what about free as in speech? An open source library allows me the freedom to maintain and modify it myself, as a last resort I can always fork it. This is an advantage since it means I can always assume the maintenance of that library myself, and that means I can take responsibility for it. But is that the primary advantage? Let’s consider you had a choice of two libraries. One of them was open source, let’s say ASF licensed, but it had no active community around it, the maintainer didn’t accept patches, and only worked on things that he or she was interested in. The other library was commercial, you had to pay for it, and you were restricted in how you could use it, however once you paid for it, you were given full access to source code, and access to an active community of developers, not only within the company that produced the library but also developers from other companies that were using the library and building on it. Which would you go with?

Now I don’t expect everyone to answer the same here, there’s no one right answer, but my answer is the commercial one. Why? Because I think the biggest value in open source is the community. And so an open source project with no community and no opportunity for community involvement is missing the biggest value of open source, which means a non open source project that has commmunity and active community involvement will trump it.

Why is a community so valuable? As I said before, when you decide to use a library in your system, you assume responsibility for maintaining that library, regardless of whether it’s open source or not. When that library comes with an active community with the like minded goal of improving that library, then when you decide to take responsibility for it, you are immediately joined by a community of people who will help you to do that. Ask any open source maintainer, a community member who sees themselves as part of the solution to any problems they have with an open source library is a peer. They are someone who the maintainer wants to work with, and so when you take that responsibility, you get a whole community of people who are wanting to work with you to solve your problems. This is the great thing about open source, and in my opinion, it’s far greater than the freedom to use the library in any way you want.

Now perhaps you might be thinking that the example of a commercial library with an active developer community is contrived - that would never happen, it’s only because of the freedom that open source affords that such communities exist. Well, that’s simply not true, I have worked in such a community before. Atlassian is a company that sells commercial software with commercial licenses, you are not free to do with Atlassian’s software as you please, you may only use it as the license stipulates. But when you do purchase a license, you get full access to their source code, and there is a vibrant plugin development community that you can join, one that is fostered by Atlassian, where knowledge is shared by both outside and Atlassian community members, and people work together to make the entire platform better. This meets all my of requirements of a developer community, and addresses the risk of including an Atlassian product as part of my system.

So if an Atlassian product was the right fit to be part of a system that I was maintaining, I would not hesitate to choose it. The fact that it isn’t free as in speech or beer wouldn’t phase me in the slightlest, I would choose it for the same primary reason that I choose most open source software - that is that I get a whole community of like minded developers working to help me take the responsibility of maintaining my system. And that’s how I know that the primary value of open source to me isn’t that it’s free as in speech, it’s the community that comes with it.

OSS diversity - A thought experiment

On the topic of diversity in open source software, I have a thought experiment - what if an open source project was structured more like a company? This is only a thought experiment, I’m definitely not advocating that we structure open source projects like companies, but I am looking for ways that we can improve diversity in OSS, and this is a big problem that many are struggling to find answers to.

The big difference between a company structure and an open source project structure is that in a company, there are many different roles, and each role has a number of responsibilities specific to that role. People are hired based on their competency to fulfill the responsbilities required by those roles. In contrast, an open source project tends to have only one role - the committer, and then all the responsibilities of the project are organically fulfilled by the various people who are in that role according to need and interest. When people are selected for the committer role, they are selected on broadly the same criteria, which is based on their technical contributions to the project.

Does this difference have any impact on diversity? Of course there is some impact, since each person in the committer role must have a passion for technical contribution to the project, whereas in a company structure, there will be some people with a passion for technical work in the technical roles, and other people with passion for communication in the marketing roles. But this is a lack of vocational diversity. The issue that we’re concerned about in the OSS world is not a lack of vocational diversity, it’s a lack of diversity of gender, age, race, sexual orientation, among other things. For lack of a better word, I’ll call this type of diversity “identity” diversity, as opposed to “vocational” diversity. At face value, if the pool of people that are passionate about technical contribution are diverse in identity, then there likewise should be identity diversity among committers in open source projects.

In practice we see things are not like this in open source projects. In contrast, and in my experience, big companies with a broad range of very specialised vocational roles tend to be very identity diverse - at very least outside of executive management. Also in my experience this identity diversity exists within vocational roles, it’s not just that each vocational role is filled with people of one type diversity that in aggregate makes the whole company identity diverse.

So could it be that a lack of diversity of roles - which are vocational based - fuels a lack in identity diversity in people that fill these roles? At this stage I haven’t done enough research to be able to answer this question, but perhaps it’s something that we could start experimenting with in open source projects.

ยงPutting it into practice

So if we were to try to introduce diverse vocational roles into open source projects, what roles would they be, and what would they look like? To make things a little more concrete, let’s start by thinking about the introduction of a marketing role. Every large open source project does a lot of marketing, from publishing blog posts to tweeting to organising conference engagements to engaging with the community on mailing lists. This is something that it would be quite straight forward to create a role for. This wouldn’t mean that everyone else in the project stops blogging/tweeting etc, that doesn’t happen in a company. It does mean that someone is given the responsibility of coordinating and ensuring that marketing happens.

In order for such a role to work, it needs to be well defined - open source projects are not used to having such roles, so it needs to be made clear to everyone on the project, not just the person in this role, what the responsibilities of the role are. This is necessary to empower that person to be and feel effective in the project. The responsibilities are going to vary from project to project, but in the context of my project, Play Framework, and without having put too much thought into it yet, this is what I’d envision:

  • Be the public face of the project, the first point of contact for people that want to engage the Play community.
  • Manage the Twitter account, including keeping an eye out for tweets to retweet and responding to mentions where necessary.
  • Keep an eye out for community related questions on the mailing list, and respond or coordinate responses to those emails.
  • Keep a look out for conferences that Play contributors, both in the core team and the broader ecoystem, could speak at, and link the appropriate people up with appropriate conferences.
  • Write or coordinate project announcements, such as new releases, security vulnerabilities, etc.
  • Seek ways to make Play a more attractive project to contribute to, through talking to people about their contribution experiences, and improving or coordinating improvements to documentation and the processes in place for receiving contributions.
  • Seek ways to make Play an easier framework to get started with, through talking to people about their usage experiences, and improving or coordinating improvements to documentation and other resources.

Having defined the role, the next step is probably the hardest - finding someone to fill it. Finding people to fill a committer role of an open source project is generally easy, many developers that love using the software would also love to be a committer on it. Finding someone with skills and a passion for marketing who is willing to volunteer their time towards an open source project I would imagine would be a lot harder. This is possibly where my whole thought experiment falls down.

For Play though it’s not too hard to solve - Play is a project that is driven by a company, and that company already has a marketing team that does a lot of behind the scenes work in engaging the Play community. For us, we could make our marketing staff more public facing in the Play community, getting them to take on the responsibilities listed above (some of them they already do), and publicly acknowledging them as a member of the Play community, not just a member of the Typesafe marketing team, on the Play website. This may be a good start.

This is just one role that we could look at introducing to open source projects. There are many other possible roles, including leadership roles (in most companies these are not filled by technical people), product management, and a variety of HR roles. I’m not sure how well (if at all) these will work in an open source project, but in order to improve diversity, OSS projects are in need of big changes, and whatever changes are found to work will probably appear unworkable at first. So I think these are worth trying.

Open Source Developers

Since everyone's doing it, I thought I'd do it too. My contribution to the What people think I do meme:

About

Hi! My name is James Roper, and I am a software developer with a particular interest in open source development and trying new things. I program in Scala, Java, Go, PHP, Python and Javascript, and I work for Lightbend as the architect of Kalix. I also have a full life outside the world of IT, enjoy playing a variety of musical instruments and sports, and currently I live in Canberra.