This is a crosspost from VanessaSaurus, dinosaurs, programming, and parsnips. See the original post here.
Let’s talk about transparency. It’s having an openness in culture that makes a community, practice, or tangible thing honest and accountable. On a personal level, transparency also means vulnerability, because if you put yourself out there, you also risk judgment of the exposure.
Open source is arguably the poster child of transparency. By stamping a project as open source, it’s implied that the community fosters a culture of openness.
But does open source also mean transparent?
On the surface, yes. But when you really think about it, no. And herein lies the problem. When I contribute to a project, I feel good about that contribution because I understand it. I might share the values of the project, or want to help the intended audience. Like a good relationship, shared values and communication between the people helping the project give the project life and strength. But have you ever contributed to a project, and noticed that you might not completely understand the incentive structure? Have you ever questioned if the open source stamp is serving as a pacifier to distract you from this feeling? I have, and this is what I want to talk about today. Open source is not transparent anymore, and this is a problem.
This observation has been caused by and led to a shift in the open source culture that has happened in the last few years. At some point companies started embracing (other) open source software. Then they started putting resources into releasing their source code to the community, and having their own employees play significant roles in contribution. It’s a win-win strategy, because embracing the open source culture and stamping it onto your brand says good things about you. From my standpoint, I started to feel conflicted. I started to label a company effort as transparent without second thought. But does open source mean completely transparent?
I’ve been troubled by this for quite some time. There is a distinct difference between software that is intended for the greater good, with no ultimate financial goals, and software that is shifting it’s development toward the latter. Here are some quick observations.
The definition of community is different. Instead of a group of passionate linux nerds [1], these new communities come with branding, marketing departments, and flashy conferences with stickers and (more commonly) ticket prices going over 1K. You can create the illusion of a community if you can pay for it. People that participate in open source can’t really tell the difference, and so they will get excited too. It’s showing me a beautiful cake in the display window that I’ll never be able to taste, but I’ll take pictures of it and send to my friends because I believe that it tastes good.
It’s not to say that creating a model to sustain development is not needed or admirable - I’m well aware that projects need this to survive. There have been prominent efforts by companies [2][3] that want to help projects without (in turn) hoping to then make money off of the projects directly. I applaud these efforts, because they create a model to sustain open source software.
Whenever I’ve encountered projects that have transitioned into trying to fold them into a product, they try to convince me that it’s essential or the effort will die. I’m skeptical. An open source project that serves a need, and builds community around that need, does not necessarily need to be a company. Those that need it (individuals and companies included) will find ways to support it. Maybe the survival is like natural selection, and the truly needed software packages will endure? Or maybe the missing component is again transparency. It’s actually okay if you want to make money off of your project. If you keep the main software as open source, you must be transparent about this: stating your goals, your intended audience, and use cases. It’s also okay if you want to make money and keep your software closed source. But don’t do the middle ground thing - wanting to make money, keeping the software open source, and not being transparent about your goals.
Could it be that projects that might not have grown to have this kind of support, by getting company support, branding, and attention, are almost tricking us into thinking that they are on the same level as the others? I’m not sure, but given what seems to be an explosion of these projects, company support for them, and conferences with tickets way too expensive for anyone, I know something strange is going on.
Have you also noticed how easy it is for companies to get some majority of seats in prominent foundations? Or how easy it is to start an entire new foundation or conference? What I see happening is that companies are taking control of prominent venues and groups that make decisions about open source. Do you really think these decisions can be unbiased to what the company needs?
The above points are scary. A culture that we know and trust is sometimes being used to manipulate us. Something that used to feel so special and community focused is being branded, controlled, and harnessed for efforts that I don’t completely understand. The question that keeps me up at night is how this change in open source is going to influence smaller communities. For example, science is about workflows. If workflow engines are being more catered to cloud technologies, and services that are so hard to set up you either need to pay for them or invest substantially to build your own, what happens to simple things like scientific discovery? The change in open source culture is creating a dependency not on a software suite, but on a company service. The big scientific discoveries of the future, by way of requiring scaled compute, workflows, and reproducibility, are slowly falling into the domain of corporate pursuits. The storage of immensely large datasets is falling into long term funding contracts with cloud providers. Perhaps we get some discount or deal today, but tomorrow, or in ten years, it’s yet another dependency. We need the hearts and minds of open source contributors to focus on some of these problems, but without the distraction of hyped projects. If users and scientists are told by social media to use a software or service to conduct their science, this is what they will ask for. Groups at academic institutions will invest millions and years of work into creating the resource that is asked for. Does the media hype that leads to user requests mean that its the right or best thing to do? Is it a decision through indecision - you don’t have any better idea, so you jump on the train of whatever you are told is the best idea?
I do not mean to make a value statement, and say that some projects are good, and some are bad. I mean to say that there is something missing across projects - complete transparency.
Finding out that a project is really only useful in context of paying a cloud provider, or that 98% of a contribution base comes from one or two companies, changes how I think about my contribution. If you tell me off the bat, I might be okay with it. If you don’t tell me and I find out later, I feel squirmy inside.
What is the opportunity cost of this glamorized, corporate controlled open source culture? It’s the work that isn’t done, of course. If there is some limited size of developers out there that regularly contribute to open source, if they are distracted with the “branded open source” projects that they are told are the hottest and best thing to contribute to, they aren’t going to notice the work that might be done to help a cause that has no funding model. It takes quite a bit of introspection to be able to turn off social media and general hype to ask harder questions about what the communities that we care about actually need. Many times an obsessive focus on a technology or trend distracts us from (much simpler) problems that warrant attention.
When we open pull requests, there are many things asked of us. We are implored to read and agree to the rules of contributing. We must sometimes sign license agreements. We have to jump through hoops to ultimately feel good about our contribution. This interaction needs to go both ways. When I open a pull request, along with my agreement, I want to have a statement that tells me about the project goals, the contributor makeup, and the intended audience. If there is any service or related product that drives some money making machine, that needs to be stated clearly. It’s really not much different than politics, when you think about it. If I support a candidate, I want to know who is paying the candidate that might ultimately drive the decisions he or she makes. These are the requirements of transparency, and any project that isn’t clear in these points, as far as I’m concerned, cannot claim the open source label.
We are all so blind to the embedded definition of transparency in open source that we’ve stopped questioning if it’s there. We’ve forgotten that there are different kinds of transparency. Transparency of code doesn’t mean transparency of incentives or goals. So what’s the fix to this problem? Be transparent, of course! Put it all out there. I want to propose the addition of two documents to the organization-wide community health files, AUDIENCE.md and GOALS.md.
Every repository should serve a GOALS.md file that clearly states goals, along with making disclosures about financial incentives.
The above provides transparency of incentives. If I contribute to your project, I want to understand the effort that I’m helping from all angles.
Along with the goals of the project, I want to know, in detail, what the intended use cases are. I want to understand the real level of impact that the software intends. For example, an entire infrastructure is most geared toward enterprise or large institution deployment, it’s not really geared for an individual to deploy. A project focused around a deployment API is more geared towards developer contributors than scientific contributors. Every project will be different, and generally the audience.md should communicate:
What is currently happening, without many of us realizing, is that open source is dividing into different things, but all are still being viewed under the same label. There is the traditional model, and there is now a growing set of projects that are focused around privilege. You need some substantial investment to participate in development, conferences, or even simple use. This is okay, but it needs to stop parading as something that it’s not. We need transparency of goals so that we can again make sound decisions about what we invest our time in. We need transparency so we can go back to thinking for ourselves. Instead of jumping onto every hype train, we need to stop for a second and ask where it’s going, and if that’s somewhere that we want to be.