Steve Klabnik — Open Source & Community

The culture war at the heart of open source

Sun, 26 May 2019 00:00:00 GMT

There’s a war going on. When isn’t there a war going on? But I’m not talking about a physical war here: I’m talking about a war over meaning. This particular war is a fight over what “open source” means.

Let’s take a few steps back.

The Free Software Foundation

People organize into groups for many reasons. This story starts with the story of an organization called the “GNU Project.” It was started in 1983, and here’s the initial announcement on USENET. I’ve pulled out four important paragraphs:

Starting this Thanksgiving I am going to write a complete Unix-compatible software system called GNU (for Gnu’s Not Unix), and give it away free to everyone who can use it. Contributions of time, money, programs and equipment are greatly needed.I consider that the golden rule requires that if I like a program I must share it with other people who like it. I cannot in good conscience sign a nondisclosure agreement or a software license agreement.So that I can continue to use computers without violating my principles, I have decided to put together a sufficient body of free software so that I will be able to get along without any software that is not free.If I get donations of money, I may be able to hire a few people full or part time. The salary won’t be high, but I’m looking for people for whom knowing they are helping humanity is as important as money. I view this as a way of enabling dedicated people to devote their full energies to working on GNU by sparing them the need to make a living in another way.

There’s a lot of different ways to talk about this post: it contains all sorts of interesting things. But for right now, I want to talk about the motivations here. There were two main ones: first, to produce a software artifact. The second one is the big one: the artifact needs to be produced because existing ones were not compatible with a particular set of values.

The word “ideology” is a tricky one, but let’s go with this definition, which is from Wikipedia:

An ideology is a collection of normative beliefs and values that an individual or group holds for other than purely epistemic reasons.

The GNU project was formed to produce software according to a particular ideology: one of sharing software. I’m not here to argue if the project has accomplished this goal, or if this goal is good or not. My point is that the origins of the GNU project were specifically motivated by a collection of normative beliefs and values.

Two years later, the Free Software Foundation would be formed to support the GNU project, and promote the concept of Free Software. Free Software was software that aligns with the ideology of the GNU project, and a definition of Free Software was created and published in Feburary of 1986. Here is that definition:

The word “free” in our name does not refer to price; it refers to freedom. First, the freedom to copy a program and redistribute it to your neighbors, so that they can use it as well as you. Second, the freedom to change a program, so that you can control it instead of it controlling you; for this, the source code must be made available to you.

Since then, the Free Software Definition has expanded to four points. You can read the current definition here.

Open Source appears

A decade passes, and trouble is brewing. I’ll quote wikipedia again:

[T]he modern meaning of the term “open source” was first proposed by a group of people in the free software movement who were critical of the political agenda and moral philosophy implied in the term “free software” and sought to reframe the discourse to reflect a more commercially minded position. In addition, the ambiguity of the term “free software” was seen as discouraging business adoption. The group included Christine Peterson, Todd Anderson, Larry Augustin, Jon Hall, Sam Ockman, Michael Tiemann and Eric S. Raymond. Peterson suggested “open source” at a meeting held at Palo Alto, California, in reaction to Netscape’s announcement in January 1998 of a source code release for Navigator. Linus Torvalds gave his support the following day, and Phil Hughes backed the term in Linux Journal.

Here we see the creation of the open source movement. If you’d prefer a primary source, here’s Eric S. Raymond:

Specifically, we have a problem with the term “free software”, itself, not the concept. I’ve become convinced that the term has to go.The problem with it is twofold. First, it’s confusing; the term “free” is very ambiguous (something the Free Software Foundation’s propaganda has to wrestle with constantly). Does “free” mean “no money charged?” or does it mean “free to be modified by anyone”, or something else?Second, the term makes a lot of corporate types nervous. While this does not intrinsically bother me in the least, we now have a pragmatic interest in converting these people rather than thumbing our noses at them. There’s now a chance we can make serious gains in the mainstream business world without compromising our ideals and commitment to technical excellence – so it’s time to reposition. We need a new and better label.

Shortly after, the Open Source Initiative was founded. A sort of mirror of the FSF, the OSI would support the promotion of the term “open source,” and the ideology behind it. Like the Free Software Definition, the Open Source Definition was created, derived from Debian’s guidelines for producing Free Software.

Again, we have an organization that’s created along ideological lines. But in this case, slightly different ones. An older version of the OSI website says:

The conferees decided it was time to dump the moralizing and confrontational attitude that had been associated with “free software” in the past and sell the idea strictly on the same pragmatic, business-case grounds that had motivated Netscape.

Today’s version says:

The conferees believed the pragmatic, business-case grounds that had motivated Netscape to release their code illustrated a valuable way to engage with potential software users and developers, and convince them to create and improve source code by participating in an engaged community. The conferees also believed that it would be useful to have a single label that identified this approach and distinguished it from the philosophically- and politically-focused label “free software.”

The idea here is plain: Free Software, but for business.

For more on this topic, I recommend The Meme Hustler.

Moving forward

Twenty years have passed, and a lot has changed.

For ideological movements to persist, they need to re-produce their values in new membership. And for a while, the FSF and OSI did a good job of that. These two movements, Free Software and Open Source, produced a lot of software, and gained a lot of new converts. But then… something happened.

I’m not sure exactly how it happened. I think the lazy answer is “GitHub!!!!”. I do think GitHub played a role, but I think the answer is more complex than that. I personally think that gender plays a huge role. But that’s a different essay. Regardless of why it happened, something did happen.

Somewhere along the way, Open Source ran into a problem that many movements face: the members of the movement no longer understood the ideology that created the movement in the first place.

If you ask a random developer what “open source” means to them, you won’t often hear “software that follows the Open Source Definition.” If you ask them “what’s the difference between free software and open source software,” you’ll often hear “aren’t those the same thing?” or “you can charge money for open source software, it’s not always free.” You may even hear “it’s on GitHub.”

In talking to developers about open source, you’ll also hear something else. Something that’s been itching in the back of my brain for a while, and the thing that led me to write this. You’ll often hear developers talk about how the relationship between businesses and open source developers is messy. Complicated. Businesses don’t “give back enough,” won’t pay people to work on open source. That it’s all take and no give. I’ve said a lot of that stuff myself.

But here’s the thing: that’s why the concept of open source was created in the first place. It’s there, plain as day, if you ask the folks who actually created it. Open source was a way to make free software better for businesses. This doesn’t mean that it’s beyond critique, of course. The relationship can improve. I don’t think these developers are stupid, or hypocrites.

This is where we’re at today. The Free Software movement was created, and then Open Source was created as a reaction to that. Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. And so we have a war. A war for the meaning of open source. A war fought with tweets, and blog posts, and discussions. A war between the old guard, those who created the idea originally, and the new generation, the developers who have taken the base idea and run with it.

I think history will repeat itself. In the same way that the open source movement said “we’re like free software, but with these changes,” I think we’ll end up with a new movement. For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name. I’m not sure what that movement will look like, and I’ll explore why in another post. To give you a teaser: the problem is in the way that both Free Software and Open Source are formulated. The rot is in the roots, and I’m not yet sure what will replace it.

What comes after open source

Tue, 02 Apr 2019 00:00:00 GMT

In a previous post, I discussed the history of open source, and ended with this claim:

Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. … For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name.

We talked about the ideological history of open source, but that’s not what developers object to, really. I don’t think developers are moving back towards a world of making source code private. Instead, it’s something related to a very old discussion in free software. To quote the FSF:

“Free software” means software that respects users’ freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software. Thus, “free software” is a matter of liberty, not price. To understand the concept, you should think of “free” as in “free speech,” not as in “free beer”. We sometimes call it “libre software,” borrowing the French or Spanish word for “free” as in freedom, to show we do not mean the software is gratis.

In a similar fashion, I don’t think that developers are turning against the concept of “free as in free speech”. I think that they don’t believe that the current definitions of free software and open source actually produce software that is “free as in speech.”

What does “freedom” mean anyway?

What “free as in speech” means is itself, yet another schism between different camps in the Free Software/Open Source camp. At the root of this schism is a difference of strategy. The specific tactic is straightforward: there are two kinds of licenses, and which do you choose? The two kinds are:

permissive licenses
viral licenses if you don’t like them, copyleft licenses if you do

Open Source advocates prefer permissive licenses, and Free Software advocates prefer viral/copyleft licenses. What’s the difference? Well, that’s the topic for a different essay. Because here’s the thing: the jig is up. We’ve already hit the root problem. And you probably didn’t even notice. It took me a long, long time to notice.

Before I expose the trick, a personal interlude.

Losing my religion

I used to be a card-carrying, passionate advocate of Free Software. I used to see Stallman lecture. I only installed Free Software on my personal computer to the fullest extent possible. I was considering buying a coreboot-compatible laptop to get rid of the rest.

But then, slowly, I stopped caring. It felt really similar to when I decided to leave Catholicism. I started noticing that these beliefs weren’t really actually helpful, and were frankly, mostly harmful. Now, I’m not saying Free Software is harmful. What I am saying is that refusing to use a wifi connection on your laptop because there aren’t free drivers helps basically nobody, and only harms yourself. Even if you are the kind of person who thinks boycotts work, there just aren’t enough free software diehards to form a large enough boycott. And even then, you’d want to do the boycott together, simultaneously, to send the message.

I realized that a lot of software I used was under a permissive license, and even though I’d prefer if it was GPL’d, the only way to actually realize the dream of improving the software thanks to the source I’d been given was to contribute back, under those terms. So I started to.

And at that point, the permissive licenses were becoming more and more popular. So I found myself in a strange position: even though I was a passionate Free Software advocate, I found myself near-exclusively building Open Source software. All around me, I saw the assault on copyleft, from all sides. And copyleft was losing.

At some point, I just stopped caring.

Free Software and Open Source are about licenses

With that out of the way, here’s the problem: note that I seamlessly switched above from talking about what Free Software and Open Source are, to immediately talking about licenses. This is because these two things are effectively synonymous. Quoting the FSF again:

Published software should be free software. To make it free software, you need to release it under a free software license.

The OSI:

Open source doesn’t just mean access to the source code. The distribution terms of open-source software must comply with the following criteria:

Both Free Software and Open Source are, almost by definition, specific design constraints on software licenses. While this started with the FSF, the OSI inherited it.

I think a quote from further down the FSF’s page really drives this home:

To copyleft a program, we first state that it is copyrighted; then we add distribution terms, which are a legal instrument that gives everyone the rights to use, modify, and redistribute the program’s code or any program derived from it but only if the distribution terms are unchanged. Thus, the code and the freedoms become legally inseparable.

The FSF sees copyleft, and therefore copyright, as a legal tool to enforce software freedom. This sometimes puts Free Software advocates in a strange position. For example, they will campaign passionately against Digital Rights Management, a software tool to protect copyrighted works, yet also support expanding copyright in some cases when it can further the aims of the GPL. (Due to controversy, they’ve stepped back a bit since, as you can see.)

Licenses are not sufficient

So why is it a problem that the concepts of free software and open source are intrinsically tied to licenses? It’s that the aims and goals of both of these movements are about distribution and therefore consumption, but what people care about most today is about the production of software. Software licences regulate distribution, but cannot regulate production. (technically they can, but practically, they can’t. I get into this below.) This is also the main challenge of whatever comes after open source; they cannot rely on the legal tactics of the last generation. I don’t have solutions here.

Let’s talk about what developers want first, and then we’ll get into why licenses can’t accomplish this.

Developers care about production

In my previous post, I claimed that developers are mostly confused about open source. Here’s two concrete examples of what I mean.

Imagine a company wants to take some open source software using a permissive license and use it in their product. In the course of using it in their product, they improve it. Are they required to give anything back to the parent project, at all?
Imagine a company wants to take some free software using a copyleft license and use it in their product. In the course of using it in their product, they improve it. In order to comply with the license, they include a line in their 250 page “software license agreement” that says “Certain components of the software, and third party open source programs included with the software, have been or may be made available by $COMPANY on its Open Source web site (http://www.opensource.mycompany.com/”. That web site contains zip files with the contents of the (heavily modified) source. Are they required to do anything more than that?

When presenting these questions to most developers today, I suspect you’d get these answers to these two questions, in order:

Yes, and that’s why open source is such a mess; companies take and don’t give back.
… that sounds extremely shady.

Those of you who have been around the block a few times may recognize situation number two: it’s what Apple did with WebKit. There was a project called KHTML, which was licensed under the GPL. Apple forked it, and in order to comply with the GPL, did exactly the above. They were completely within their rights to do so. Yet, many recognized, even at this time, that this “wasn’t in the spirit of open source.” These tactics are sometimes called “source dumps” or “code bombs.”

But that attitude, that this may be following the letter, but not the spirit, is the crux of it here. Most developers don’t understand open source to be a particular license that certain software artifacts are in compliance with, but an attitude, an ideology. And that ideology isn’t just about the consumption of the software, but also its production. An open source project should have a public bug tracker. There should be a mailing list, for discussion. You should be able to observe, and ideally participate in, the development of the software. Focusing on the code being open is putting the cart before the horse. In fact, this is one of the reasons why there’s such derision for “source available” licenses; it’s not about the source being open. It’s that the source being open is a necessary, but not sufficient, component of being open source. Now, this is normally framed as a problem of distribution, but I think that many also understand it as a problem of production.

I believe that this focus on process is why the GNU project has fallen out of favor as well. The tools that the GNU project uses to develop its Free Software are arcane, ugly, and unique to them. It’s the same with many of the old-school Open Source projects as well. If I never have to look at a Bugzilla instance again, I will die happy. This is why GitHub took off; it provided a significantly nicer developer experience for building software. You may not personally agree, but the numbers speak for themselves. The FSF didn’t move to GitHub because GitHub is proprietary, and they see that as inconsistent with their values. Most developers see that you can use it for no money, and that the software produced with it is open source. They see this as consistent with their values.

When developers talk about problems they see in open source, it’s often that there are production problems. Companies don’t “give back” money or developer hours. Programmers today don’t seem to be upset that, if they’ve developed any proprietary extensions to their open source software, that those extensions are not shared back with the community. They care that the production process is impeded by additional pressure, without providing resources. If a company were to add a proprietary feature to an open source project, yet pays five employees to develop the open source part further, the FSF sees this as a tragedy. The commons has not been enriched. The new generation of open source developers sees this as a responsible company that thankfully is contributing to the development of something they use and care about.

Software licenses can only restrict what people can do when they distribute the source code, and that’s it. It cannot force someone to have a bug tracker, or a code of conduct, or accept your patch. Copyleft can force an absolute minimal “contribution” back to your project, but it can’t force a good-faith one. This makes it an inadequate tool towards building something with the kinds of values that many developers care about.

The challenges

How would one go about building a software movement around the open production of software? There are a number of challenges.

First of all, do you even want the legal system to enforce such a thing? There are pros and cons. Without legal teeth, companies are unlikely to comply. That’s part of why we’re in this mess to begin with!

Okay, so you want the legal system to somehow enforce this kind of control over the production of software. But how? If we look closer at the strategy used by Free Software and Open Source, they use licenses, which are a form of intellectual property law, which is modeled after property law. Earlier, I said that you can’t use licenses to regulate production, and that’s technically not true. For example, say that I own a brand, like McDonalds. I own the intellectual property surrounding that brand. I can licence that intellectual property to others, contingent on them producing hamburgers (and whatever else) in a certain way, according to my specification.

This doesn’t really work with the way that open source is set up. The entities are in reverse here; it’s the software developers that want to be able to dictate things, but it’s the project that sets the license terms.

That got me thinking about a different pattern for doing this kind of thing. Have you ever seen one of these?

This image on a product is part of a process called “certification.” The image itself is referred to as a “certification mark.” In order to use this image on your product, you apply to a “certification body”, in this case, the USDA. This body has set up some kind of tests, and if your product passes them, you gain the ability to say that you’ve passed the certification. I chose organic food on purpose here; most aspects of this certification are about the process by which the food is produced.

Technology is no stranger to these kinds of processes:

So in theory, one could imagine an organization that produces a different kind of document. Instead of a license for the source code, they would provide a way to say uh, let’s go with “Open Development Certified.” Projects could then submit for certification, they’d get accepted or rejected.

I’m not confident this solution would work, however.

For one, even though some parts of our industry have accepted certifications, I feel like software developers have a pretty big bias against them. Beyond that, there’s two other major questions: who would do this certification, and how would they determine the criteria? It’s not clear who has the moral authority to suggest that they are the arbiter of what is correct here, and that a majority of people would agree. And then they would have a big job of actually determining what those sets of rules would be. It does have a nice property of a built-in business model; you can charge for application for the certification. But that also has a host of issues. And even if you sort all of that out, it runs afoul of the same “boycott” problems that I talked about above with Free Software. This certification only makes sense if people demand that the software they use is Open Development Certified. I’m not sure that this would be the case.

Another option is some sort of “Developer Union,” which would put pressure on the companies that those developers work at to contribute back to open source projects. Many developers seem rabidly anti-union, and tech companies are as well. I’m not sure this is a viable path, today.

So where do we go from here?

I’m still, ultimately, left with more questions than answers. But I do think I’ve properly identified the problem: many developers conceive of software freedom as something larger than purely a license that kinds in on redistribution. This is the new frontier for those who are thinking about furthering the goals of the free software and open source movements. Our old tools are inadequate, and I’m not sure that the needed replacements work, or even exist.

Something to think about, though.

What's next for SemVer

Mon, 11 Feb 2019 00:00:00 GMT

On December 18, 20091, the Semantic Versioning specification was announced. More commonly known as SemVer, it provided guidance for developers regarding software versions:

I propose a simple set of rules and requirements that dictate how version numbers are assigned and incremented. For this system to work, you first need to declare a public API. This may consist of documentation or be enforced by the code itself. Regardless, it is important that this API be clear and precise. Once you identify your public API, you communicate changes to it with specific increments to your version number. Consider a version format of X.Y.Z (Major.Minor.Patch). Bug fixes not affecting the API increment the patch version, backwards compatible API additions/changes increment the minor version, and backwards incompatible API changes increment the major version.I call this system “Semantic Versioning.” Under this scheme, version numbers and the way they change convey meaning about the underlying code and what has been modified from one version to the next.

The author came from the Ruby world, and around the same time, some big things were brewing. Bundler was being brought into the world. Bundler brought a few new things, but the most important features involved specifying dependencies for your projects. Normally, you’d specify which version of a gem your project needed. Instead, Bundler allowed you to specify a range of versions you’d accept, and it would help figure out which version you should use, according to the SemVer rules.

In January of 2010, npm, a similar program for the shiny new Node.js, also shipped, and also used SemVer.

Thanks to these tools, Ruby and Node enjoyed (and still do, to this day) a vibrant open source ecosystem. SemVer is the underlying concept that made this possible.

However, SemVer is not perfect. In some ways, I think of SemVer as the gofmt of versioning:

Gofmt’s style is no one’s favorite, yet gofmt is everyone’s favorite.Go Proverbs

SemVer has its flaws, but it empirically works better than previous attempts at such a thing. That such a thing exists is far more important than how perfect it is. Fundamentally, version numbers are a communication tool, and SemVer provides a framework for said communication.

That said, SemVer does have flaws. Bundler, npm, Rust’s Cargo, NuGet, and a variety of tools have implemented the specification. But there are edge cases, as well as gaping holes, in the spec, and so in some ways, these tools are incompatible.

Over the last few months, several of us2 have been talking, and today, we have an announcement to make. We’ve formed a semver team, and we intend to start work on producing a new version of the spec.

Who is “we”? The team roster will be in a GitHub team 3, but for posterity’s sake, here’s the initial list of members, in the same order GitHub lists them:

Anand Gaurav, NuGet
Dave Herman, Notion
André Arko, Bundler
isaacs, npm
Samuel Giddins, CocoaPods
Steve Klabnik, Cargo

(I’m putting myself as “Cargo,” because Cargo is the main client of my SemVer implementation. In most of these projects, the person who maintains the SemVer implementation is the same as the project, but in Rust, they’re separate. Cargo is the higher-order bit here, though.)

We have not yet started the work on the next iteration of the specification, but we have agreed on a governance model for SemVer. Described in the new CONTRIBUTING.md, we’re going with an RFC process, similar to the Rust programming language and many other projects. In short, “substantial” changes to the SemVer spec need to go through a process by which a written description of the changes are proposed, and the team reaches consensus on acceptance.

It also contains some principles we have for the future of SemVer. I’ve copied the important ones for the specification itself here:

No RFC will be approved if it is deemed to cause significant breakage to any of the SemVer-using communities represented by the SemVer Maintainers group.
RFCs will be considered formally adopted only when they are approved by the SemVer Maintainers group, and implemented in a simple majority of represented implementations.
Implementations may add functionality in advance of an approved RFC but all such functionality must be flagged as “experimental”, so that users understand it may change in the future.

In essence, we don’t intend to re-write the spec, but instead, fill in holes in the spec, find out where our implementations differ, and unify all of our implementations as much as is feasible. We believe that this will not only help all of us, but also help new tools as well. Being compatible with each other is an important part of the value of these tools.

I look forward to letting you know when our work is done! If you’d like to get involved, please watch that governance repository, and submit RFCs! While we’re all excited about the future of SemVer, it’s nobody’s job to do this work. As such, we’ll be moving fairly slowly. Please be patient. Thanks :)

You may also want to read Phil’s post, reflecting on this transition from the other side. Thanks for being so much of a good steward of SemVer so far, Phil!

At least according to the WayBack Machine ↩
I maintain Cargo’s semver implementation.↩
Ugh, I’m told apparently this can’t be made public? Even better that I put the list here, then.↩

The language strangeness budget

Fri, 26 Jun 2015 00:00:00 GMT

I’ve always loved programming languages. I’ve spent plenty of times with many of them, and am fortunate enough that language design is now part of my job. In discussions about building Rust, I’ve noticed myself making a particular kind of argument often in design discussions. I like to call it ‘the language strangeness budget’.

When building anything, it’s important to understand why you are building it, and who you are building it for. When it comes to programming languages, building one is easy, but getting people to use it is much, much harder. If your aim is to build a practical programming language with a large community, you need to be aware of how many new, interesting, exciting things that your language is doing, and carefully consider the number of such features you include.

Learning a language takes time and effort. The Rust Programming Language, rendered as a PDF, is about 250 pages at the moment. And at the moment, it really covers the basics, but doesn’t get into many intermediate/advanced topics. A potential user of Rust needs to literally read a book to learn how to use it. As such, it’s important to be considerate of how many things in your language will be strange for your target audience, because if you put too many strange things in, they won’t give it a try.

You can see us avoiding blowing the budget in Rust with many of our syntactic choices. We chose to stick with curly braces, for example, because one of our major target audiences, systems programmers, is currently using a curly brace language. Instead, we spend this strangeness budget on our major, core feature: ownership and borrowing.

On the other hand, if you’re trying to accomplish something different with your language, purposefully blowing this budget can be useful too. I really enjoy Haskell, and for a long time, their slogan has been ‘avoid success at all costs’:

As for the second, I don’t know if you know this, but Haskell has a sort of unofficial slogan: avoid success at all costs. I think I mentioned this at a talk I gave about Haskell a few years back and it’s become sort of a little saying. When you become too well known, or too widely used and too successful (and certainly being adopted by Microsoft means such a thing), suddenly you can’t change anything anymore. You get caught and spend ages talking about things that have nothing to do with the research side of things.I’m primarily a programming language researcher, so the fact that Haskell has up to now been used for just university types has been ideal

This is Simon Peyton-Jones knowing exactly what kind of language he wants to build, and in what way. Haskell’s embrace of strangeness in this sense is what makes Haskell Haskell, and actually furthers its goals.

If you include no new features, then there’s no incentive to use your language. If you include too many, not enough people may be willing to invest the time to give it a try. Language designers should give careful thought to how strange their language is, and choose the right amount to accomplish what they’re trying to accomplish.

How to be an open source gardener

Mon, 14 Apr 2014 00:00:00 GMT

I do a lot of work on open source, but my most valuable contributions haven’t been code. Writing a patch is the easiest part of open source. The truly hard stuff is all of the rest: bug trackers, mailing lists, documentation, and other management tasks. Here’s some things I’ve learned along the way.

It was RailsConf 2012. I sat in on a panel discussion, and the number of issues open on rails/rails came up. There were about 800 issues at the time, and had been for a while. Inquiring minds wished to know if that number was ever going to drop, and how the community could help. It was brought up that there was an ‘Issues team,’ whose job would be to triage issues. I enthusiastically volunteered.

But what does ‘issue triage’ mean, exactly? Well, on a project as large as Rails, there are a ton of issues that are incomplete, stale, need more information… and nobody was tending to them. It’s kind of like a garden: you need someone to pull weeds, and do it often and regularly.

But before we talk about how to pull the weeds, let’s figure out what kind of garden we even have on our hands!

What are Issues?

The very first thing your project needs to do is to figure out what Issues are supposed to be for. Each project is different. For example, in Rails, we keep Issues strictly for bugs only. Help questions go to Stack Overflow, and new feature discussion and requests go to the rails-core mailing list. For Rust, we have issues for feature requests, meta-issues… everything. For some repositories, closing all of the issues is not feasible, and for others, you’re shooting for zero. (If you don’t believe that this is even possible, check out Sequel. Issues are rarely even open for more than a few days!)

My personal favorite is to follow the Rails way. Ideally, you’d be at zero defects, and you can still have a place to discuss features. But really, having some plan is a necessary first step here.

Regular tending

So how do you tackle 800 issues? The only way I knew how: read all of them. Yep. Here’s what I did: I took a Saturday (and a Sunday), and I went to the list of open Issues, then control-clicked on each one in turn to open them in a new tab. Finally, I also control-clicked on page 2. Then I closed this tab. Now I had 31 open tabs: 30 issues, and the next page. I read through the whole issue, including comments. When I got to the last tab, I was ready to repeat the process: open 30 issues, open page 3, click close. Next!

See, people think working on open source is glamorous, but it’s actually not. Working on open source is reading 800 issues over the course of a weekend.

Anyway, once I read all of those issues, I was significantly more informed about the kinds of problems Rails was facing. I had a whole bunch of common questions, comments, and problems.

The next step was to do it all again.

Wait, again? Why? Well, now that I had a handle on things, I could actually take on the task of triage-ing the issues. If I’d tried to do it before I had the context, I might not have seen the duplicate issues, I wouldn’t know what the normal kinds of comments were on issues, I wouldn’t have known some common questions that maintainers had on pull requests, and in general, things would have just been worse.

This time, when reading the issue, I went through a little algorithm to sort them out. It looked a little like this:

Is this issue a feature request? If so, copy/paste an answer I wrote that pointed them to the mailing list, and click close.
Is this issue a request for help? If so, copy/paste an answer I wrote that pointed them to StackOverflow, and click close.
Was this issue for an older version of Rails than is currently supported? If so, copy/paste an answer I wrote that asks if anyone knows if this affects a supported version of Rails.
Did this issue provide enough information to reproduce the error? If no, copy/paste an answer I wrote that asks if they can provide a reproduction.
If the issue has a reproduction, and it wasn’t on the latest Rails, try it against HEAD. If it still happened, leave a comment that it was still an issue.
If we got to this point, this issue was pretty solid. Leave a comment that I had triaged it, and cc the maintainer of that relevant sub-system of Rails, so they could find issues that pertain to the things they work on.

At the same time I did this, I clicked this button on the GitHub interface:

And then set up a Gmail filter to filter all of the emails into their own tag, and to skip my inbox:

Why do this? Well, I didn’t do all 800 immediately. I decided to do one page per day. This kept it a bit more manageable, rather than taking up entire days of my time. I need these emails and filters for the important second part of the process: tending to the garden regularly.

Each morning, before I go to work, I pour a cup of coffee and check my emails. I don’t handle all of them before work, but I made an effort to tackle Rails’ emails first. There would usually be about 20 or 25 new emails each morning, and since it was largely just one new comment, they’d be pretty fast to get through. 15 minutes later, I was back to current on all issues. At lunch, I’d do it again: ten minutes to handle the ten or so emails by lunch, and then, before I’d go to bed, I’d do it again: 15 more minutes to handle the next 20 notifications. Basically, I was spending a little under an hour each day, but by doing it every day, it never got out of hand.

Once I got through all of the issues, we were down to more like 600. A whole fourth of the issues shouldn’t even have been open in the first place. Two weeks in is when the next big gain kicked in. Why two weeks? Well, two weeks is the grace period we decided before marking an issue as stale. Why two weeks? Well, that’s kind of arbitrary, but two weeks feels like enough time for someone to respond if they’re actively interested in getting an issue fixed. See, issues often need the help of the reporter to truly fix, as there just isn’t enough information in many bug reports to be able to reproduce and fix the problem.

So, after two weeks, I did one more thing each evening: I filtered by ‘least recently updated,’ and checked to see if any of those issues were stale. You just go back until they say ‘two weeks,’ and then, if you haven’t heard from the reporter, mention that it’s stale and give the issue a close. This is one of the other things I had to kind of let go of when working on a real project: closing an issue isn’t forever. You can always re-open the issue later if it turns out you were wrong. So when trying to get a handle on 800 open issues, I defaulted to ‘guilty until proven innocent.’ Terminate issues with extreme prejudice. Leaving old, inconclusive issues doesn’t help anyone. If it’s a real bug that matters to someone, they’ll come along and help reproduce it. If not, maybe someone else will later.

After a month or two, keeping on it, we got down to 450 or so issues. Members of the core team joked that they had to set up extra email filters from me, because they could tell exactly when I was doing triage. Slow and steady wins the race!

At this point, I knew enough about Rails to actually start writing some patches. And I happened to be familiar with basically every open bug. So it was easy to start picking some of them and try to reproduce them locally. So I’d do that, and then try to write a patch. If I couldn’t, I’d at least upload my reproduction of the issue, and then leave a note on the Issue, pointing to my reproduction. That way, another team member could simply clone my repository and get to it. The only thing better than reproduction instructions are when those instructions say git clone.

But I managed to get a few patches in, and then a few more. Doing all of this janitorial work directly led the way towards attaining a commit bit on Rails. It was a long slog at first, but it just got easier the more I did it. A lot of work in open source is this way: it’s really easy once you’ve done it a lot, but is hard for newbies. I’m not yet sure how to tackle this problem…

I’ve since taken this approach on basically every repository I’ve worked on, and it’s worked really well. But it only works if you keep at it: if you don’t tend your garden, you’ll get weeds. I haven’t had as much time for Rails over the last few months, and it’s back to 800 issues again. I’m not sure if these are real issues or not, as I’ve stopped tending. But without someone actively paying attention, it’s only a matter of time before things get unseemly. If you’re looking to help out an open source project, it’s not a glamorous job, but all it takes is a little bit of work, and developing a habit.

(Oh, and I should take a moment to mention Code Triage here. It’s super useful, and can also help you find projects that need help.)

Announcing Emoji 1.0

Mon, 10 Feb 2014 00:00:00 GMT

A long time ago, I wanted an emoji gem. So I set out to make one. Turns out, emoji are damn complex. Emoji Licensing was one of the results. Who’d have guessed the hard part of making a gem was the legal part! Next up was that the ‘emoji’ gem was waaaay out of date: it was a library for converting between Japanese telecoms’ emoji, which was important years ago, but not now that they’re just UTF-8. So I contacted the owner, who graciously gave me the gem.

Well, after doing the legal and social work, it turns out that some other people were interested in the idea as well! In the end, I didn’t write very much actual code, and Winfield really stepped up and made the code side happen.

So today, I’m very happy to have just released version 1.0 of the emoji gem, and to officially hand off maintainership to Winfield. Thanks for all your hard work!

Install it with

$ gem install emoji

And find the source on GitHub: https://github.com/steveklabnik/emoji

redis-namespace 1.3.1 security release

Sat, 03 Aug 2013 00:00:00 GMT

TL;DR: if you use redis-namespace and you use send, Kernel#exec may get called. Please upgrade to 1.0.4, 1.1.1, 1.2.2, or 1.3.1 immediately.

Link to the fix: https://github.com/resque/redis-namespace/commit/6d839515e8a3fdc17b5fb391500fda3f919689d6

The Problem

Redis has an EXEC command. We handle commands through method_missing. This works great, normally:

  r = Redis::Namespace.new("foo")  r.exec # => error, not in a transaction, whatever

Here’s the problem: Kernel#exec. Since this is on every object, when you use #send, it skips the normal visibility, and calls the private but defined Kernel#exec rather than the method_missing verison:

  r = Redis::Namespace.new("foo")  r.send(:exec, "ls") # => prints out your current directory

We fix this by not proxying exec through method_missing.

You are only vulnerable if you do the always-dangerous ‘send random input to an object through send.’ And you probably don’t. A cursory search through GitHub didn’t find anything that looked vulnerable.

However, if you write code that wraps or proxies Redis in some way, you may do something like this.

The official Redis gem does not use method_missing, so it should not have any issues: https://github.com/redis/redis-rb/blob/master/lib/redis.rb#L2133-L2147

I plan on removing the method_missing implementation in a further patch, but since this is an immediate issue, I wanted to make minimal changes.

Testing this is hard, #exec replaces the current running process. I have verified this fix by hand, but writing an automated test seems difficult.

:heart::cry:

Announcing security_release_practice

Fri, 08 Mar 2013 00:00:00 GMT

Security is hard. One of the skills any OSS maintainer needs is how to do releases, and security releases are a special kind of release that needs special git-fu to make it work.

I’ve created a repository so that you can practice this particular skill. It’s called ‘security_release_practice’.

The problem

‘security_release_practice’ provides a binary, omg_insecure, which has a security issue. Basically, the super_secure_calculation method converts user input to a symbol. Since Ruby does not garbage collect symbols, the longer you run omg_insecure, the more memory it will use, until your computer runs out of memory and the box grinds to a halt. Seems bad.

So, just fix super_secure_calculation and release, right? Well, here’s the problem: the last release of security_release_practice was 1.0.0. Since then, we’ve had a new feature, and a backwards incompatible change with super_secure_calculation. You can see the two commits here.

This is a problem: if we fix the issue and release, people who are relying on the + 5 behavior can’t upgrade: they’ll now be getting + 6. Also, the new feature (another_new_calculation) may have conflicts or weirdness with their code. That’s bad! So what we really want is a relase that’s exactly the same as 1.0.0, but with the security fix applied.

Let’s give that a shot.

The answer

If you think you’re good with git, you can try this out right now. If you’ve done it correctly, you should end up with the following:

A 1-0-stable branch
That branch should contain a new commit that fixes the issue
That branch should contain a new tag, v1.0.1 that fixes the issue
Master should have a backported version of the commit in #2.

The repository as it exists has all of this stuff, so check your work against it!

Practice!

If you don’t know how to do this, or you get stuck, you’ve come to the right place! Here’s what you need to do:

First, some setup work. Fork the repository and clone it down. Or, just clone mine, whatever:

$ git clone https://github.com/steveklabink/security_release_practice
$ cd security_release_practice

Next, since this repository has the backported fix involved, you need to remove that commit:

$ git reset --hard HEAD~1

This basically backs our branch out by one commit. Now we’re ready to go.

The first thing in actually doing the work is to check out the tag that we last released from. In our case, that tag is v1.0.0. So let’s do that now:

$ git checkout v1.0.0

git will give you a message:

Note: checking out 'v1.0.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 47a8bfb... Initial release.

We want to take git’s advice: let’s make a new branch. Since it’s going to be all our fixes for 1.0.x, let’s call it 1-0-stable:

git checkout -b 1-0-stable

Now we have our stable branch. Awesome! Master is the work that will go into 1.1.x, and this branch will be for 1.0.x.

Next, we need to fix our bug. You need to remove this one line:

--- a/lib/security_release_practice.rb+++ b/lib/security_release_practice.rb@@ -3,7 +3,7 @@ require "security_release_practice/version" module SecurityReleasePractice   def super_secure_calculation(input)-     input.to_sym     input.to_i + 5   end   def another_new_calculation(input)

We don’t even use that symbol, what a waste! Commit this, with a descriptive message. You can find mine here. Note the first few characters of the hash: mine is 168d5f756221.

Next, we need to release. Go ahead and increment the version number in lib/security_release_practice/version.rb, commit that, and then try rake install. Everything should work. Great! If you were actually releasing this gem, you’d be running rake release instead, but it’s my gem, not yours. ????

Okay, now we’ve released a version with our fix, but master still has a vulnerability: we need to port the fix. So let’s go back to master:

$ git checkout master

And then cherry-pick our fix over:

$ git cherry-pick 168d5f756221

There will be a conflict. The diff is a little weird, such is life. Go ahead and fix the conflict, then commit:

$ git commit

And you’re done!

Other considerations

If we had a CHANGELOG, we’d need to udpate that as appropriate, including creating new sections for our 1.0.1 release.

Sometimes it’s easier to fix things on master first, then backport to the new branch. I prefer to do it the way I showed here.

You should probably try to get as many people to know that you’ve fixed the bug. Tweet, blog, rabble-rouse, and possibly post to the ruby-security-ann mailing list, which was created to help Rubyists know about security releases of their gems.

If your gem is really widely used, you may want to actually register a CVE. You can find information on this process here. I made one here.

Going vimgan

Wed, 06 Mar 2013 00:00:00 GMT

I’m going vim-gan. Basically, I don’t want to consume any source code which doesn’t come from vim, or products that contain non-vim source code.

While I won’t, some people I’ve talked to will also consume code produced in Eclipse, since recent scientific studies show that Eclipse users can’t actually feel pain. This is enough of an ethical gray area that I’m staying away from it, but I can see that perspective.

I just can’t stay silent while the software industry brutalizes source code like this. Not cool, everyone.

Announcing request_store

Mon, 17 Dec 2012 00:00:00 GMT

Last night I had some insomnia, so I wrote a gem.

Here it is: https://github.com/steveklabnik/request_store.

TL;DR:

If you’re using Thread.current in your Rails app to store global-ish data, don’t do it! If you use Thin or Puma or a threaded web server, it won’t get reset between requests, and you’ll end up with subtle bugs. So do this instead:

gem "request_store"

and replace

Thread.current[:foo] = 1

with

RequestStore.store[:foo] = 1

And things will be peachy keen.

No Rails?

If you’re not using Rails, use RequestStore::Middleware to make it work. If you are using Rails, a Railtie takes care of it for you.

Software and community

Tue, 18 Sep 2012 00:00:00 GMT

“Are you sure you’re not vegan? You look like a vegan.”

Chad Fowler made me laugh pretty hard that day. We were in Austin, at a Ruby conference, and getting some very non-vegan food at the excellent Iron Works BBQ. I had only talked to Chad a few times before, and while chowing down on some excellent ribs, we talked about something that we’d discussed in a previous conversation: “the Ruby community.”

I gave a talk at Lone Star Ruby last year about my involvement with Hackety Hack and Shoes, and at the end, discussed some social issues I thought Ruby has:

The fun starts at 27:49. But ultimately, this whole section relies on the idea that in Ruby, we have a cohesive ‘community’ of people. After I gave my talk, Chad mentioned to me that he didn’t like the idea of “the Ruby community.” At the time, I was so starstruck that I said something like “sure, okay, I’ll think about that,” not even realizing the irony. So, while trying valiantly to eat barbecue in a not completely messy way, I asked Chad more about his thoughts on this.

If you were wondering, if it’s good barbecue, it’s impossible to eat it neatly.

I don’t remember his exact words, but Chad’s point was that if we talk about Ruby like a big cohesive group, we forget about all the people who use it who don’t post on blogs, or Twitter, or contribute to GitHub. Those people are people too, but they’re effectively outside ‘the community,’ either by choice or by ignorance.

Kill Your Idols

It seems to me that much of software is driven by fashion, celebrity, and superstition. This shouldn’t surprise anyone who’s not a software developer, as programmers are just people, and much of the human world is driven by fashion, celebrity, and superstition. Programmers, though, regard themselves as bastions of rationality.

I don’t argue about sports, it’s stupid. I prefer to argue about important things, like whether or not to use semicolons in JavaScript.

— Matthew Nielsen (@xunker) September 15, 2012

Right? This whole idea makes us, or at least me, very uncomfortable. I’m a scientist, dammit! I measure twice, cut once. I use the right tool for the job. I swear, MongoDB, CoffeeScript, and Ember.js fits my use case perfectly!

Ideally, we’d make technical decisions on technical means only. However, we’re just humans. I’m not sure it’s possible to divorce our decision making from our squishy human tendencies, and after typing that sentence, I’m no longer sure that it’s a good idea, either.

Leaders vs. Bosses

Does it follow that I reject all authority? Far from me such a thought. In the matter of boots, I refer to the authority of the bootmaker; concerning houses, canals, or railroads, I consult that of the architect or the engineer. For such or such special knowledge I apply to such or such a savant. But I allow neither the bootmaker nor the architect nor the savant to impose his authority upon me. I listen to them freely and with all the respect merited by their intelligence, their character, their knowledge, reserving always my incontestable right of criticism and censure. I do not content myself with consulting a single authority in any special branch; I consult several; I compare their opinions, and choose that which seems to me the soundest. But I recognise no infallible authority, even in special questions; consequently, whatever respect I may have for the honesty and the sincerity of such or such an individual, I have no absolute faith in any person. Such a faith would be fatal to my reason, to my liberty, and even to the success of my undertakings; it would immediately transform me into a stupid slave, an instrument of the will and interests of others.Mikhail Bakunin, “What is Authority,” 1882

There is a difference between a leader and a boss. There’s also a difference between a leader and a celebrity. What I’m concerned with is that we occasionally turn leaders into bosses, and we turn celebrities into leaders.

One other Ruby person that straddles this line is Aaron “Tenderlove” Patterson. He is, for good reason, almost universally beloved by Rubyists. He’s known to all for two different reasons: as a leader, and as a celebrity. Aaron is the only person on both the core Ruby as well as core Rails teams. He is one of the most technically brilliant people I know.

He also posts amazing pictures of his cat on Tumblr. (The caption is “My new Yoga outfit.”)

gorbypuff

Another amazingly technical thing Aaron has done is re-writing the AST -> SQL generation stuff in ActiveRecord, which sped up ActiveRecord by 5x. He deserves every single last bit of his popularity, on both sides. I happen to find the split reasons interesting.

“The community” as useful abstraction

While I think ‘the community’ may help drive a celebrity culture over a culture of leadership, there’s no denying that we do, in fact, have a community. If I had a nickel for every time someone in Ruby did something nice for me just because, I’d have a lot of nickels. I’m writing this post from the couch of a Rubyist who happened to be on the same plane as me to Helsinki, and rather than stay at the airport for 10 hours overnight (checkin isn’t for another few hours yet), I crashed here. On my last trip to Germany, Konstantin and Josh from Travis not only gave me their couches to sleep on, but bought me food when I fucked up my debit card situation. One Saturday, Luis Lavena spent 8 hours on a Saturday randomly pairing with me on some Ruby on Windows stuff, and left with an “I’m sorry, but my fiance is bugging me, I told her we’d get dinner an hour ago, I need to leave.” That’s crazy, I’m some dude from another hemisphere, hang out with your fiance! Aaron helped me debug an issue in Rails the other day, even though I kept asking him stupid questions. I have tons of stories like this, and so do many other people who participate in open source.

In “Debt: The first 5,000 Years,” Graeber defines “everyday communism” as “any community who follows the maxim ‘from each according to their abilities, to each according to their needs.” Of course, communism and community share a root for a reason, and all of these examples above are great descriptions of ’everyday communism.’ Not every exchange must go through some sort of monetary exchange. It’s often simpler to just do each other favors, and know that it will all work out in the end. That’s the basis of a gift economy.

In this way, I think community is a useful abstraction; we just need to make sure that we don’t get too caught up in ourselves. Focus on the friendships, helping each other out, and writing great software. Try not to get too trendy. Pay attention to what matters.

In the end, I wouldn’t give up the real friendships I’ve made via the Ruby community for the world.

Introducing issue2pr

Fri, 29 Jun 2012 00:00:00 GMT

Ever had an issue on GitHub, and then you fixed the bug? Opened a brand new pull request, which becomes a new issue, and then you have to remember to link the two via a comment, or manually go back and close the issue as well?

Turns out that GitHub has the power to turn Issues into Pull Requests in the API, but not in the web UI. So I wrote a tiny web app that lets you use the API call yourself: http://issue2pr.herokuapp.com/

Of course, the code is up on GitHub, here: https://github.com/steveklabnik/issue2pr

Issues (and pull requests) welcome!

More rstat.us refactoring

Fri, 23 Sep 2011 00:00:00 GMT

Hey everyone! I just wanted to share One More Thing with you about this rstat.us refactoring.

The main thrust of the last article I posted was to show you a technique for extracting a class, getting it under some kind of test, and then refactoring it a bit. Refactoring is always an iterative process, and Ryan Bates from the always awesome Railscasts asked me why I made the Salmon class into a module, especially given my recent rant against modules. The answer was, ‘because it’s simpler, and the first thing I thought of.’ He shared with me an alternate implementation that I like too, and I wanted to share with you. Check it:

class SalmonNotifier
	def initialize(user, feed)
		@user = user    
		@feed = feed  
	end

	def mention(update)  
		deliver OStatus::Salmon.new(update.to_atom("http://#{@user.author.domain}/"))  
	end

	def follow   
		deliver OStatus::Salmon.from_follow(@user.author.to_atom, @feed.author.to_atom)  
	end

	def unfollow   
		deliver OStatus::Salmon.from_unfollow(@user.author.to_atom, @feed.author.to_atom)  
	end

	protected 

	def deliver(salmon)  
		send_envelope_to_salmon_endpoint salmon.to_xml(@user.to_rsa_keypair) 
	end

	def send_envelope_to_salmon_endpoint(envelope)
		uri = URI.parse(@feed.author.salmon_url)
		http = Net::HTTP.new(uri.host, uri.port)  
		http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})  
	end 
end

This follows a more OO style, and is a bit more well-factored. Here’s his description of what he did:

Switched to a class and gave it a name that is a noun (maybe name should be different though).

Moved common method params into instance variables.
This simplified the mention/follow/unfollow methods enough to be on one line.
Renamed “send_to_salmon_endpoint” to “deliver” because it feels nicer, the “salmon endpoint” can be assumed due to the name of the class. I generally don’t like to put the class name in the method name.
Extracted commented out into its own method with the same name. I don’t know if this is really necessary though (same reason as above).

The only thing that bothers me now is the constant access of @user.author and @feed.author. This makes me think it should be interacting directly with authors. However you still need access to @user.to_rsa_keypair but maybe that method could be moved elsewhere more accessible.

All of these changes are pretty good. Ryan’s suggestions for moving forward are pretty good, as well, but I’d add one more thing: we’re starting to collect a bunch of methods all related to delivering more generic HTTP requests in the protected section. This also might be worth pulling out, too. Protected/private in general is a minor smell that indicates there’s code being used multiple times that’s not part of the main objective of the class. It might not be worth it yet, but then again, it might. Speaking of protected methods, you might wonder why I ended up mocking out my protected method in the last post, as well. There are no more tests that actually test the method is working properly. This is true, and it’s because I was leading into thinking of something like this. Mocked tests are almost always unit tests, and two methods are two separate units, so I had just ended up doing it out of habit. This further shows that maybe an extraction of another class is worthwhile.

Matz is nice so we are nice

Fri, 19 Aug 2011 00:00:00 GMT

Rubyists have a saying, or at least, we used to. “Matz is nice so we are nice.” This has sort of been lost over the years… some people who have been around Ruby for a long time still say this, but it’s something that’s gone in and out of popularity.

Nice should be the default

Most of being nice, in a software context, is about being considerate. Not unnecessarily trash talking other people’s stuff. Not going out of your way to be a jerk. Try to be helpful. Treat others how you want to be treated.

Sometimes, the truth hurts

Being considerate of others, however, can only go so far. While a lot of what we do is subjective, a lot of is objective, too. If you’re building a new project, not saying “this is why we’re awesome” is silly. Old code is often bad. Other people’s code is often bad. The key in “not unnecessarily trash talking’ is the neccesary part. There’s a difference between”My library is better” and “My library is better and their library sucks.”

The last thing I’ll say about RVM Drama

After talking to a few people, I guess I committed a bit of the sin I was trying to rail against: I got some feedback that it was thought that I was villainizing Sam, and a little easy on Wayne. This is natural, because I’m friends with Wayne and I don’t really know Sam, but in the interest of being nice myself, I just want to say this: I think that all that drama last week was largely a perfect storm of a few different factors, and I don’t think that Sam set out to hurt anyone specifically. If and when I ever have an issue with RVM, I’m totally going to try out rbenv. I like having options. I like there being multiple options. TMTOWTDI. And Sam builds a ton of other software that I really enjoy, like Sprockets, and pow. There’s a big difference between “that guy is an asshole” and “I think that that situation could have been handled better.”

Let’s all build lots of cool software together.

We forget that open source is made of people

Fri, 12 Aug 2011 00:00:00 GMT

Programmers like to think that software is the purest form of meritocracy. We like to consider ourselves scientists; what we do is entirely impersonal. I’m at Lone Star Ruby right now, and the first two talks are entirely about software as a science.

But this is a fallacy. There’s a human element to open source that’s often totally forgotten, and it drives people away.

Software Celebrity and Hero Worship

We sort of acknowledge this, in a way. There are most certainly software celebrities, even if their celebrity is localized to the community in which they operate. I heard someone say this today:

If I can go to one Ruby conference and not hear anyone mention Zed Shaw or _why, I can die happy.

If I say “jnicklas” you say “Capybara.” If I say “DHH” you say “Rails.” If I say “matz” you say “Ruby.” We have our heroes, villains, and foils, just like anybody else. But we forget that in every celebrity, there is a real person in there. Actual people, who have real thoughts, feelings, ideas, emotions, and that spend all of their time working for free just to make your lives better out of the goodness of their heart. Often after working at day jobs, often at the expense of a personal life, often straining relationships with people they care about.

Competition Means Somebody Loses

I used to think that Git was great because it made forks trivial. It changed what we mean by ‘fork.’ This is largely a good thing. But ‘the eff word’ used to be a last resort. The nuclear option. You’re fed up, can’t take it anymore, and so this is what they’ve driven you to do. Forking was Serious Business.

Now, GitHub has made it so easy to fork a project and distribute it to tons of people, people just fork things willy-nilly. And, from the outside, competition is good: in theory, it means that the good ideas work and the bad ideas don’t.

(A small aside: forking in git is different than ‘forking’ in svn. This overloading of terminology is unfortunate. The fork button is actually one of my favorite features of GitHub, because it makes contribution easier. But as it’s easier to publish new software, it’s easier to ‘fork’ in the svn sense. It used to be a pain to start a new project, and GitHub’s made that so easy that it happens more often than I think it should.)

But guess what: competition sucks. Cooperation is much better. One of the biggest problems with competition is that it means that somebody has to lose. For every winner, someone is a loser. That person is going to be hurting. In the case of open source, someone has spent (possibly) years of their life working on something, and now, all of that is gone. That shit stings.

Sometimes, forks are necessary. Sometimes, there are approaches that are fundamentally different. Forking is still an option, even if it’s a last resort. I’m also not saying that feelings should totally trump technical merit. All I’m saying is this: consider the blood, sweat, and tears of others before you act. Just in general.

It’s also not a zero-sum game, either. The pie can be grown. But that also doesn’t mean that it’s grown in a way that makes anyone feel good about it. There’s a difference between “We represent two sides of some coin” and “Those fuckers make shitty software.” Both approaches can grow the pie. One is far better than the other.

What’s the human cost of rapid competition between software projects? Is this cost worth it?

Open Source: All take, no give

The Pareto Principle: 80% of the effects come from 20% of the causes. This is absolutely true in OSS, and it might be more imbalanced. A few people contribute, and lots of people use. Sometimes, this is a good thing: it’s actually what draws me to open source. I can help people in a scalable way. My code is currently being used by tens of thousands of people. This is awesome. Some documentation that I’ve written is being read by millions of people. This is way more awesome.

But I rarely get any help with my projects. It’s really easy to let that get to a person.

Do you know how many people have told me that what I do is important? I got given a Ruby Hero award for my work on Hackety and Shoes. Do you know how many commits I have in on them? Probably less than 200. Do you know why? I’ve spent weeks of my life trying to track down bugs, but some of them are too hard for me, at my current stage of programming-fu. And it gets depressing. And everyone that’s ever offered to help has flaked on me. And all the recognition for something that was largely done by someone else gets to me, really deeply. And I get tons of people screaming about Lion support, and ‘this doesn’t work,’ and all sorts of things… and no patches. It’s why the ‘patches accepted’ saying was coined: this shit grinds on you after a while.

All of my Heroes are Dying

Today, this situation with rbenv and rvm made me realize something: everyone that I truly, deeply respect in open source has either burned out or left. I’ve said once before that there are four people that I truly respect in software: Zed Shaw, _why, Yehuda Katz, and (more recently) Wayne Seguin (and also Aaron Patterson. Dammit. This list is getting long). All of these developers are brilliant, work hard, and are tirelessly supporting their communities. That is, until they were driven away. This isn’t totally applicable to Yehuda, but it happened to Zed. It happened to _why. And I can see it happening, right now, to Aaron and Wayne. And Yehuda isn’t really doing Ruby…

But seeing this happen makes me deeply sad. For them personally, for my own selfish reasons, and that that’s the state of our communities.

Where Does This Leave Us?

I don’t have any answers. I only have a bunch of questions:

Why do we drive people away from Open Source?
Can we learn to work together, instead of competing?
How can we better recognize this human element?

If you don’t contribute to Open Source: please consider thanking someone that does open source, in a project that you love. Consider helping out, in even the smallest of ways.

Thanks.

My next post is a bit of a follow-up to this.

The first week of rstat.us: users, press, and scaling, oh my!

Wed, 30 Mar 2011 00:00:00 GMT

Hey everyone. A lot of people have been asking me about rstat.us lately, so I figured I’d tell a little bit of the story as it’s gone down so far.

Stats

First, here’s some numbers:

Users: 4553
Uniques: 25,000
Pageviews: 119,385
Pages/visit: 4.77
Statuses posted: 9387

Here’s some fun info: stats on statuses by language:

german 118
russian 1355
none 97 <- ha!
english 4836
spanish 1412
dutch 98
french 1155
portuguese 272
farsi 66
pinyin 1

Code

Lines, by committer:

28 AndrewVos
124 BRIMIL01
6 Blaine Cook
107 Brendan Taylor
924 Brian Miller
45 Caley Woods
162 Carol Nichols
20 Claudio Perez Gamayo
1347 Dominic Dagradi
4 James Larkby-Lahet
99 Jorge H. Cuadrado
258 Kat Hagan
10 Lauren Voswinkel
143 LindseyB
16 MenTaLguY
3 Michael Stevens
3 Murilo Santana
10 Nate Good
1 Peter Aronoff
24 Shebanian
44 Stephen Paul Weber
1409 Steve Klabnik
8 Tony Arcieri
478 Zachary Scott
104 burningTyger
28 james cook
56 kat
200 wilkie
10 wolfwood

Also, wilkie has a lot more code in the 3 gems that we build off of, for our ostatus implementation.

That makes for 29 unique committers. These stats were generated by git-blame.

Just over 200 pull requests

872 commits

The deal with Heroku

Crazy! Such good stuff. I’m really excited as to how it’s going, but there were a few rocky points along the way as well. Some people saw me tweeting in frustration with Heroku, so here’s the skinny:

On Sunday, for some reason, our 3 line app.js file, as well as our favicon.ico, started to take 60 seconds to respond. This meant that a dyno was getting tied up, and since Heroku changed the way their mesh works, they kept routing requests to these screwed up dynos. This caused a rash of 500 errors, and I frantically filed a support ticket. This was the first time I’d had a paid app on Heroku, and they’ve always been fantastic. The Hackety Hack site was on the free account, and performed beautifully, with similar traffic.

What it boils down to is this, though: Heroku took about 22 hours to respond to my support ticket, in any way. Not like “we’re looking at it, but it’s Sunday, so it’ll take a while.” Crickets. Meanwhile, the errors kept happening. Eventually, I heard something from them, where they pretty much said ‘I see other errors, try fixing those.’ with no response on the actual root cause. I got an email from a Level2 person who promised to investigate and escalate, but by now, I still haven’t heard.

I also had a smaller issue with MongoHQ. To their credit, I heard back Sunday night, and the issue with them was an annoyance, not a showstopper. I ended up hearing from the CEO, who ended up following up with me multiple times, and in a pretty timely fashion.

So, we moved our hosting away from Heroku and onto Duostack. Now, full disclosure: I’m friends with the guys from DuoStack. That said, they took care of me. I found a small bug in their platform when deploying the app, but everything’s gone really well. Bunches of users have commented on how fast the site is now, and I’ve been really happy with it so far. We’ll see how it goes.

The Future

Anyway, here’s what’s next for rstat.us: The biggest thing is getting to 100% ostatus compliance. We spent lots of time this week getting screwed over by a bug in Google’s hub implementation, but now that it’s all taken care of now. While we work on that, we’re also getting ready to deploy a statusnet compliant API, so people can use statusnet clients on mobile and desktop. This is huge for me, as I’m really missing notifications.

Other than that, just keep using it! Tell your friends! If you code, come join us on the IRC (#rstatus on freenode), and help file bugs and patch them!

Announcing rstat.us

Wed, 23 Mar 2011 00:00:00 GMT

Whew.

If you’ve been following me on Twitter at all lately, you’ll know that I’ve been working hard on a new project lately. Tonight, even though it’s late, I’m finally getting it out there. Please welcome http://rstat.us/ to the world! (as soon as the DNS updates. ;) )

rstat.us is a micro-blogging service, just like Twitter. There’s one big difference, though: it is distributed. Oh, and it’s 100% open source. So two things. It’s also integrated with Facebook and Twitter, so you can just log in with either of those accounts, and it’ll syndicate your posts. Well, just to Twitter right now, but we’re getting there.

Here’s the other cool thing: Want to own your data? You can actually get the code and run a copy yourself! But you won’t get left out: you can follow anyone else that uses the ostatus protocol, like http://identi.ca/, and it’ll Just Work. You just have to grab the URL to their feed, put it in, and it’s all good! But you probably shouldn’t do that yet. We’re still working on some features.

rstat.us is still very much alpha quality. 7 or 8 of my best friends and I have been fighting off sleep, food, and tests for the last two weeks to put this together. Please give it a try and let me know what you think, tell me if you can break it, or anything else that’s particularly interesting. If you’re the coding type, please check out the wiki for a big old list of stuff we’d like to do.

Oh, and of course, you should follow me on rstat.us.

Reddit drama: an interesting look at community values

Mon, 01 Mar 2010 00:00:00 GMT

I bet social psychologists are having a field day with the Internet. I know nothing about the field, so I don’t know if there are thousands of papers about it already written or not, but there should be. I can’t image that we’ve ever had a period in history when more communities are being formed; they’re also being formed out in the open.

I’ve been involved in many different online communities over the years. I’ve already written before about the seeming decline of Reddit’s content… but this past week, something really interesting has happened. Reddit has had two major events occur that have caused it to be increasingly self-reflexive on the values and norms that the community desires for itself. This is particularly interesting because normally, cultural norms have to be observed, not codified. But here, we’re seeing completely open discussion about “The community says this is not okay.” It’s really interesting stuff.

Speed Issues

I won’t comment on the technical details involved, but Reddit has gotten significantly slower over the past few months. This is normal, as the community is growing. But it’s caused quite a bit of a stir lately. Many users are seeing increased loading times, error messages, missing posts, and various other problems.

What’s a user to do in this situation? Post about it. See the posts about the search feature, or posts about the slowdown.

For example, as I look for the links to these posts, I get this:

Our search machines are under too much load to handle your request right now. :( Sorry for the inconvenience.

Amusing. But unfortunate. The interesting part, though, is the backlash against the complaints. See this thread, “Reddit, fix your fucking users. They are unbearably bitchy.” (The cursing is emulating the posts made complaining about Reddit’s lack of speed.)

There’s a huge discussion about what it means to be part of a free online community. There’s a contingent of people who say that people aren’t entitled to complain, because Reddit is free. Another points out that complaining is better than silently leaving the site, and that while people don’t pay for Reddit accounts, the eyeballs Reddit users provide enable the site to make money.

Some choice comments:

Hey, my incredibly expensive Reddit subscription fees entitle me to 100% perfect service! LEAVE ME ALONE!

Man, I’ve been loving this particular strawman for over a decade now. Thanks for breaking out the classics.

The reality is that Reddit is a business. The owners get paid because the users keep coming back to the site. Having users bitch and moan is not a problem for the admins - this isn’t their personal blog, they’re not mopy teenagers who are gonna cut themselves when they see how mean people are being to them on the interwebs.

The problem for the admins is when users stop visiting Reddit altogether because of constant technical problems, or fishy administrative decisions about deleting threads, or too many spammers, or etc. They would much rather have us bitch about these things on the front page, create giant threads about them that keep thousands of posters and tens of thousands of readers engaged and on the site, and provide catharsis and a public forum to address the issues.

tl;dr: Reddit is a business, they’d rather have us complain than have us leave.

Meta-bitching about bitching time

The OP posted a well thought out comment simply suggesting that Reddit’s users should be polite and respectful when addressing each other and the people that make using this site possible, and 90% of the responses here just prove his point. It seems like people are more concerned with gaining comment karma by posting bad one-liners than actually participating in intelligent conversation.

Is anyone else absolutely done with the reddit user base? I mean… I was told that reddit was this great intellectual news aggregator that through the use of ‘karma’ was able to bad submissions off the front page.

Over the past few months I’ve been here all that has been up on the front page has been Glen Beck, Conan O’Brian, Tiger Woods, weekly “Reddit is slow” posts and now this Saydrah BS.

I’ll say what I said when Conan O’Brian was on the front page for a week longer then Haiti ever was: I DON’T CARE. The Reddit admins don’t think Saydrah has abused her power as a moderator. At the very least (I personally don’t think she is a spammer) go flame some of the other 1000000000000000 spammers on reddit FFS. This is boring uneducated dribble.

KEEP THIS TRASH OFF THE FRONT PAGE.

And more. Interesting stuff, overall. There’s still quite a few trash comments, though.

The Saydrah Situation

The other event involves a user named Saydrah. She’s been accused of promoting links on Reddit for money, and abusing her moderator powers in conjunction with those offenses. This has spawned a huge amount of discussion on if these actions were inappropriate or not. Here’s the first big thread. Then an AMA with moderators about their opinions. Lastly, Saydrah does an AMA herself and explains her side of the story.

I won’t show you a bunch of comments, only one. And it echoes my opinion on the matter:

Two things: Frankly, I don’t care if people are paid to submit links…if they’re interesting, I upvote. If not, I ignore them.

Second, I have lurked around AR and RA a fair amount, and consistently find myself thinking, “That’s a helpful and well-reasoned response! Oh, it’s Saydrah again!” Whatever else people may say, I feel that you at least do try to contribute positively to this site.

…I guess I don’t have a question, so I’ll just leave my un-asked-for opinion here.

There’s a lot more, on both sides of the conflict. Regardless, people are hashing it out.

Growth Hurts

There’s a reason they’re called “growing pains.” Change is always a painful time in any organism’s life, and even though there’s a lot of drama, the current Reddit staff can take pride that they’ve created something that’s so important to people that they feel the need to scream about it for hours. It’s unfortunate that their baby is being embroiled in a flamewar, but these things happen.

We’ll see what the community ends up deciding is acceptable. I’ve managed to not get involved in these particular conflicts, but it sure is interesting to watch!

Edit: Jen added some interesting links on my facebook feed: http://j.mp/baRqdy