Steve Klabnik — Technology

I see a future in jj

Wed, 22 Oct 2025 00:00:00 GMT

In December of 2012, I was home for Christmas, reading Hacker News. And that’s when I saw “Rust 0.5 released."" I’m a big fan of programming languages, so I decided to check it out. At the time, I was working on Ruby and Rails, but in college, I had wanted to focus on compilers, and my friends were all very much into systems stuff. So I decided to give Rust a try.

And I liked it! But, for other reasons I won’t get into here, I was thinking about a lot of things in that moment. I was looking to shake things up a bit. So I asked myself: is Rust going to be A Thing?

Why I chose Rust

So, I thought about it. What does a programming language need to be successful? It needs some sort of market fit. It needs to have people willing to work on it, as bringing a new language into the world is a lot of work. And it needs users.

When I considered all of these things, here’s what I saw with Rust:

Market fit: there was basically no credible alternatives to C and C++. I had been involved in the D community a bit, but it was clear that it wasn’t going to take off. Go was a few years old, and hit 1.0 earlier that year, but for the kinds of work that C and C++ are uniquely able to do, I saw the same problem that I did with D: garbage collection. This doesn’t mean Go isn’t a good language, or that it’s not popular, but I didn’t see it as being able to credibly challenge C and C++ in their strongholds. Rust, on the other hand, had a novel approach to these problems: memory safety without garbage collection. Now, I also need to mention that Rust back in those days was much closer to Go than it even is today, but again, I had just learned about it for a few hours, I didn’t really have a deep understanding of it yet. If I had, I actually might have also dismissed it as well, as it wasn’t really GC that was the issue, but a significant runtime. But again: I hadn’t really come to that understanding yet. Point is: low-level programming was a space where there hadn’t been much innovation in a very long time, and I thought that meant that Rust had a chance. Check.

For a team: well, Mozilla was backing it. This is a big deal. It meant that there were folks whose job it was to work on the language. There’s so much that you need to do to make a new language, and that means a ton of work, which means that if you’re going to be able to get it done in a reasonable amount of time, having paid folks working on it is certainly better than the alternative. Check.

And finally, how does this translate into users? Well, Mozilla was planning on using it in Firefox. This is huge. Firefox is a major project, and if they could manage to use Rust in it, that would prove that Rust was capable of doing real work. And, more importantly, it would mean that there would be a lot of folks who would need to learn Rust to work on Firefox. This would create a base of users, which would help the language grow. Check.

Finally, even though it wasn’t part of my initial assessment, I just really liked the Rust folks. I had joined IRC and chatted with people, and unlike many IRC rooms, they were actually really nice. I wanted to be around them more. And if I did, other people probably would too. So that was also a plus.

So, I started learning Rust. I decided to write a tutorial for it, “Rust for Rubyists,” because I’m a sucker for alliteration. And I eventually joined the team, co-authored The Book, and if you’re reading this post, you probably know the rest of the story.

Enter jj

For some background, jj is a new version control system (VCS), not a programming language. It is written in Rust though! While I talked about how I decided to get involved with Rust above, my approach here generalizes to other kinds of software projects, not just programming languages.

I have a rule of thumb: if Rain likes something, I will probably like that thing, as we have similar technical tastes. So when I heard her talk about jj, I put that on my list of things to spend some time with at some point. I was especially intrigued because Rain had worked at Meta on their source control team. So if she’s recommending something related to source control, that’s a huge green flag.

It took me a while, but one Saturday morning, I woke up a bit early, and thought to myself, “I have nothing to do today. Let’s take a look at jj.” So I did. You’ll note that link goes to a commit starting a book about jj. Since it worked for me with Rust, it probably would work for me for jj as well. Writing about something really helps clarify my thinking about it, and what better time to write something for a beginner than when you’re also a beginner?

Anyway, people seem to really like my tutorial, and I’m thankful for that.

The future of jj

So, what do I see in jj? Well, a lot of it kind of eerily mirrors what I saw in Rust: a good market fit, a solid team, and a potential user base.

But the market fit is interesting. Git has clearly won, it has all of the mindshare, but since you can use jj to work on Git repositories, it can be adopted incrementally. At Oxide, Rain started using jj, and more of us did, and now we’ve got a chat channel dedicated to it. This is, in my opinion, the only viable way to introduce a new VCS: it has to be able to be partially adopted.

Google is using jj, and so that is a bit different than Mozilla, but the same basic idea. I have more to say about Google’s relationship to jj, but that’s going to be a follow-up blog post. What I will say in this post is that at the first ever jj conference a few weeks ago, Martin (the creator of jj) said that internal adoption is going really well. I’m burying the lede a bit here, because the video isn’t up yet, and I don’t want to get the details of some of the more exciting news incorrect in this post. I also don’t mean to imply that everyone at Google is using jj, but the contingent feels significant to me, given how hard it is to introduce a new VCS inside a company of that size. Well, in this case, it’s using Piper as the backend, so you could argue about some of the details here, but the point is: jj is being used in projects as small as individual developers and as large as one of the largest monorepos in the world. That’s a big deal. It can show the social proof needed for others to give jj a chance.

Outside of Google, a lot of people say that there’s a bit of a learning curve, but once you get over that, people really like it. Sound familiar? I think jj is different from Rust in this regard in that it’s also very easy to learn if you aren’t someone who really knows a ton about Git. It’s folks that really know Git internals and have put time and care into their workflows that can struggle a bit with jj, because jj is different. But for people who just want to get work done, jj is really easy to pick up. And when people do, they often tend to like it. jj has developed a bit of a reputation for having a passionate fanbase. People are adopting it in a skunkworks way. This is a great sign for a new tool.

And finally, the team. Martin is very dedicated to jj, and has been working on it for a long time. There’s also a small group of folks working on it with him. It recently moved out from his personal GitHub account to its own organization, and has started a more formal governance. The team is full of people who have a deep history of working on source control tools, and they know what they’re doing. The burgeoning jj community reminds me of that early Rust community: a bunch of nice folks who are excited about something and eager to help it grow.

Basically, to me, jj’s future looks very bright. It reminds me of Rust in all of the best ways.

Putting my money where my mouth is

Speaking of burying the lede… I’ve decided to leave Oxide. Oxide is the best job I’ve ever had, and I love the people I work with. I was employee 17. I think the business will do fantastic in the future, and honestly it’s a bit of a strange time to decide to leave, since things are going so well. But at the same time, some of my friends have started a new company, ERSC, which is going to be building a new platform for developer collaboration on top of jj. Don’t worry, “errssk” isn’t going to be the name of the product. It’s kind of like how GitHub was incorporated as Logical Awesome, but nobody calls it that.

This won’t be happening until next month, I have some stuff to wrap up at Oxide, and I’m going to take a week off before starting. But as sad as I am to be leaving Oxide, I’m also really excited to be able to spend more time working in the jj community, and helping build out this new platform. For those of you who’ve been asking me to finish my tutorial, well, now I’ll have the time to actually do that! I’m sorry it’s taken so long! You’ll see me talking about jj even more, spending even more time in the Discord, and generally being more involved in the community. And I’ll be writing more posts about it here as well, of course.

I’m really excited about this next chapter. 2025 has been a very good year for me so far, for a number of reasons, and I am grateful to be able to take a chance on something that I’m truly passionate about.

Here’s my post about this post on BlueSky:

Steve Klabnik @steveklabnik.com

I see a future in #jj-vcs: steveklabnik.com/writing/i-se...

I see a future in jj

Blog post: I see a future in jj by Steve Klabnik

steveklabnik.com

October 22, 2025 at 5:22 PM

384 53

Read 67 replies on Bluesky

Choosing Languages

Thu, 13 Mar 2025 00:00:00 GMT

I tried to write this post yesterday, but I didn’t like my approach there. Last night, I had a conversation with one of my best friends about it, and he encouraged me to write this, but in a different way. I think this way is better, so thanks, dude.

The other day, Microsoft announced that they are re-writing the TypeScript compiler in Go. A lot of people had a lot of feelings about this. I did too, but in a different way than many other people.

The biggest questions people had were “Why not C#?” and “Why not Rust?”. To be clear, I do think that asking why someone chooses a programming language can be valuable; as professionals, we need to make these sorts of decisions on a regular basis, and seeing how others make those decisions can be useful, to get an idea of how other people think about these things.

But it can also… I dunno. Sometimes, I think people have little imagination when it comes to choosing a particular technology stack.

Steve Klabnik @steveklabnik.com

Here’s my opinion on the new typescript compiler: You should write programs in the language you want to.

March 12, 2025 at 1:34 PM

385 43

Read 22 replies on Bluesky

It is true that sometimes, there are hard requirements that means that choosing a particular technology is inappropriate. But I’ve found that often, these sorts of requirements are often more contextually requirements than ones that would be the case in the abstract. For example, you can write an OS in Lisp if you want. But in practice, you’re probably not working with this hardware, or willing to port the environment to bare metal.

But it’s also the case that you often don’t know what others’ contextual situation actually is. I would have never guessed the main reason that Microsoft chose Go for, which is roughly “we are porting the existing codebase, rather than re-writing it, and our existing code looks kinda like Go, making it easy to port.” That’s a very good reason! I hadn’t really thought about the differences between porting and re-writing in this light before, and it makes perfect sense to me.

At the same time,

Jake Bailey @jakebailey.dev Mar 12, 2025

Replying to Steve Klabnik

Funnily I'm pretty sure I'm the only team member who could actually claim their favorite language is Go, but I tried so hard to prove rust out early on. It sure wasn't my decision either, Go just kept working the best as the hacking continued

Jake Bailey @jakebailey.dev

I think it's going well, though, besides the terrifying infohazard that is the "Why Go?" GitHub discussion thread, which contains some of the weirdest and most frustrating takes I've ever seen

March 12, 2025 at 2:46 PM

37 1

Read 4 replies on Bluesky

people are going way overboard here.

Jake Bailey @jakebailey.dev Mar 12, 2025

Replying to Jake Bailey

I think it's going well, though, besides the terrifying infohazard that is the "Why Go?" GitHub discussion thread, which contains some of the weirdest and most frustrating takes I've ever seen

Michael Pratt @prattmic.com

I peeked briefly yesterday (and quickly ran away). Just took another look. 370 comments in 24 hours?? Wow. The proposal to add generics to Go (mildly contentious, to put it lightly) only got ~450 comments over a few months.

March 12, 2025 at 3:42 PM

11 0

Read 1 reply on Bluesky

There’s some sort of balance here to be struck. But at the same time, I hate “choose the right tool for the job” as a suggestion. Other than art projects, has anyone ever deliberately chosen the wrong tool? Often, something that seems like the wrong tool is chosen simply because the contextual requirements weren’t obvious to you, the interlocutor.

I think there’s two other reasons why this situation is making me feel complicated things. The first is RIIR, and the second is a situation from a long time ago that changed my life.

Many of you have probably heard of the “re-write it in Rust” meme. The idea is that there’s this plague of programmers who are always suggesting that every single thing needs to be re-written in Rust. While I am sure this happens from time to time, I have yet to see real evidence that it is actually widespread. I almost even wrote a blog post about this, trying to actually quantify the effect here. But on some level, perception is reality, and if people believe it’s true, it’s real on some level, even if that’s not actually the case. Regardless of the truth of the number of people who suggest this, on every thread about Rust, people end up complaining about this effect, and so it has an effect on the overall discourse regardless. And while I don’t believe that this effect is real overall, it absolutely has happened in this specific situation. A bunch of people have been very aggressively suggesting that this should have been in Rust, and not Go, and I get where they’re coming from, but also: really? Go is an absolutely fine choice here. It’s fine. Why do you all have to reinforce a stereotype? It’s frustrating.

The second one is something that’s been on my mind for over a decade, but I’ve only started talking about it in places other than conversations with close friends. Long-time readers may remember that one time where I was an asshole. I mean, I’ve been an asshole more than once in my life, but this one was probably the time that affected me the most. The overall shape of the situation is this:

Someone wrote grep, but in Node.
I made fun of it on the internet, because why would someone write a CLI tool in Node?
The author found out that I did and felt bad about it.

Looking back at my apology, I don’t think it’s necessarily the best one. It was better than the first draft or two of what I wrote, but I’ve gotten better at apologizing since then. Regardless, this incident changed the trajectory of my life, and if you happen to read this, Heather, I am truly, unequivocally, sorry.

There was a couple of things that happened here. The first is just simply, in 2013, I did not understand that the things I said had meaning. I hate talking about this because it makes me seem more important than I am, but it’s also important to acknowledge. I saw myself at the time as just Steve, some random guy. If I say something on the internet, it’s like I’m talking to a friend in real life, my words are just random words and I’m human and whatever. It is what it is.

But at that time in my life, that wasn’t actually the case. I was on the Rails team, I was speaking at conferences, and people were reading my blog and tweets. I was an “influencer,” for better or worse. But I hadn’t really internalized that change in my life yet. And so I didn’t really understand that if I criticized something, it was something thousands of people would see. To me, I’m just Steve. This situation happened before we talked about “cancel culture” and all of that kind of stuff, but when the mob came for me, I realized: they were right, actually. I was being an asshole, and I should not have been. And I resolved myself to not being an asshole like that ever again.

And to do that, I had to think long and hard about why I was like that. I love programming languages! I love people writing programs that they find interesting! I don’t think that it’s stupid to write a tool in a language that I wouldn’t expect it to be written in! So why did I feel the reason to make fun of this in that situation?

The answer? A pretty classic situation: the people I was hanging out with were affecting me, and in ways that, when I examined it, I didn’t like very much.

You see, in the Rails world at the time, there was a huge contempt culture at the time, especially around JavaScript and Node. And, when you’re around people who talk shit all the time, you find yourself talking shit too. And once I realized that, well, I wanted to stop it. But there was a big problem: my entire professional, and most of my social, life was built around Ruby and Rails. So if I wanted to escape that culture… what should I do?

If you clicked on the link to my apology above, you probably didn’t make note of the date: Jan 23 2013. Something had just happened to me, in December of 2012: I had decided to check out this little programming language I had heard of called Rust.

One of the things that struck me about the “Rust community” at the time, which was like forty people in an IRC room, was how nice they were. When I had trouble getting Hello World to compile, and I typed /join #rust, I fully expected an RTFM and flames. But instead, I was welcomed with open arms. People were chill. And it was really nice.

And so, a month and a day later, I had this realization about the direction my life was heading, and how I wasn’t really happy about it. And I thought about how much I enjoyed the Rust stuff so far… and combined with a few other factors, maybe of which I’ll write about someday, this is when I decided that I wanted to make Rust a thing, and dedicated myself to achieving that.

And part of that was being conscious about the culture, and how it developed, and the ways that it would evolve over time. I didn’t want Rust to end up the way that Rails had ended up. And that meant a few things, but first of all, truly internalizing that I had to change. And understanding that I had to be intentional with what I did and said. And what that led to, among other things, was a “we talk about Rust on its own merits, not by trash talking other languages” culture. (That’s also partially because of Nietzsche, but I’m already pretty far afield of the topic for this post, maybe someday.)

Anyway, the Eternal September comes for us all. The “Rust Community,” if it ever were a coherent concept, doesn’t really make sense any more. And it is by no means perfect. I have a lot of criticism about how things turned out. But I do think that it’s not a coincidence that “Rust makes you trans,” for example, is a meme. I am deeply proud of what we all built.

So yeah, anyway: people choose programming languages for projects based on a variety of reasons. And sure, there may be better or worse choices. But you probably have a different context than someone else, and so when someone makes a choice you don’t agree with, there’s no reason to be a jerk about it.

Here’s my post about this post on BlueSky:

Steve Klabnik @steveklabnik.com

Choosing Languages steveklabnik.com/writing/choo...

Choosing Languages

steveklabnik.com

March 13, 2025 at 5:12 PM

304 91

Read 29 replies on Bluesky

Against Names

Mon, 12 Aug 2024 00:00:00 GMT

There’s an old saying:

There are only two hard things in Computer Science: cache invalidation and naming things.

― Phil Karlton

I also appreciate the joke version that adds “and off by one errors.”

Lately, I’ve been thinking about this saying, combined with another old joke:

“The patient says, “Doctor, it hurts when I do this.” The doctor says, “Then don’t do that!”

― Henny Youngman

Specifically, if naming things is so hard… why do we insist on doing it all the time?

Now, I am not actually claiming we should stop giving things names. But I have had at least two situations recently where I previously found names to be kinda critical, and then I changed to systems which didn’t use names, and I think it improved the situation.

One of the most famous examples of not giving something a name, lambdas/closures, took some time to catch on. But many folks already recognize that naming every single function isn’t always neccesary. I wonder if there are more circumstances where I’ve been naming things where I didn’t actually have to.

Anyway, here’s my two recent examples:

Nameless Branches

I haven’t written much about it on my blog yet, but I’m fully converted away from git to jj. I’ll say more about this in the future, but one major difference between the two is that jj has anonymous branches. If, like me, you are a huge fan of git, this sounds like a contradiction. After all, the whole thing about branches are that they’re a name for some point in the DAG. How do you have a nameless name?

Here’s some output from jj log:

Here’s some sample output from jj log:

    $ jj log --limit 5
    @  pzoqtwuv steve@steveklabnik.com 2024-03-01 15:06:59.000 -06:00 9353442b
    │  added some cool new feature
    │ ◉  xrslwzvq steve@steveklabnik.com 2024-02-29 23:06:23.000 -06:00 a70d464c
    ├─╯  create hello and goodbye functions
    │ ◉  yykpmnuq steve@steveklabnik.com 2024-02-29 23:03:22.000 -06:00 210283e8
    ├─╯  add better documentation
    ◉  ootnlvpt steve@steveklabnik.com 2024-02-28 23:26:44.000 -06:00 b5db7940
    │  only print hello world
    ◉  nmptruqn steve@steveklabnik.com 2024-02-28 23:09:11.000 -06:00 90a2e97f
    │  refactor printing

Here, we are working on change pzoqtwuv. (@ means the working copy.) There are colors in the real CLI to make the differences more obvious, and to show you unique prefixes, so for example, you probably only need p or pz instead of pzoqtwuv to uniquely identify the change. I’ll use the full IDs since there’s no syntax highlighting here.

We have two anonymous branches here. They have the change IDs of xrslwzvq and yykpmnuq. The log output shows the summary line of their messages, so we can see “create hello and goodbye functions” on one branch, and “add better documentation” on the other.

You don’t need an additional branch name: the change ID is already there. If you want to add even more better documentation, jj new yykpmnuq (or again, likely jj new yy in practice) and you’re off to the races. (jj new makes a new change off of the parent you specify.)

(And if you’re in a larger repo with more outstanding branches, you can ask jj log to show specific subsets of commits. It has a powerful DSL that lets you do so. For example, say you only want to see your commits, jj log -r 'mine()' can do that for you.)

That’s all there is to it. We already have the commit messages and IDs, giving an additional identifier doesn’t help that much. In practice, I haven’t missed named branches at all. And in fact, I kind of really appreciate not bothering to come up with a name, and then eventually remembering to delete that name once the PR lands, stuff like that. Life is easier.

Utility CSS

Another technology I have learned recently is tailwind. But Tailwind is just one way of doing a technique that has a few names, I’m going to go with “utility CSS”. The idea is in opposition to “semantic CSS.” To crib an example from a blog post by the author of Tailwind (which does a better job of thoroughly explaining why doing utility CSS is a good thing, you should go read it), semantic CSS is when you do this:

<style>
  .greeting {
    text-align: center;
  }
</style>

<p class="greeting">Hello there!</p>

Whereas, with Tailwind, you end up instead having something like this:

<p class="text-center">Hello there!</p>

We don’t have a new semantic name greeting, but instead describe what we want to be done to our element via a utility class.

So the thing is, as a previous semantic CSS enjoyer, this feels like using inline styling. But there’s a few significant differences. The first one is, you’re not writing plain CSS, you are re-using building blocks that are defined for you. The abstraction is in building those utility classes. This means you’re not writing new CSS when you need to add functionality, which to me is a great sign that the abstraction is working.

It’s also that there is some sleight of hand going on here, as we do, on another level. An objection that gets raised to doing things this way is “what happens when you need to update a bunch of similar styles?” And the answer for that is components. That is, it’s not so much that utility CSS says that semantic names are bad, it’s that semantic names at the tag level are the wrong level of abstraction to use names. To sort of mix metaphors, consider the lambda/closure example. Here’s a random function in Rust:

fn calculate(list: &[i32]) -> i32 {
    list.into_iter()
        .filter(|i| **i % 2 == 0)
        .sum()
}

The ** is unfortunate, but this function takes a list of numbers, selects for the even ones, and then sums them. Here, we have a closure, the argument to filter, but it’s inside a named function, calculate. This is what using Tailwind feels like to me, we use names for higher level concepts (components), and then keep things semantically anonymous for some of the tasks inside of them (markup).

Heck, even the most pro-semantic-styles folks don’t advocate that you must give every single element a class. Everyone recognizes the value of anonymous things sometimes, it’s just a matter of what level of abstraction deserves to get named.

Here’s my post about this post on BlueSky:

Steve Klabnik @steveklabnik.com

Against Names: steveklabnik.com/writing/agai...

Against Names

steveklabnik.com

August 12, 2024 at 7:34 PM

33 5

Read 9 replies on Bluesky

Using the Oxide Console

Sat, 17 Feb 2024 00:00:00 GMT

A very, very long time ago, I was introduced to Gerrit. To be honest, I hated it. However, lately I have become interested in divesting from git and GitHub, and so have decided to re-visit various “forges” to see what’s out there. The “why” for this will come later, and I’m not leaving them just yet, just, you know, doing some exploring.

Anyway, in order to play around with Gerrit, I need a server somewhere. This website runs entirely on Vercel at the moment, so I don’t happen to have a more traditional server lying around to give Gerrit a shot. But as it just so happens, I work at Oxide, and so have access to a rack that I can play around with. So I thought it would be fun to share what that looks like, given that people are interested in what we do, but don’t actually have access to one of our machines.

If you’d like to poke at the console yourself, there’s a live demo! Quoting the README:

At https://oxide-console-preview.vercel.app the console is deployed as a static site with a mock API running in a Service Worker. You can create mock resources and they will persist across client-side navigations, but they exist only in the browser: nobody else can see them and the mock “DB” is reset on pageload. Request and response bodies in the mock API match the Oxide API’s OpenAPI spec, but behavior is only mocked in as much detail as is required for development and testing of the console and is not fully representative of the real API.

I happen to think this is extremely cool. If you want to, you could stop reading this post and go play with that instead. It’s also, like the rest of Oxide’s software stack, fully open source!

Before we begin, I would like to say that I am going to be doing this “manually” via the console and a web browser rather than via the CLI. I am doing this for two reasons:

I think the console is pretty and I would like to show it off. The folks who work on it do a fantastic job.
I don’t have many images on this blog and so want to write a post with some images and this will look nicer than screenshots of a terminal.

To be clear, I imagine a lot of real production usage of the Oxide rack will be driven by the API, either via our CLI tools or terraform or whatever. Heck, the console is built on those APIs, so on some level, that’s the only way to do it. My point is, don’t think of this as being the only workflow to interact with Oxide. Maybe if people like this post I’ll make some more covering the CLI driven workflow. Let me know.

After you SSO into the console, you’re put on the Projects page:

We’re in a “silo” named Oxide. A silo is a grouping of users, projects, and resources that are isolated from one another. But we don’t care about that: we want to make a new project. As you can see, I have censored my co-workers’ projects, as I didn’t tell anyone I was making this post, and even though the description here is hilarious, it wouldn’t be right to share without asking, and I’m already like ten layers deep in this yak shave.

Let’s make a new project:

It’s just that easy. After we push the button, we’re put on the instances page:

There’s a nice little toast in the bottom left that I didn’t manage to get a screenshot of because I was re-sizing my window too. Anyway, now that we have our project, we can add one or more instances to it. Instances are virtual machines, and so setting one up looks like you might expect. You can give it a name and description, but for this screenshot I’m only showing off one part of this page because this is already a ton of screenshots:

We can size instances however we’d like. Even though Gerrit is a Java application these days, I’m going to keep the instance small, because it’s just for me and performance isn’t a huge deal. Yes, I have made massive instances for fun in the past.

We’re using a Debian image that’s been provided by this silo, because I am lazy and that’s fine. Is four gigabytes of disk enough? I don’t know, let’s find out.

A few seconds later, our instance is up and running:

I am obscuring the external IP because, well, no reason to give that out. Anyway, that’s it! We can ssh in:

… how, you may ask? Well, I didn’t show off the entire “Create instance” page: I have set up my ssh keys in my account previously, and so by default, because this image uses cloud-init to configure itself, my ssh keys are already injected into the image. Furthermore, I left it on the default VPC, which is currently configured with a firewall rule that allows for incoming ssh connections. All of this is of course absolutely customizable, but given what this rack is being used for, these defaults make sense for us, so I didn’t have to actually do much setup at all.

That being said, I do need to open port 8080 so that I can poke at Gerrit in a web browser. If I go into the VPC configuration, I can add a new firewall rule:

Further down, I can pick port 8080, and that I want to allow TCP. After setting up Gerrit, we have success!

And that’s it! It’s just that easy. There’s of course a ton of stuff I didn’t talk about, but easy things should be easy, and for basic usage, it’s just that easy.

You're probably learning a technology in its seventh season, not its pilot

Fri, 27 Jan 2023 00:00:00 GMT

I first heard this adage in the Rails space. I don’t know who to attribute this to, but I also never wrote it down. So, time to write it down, and if you know of the origin of this aphorism, let me know and I’d love to put it here!

I think that the first formulation of this sentiment goes something like this:

Rails is like a soap opera. You’re getting started with it in a later season. The cast of characters has already been established, there’s a lot of history. You kinda just gotta dive in and go with the flow and catch up on older stuff at your leasure.

That’s obviously a bit long, so I think I’ll be using something like this in the future:

You’re probably learning a technology in its seventh season, not its pilot.

I was thinking about this today because I happened to come across this reddit post from a few months ago.

In fact, there is hardly any analysis of the downsides of Rust’s error model in the first place, which is quite disheartening.

Not really planning to add much to the convo, but back in mid 2019 when I first started learning and writing Rust, about half the posts I saw here in the subreddit were related to errors, how to do them right, how to minimize the boilerplate, and so on and so forth…

Over the last couple years, the convo has died down because we’ve all more or less agreed that the major pain points around error handling have been solved with a simple rule: thiserror for library code, anyhow for application code.

There’s obvs still more things that can be done around error handling. try is one such example, along with standardizing thiserror and anyhow so its part of the stdlib.

But to say there is no discussion about Rust and it’s error handling story is very dishonest imo as that discussion was raged for years before I saw it and only recently came to a proper conclusion for the majority of cases, and thus theres now much less talk about it. I even learned 3 different “defacto standard error libs” (that are all now defunct and no one cares about) out of a much longer list of them that came and went before them due to better and better patterns being being found over the years!

There was plenty of healthy and constructive discussion on this topic for an insanely long time and just because you didn’t see it doesn’t mean everyone was satisfied with the error handling story as it is in std from the 1.0 of Rust. Rust users actually engage with the system constantly, so ofc they’d discuss the problems with it and work towards fixing it! Hell, there’s still people that prefer things other than thiserror and anyhow and thats part of why those havent been standardized yet too, and why there still might seem to be “no progress” on fixing things.

TL;DR: Rust developers use Rust’s error handling literally all the time. Your assertion that they do not understand its shortcomings and pain points and refuse to even discuss them is weird given that there’s been a multi-years long battle between all kinds of error handling crates that only recently resulted in a proper and clear winner.

Now, I don’t want to take a position on the original thread, or what they have to say, or if they actually are falling afoul of this aphorism. But I think that this particular text shows the meaning of what I mean by “you’re in the seventh season” beautifully. Sometimes, when we learn a new technology, we go “wtf, why is this like this?” And sometimes, that outside perspective is good! But sometimes, it just means that you weren’t there for the discussion, you either don’t know or aren’t in the same contexts, or a variety of other reasons why something is the way that it is, even if that way is surprising to you at first.

In that sense, I think this idea is pretty closely related to Chesterton’s Fence. It may even just be the same thing, formulated in a slightly different way.

"C is how the computer works" is a dangerous mindset for C programmers

Tue, 31 Mar 2020 00:00:00 GMT

A long time ago, I wrote “Should you learn C to “learn how the computer works”?”. This was part 1 in a three part series. A year later, I got around to part 2, ““C is not how the computer works” can lead to inefficient code”.

It’s been five more months. I don’t think I really want to write out the third part; I expected this to be a quick series of blog posts, not something that was on my plate for a year and a half. Here’s the thesis of part 3:

Part three is going to show what happens if you make a mistake with the ideas from part two. If you incorrectly assume that C’s abstract model maps directly to hardware, you can make mistakes. This is where UB gets dangerous. While you can take advantages of some properties of the machine you’re relying on, you have to know which ones fit within C’s model and which ones don’t.

I honestly don’t feel like writing more than that, so I’m just gonna leave it there. Basically, the overall thrust of this series has been this: C is not the hardware, it’s an abstract machine. But that machine runs on real hardware, and abstractions are leaky. If you go too far into “purely only the abstract machine,” you may not be able to accomplish your tasks. If you go too far into “C runs directly on the hardware,” you may be surprised when a compiler does something that you didn’t expect.

"C is not how the computer works" can lead to inefficient code

Fri, 25 Oct 2019 00:00:00 GMT

A little over a year ago, I wrote “Should you learn C to ‘learn how the computer works’”. It was a bit controversial. I had promised two follow-up posts. It’s taken me a year, but here’s the first one.

In that post, I argued that C doesn’t “work like the computer”, but rather, by a concept called the “C abstract machine.” It happens to be close to how computers operate in some sense, but that when you write C, you’re writing for the abstract machine, not the computer itself. Here’s the trick, though: the C abstract machine doesn’t define every single aspect of how computation happens. And so sometimes, you can write code in two different ways, that the C abstract machine says are equivalent, yet are not equivalent in performance. This means that, even if you buy into my original point, you still need to know the boundaries of this definition, and can exploit those boundaries in important ways.

Let’s start with a made-up example, and then we’ll show how this happens in real code. Imagine that we have a very simple programming language, built on an abstract machine. It defines only the semantics for the multiplication of two 32-bit integers. It also ignores things like overflow. You could write a program like “3 * 4” and this machine would say “hey the result is 12.” Let’s also imagine we have a very simple computer: it has a single instruction, “add two numbers together.” This takes one clock cycle to execute. If we wrote a compiler for this machine, and didn’t apply any optimizations, it would execute “4 + 4”, and then take that result and add four to it. We have two add operations, and so our program takes two clock cycles.

Let’s imagine a new version of our computer comes out. It adds a new instruction: multiplication. This also takes one cycle, but can multiply two numbers directly. Our compiler adds a new back-end, and when compiling our “3 * 4” program for this fancy new machine, it emits a single multiplication instruction. Our program now takes one clock cycle.

Now imagine a new version of our compiler. It is able to do “constant folding”, an optimization. Our program does no computation at all, only produces a “12” directly. This now means our program takes zero cycles on both bits of hardware.

Is all of this okay?

Yes! The machine only defines how the * operator works on numbers in a “what number comes out the other side” sense. It doesn’t specify how to actually accomplish this task. Compilers for our little language are free to implement the details in any way they choose, as long as they respect the rules of the abstract machine. Other details are fair game. As long as the program “3 * 4” produces “12”, the rest of the computational process doesn’t actually matter.

This is how you can write “portable” code that is portable in one sense, but not at all in another sense. As long as you respect the rules of the abstract machine, you’ll be guaranteed (we live in a world with no compiler bugs, of course) to get the correct result. If it takes too long, or takes no time at all, that’s not the language’s concern, strictly speaking.

If this idea seems silly to you, well, I can assure you it’s real. For example, let’s consider the actual C language, and some actual hardware, specifically, x86.

Imagine we have an array of arrays, four by four:

There are two naive ways to process it: row by row:

1	2	3	4
5	6	7	8
9	10	11	12
13	14	15	16

or column by column:

Here’s a fun question: is one of these faster than the other?

At this small size, probably not. But there’s a way to do this that’s faster than either of these ways, and that’s called “blocking” or “tiling”. It means that you process a sub-set, or “block”, of the array each time. Something like this:

1	2	5	6
3	4	7	8
9	10	13	14
11	12	15	16

On x86, this is meaningfully faster in many circumstances. This has to do with the way that the CPU caches information. For more, see this page from Intel, including a visualization that’s probably better than mine.

This raises an interesting point: is this code portable? Imagine a machine that didn’t have the cache behavior of x86 chips here. This code would run on them, but it may be 14 times slower. Strictly speaking, this program is “portable” in the sense that you’ll get the right computation in the end, but maybe this speed is part of the user requirements for this software to be successful, and so being super slow means that it “doesn’t work” according to those requirements, even though from a programming language point of view, it’s all good.

Because C’s abstract machine is so thin, these kinds of details can really, really matter, as we’ve seen. And this is where the good part of the “C teaches you how the computer works” meme comes in. Because the machine is thinner, you can learn more about the details, and as long as they’re not details that the C abstract machine cares about, exploit them. And there’s a good reason I mentioned an optimizing compiler above; it’s also the job of compiler authors to realize the difference between a given computer and the C abstract machine, and then exploit the difference to make your code go as fast as possible. But this really blurs the line between abstract machine and physical machine. And that’s why we all argue online about this all the time.

I have a third post to wrap up this series. I hope it won’t take me another year, but no guarantees…

Why is Clojure so stable?

Tue, 18 Dec 2018 00:00:00 GMT

There’s been some Programming Language Discourse lately, and I have some thoughts. It’s mostly centered around a comment on the orange website:

I would put it simply: Clojure was designed by a thinker, who creates when he is away from the keyboard, not in front of it. When one releases and breaks the code in his head first, very few breaking changes are left for the public releases.

I think, regardless of this person being right or wrong, there’s an interesting question here. Let’s look at the comment this is ultimately responding to:

From the perspective of a (fairly large-scale at this point) app developer: I find it great that Clojure places such emphasis on backwards compatibility. In general, migration to newer Clojure versions is completely painless.The language has been designed by experienced and mature people and doesn’t go through “let’s throw everything out and start again” phases like so many other languages do.

Clojure has a well-deserved reputation for stability. The argument is not about that, but about why Clojure is so stable.

Individuals and their greatness

A lot of people, like the original commentator, believe that Rich, the BDFL of Clojure, is responsible. I think Rich deserves a lot of credit, but at the same time, this explanation is a little bit too close to the “great man theory of history.” The Great Man theory was originally proposed in the mid 1800s by Thomas Carlyle, and goes something like this: ““the history of the world is but the biography of great men.”

Don’t get me started about Carlyle, who was a pretty terrible guy even in his own day. A lot of the alt-right, “Dark Englightenment”, and Neo-reactionary folks consider him a really important thinker, and that’s all I’ll say about that.

What this means is that Important Men make big decisions, and that the lives of the rest of us are irrelevant in the scheme of things. He cited people like Shakespeare, Martin Luther, and Napoleon. In Carlyle’s mind, history was changed by these men and them alone. The French Army didn’t win wars, Napoleon won wars.

I tend to subscribe to the opposite, that is, “history from below,” or “a people’s history,” which emphasizes the army over Napoleon, the worker over the CEO. A critic of Carlyle’s, Herbert Spencer, said this:

You must admit that the genesis of a great man depends on the long series of complex influences which has produced the race in which he appears, and the social state into which that race has slowly grown. … Before he can remake his society, his society must make him.

That is, we have to look at historical context to see what created these changes, not just a few famous individuals.

Two sidebars here: first, Spencer is the person who coined the phrase “survival of the fittest.” Second, there is an irony in framing this particular philosophical debate about if individuals or history cause historical change by citing well-known opinions of two individual men. This idea was so popular, and still is, that it’s something many of us latch onto, sometimes subconsciously, even those who don’t believe it!

So, beyond Rich, who as I said before, does deserve a lot of credit… why might Clojure be so stable? I can think of three reasons. There may be more, but these are what comes to my mind.

Clojure is a Lisp

First of all, Clojure is a Lisp. This has a wide array of reasons why stability is a bit easier than in other languages.

The first is Lisps’ lack of syntax. Most languages have updates that introduces new syntax. As this grows larger and larger, the design space gets more and more constrained, making updates harder and harder. Lisps, on the other hand, rarely grow syntactically. Clojure has more syntax than most Lisps, but it still has far less syntax than non-Lisp languages.

Another perspective of this is homoiconicity. This property means that “new syntax” for Lisps can be introduced through macros, and not through the base language. In most other languages, users cannot extend the language to provide new syntax, but in Clojure, they can.

Lesson: the smaller your core language, the more stable you can be.

Clojure is dynamically typed

There are many, many, many ways to debate the virtues of static typing against those of dynamic typing. However, one area in which dynamically typed languages clearly win out, in my mind, is that they have a wider set of tools at their disposal to make changes without breaking code. This cuts both ways; they also make it much, much harder to know if you’ve broken code as well. Those are able to be mitigated through a variety of techniques, and we can argue about the tradeoffs here, but I think the possibility of advantage is plainly clear.

Lesson: it can be easier to change a dynamic system than a static one. This is both good and bad, but for now we’re focused on the good.

Clojure’s development is pretty slow

Clojure famously optimizes contributions around Rich and his process, at the expense of others. Debating this has been a real sore spot lately, and I don’t mean to pile onto this part of the discussion. I’m not interested in suggesting, in this blog post, if this is good or bad. I’m interested in the effects it has.

If we ask GitHub how many commits were between 1.0 and master, the answer is 1,957. If we look at commits between 1.9.0 and 1.10.0, we get 138 commits. Clojure 1.9 was released December 8, 2017. Clojure 1.10 was released December 17, 2018. That’s 138 commits for that release, for a year.

In contrast, between 1.30 and 1.31, six weeks of development, Rust had 1,694 commits. that’s almost as many as Clojure has had in the last nine and a half years.

Lesson: It’s much easier to keep things stable when you don’t change as often.

All of these factors tie into each other. Clojure doesn’t need to change as much, since the core language is a Lisp. When it does need to change, it has good tools at its disposal to change. Also, note that Rich was the one who made a lot of these decisions! He absolutely deserves credit for a job well done, even if I think that there are many other factors at play.

Furthermore, ascribing this entirely to Rich makes it harder for other languages to learn good lessons from Clojure’s successes; they can’t just get a Rich. By digging in a bit deeper, we can apply these lessons a bit more broadly. Good language design requires taking a sober look at various tradeoffs, and making the best decision you can make, regardless if you’re a BDFL or part of a language design team.

Borrow checking, escape analysis, and the generational hypothesis

Wed, 10 Oct 2018 00:00:00 GMT

There’s a fairly common argument that pops up on programming forums, and it’s about the nature of what “garbage collection” means. In the research world, this is what GC means:

Garbage collection (GC), also known as automatic memory management, is the automatic recycling of dynamically allocated memory(2). Garbage collection is performed by a garbage collector which recycles memory that it can prove will never be used again. Systems and languages which use garbage collection can be described as garbage-collected.

Historically, there has been two major forms of GC: reference counting, and tracing. The argument happens because, with the rise of tracing garbage collectors in many popular programming languages, for many programmers, “garbage collection” is synonymous with “tracing garbage collection.” For this reason, I’ve been coming around to the term “automatic memory management”, as it doesn’t carry this baggage.

I’ve often seen people make statements like this one, from the Rust subreddit this morning:

Manual memory management requires more work than garbage collected. Its a trade off of course to be more performant or use lower resources. When and where should Rust be used or not used according to you?

While I don’t completely disagree with this sentiment, it’s also never quite sat right with me. Yes, Rust is a bit harder at the start, but once you get over a hump, I don’t generally find writing Rust to be significantly harder than using a GC’d language. I’ve been trying to figure out why that is.

The idea that it’s easy for me because I’m an amazing programmer is not an answer I’m willing to accept here, both because I don’t think that’s particularly true, but also because I’ve seen Rust have a pretty wide appeal to a fairly large group of people, and they can’t all be amazing programmers. It’s just math.

My first clue comes from my own programming history. Some of my first programming languages were C, C++, and Java. These languages are statically typed, that is, they require you to declare (or have inferred) the type of variables up front, and check that the type holds. That is, they’re known at compile time. I can remember the first time I came across Perl, and the concept of dynamic typing. Wow, this is so much easier! I could do anything. From then on, I pretty much focused on dynamically typed languages.

While doing my work in Ruby, one thing I worked hard on was testing. Rubyists are big on tests; with a language that’s so dynamic, you really need them to make sure your stuff is working properly.

When I decided to start writing a bunch of Rust, I had to get used to using static types again. But I also wrote less tests. Many of the things that I was testing for were things that were taken care of by the type system. In other words, my tests were sort of like “dynamic type checking.” Instead of being checked by a compiler at compile-time, these behaviors of my system were checked at run time, by my tests. This isn’t a novel insight; many others have stated this relationship between tests and types.

We often acknowledge that writing tests is a skill. When I first started with Ruby, I was bad at testing. I got better with practice. But I don’t think we collectively acknowledge that working with types is also a skill. At first, it’s tough. But as you develop those skills, it becomes easier and easier.

Let’s switch gears a little, to garbage collection.

In garbage collection research, there’s a term, the “generational hypothesis”. It states

In most cases, young objects are much more likely to die than old objects.

If that succinct description doesn’t make sense, here’s a slightly longer one: there’s a kind of GC called a “generational garbage collector.” There are a few options when implementing one, but I’m going to describe one form in particular. What it does is this: new objects are allocated in a section of memory called the “nursery,” an analogy to where human children go shortly after they’re born. When the GC runs, it looks through the nursery. Any objects that survive are promoted to a new section of memory. Some collectors may have multiple of these “generations.” The generational hypothesis states that most objects die in the nursery. The GC can then take advantage of this fact by scanning the nursery more often than the older generations, which allows you to not scan every object every collection.

Empirically, the generational hypothesis is true. But I haven’t seen much investigation into why it is true. That said, I’m also not a GC expert. If you have references here, please get in touch.

There’s another concept related to GCs called “escape analysis.” This is an optimization which several languages do. The idea is fairly straightforward: if something is declared in a function, and never leaves (“escapes”) that function, you don’t need to put the object on the heap, you can stack allocate it instead.

“It isn’t that the generational hypothesis isn’t true for Go, it’s just that the young objects live and die young on the stack.The result is that generational collection is much less effective than you might find in other managed runtime languages” #golang https://t.co/aXEleY2yQ6 — Dave Cheney (@davecheney) July 18, 2018

If you squint a little at this tweet, you can almost think of escape analysis as a stronger version of the generational hypothesis. The claim is that escape analysis applies so often to Go code that any object that reaches the heap is effectively in the older generation.

Why is this? I’m not totally sure, but I have a feeling that is has to do with the core concept of functions and a call stack. Additionally, the value/reference type distinction. The more you program with values in functions, rather than references, the less likely objects are to escape past the call stack. in generational hypothesis terms, values allocated lower in the call stack tend to live longer, and ones higher in the call stack tend to live shorter.

There’s an interesting area of type research called “gradual typing.” The idea is that typing is not an either/or, static vs dynamic. It’s a spectrum. Some things are statically typed, some things are dynamically typed, and it all works together. The idea is that you can start off writing things that are dynamically typed, and then once you’re more sure of the behaviors you want, you can start to statically type them.

TypeScript is a great example:

// person is dynamically typed
function greeter(person) {
    return "Hello, " + person;
}

// person is statically typed
function greeter(person: string) {
    return "Hello, " + person;
}

TypeScript is really, really growing in popularity. I think one of those factors is the gradual typing; you can slowly work on your type-using skills, and you don’t have to use them everywhere.

Interestingly enough, many JIT compilers rely on the fact that in practice, programmers tend to pass the same types of things to functions, even if the language is dynamically typed. This is called “type specialization”. This is similar to the generational hypothesis to me, but in a different domain. And as I said about it, empirically, this is true. But I haven’t seen much investigation into why it is true. That said, I’m also not a JIT expert. If you have references here, please get in touch.

So what’s all this have to do with Rust?

Similar to the dichotomy between static and dynamic typing, I think that Rust is proving out a new niche in the concept of garbage collection. That is, historically, we think of GC as something dynamic, that is, it involves a run-time component that does stuff at runtime. However, the idea of automatic memory management doesn’t inherently mean that it has to execute at runtime.

I should mention that C++ paved a lot of the path here too, so maybe the above is worded too strongly. I do think that Rust is significantly different here, but you may not. If you don’t, please replace the above with “Rust is continuing to prove out a niche pioneered by C++,” and that’s fine by me.

This also opens up two more things about why I don’t feel that “Rust is too hard and so you should use a GC’d language when you can afford it.” First of all, Rust’s strong, static checking of memory provides me a similar peace of mind to a runtime GC. To make an analogy to types and tests, when I program in a language like C, I have to think a lot about the details of memory management, because I don’t really have anything at all checking my work. Its memory management is completely unchecked, similar to (compile time) types in a Ruby program. But with automatic memory management, I don’t have to worry about this, as the memory management system has my back. I won’t get a use after free, I won’t get dangling pointers.

I further think this line of thought is on the right track because lifetimes are generic types, and the ownership system is affine types.

This also relates to the “difficulty” of Rust. In the same way that learning to use static types takes time, learning to use static memory management takes time. But once you’ve developed that skill, you can rely on those static checks to free up your brain to instead focus on your program itself, and not the house-keeping tasks that these systems are checking for.

And, just like with static types, different people will make different choices of tradeoff for effort. Some people refuse to use dynamically typed languages, because they’ve invested the time into effectively using static types and prefer to have these things checked. Others don’t want to waste time on learning these kinds of things, and are more focused on shipping things. They have their own set of skills to ensure reliability. I think both approaches make sense depending on what you’re doing.

In some ways, you can think of the borrow checker as an even stronger form of escape analysis; that is, instead of saying “oh, this escapes, so allocate it on the heap”, the borrow checker says “oh, this escapes, so error at compile time.” For example, this code:

fn bar() -> i32 {
    5
}

fn foo() -> &'static i32 {
    &bar()
}

gives this error:

error[E0716]: temporary value dropped while borrowed
 --> src/lib.rs:6:6
  |
6 |     &bar()
  |      ^^^^^ creates a temporary which is freed while still in use
7 | }
  | - temporary value is freed at the end of this statement
  |
  = note: borrowed value must be valid for the static lifetime...

bar() produces a value, and so &bar() would produce a reference to a value on foo()’s stack. Returning it would be a dangling pointer. In a system without automatic memory management, this would cause a dangling pointer. In a system with dynamic automatic memory management, this would heap allocate the return value of bar(), so that everything works out. But Rust’s automatic memory management is fully static, and so it has to error in this situation.

Similar to gradual typing, this means that you can also have gradual GC. For example, Rust does have Rc<T> and Arc<T>, which are reference-counted types, clearly GC according to the literature. This moves the memory management from static to dynamic.

I don’t think that the transition here is as seamless as in TypeScript, though that’s a whole other post… additionally, it’s not like gradual typing is perfect.

One last thing before I go: the lifetime elision rules are also an empiric statement about how programmers write code, but I’m not sure what the underlying reasons are. From the RFC:

We found that of the 169 lifetimes that currently require annotation for libstd, 147 would be elidable under the new rules, or 87%.

If it were 100%, we could never require you to write annotations. But with 87%, we can let you not write annotations, and most code that follows most normal coding practices never needs to write explicit lifetimes.

Should you learn C to "learn how the computer works"?

Tue, 02 Oct 2018 00:00:00 GMT

I’ve often seen people suggest that you should learn C in order to learn how computers work. Is this a good idea? Is this accurate? I’m going to start with my conclusion right upfront, just to be crystal clear about what I’m saying here:

C is not “how the computer works.”
I don’t think most people mean this phrase literally, so that is sort of irrelevant.
Understanding the context means that learning C for this reason may still be a good idea for you, depending on your objectives.

I plan on making two follow-up posts as well, exploring more implications of this idea, but this is already quite a lot. I’ll update this post with links to them when I make them.

I’ve often seen people suggest this:

By learning C, you can learn how computers work.

I think that this idea is not inherently wrong, but does come with some caveats. As long as you keep those caveats in mind, I think this can be a viable strategy for learning new and important things. However, I rarely see people discuss this slogan in detail, so I’m writing this post so that I can provide that context that I believe is sorely needed. If you’re thinking about learning C for this reason, this post is written for you. I hope it’ll assist you as you learn.

Before we truly begin, I’d like to say one more thing: if you want to learn C, then learn C! Learning things is great. Learning C was quite important to my understanding of computing and my career. Learning C and about its place in programming language history will make you a better programmer. You don’t need a justification. Learn stuff for the sake of learning it. This post is intended to be a guide to your quest for understanding, not a suggestion that you should or should not learn C.

First of all, let’s talk about who is generally recommended this idea. If you’re trying to “learn how computers work,” then it stands to reason that you don’t currently understand that. Which programmers don’t understand how computers work? I’ve mostly seen this sentiment come from people who primarily program in dynamically typed, “scripting” languages, like Ruby, Python, or JavaScript. They “don’t know how computers work” because these languages operate inside of a virtual machine, and therefore, the semantics of the virtual machine are what actually matters. After all, the whole idea of a virtual machine is to provide portability. The goal is to only require knowledge of what’s portable, rather than rely on the details of the hardware that the VM is running on.

There’s just one problem with this: C also operates inside of a virtual machine.

The C abstract machine

From the C99 spec, Section 5.1.2.3, “Program Execution”:

The semantic descriptions in this International Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant.

In my opinion, this is the most important concept to understand when learning C. C does not “describe how the computer works,” it describes how the “C abstract machine” works. Everything else of importance flows from this concept and its implications.

A further note: I’ve chosen C99 here, which is not the latest C standard. Why? Well, MSVC has an …interesting amount of C support, and I’m a Windows user these days. Yes, you can run clang and gcc on Windows. There is not a ton of difference between C89, C99, and C11 with regards to what we’re talking about in these posts. At some point you have to pick. The version I’ve linked here includes some amendments to the initial spec.

You may have heard another slogan when talking about C: “C is portable assembler.” If you think about this slogan for a minute, you’ll also find that if it’s true, C cannot be how the computer works: there are many kinds of different computers, with different architectures. If C is like a version of assembly language that works on multiple computers with multiple architectures, then it cannot function exactly how each of those computers work simultaneously. It must hide details, or else it wouldn’t be portable!

That said, I think this fact is sort of irrelevant, because I don’t think people mean the phrase “C is how the computer works” literally. Before we can talk about that, let’s talk about the C abstract machine, and why many people don’t seem to understand this aspect of the C language.

Aside: why do people get this wrong?

I can only relate my own experience, though I think it generalizes to many others.

I learned GW-BASIC, then C, then C++, then Java. I had heard about Java before I started writing it in roughly 1999, four years after it was introduced. Java’s marketing at the time was very anti-C++, and it focused on the JVM as a platform, and how the machine model was what differentiated it from C++, and therefore C. Sun Microsystems no longer exists, but a mirror of the press release has this to say:

Java-based applications are platform-independent; only the Java Virtual Machine needs to be ported to each platform. It acts as an interpreter between an end user’s computer and the Java-based application. An application written in the Java environment can run anywhere, ending the need for porting applications to multiple platforms.

“Write once, run everywhere” was the big slogan. These two sentences were how I (and many others) came to understand Java, and how it differed from C++. Java has an interpreter, the Java Virtual Machine. C++ does not have a Virtual Machine.

With this huge marketing push, “virtual machine” became synonymous with “large runtime and/or interpreter” in the minds of many. Languages without this feature were too tied to the particular computer, and required porting between platforms, because they are not truly platform independent. Java’s very reason to exist was to change this aspect of writing applications in C++.

“Runtime”, “virtual machine”, and “abstract machine” are different words for the same fundamental thing. But they’ve since gained different connotations, due to non-essential variance in different implementations of these ideas.

I personally believe that this marketing, in 1995, is why programmers still misunderstand C’s nature today.

So is this claim false? Why would Sun Microsystems spend millions and millions of dollars promoting a lie? If C is also based on an abstract machine that offers portability between platforms, why did Java need to exist? I think this is key to understanding what people really mean when they say “C is how the computer works.”

What do people actually mean?

Even though C operates in the context of a virtual machine, it still is significantly different than something like Java. Sun was not lying. In order to understand why, you have to understand C’s history.

An operating system was written in assembly language for a computer called the “PDP-7” in 1969 at Bell Labs. In 1970, it was christened UNIX. As time went on, Bell Labs bought more and newer computers; including the PDP-11.

When it came time to port Unix to the PDP-11, they considered using a higher-level language, which was a pretty radical idea at the time. Imagine that I said to you “I’m going to write an OS in Java” today, you’d probably laugh, even though it is possible. The situation (in my understanding, I wasn’t alive then) was pretty analogous. They considered a language called B, but it didn’t support some of the features that the PDP-11 had, and so they created a successor, naming it “C”, since C was the next letter in the alphabet.

There was no “A”; B was a successor to BCPL, the “Basic Combined Programming Language.”

In 1972, on a PDP-11, they wrote the first C compiler, and simultaneously re-wrote UNIX in C. Initially, portability wasn’t the actual goal, but C did resonate with a lot of people, and C compilers were ported to other systems.

In 1978, the first edition of “The C Programming Language,” a book on C, was published. Affectionately known as “K&R,” after its authors, the book was very much not a specification, but provided enough of a description of the language that others attempted to write C compilers. This would later be called “K&R C.”

As UNIX spread, so did C, and both were ported to many computers. In the 70s and 80s, the ports grew and grew. In the same way that C was created because B couldn’t support all of the features of the PDP-11, many compilers had extensions to the language. Since there was only K&R, and not a spec, as long as they were close enough, that was considered acceptable. By 1983, the lack of any sort of standardization was causing issues, and so a group was created at ANSI to produce a spec. C89 was published in 1989, and is sometimes called “ANSI C.”

As such, the C specification was attempting to unify these diverse implementations, on diverse hardware. And so the C abstract machine is sort of the minimal possible specification that would allow the same code to run the same on all platforms. Implementations of C were compiled, not interpreted, and so there was no interpreter, so there was no “VM” in that 1995 sense. However, C programs are written against this abstract, non-existent machine, which is then translated into assembly that is specific to the actual computer that the program is running on. You could not rely on some specific details in order to write portable C code. This makes writing portable C very tricky, as you may have made a platform-specific assumption when writing the initial version of your code.

This is best illustrated with an example. One of the fundamental data types in C is a char, named after “character.” However, the C abstract machine does not define how many bits a char is. Well, it defines it, but not with a number; it defines it as CHAR_BIT big, which is a constant. Section 5.2.4.2.1 of the spec:

The values given below shall be replaced by constant expressions suitable or use in #if preprocessing directives. … Their implementation-defined values shall be equal or greater in magnitude (absolute value) to those shown, with the same sign.CHAR_BIT: 8

So in other words, you know that a char must be at least 8 bits in size, but implementations are allowed to be larger. In order to properly code against the “C abstract machine”, you must use CHAR_BIT instead of 8 as the size when doing char-by-char processing. But this isn’t some sort of interpreter feature, in the way that we think of virtual machines; it’s a property of how the compiler turns your source code into machine code.

Yes, there are systems where CHAR_BIT isn’t 8.

So this “abstract machine”, while still technically being the same idea as the Java VM, is more of a compile-time construct to guide compilers in the task of emitting assembly code than some sort of runtime check or property. The equivalent type in Java is byte, which is always 8 bits, and it’s up to the implementation of the JVM to decide how to make that happen on platforms where a byte is larger. (I am not sure if the JVM runs on any of those platforms, but that’s how it would work.) The C abstract machine was created to be a minimal papering over of various hardware, not to be some sort of platform created from whole cloth that’s written in software that your code targets.

So while Sun is technically incorrect, in practice, they mean something different than what they’re literally saying, and what they mean is accurate. It’s the same with “learn C to learn how computers work.”

Learn C to learn more about how computers work

What do people actually mean? In the context of “should a Rubyist learn C to learn about how computers work”, this desire to drop “down to the metal,” as it were, is an interest in understanding not only their program, and how it works inside the VM, but to understand how the combo of their program + their VM operates in the context of the machine itself.

Learning C will expose you to more of these kinds of details, because the abstract machine maps so much more closely to hardware, as well as abstractions provided by operating systems. C is very different than these sorts of languages, and so learning C can teach you a lot.

But it’s also important to remember that C is fundamentally an abstraction of hardware, and abstractions are leaky. Don’t conflate what C does or how it operates with the machine itself. If you do, you’re bound to run afoul of these differences, which can cause problems. Most learning resources for C, especially today, as hardware becomes more and more homogeneous, will promote the idea that this is how the computer works. So it can be hard, as a learner, to know what’s going on under the hood, and what’s an abstraction provided by C.

We haven’t even touched on other factors in this discussion, like that due to C’s tremendous popularity, hardware has become more homogeneous because it tends towards being similar to the semantics of C’s abstract machine. If your architecture deviates too much from C’s semantics, C programs may run much more slowly than others, and C programs are often how the speed of hardware is tested. This post is long enough…

For this reason, I think a more accurate version of this statement would be “By learning C, you can learn more about how computers work.” I do think that a rough familiarity with C can be useful to many programmers, and will serve them well even if they don’t write any C themselves. Learning more about C can also give you insight into the history of how our field has developed.

There are other ways to learn this stuff as well; C is not inherently the way to learn these topics, but it is a good option. My next two posts will explore some of the implications of this intersection between portability, the C abstract machine, and how people think about C programs.

There’s so many things to learn in computing. I wish you well on your journey.

User-Agent: moz://a

Tue, 12 Dec 2017 00:00:00 GMT

Every six months, Mozilla gets all of its employees from around the world together in one place, for a week. This morning was the plenary session, with Mozilla leadership talking about our successes this year, as well as plans for the future. I noticed a theme that, while not super overt, I find very interesting, and bodes well for Mozilla’s future.

The words “user agent” kept coming up in various aspects of the plenary. The User-Agent header has a… shall we say storied history. When most people hear the term “user agent”, they mostly think of the “user agent string”. This is passed through the User-Agent header to identify which browser you’re using.

Well, that’s how most people think of it, but that’s not the whole story. If it is supposed to identify which browser you’re using, why is it not the Browser header? What is this “user agent” term anyway?

The concept is a bit more broad: a “user agent” is a bit of software that acts on behalf of its user. A web browser is a “user agent” for the web. But an email program is also a “user agent” for email, for example. So “user agent” and “web browser” are not synonymous; a web browser is a specific form of user agent.

Words have power, however. Framing matters. The way you describe a problem affects the way you think about the solutions. So the shift in language I picked up on was, Mozilla is not building a web browser, Mozilla is building a user agent. So what’s the difference here?

A web browser is a dumb terminal. A user agent advocates and works for you, to better your experience.

There’s a lot more to say about what all this framing implies: for security, for privacy, for customization, for openness, for advocacy, for social impact. But I have some meetings to go to, so I’ll just leave you with a question:

What if your browser worked for you, rather than against you?

In my mind, this “user agent” framing is a very useful and interesting way to think about Mozilla’s mission, and the goals we pursue in the future.

Ouroboros

Mon, 18 May 2015 00:00:00 GMT

One of the things that I love about open source is its ability to send improvements across projects. Sometimes, an improvement in one project ends up improving its upstream. This kind of network effect really gets to the core of it for me: steady, constant improvement.

I saw an example of this today that makes me smile. Before Rust’s 1.0 release, I wrote a chapter about the Dining Philosopher’s problem. Incidentally, I’m really happy to use classic CS concepts in the Rust docs, hopefully exposing them to people who didn’t get the opportunity to get a degree in computer science. Anyway, towards that end, I decided to cite Tony Hoare’s classic CSP paper. Other funny aspect of that: I started my implementation using channels, but ended up using mutexes instead.

Anyway, today, a new contributor spotted a typo. The PR message made me smile:

Looks like this was an issue in the source material. I’ve let the editor know and he has told me he is correcting it there as well.

In my own small way, a thing I did led to someone else improving one of the most classic and well-known CS papers ever written. And it’s just Monday.

Update GitHub Pages with Travis CI

Tue, 16 Dec 2014 00:00:00 GMT

Ever wanted to have your project automatically update Pages when you have a successful build?

I have a demo repository here: https://github.com/steveklabnik/automatically_update_github_pages_with_travis_example

It, of course, uses Travis to update its own Github Pages, which is here: http://steveklabnik.github.io/automatically_update_github_pages_with_travis_example/

I hope that helps!

SF

Tue, 20 May 2014 00:00:00 GMT

@steveklabnik including you?

— Nick Quaranto (@qrush) March 5, 2014

I was always not quite sure about moving to San Francisco. I’ve heard about this place for my entire life: back in high school, during the first bubble, I would read Slashdot and wonder about life out west. But San Francisco is like a vortex for software developers. It’s got this gravitational pull that’s hard to escape. When I made the choice to move here, I wasn’t sure if I’d be able to handle it. Turns out I can’t.

So I’m leaving. My job is great, but it’s not worth staying in this city. I haven’t done anything outside of work in a few months. I go to work, work hard, go home, and then go to sleep. I don’t go home late. I go home at five. It’s not work/life balance that’s the issue. I’m just not psyched for the ‘life’ part. San Francisco has sucked out all my creativity, most of my passion, and has made me increasingly frustrated with things. Elaborating further would just be repeating Tom Dale and tef with not much difference, so I’ll leave it at that.

So, at the end of the month, I’m leaving Balanced and the city. I’ll be arriving in Brooklyn, a place I’ve been trying to live for the past few years. I’m going to deeply miss my friends and my (ex-)coworkers, but I have a lot of friends in NYC, too. Every hour I spend in New York, I feel excited, and every hour I spend in San Francisco, I feel drained.

So here’s to new adventures. I can’t wait to be happy again.

Is npm worth 26MM?

Wed, 12 Feb 2014 00:00:00 GMT

Yesterday, npm, Inc. announced that it had closed a $2.6MM round of funding. This was met with a bunch of derision on Twitter. I feel this is not only unwarranted, but backwards: I think this investment signals a nascent trend within the VC community, and a more honest view of how open source actually works.

Further disclaimer: I work for a company which has received investment from people mentioned in this article. I have never talked to those firms directly either, and I have no privileged information as to why they invested in us, either. It all happened before I came on board.

Let’s talk about VC, and then about some odd investments, and then we’ll get around to npm. I apologize for the length of this post, I didn’t have time to make a shorter one.

How VC investment works

First, a quick refresher on the actual mechanics of VC investment: the answer to this headline is ‘no.’. npm, Inc (‘npm’ for the rest of this post) received an investment of $2.6MM, which means that their company is ‘worth’ some multiple of that. At least, it is to True Ventures and friends.

As the saying goes, “follow the money.” A VC firm is, in many ways, a marketplace: they take money from investors (“limited partners,” or “lp’s”) and give it to entrepreneurs, and take a cut off the top. Let’s go through an example:

The Vulture Capital company is a venture capital firm. They announce a new fund. This fund, like many, has a mission statement: to invest in new SaaS technologies. They go to various wealthy individuals, insurance companies, funds-of-funds, endowments, and other companies, and pitch them on this investment: “The VC firm is going to invest $100MM into new SaaS technologies. Our last fund turned $10MM into $20MM. We think we can turn this $100MM into $300MM in the next ten years. We just need your money.” This motley band of the 1% scrapes together $100MM total out of their couch, and this new fund is good to go! The fund is now ‘closed.’ The VC firm now goes to work: the individuals who work for the firm now figure out who to actually give the money to. This generally takes three to five years. In this story, to make the math easy, VC invests in ten companies, $9.75MM each. For this, they get some stake in each company: we’ll call it 30% here. Since they purchased 30% of the company for $9.75MM, each company is ‘worth’ $29.25MM. In order to actually monetize this stake, generally, these companies need to experience a liquidity event: someone has to purchase them, or they need to go public. Because early-stage investing is exceedingly risky, when we check in on Vulture Capital ten years later, six companies have gone bankrupt, three sold for $67MM, and one went public and investors sold off $100MM. 3 * 67MM + 100MM = 300MM: success! The firm has turned $100MM into $300MM. The firm takes a chunk of this money, and the rest is given to those individuals that put up the money in the first place. A new fund is announced, and the cycle repeats itself.

There are variations at all points in this story, but that’s basically how it works. I’ve also ignored that VC firms generally have multiple funds going at once, that there are different kinds of employees in VC funds with different roles, and a whole ton of other things. This is a simplification, but a representative one.

Two big things matter here: It’s a numbers game. Over half of our example companies bit the dust, but the firm still made their money. Things have been successful overall. Secondly, companies must exit. A business that’s making $1MM a year in profit off of our $9.75MM investment will still make money eventually, but it won’t in our ten year period, which is when our limited partners want their money back. In another context, a million dollars of profit a year is a success, but for us, it’s a failure.

We’ll come back to these economics in a bit.

Open Source and $$$

It’s time for some #realtalk about open source. Open Source is big business. And it’s always been that way. Someday I will write more about this, but for now, I’ll just make that statement and not really back it up. For now, let’s trace the flow of labor in the open source world, and how it relates to businesses.

Open Source is pretty much the de-facto way to run a startup company these days. Even people on the Microsoft stack are increasingly utilizing open source. And there’s a good reason: the initial monetary cost of open source is zero. It’s also a form of DRY: a bug fixed in an open source component fixes a bug in every firm that uses that particular component.

Here’s a (sorta kinda) real world example:

A company, “Adequate HQ” builds a web application (RecruiterSpam) in a hot new language, “Phuby.” They decide to release a bunch of the code from this application as open source, and call it “Phuby on Phails.” There are a number of reasons to do this, but here’s a big one: bugs fixed in Phails will be fixed in RecruiterSpam, at no cost to Adequate! Furthermore, there’s a market opportunity here: by controlling Phuby on Phails, Adequate secures its reputation as a leader in the space, which leads to an invaluable amount of publicity, book deals, and eventually, pro subscriptions to RecruiterSpam. Adequate’s CEO, Gorbachev “Puff Puff” Thunderhorse, can just lay around all day and post photos of himself to Instagram. Other companies see the success of Adequate, and also decide to build their application on Phails. After all, Phails is pretty awesome, and is (truly) significantly better than all the other frameworks out there. Their new use exposes bugs in Phails, so they fix them and submit patches. Eventually, there’s an entire Phails ecosystem, with conferences, books, ninjas, and rockstars. Tons of gravy for everyone, and everyone rides their sports cars into their various sunsets.

This is a great story. Everyone wins. Everyone helps each other out. Costs go down, quality goes up. But there’s something that needs to be acknowledged here: Adequate had to exist. Someone had to do the initial work of actually building Phails (and all the other related tools I didn’t explicitly mention: deployment tools, testing tools, dependency management tools, the list goes on and on). In this case, it was because they had a kick-ass product and made boatloads of cash.

But fundamentally, programmers must pay rent and eat. Yes, they may not directly work on open source for free. But someone is paying them, and it has to come from somewhere. There is no such thing as a free lunch.

There are, however, different ways to get the money to buy lunch.

Infrastructure investment

So we have this trend of open source in general, and that for each language, we need an ecosystem to support building software. That ecosystem has to be bootstrapped somehow. Previous communities effectively got lucky, or put in long, long hours, only to be forgotten. I still cry tears for CPAN every once in a while. But I digress. The real question is this:

I’m a VC. I need a large number of companies to exist to make my money. I also need them to get as big as possible to make my money. I need costs to be low and profits to be high. I need a healthy open source ecosystem to make this happen. So what do I do?

So here’s what I’m seeing slowly happen: VCs are realizing this story, and are starting to invest in infrastructure. Remember, failure is built into their model. If you had to make ten investments, and I told you that I would take one of them, and you’d lose money on me, but I’d build an ecosystem that would help be a multiplier on all your other investments, would you take it? Remember, you just need one really, really big payday. And a bunch of your investments will already fail.

This is what I see happening with npm. And it’s not just npm: there’s a number of investments in the last few years which fit this pattern. I’m pretty sure that a16z is the only VC firm that explicitly understands this model, and True Ventures may have stumbled upon it by accident.

Because Rails (If the sarcasm wasn’t heavy enough, Phails is Rails) isn’t exactly repeatable. It’s a fluke. A wonderful fluke that’s changed my life (I have a Ruby tattoo), but I wouldn’t bet on it happening that way again. If I told you, “I have a design firm. No programming skill. We’re going to hire an offshore contractor to build an application and let them do whatever they want.” Would you invest? I wouldn’t.

If you want to repeatably manufacture an open source ecosystem, you need capital to do so. And a firm that’s progressive enough to understand the indirect payoffs of investing in infrastructure is poised to have a huge advantage.

While my VC story above is a generic, average one, there are also oddball investments. Before we get to NPM exactly, I want to talk about a few others. The first one is not actually a VC investment itself, but it is a total curveball that caused a lot of waves. Let’s talk about 37signals and Jeff Bezos. In many ways, 37signals is the anti-VC. A boostrapped, incredibly profitable company. So why take money from an investor? Well, money isn’t the only things investors have to offer. From the announcement:

Since we launched Basecamp we’ve been contacted by nearly 30 different VC firms. We’ve never been interested in the typical traditional VC deal. With a few exceptions, all the VCs could offer us was cash and connections. We’re fine on both of those fronts. We don’t need their money to run the business and our little black book is full. We’re looking for something else.What we’ve been looking for is the wisdom of a very special entrepreneur who’s been through what we’re going through. Someone who sees things a little differently and makes us feel right at home. Someone with a long term outlook, not a build-to-flip mentality. We found a perfect match in Jeff. Jeff is our kinda guy.

37signals themselves have said many, many times that they would never sell the company. So where’s Bezos going to get his liquidity event from? Jeff doesn’t actually need Basecamp (which 37signals abruptly re-branded to last week, so let’s abruptly re-brand them here) to experience a liquidity event to get what he wants out of this investment. Bezos invests in all kinds of things. And, as Basecamp mentions, his advice and connections are invaluable.

The next two investments that fall in this category of “force multiplier for other things I’m doing that I don’t have to worry about getting my money back directly” are both from Andreessen Horowitz. a16z is in my opinion the most progressive and interesting VC firm in the valley. They make a lot of weird, oddball, long-shot investments, and they tend to pay off. There are two (and a half) that directly fall into the category of investment I’m talking about in this post, and that’s GitHub and Meteor (and CoinBase). Let’s talk Meteor first.

Meteor Development Group got $11.2MM in a round led by a16z. While MDG claims they’ll eventually release some sort of commercial product, I don’t think that’s what really matters. If Meteor is the new Rails, then a16z gets the most inside track possible to that entire new ecosystem. And given that most VC investments fail anyway, this is a really good deal for a16z. They’re purchasing a lottery ticket, but it’s more than that: it’s a lottery ticket that makes future lottery tickets worth more money. They can invest in future Meteor-based startups, knowing that Meteor development will continue for a long time. They’ll always have their nose to the ground on any promising companies built around Meteor, earlier than everyone else. Knowledge and information is power.

Secondly is GitHub. GitHub raised $100MM from a16z, and a16z alone. At the time, I was super mad, because I said exactly what everyone has been saying about the NPM investment. As time goes on, though, I don’t think that GitHub needs to experience a liquidity event. While we don’t know the valuation of GitHub, I do know that an acquisition doesn’t make any sense, and I’m increasingly of the opinion that an IPO doesn’t either. This is based primarily on subjective opinions that I’ve gleaned from being friends with so many GitHubbers over the years, and I don’t think an IPO is impossible, but unlikely. So why invest?

a16z famously believes that software is eating the world. They even reference this article in the title of the GitHub announcement. Investing in GitHub makes a ton of sense when viewed through this lens. In fact, if I had to guess, I’d bet that a16z got a small enough chunk that GitHub’s valuation is insane. (And I try not to use that word.) This means that a16z and GitHub management are protected from a acquisition, and can retain control of GitHub. Why does that matter? Well:

“Scumbag Steve image saying”Open source infrastructure should be free” / “pushes all code to GitHub”” Apparently memegenerator got hacked, and it was giving this page a warning.

Increasingly, GitHub is where all the open source code is hosted. And open source is the way business is going. And GitHub makes money off of the code that’s not open. In many ways, the open source stuff is a loss leader, so that you get hooked on the GitHub interface and then use it for everything, even your closed source code. This is a great business strategy, but it also places GitHub at the center of this new universe. So for similar reasons, a16z needs GitHub to be awesome to bootstrap every open source ecosystem that will exist into the future. It’s the infrastructure for the infrastructure. The primordial ooze of the 10x return. And a16z has the money to ‘throw away’ on something they won’t get a direct return out of, because they’re smart enough to invest some of their fund in ecosystem development.

The extra half-investment I’m talking about is CoinBase. They got $25MM from a16z and others, and Andreesen himself has written about BitCoin. By way of analogy:

“Software will eat the world” : GitHub :: “Why Bitcoin Matters” : CoinBase

I only give them a half in this case because it’s not clear to me that CoinBase will be dominant in the way that GitHub is dominant, and that Bitcoin will build a new payments ecosystem the way that open source has built a new ecosystem. But some entity like CoinBase is necessary for Bitcoin to succeed. More on that in a future post, someday.

npm?

So by now, I hope it’s clear how I view this NPM investment: True Ventures puts in money because the ecosystem around Node.js is still young, and a package manager is one of the biggest parts that requires money. Someone has to pay that S3 (I guess CouchDB) and EC2 bills. So they bootstrap the ecosystem, and also give a multiplier on their future investments. They get insight and access at unprecedented levels into the future of Node, at the cost of something they can already afford to lose.

We’ll see if I’m right in ten years.

How Dogecoin changed my perspective on cryptocurrency

Sat, 21 Dec 2013 00:00:00 GMT

I have long been a Bitcoin hater, for many reasons. But then, a fortnight ago, something happened. Dogecoin came into existence, and for some reason, I just couldn’t get mad about it. After some thoughtful self-evaluation this week, I’ve realized that Dogecoin has actually reversed my position on cryptocurrency. Here’s why: basically, I’ve come to see Dogecoin as an absurdist art project, mocking the entire global financial system.

Investment Disclaimer: This webpage is provided for general information only and nothing contained in the material constitutes a recommendation for the purchase or sale of any security. Although the statements of fact in this report are obtained from sources that I consider reliable, I do not guarantee their accuracy and any such information may be incomplete or condensed. Also views expressed in this Report are based on research materials available from sources considered reliable. Views are subject to change on the basis of additional or new research, new facts or developments. The investment risks described herein are not purported to be exhaustive, any person considering an investment should seek independent advice on the suitability or otherwise of the particular investment. Investment products are not insured by government or governmental agencies. Investment and Treasury products are subject to Investment risk, including possible loss of principal amount invested. Past performance is not indicative of future results: prices can go up or down. Investors investing in investments and/or treasury products denominated in foreign (non-local) currency should be aware of the risk of exchange rate fluctuations that may cause loss of principal when foreign currency is converted to the investors home currency. I personally only hold about 40,000 DOGE.

Okay, maybe I’m taking something that’s silly way too seriously. Do I really believe that sentence? This question was key for my own understanding of Dogecoin and where it fits into the world. It’s the first question many people ask when they first hear about Dogecoin:

Wait. Is Dogecoin a real thing?

— horse >>= js (@jcoglan) December 18, 2013

Searching for “is dogecoin real” on Twitter reveals a lot of people who harbor the same confusion. Here’s the thing: if you are confused if Dogecoin is ‘real’ or not, you’re basically on a wild ride. Critically investigating the ‘realness’ of Dogecoin leads you to realize that it’s basically just as ‘real’ as ‘real money,’ whatever that means. And if Dogecoin is absurd, then so is the entire financial system.

Dogecoin as symptom of the postmodern condition.

— Steve Klabnik (@steveklabnik) December 19, 2013

Wait, let’s back up here. What do I mean when I say Dogecoin is ‘just as real’ as, let’s say, Euro? And in economics, most questions end up leading to “What is the meaning of ‘value’?” It’s one of the most fundamental questions of the field. And there are many different positions. For the purposes of this post, I’m going to explain things from the perspective of the “subjective theory of value,” which is arguably the most dominant explanation of value in the field today. I’m painting this concept with a pretty wide brush. Please note two things: I’m not an economist by training, I just enjoy reading lots about it, and I don’t subscribe to the subjective theory of value. That said, I am trying to represent it as accurately as possible in this post.

The simplest formulation of the subjective theory of value is this: something has whatever value a person says it has. Put another way, my coffee cost $3 because a bunch of people all agree that it’s worth $3. I, as an individual, have goals that I want to accomplish. In this case, it’s to write this blog post. To accomplish this, I made a rational decision: make some coffee at home, or go to the coffee shop. I have various resources that allow me to accomplish this goal, namely, that I have three United States dollars. So I take all the information that’s available to me, and make a decision: three dollars is a pretty small amount of money to me (I’m lucky in this regard), I don’t have anything else I need to do at home, the coffee shop hasn’t changed its prices at all in the last week, so I can be pretty sure it’s still $3. So I decide that getting out of the house is worth at least three dollars to me, and I got to exchange my 3 USD for one latte. You can see how endlessly complex this is: if I had chronic pain, for example, I may not want to walk down the steps, so that changes my decision. Maybe I needed to do laundry at home, so staying at home and drinking regular coffee makes more sense. I can’t make a latte at home, so maybe I get out of the house anyway because I’d prefer it to black coffee today.

Anyway, that’s it. Seems pretty simple, right? So what about Dogecoin?

Well, if you think about my example more, I actually have a number of commodities that I’m using to exchange with others to achieve my goal. For example, I also ate a sandwich here at the coffee shop, so I actually spent $6, and that’s partially because I didn’t have any food at home. The coffee shop I went to has good sandwiches, so I chose it over other places that may also have decent lattes. So in my example, I’m trading $6 USD for a latte and a sandwich, and for the right to sit here and use up a seat and some wifi. The woman working behind the counter is trading her time and labor for a wage paid by the company who owns the shop, and the company is trading some coffee beans, a bagel, and some other things with me, through the woman, in exchange for my money.

TL;DR: buying a cup of coffee is surprisingly complicated.

Okay, so now that you’ve seen there are way more commodities involved in this transaction than just USD, why is USD special? If this scenario played out in Paris, I would have been using Euro instead of USD. There’s no inherent reason that a piece of paper has a certain amount of value. There is a subjective reason, though: namely, that a large enough number of people agree that USD is a worthwhile commodity. They believe that for a number of reasons, including that our Army will go off and fucking kill anybody who threatens to break the spell that makes everyone agree that USD are worth your time to accept.

Anyway.

But really, that’s the issue, right? Everyone agrees that dollars are worth something, and we trade them all the time. They’re really easy to trade. I don’t know of any coffee shop near me that takes Euros, so even though 1€ is “worth more” than $1 USD, the Euro note I have in my backpack is basically useless for me to get some coffee. Economists call this property ‘liquidity,’ and it basically means ‘how easy is it for me to get what I want with this commodity.’ USD is very liquid. Euro is very not liquid, unless I deposit it with my bank, which will exchange those Euro for me for USD, which I can then trade for my coffee. Most discussions around the ‘realness’ of cryptocurrency are really arguments about its liquidity. Bitcoins aren’t real because I can’t buy anything with them. Really, liquidity is the aspect that matters to most people when it comes to basic financial transactions.

This brings me to MtGox and CoinBase. These two companies are the biggest things to happen in cryptocurrency, and that’s because they significantly improve the liquidity of Bitcoins. MtGox is a place where you can trade Bitcoins for USD. As soon as you have an exchange like this, your Bitcoins now become incredibly more liquid: you can always trade some BTC for USD, and then trade USD with… anything. Coinbase is an online wallet that allows you to store Bitcoins, but more importantly, it provides liquidity to the BTC market because it allows merchants to ‘accept Bitcoins’ without exposing themselves to the risk of Bitcoin’s price fluctuations (this is called ‘volatility’). As a merchant, I can integrate with Coinbase, and someone can, upon checkout, pay with their Coinbase account. It’s sort of similar, from the perspective of the buyer, as paying with your PayPal account. Anyway, here’s the crucial bit: Coinbase then instantly converts the Bitcoins to USD, and then gives the USD to the merchant. This is incredibly important for the future of Bitcoin. By allowing merchants to not worry about the fact that their customers are paying in BTC, it means that more of them will ‘accept Bitcoins,’ which increases Bitcoins’ liquidity in two ways: first, that they can buy more things in BTC ‘directly,’ as well as adding a healthy amount of orders on the various marketplaces, which helps stabilize the price.

I haven’t said the word “Dogecoin” in a while, so you’re probably wondering where I’m going with this. So here’s the thing: as long as BTC and USD can be exchanged easily, it allows for a significant opening for all other cryptocurrencies. This is because BTC is incredibly liquid when it comes to other cryptocurrencies: there are a number of exchanges that will trade whatever kind of the cambrian explosion of cryptocurrencies you want for BTC. Basically, BTC is sort of the ‘world reserve currency’ of cryptocurrencies. So as long as the link to the ‘real world currencies’ exists, they can cross the gap. Imagine a “Dogecoin VISA”, a credit card where I hold a balance in DOGE. When I go to buy something with my Dogecoin VISA, the backing bank would query the DOGE-> BTC -> USD rate, and then ship the merchant that many USD, and debit me that many DOGE. What if I want to trade EURO for DOGE? EURO -> USD -> BTC -> DOGE. Done. The weakest and most tenuous link here is the BTC <-> USD bridge. As long as it’s maintained, cryptocurrency is liquid, and therefore, ‘real.’ Furthermore, the entire financial system can be re-built on the cryptocurrency side of the bridge, as well. Imagine a site that allowed you to “Dogecoin IPO,” listing your stock on the DSE (Dogecoin Stock Exchange). TWTR -> USD -> BTC -> DOGE -> MYCOMPANY. First step is on the NYSE, second step is on MtGox, third step is on Coinedup, fourth step is on the DSE. It all works. Here’s the truly scary thing: we heavily regulate the finance industry, because when we don’t, bad things happen. Hell, when we do, bad things happen. As long as cryptocurrency isn’t taken seriously, it too, will remain unregulated, which allows for a massive extraction of wealth. The cryptocurrency community is largely made up of people who have no idea what they’re doing, and people who do know what they’re doing are starting to notice. It’s all of the worst, most predatory aspects of the global capitalist system, without any consumer protection.

Regardless, there’s an initial ‘bootstrapping’ phase for anything, and I think Bitcoin is past that point now. It doesn’t matter if I think Bitcoin is a terrible idea, it’s going to be a thing regardless, so time to figure out what that actually means.

So the statement that Dogecoin poses to us is this: If things really are worth whatever anyone says they’re worth, so are memes. Do we really want to live in a world where a dumb joke about dogs that can’t talk has the same legitimacy as money? Because the more people that take Dogecoin seriously, the more serious it becomes… and the very nature of its silliness encourages us to think seriously about the nature of money and therefore value.

Before, I just laughed. Now, thanks to a joke, I’m scared.

You can tip me in Dogecoin at this address: DEJf4pmo7a87FeZP1fZAT8jPjDHnKL9r8C or in Bitcoin at this address: 13BHd8tYA41JMuoBpLTGjyXfrnc6Xi6ma5

git, history modification, and libuv

Sat, 30 Nov 2013 00:00:00 GMT

There has been a recent kerfuffle over a pull request to libuv: it was rejected, applied, reverted, and re-applied. There was some question as to the authorship of that commit, and I wanted to show you why that was, because it illustrates how git handles history.

At first, the pull request was closed. Then, it was committed in 47d98b6. Then 804d40ee reverted that 47d98b6. But when you look at 804d40ee on GitHub, you’ll see no indication of which branch it’s on. That’s because it’s not on any branch. If you clone down libuv, and try to find the commit, you’ll see it’s not there:

~/libuv(master)$ git log 804d40e                                                                                                                
fatal: ambiguous argument '804d40e': unknown revision or path not in the working tree.

What gives?

Let’s make a test repository:

$ mkdir test                                                                                                                                                       
$ cd test                                                                                                                                                          
$ git init                                                                                                                                                         
Initialized empty Git repository in /home/action/test/.git/                                                                                                        
$ touch HELLO.md                                                                                                                                                   
$ git add HELLO.md                                                                                                                                                 
$ git commit -m "initial commit"                                                                                                                                   
[master (root-commit) 646567b] initial commit                                                                                                                      
 1 file changed, 0 insertions(+), 0 deletions(-)                                                                                                                   
 create mode 100644 HELLO.md

Okay, now let’s make a second commit:

$ touch WHATEVER.md                                                                                                                                                
$ git add WHATEVER.md                                                                                                                                              
$ git commit -m "adding whatever"                                                                                                                                  
[master 7c232cc] adding whatever                                                                                                                                   
 1 file changed, 0 insertions(+), 0 deletions(-)                                                                                                                   
 create mode 100644 WHATEVER.md

Actually, that commit was a mistake. Since we haven’t pushed anywhere yet, let’s just use git reset --hard to just throw it out.

$ git reset --hard HEAD~1                                                                                                                                          
HEAD is now at 646567b initial commit

But what about that commit? Where is it?

$ git show 7c232cc                                                                                                                                                 
commit 7c232cceb94a2e7cdd95c526de785efe08da2325                                                                                                                    
Author: Steve Klabnik <steve@steveklabnik.com>                                                                                                                     
Date:   Sat Nov 30 20:19:26 2013 +0000                                                                                                                             
                                                                                                                                                                   
    adding whatever                                                                                                                                                
                                                                                                                                                                   
diff --git a/WHATEVER.md b/WHATEVER.md                                                                                                                             
new file mode 100644                                                                                                                                               
index 0000000..e69de29

It’s still in the repository. We can use git reflog to see our changes:

$ git reflog                                                                                                                                                       
646567b HEAD@{0}: reset: moving to HEAD~1                                                                                                                          
7c232cc HEAD@{1}: commit: adding whatever                                                                                                                          
646567b HEAD@{2}: commit (initial): initial commit

Git calls these kinds of commits ‘unreachable’ because, while they exist in your repository, you can’t find them unless you know their SHA. They’re not connected to any tag or branch. We can use git fsck to find these for us automatically:

$ git fsck --no-reflogs --unreachable                                                                                                                              
Checking object directories: 100% (256/256), done.                                                                                                                 
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5                                                                                                          
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325

Some Git commands will run git gc, the Git garbage collector, as part of their normal operation. However, if we run git gc ourselves, it doesn’t look like anything happens:

$ git gc                                                                                                                                                           
Counting objects: 5, done.                                                                                                                                         
Delta compression using up to 8 threads.                                                                                                                           
Compressing objects: 100% (3/3), done.                                                                                                                             
Writing objects: 100% (5/5), done.                                                                                                                                 
Total 5 (delta 0), reused 0 (delta 0)                                                                                                                              
$ git fsck --no-reflogs --unreachable                                                                                                                              
Checking object directories: 100% (256/256), done.                                                                                                                 
Checking objects: 100% (5/5), done.                                                                                                                                
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5                                                                                                          
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325

What gives? Well, git gc has some settings that indicate how long it will let an unreachable commit lay around in your repository. The gc.pruneExpire configuration variable controls this behavior, and it defaults to 14 days. So even if we try to throw away a commit, and even if we manually run the garbage collector, we’ll still have the commit for two weeks.

We can, of course, tell git gc to ignore the setting:

$ git gc --prune=now                                                                                                                                 
Counting objects: 5, done.                                                                                                                                         
Delta compression using up to 8 threads.                                                                                                                           
Compressing objects: 100% (3/3), done.                                                                                                                             
Writing objects: 100% (5/5), done.                                                                                                                                 
Total 5 (delta 0), reused 5 (delta 0)                                                                                                                              
$ git fsck --no-reflogs --unreachable                                                                                                                              
Checking object directories: 100% (256/256), done.                                                                                                                 
Checking objects: 100% (5/5), done.                                                                                                                                
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5                                                                                                          
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325

Uhhhh what? It turns out that git gc won’t touch commits that are still in our reflog. So let’s clear that:

$ git reflog expire --expire=now --all                                                                                                                             
$ git reflog        
$

Good. And now, let’s take out the garbage:

$ git gc --prune=now                                                                                                                                               
Counting objects: 3, done.                                                                                                                                         
Writing objects: 100% (3/3), done.                                                                                                                                 
Total 3 (delta 0), reused 3 (delta 0)                                                                                                                              
$ git fsck --no-reflogs --unreachable                                                                                                                              
Checking object directories: 100% (256/256), done.                                                                                                                 
Checking objects: 100% (3/3), done.

Easy! When you fetch from a remote repository, Git does not include unreachable commits. That’s why when we cloned down libuv earlier, we didn’t get the orphaned commit.

Anyway, as you can see, it’s actually really hard to lose your data with Git, even if you explicitly try to throw it away. This is why you can rebase with… well, maybe not reckless abandon, but you don’t have to worry about your data. All it takes is a quick git reset --hard HEAD@{1} (read: reset me to the last entry in the reflog) and you’re back to where you were before that operation you screwed up.

So, what happened with that commit on GitHub? Basically, one committer pushed 804d40ee, and then another did a git reset --hard to remove it, and then force pushed up to the repository. That commit still lives on GitHub, and is still viewable in their interface, but in some time, when GitHub runs git gc on the repository, it will go away.

Abstract vs concrete

Thu, 14 Nov 2013 00:00:00 GMT

When I talk to people about systemic inequalities, one of the hardest things for them to understand is the concrete nature of how these relations affect the day-to-day lives of others. For many, this topic is an abstract discussion. For many others, this is life, day to day.

In the interest of helping bridge this gap, I want to share a story with you from a few weeks ago. The setting: the West Village, Manhattan. The night: Halloween. In the early morning hours of All Saint’s Day, I got a text message from someone I care about. Here’s some screenshots of the conversation. She has seen this post and said it’s okay to share, for the same reasons I’m interested in sharing this with you. Also, if reading a story about a bad situation or a slur or two will make you upset, you may want to read another blog post.

I’m not sure how to convince random people on Twitter that yes, in 2013, in the West Villiage, in the largest city in the country, on a holiday night when everyone is out, you still can’t let your guard down. Even if you’re not actually a gay man.

And this is just one story.

New Beginnings

Tue, 01 Oct 2013 00:00:00 GMT

TL;DR:

WANTED: Proletarian/petite bourgeoisie looking for capitalist/industrialist/haute bourgeoisie. I’ve got the labor power, you’ve got the money commodity. Let’s exchange commodities at full market value in accordance with the Law of Value; I know you’ll extract some surplus and alienate my species-being, but I really need to regenerate and reproduce myself so that I can be around to labor tomorrow.

Yes, that’s right, I am officially looking for work. I loved my job at Jumpstart Lab, but all good things must come to an end. So here we are. There’s a farewell post on the JSL blog too.

So, what’s next? To be honest, I’m not fully sure. I’m open to suggestions, and I give bonus points for interesting.

Things I want out of life:

Travel. I still like doing this. I just don’t want to do it constantly.
Related: some degree of location independence. I’ll only accept positions that require me to be in a seat that’s in San Francisco or New York City. Remote would be even better. Probably.
Piles of money. I have student loan debt I’d prefer not to have.
Side project ability. I am highly motivated by doing side projects that are different than I do all day.

… I’m not saying that Ruby is a minus, but I’m not saying it’s a plus, either.

I’m also open to re-locating out of the country. That certainly counts as interesting.

So, if you think we might work well together, please, send me an email.

The CLOSURE companion

Thu, 19 Sep 2013 00:00:00 GMT

Today is the fourth _why day. So I thought I’d say a few words about his final gift to the rest of us, CLOSURE.

Be Skeptical

First, a few words about the words I’m about to write. Software developers (though you don’t have to be one to read this) love meta, so surely you’ll indulge me for these next few paragraphs. Don’t worry, it’s only a little bit of meta-data.

And if you don’t understand some of my language, things like “Open GL” and “fighting NULL” and so on, you can skip it and be fine I think. - _why, CLOSURE

Nowadays people are actually somewhat jaded by the term “postmodern’’. Well, perhaps jaded is an understatement. Nauseated might be more like it. - Larry Wall, “Perl, the first post-modern programming language”

I define postmodern as incredulity toward meta-narratives, - Lyotard, “The Postmodern Condition: a report on knowledge”

_why’s work has always taken on a strong element of the postmodern, and CLOSURE is no different. So in order to really get CLOSURE, you need to get a little postmodern. So let’s breifly examine this statement by Lyotard.

A ‘narrative’, to Lyotard, is a story that organizes tribal knowledge, the structure of our life and society. Therefore, a meta-narrative is a narrative about a narrative. As an example, let’s say you want to make sense of programming languages. A meta-narrative about programming languages would be that in the beginning, we started off with assembly language. Then we moved forward to structured programming, with Pascal and C and such. Next came object oriented languages, which is where we’re mostly at today. We’re starting to finally lean functional.

Now, this narrative works. It’s not wrong. But it also isn’t really true, either. After all, Lisp existed waaaay before the contemporary moment, yet it’s often functional. Actually, presenting Lisp as functional is in of itself not exactly true. So this meta-narrative, while it helps us understand a bit about programming languages and their history, also has a bunch of lies. It’s not black and white, it’s quite gray.

“Well, it doesn’t matter – no one’s going to listen to either of us. I can say whatever I want at this point. You can tell the truth and no one would care.“That’s amazing.”“I’m going to change my story weekly – the more confusion the better. The Jonathan story is locked.”_why, CLOSURE, HELLOYES (p56)

So what I’m saying is this: I’m about to present you with a meta-narrative about CLOSURE. So be skeptical of me! What I’m about to tell you is a lie, I swear. Don’t take what’s about to come as the final word, some sort of actual truth. It’s just my own little anti-memory that I’d like to share with you.

What CLOSURE is

CLOSURE is a few different things. 90% of what you need to know is located in DISCLAIMER, which is page 13 of CLOSURE. In it, _why mentions a few things:

It’s ‘perilous to communicate this way’
Everything he’s done belongs to us
He likes what we’ve done so far
This will be the last thing from him.

#4 is a big deal. No more waiting for _why to come back. That’s it. No more questions, no more wondering, no more guesses. This is all we have.

For me, #3 was the most important. It brought me to tears, to be honest. That’s what I personally really needed to hear. It’s very strange to take up the work of someone you’ve never met, and make their life’s work your own. I spent quite a bit of time really wondering if keeping some of his stuff alive was the correct thing to do, and so this was a big relief. It was all gravy after that. #2 is related, it’s all public domain. That’s good to know.

Identity

#1 is interesting. It could just be flavor, but I think it’s something more than that. One of the major themes of CLOSURE is identity, and _why / Jonathan ’s struggles with it. On page 19:

Now, could you please tell me the point of this ridiculous anonymity exercise, hmm?As it turns out, oddly enough, your real self is just an unknown programmer from Utah. The myth is that easily dispelled. Why not making something of your real self? (Of course I know why and can tell you: Because your fear of the world has clouded your ability to do things to improve your situation. You are stuck there in Draper, Utah, until you can cut through the paranoia!)Please, Mr. Gillette, come on in. The water’s fine. ;)Emery Pestus, CLOSURE

While many hold _why as some kind of superhero, it’s really important to remember that he’s human. We, the Ruby community, placed an indescribable amount of pressure on a human being to play a character for us, without consideration for his feelings. Think about this for a moment: could Jonathan ever write Ruby code? As Jonathan? What if it had unit tests? If it was just some old, normal, boring gem that added a little feature to Rails?

Could we let him just be him? I’m not sure. I’m not sure he could let him just be him, either.

Once, I met someone for dinner in Chicago. This was the first time we’d met, and so we were asking each other a lot of questions. One of my answers yielded an interesting response: “Oh, you’re like actually for-real. All that twitter stuff isn’t an act. That’s really you.”

Impermanance

Impermanence is possibly the biggest question raised in CLOSURE.

kafka would be a lot harder to get into if the trial only ran on a power pc. - one of _why’s last tweets

This tweet was really confusing, until CLOSURE. _why reveals that one of his biggest problems is what we call ‘bitrot’: you can’t just write a program, it must be updated. I have one of the first C programs I ever wrote, from when I was 12, and it wouldn’t compile on a modern system, due to a ‘clear screen’ function I wrote that interacted with the hardware that I owned at the time. I don’t have any of the GW-BASIC programs that I wrote as a child, because there was no way to transfer the source off of the computer: I didn’t have a disk drive, they were too expensive.

And so it is with Kafka. Right before he died, Kafka asked his friend Max Brod to burn everything he ever wrote. Brod published it instead. _why brings up that if The Trial was written for the PowerPC, he wouldn’t have needed Brod to burn it: it would have just naturally gone away.

Our industry is constantly changing, and that’s great. But we have no institutional memory. We keep making the same mistakes, over and over again. _why talks about fighting NULL, and how that was his biggest struggle as a programmer. The guy who invented null pointers, Tony Hoare, calls it “my billion dollar mistake”. Yet Go has null pointers.

It’s really easy to burn out. I won’t lie, when _why deleted himself, I thought it was a terribly silly idea. But the more stuff I do, the bigger my projects and responsibilities get, and the more of a public person I am, the more it sounds appealing.

When I had a shitty band in high school, I wrote a dumb song after a girl dumped me called “The Only Constant is Change.” I thought it was clever at the time.

The Software Industry

Speaking of the industry, and burnout, that’s the last part of CLOSURE, and the hardest. The entire end of the book is about it, and it’s a very obvious, very ridiculous allegory.

There is a cult of Steve Jobs, and they speak incorrect French. They obsess over these flutes, and the flutes can make more flutes, and when one of them gets old and dies, they forget him immediately. When a new person is born, they start worshipping him, and induct him into the cult as quickly as possible, by clothing him in blue jeans and a black turtleneck. The cult members never eat any food.

The cult is programmers, the flutes are programming. It all falls out of that. Songs on the flute always start with C#, which I found amusing.

It occured to me that they could use the flutes to describe a whole landscape, to transfer a map from one man’s mind to another._why, CLOSURE, page 84: SPAWN

There’s also a really fantastic comparison between Steve Jobs and Bill Gates, as well as an angel investor who wants to invest in _why: I highly recommend that part.

For instance, many people struggle with acceptance, feeling like they aren’t accepted by other people. But what I deal with is primarily hatred of entrepreneurs. But it’s something that I’m always working on and I’ve gotten much better.

I’ve come to realize that I’m starting to become a grumpy old man. These damn kids never learn from their elders, we solved all the world’s problems back in the 60s and 70s, and they don’t know anything. I wish we had a better way to transfer institutional knowledge and stop repeating ourselves.

Some final thoughts

There is so much more that’s enjoyable in CLOSURE. Please check out the full thing, even if you don’t get all of it. For example, the part on Neil Gaiman is amazing.

I hope these jumping off points helps you in your journey. Let’s all have some fun with programming, okay?

Just the regularly scheduled apocalypse

Tue, 10 Sep 2013 00:00:00 GMT

Every once in a while, I unfollow EVERYONE on Twitter. I did it again last night. Feels good, like a fresh start.

Part of the problem is that I feel like I should be following everyone who I know in real life. This causes Twitter to be absolutely unmanageable, as I’m following far too many. So then I just dip into the stream every so often, but then I miss some people’s tweets entirely. So why follow them all in the first place?

So, if I’ve unfollowed you, don’t worry, I still <3 you. And even as I start following people back, I’m doing it slowly, and at semi-random. Trying to get some people I hadn’t seen in a while. People from outside my own community. People that’ll make me a little less caremad.

Yeah, like that’ll last.

Emoji licensing

Tue, 06 Aug 2013 00:00:00 GMT

I recently decided to make an emoji gem. Of course, as with any project, I first look into the relevant licensing issues. What I found made me :cry:. Here’s the low-down on emoji and intellectual property law.

A history

So what are emoji, anyway? The Japanese spelling is 絵文字: 絵 (e) means ‘picture’ and 文字 (moji) means ‘letter.’ Picture letters. Simple.

Back in the day, three different Japanese phone carriers created their own versions of these smilies: docomo, au, and SoftBank. Codes would be sent down the (nonexistant) wire, which would then be turned into the final character. However, the three carriers used different characters and different codes.

Google and Apple petitioned the Unicode Consortium to include a number of these characters, and version 6.0 of the Unicode Standard ended up with 722 emoji.

The font

Apple ended up implementing a font named Apple Color Emoji that included these symbols. It came first with OS X Lion and iOS 5. These are the characters that you’re probably familliar with. Apple actually implements the font with a proprietary extension to OpenType, but that’s not particularly important. What is important is two things: typing a character that’s not on a normal keyboard, and displaying custom fonts on the web. These two problems have led to the current emoji IP situation.

Typing extended characters

On an iPhone, you can type emoji easily by enabling the extra keyboard. That’s not hard. But what about on a computer? Web sites that wanted to use emoji needed an easy way for you to type these characters in an input box.

I’m not sure who was first, but the first I remember is GitHub: with their announcement post, showed off a slick way to input emojis: use a set of characters that would get replaced with the emoji. So :heart: becomes a heart. Easy!

Custom fonts in browsers

That takes care of input, but what about output? If you don’t have the proper font installed, you’ll get those terrible boxes. That’s not very useable. Also, even if you do, Chrome won’t display them.

So what do you do? You use your good buddy <img>.

If you’re building a web application, you are already telling users to type :heart: rather than the character, you can just make :heart: be rendered as <img src="/assets/heart.png">. So that’s easy…. but where do you get the PNGs?

Apple Color Emoji

Remember that extension? The spec was published, and is being worked in to FreeType.

Anyway, the extension works by embedding PNGs into the font. I think that you can see where this is going… just rip the images out of the font, and you’re golden. Well, other than that pesky IP law…

Extensions

Furthermore, now that you’re not constrained by the unicode standard, you can make anything between two colons an emoji. Like :shipit:. Several services do this. http://emoji-cheat-sheet.com lists the licenses as such:

octocat, squirrel, shipit
Copyright (c) 2012 GitHub Inc. All rights reserved.

bowtie
Copyright (c) 2012 37signals, LLC. All rights reserved.

neckbeard
Copyright (c) 2012 Jamie Dihiansan. Creative Commons Attribution 3.0 Unported

feelsgood, finnadie, goberserk, godmode, hurtrealbad, rage 1-4, suspect
Copyright (c) 2012 id Software. All rights reserved.

trollface
Copyright (c) 2012 whynne@deviantart. All rights reserved.

All other emoji images
Copyright (c) 2012 Apple Inc. All rights reserved.

37Signals had said :neckbeard: is CC licensed, and in an email, said they’d be happy to license :bowtie: the same way.

I didn’t even bother emailing Apple.

What’s this mean for me?

Basically, if you’re distributing the images from Apple Color emoji, you are violating copyright law. And if you’re not one of the above mentioned entities, you’re violating their licenses, as well.

So that’s a big downer. What to do?

Phantom Open Emoji

At some point, there was a kickstarter for an open emoji set. And it worked! And so Phantom Open Emoji was born.

Phantom Open Emoji basically gives you images to use as all of the glyphs used in Apple Open Emoji, but you have a license to use them however you’d like. So that’s pretty awesome.

I wrote this post because I was writing a Ruby gem for emoji, and needed to know if I could distribute the emoji with the gem. It’s not done yet (it’s barely started), but if you want to use emoji with your app, it will be the simplest and safest way to make sure you don’t run afoul of this kind of thing.

Returning to Free Software, a guide

Sat, 15 Jun 2013 00:00:00 GMT

A month or two ago, I made a conscious decision that I would return to using as much Free Software as possible. The PRISM debacle of the last week confirmed my fears and reasoning, and so I made the decision to accelerate the schedule.

Here’s what I did, and how you can, too.

My GNU/Linux Background

The first computer I ever used was an Apple machine, and at various times in my life I’ve used GNU/Linux as my main OS. So I’m already pretty familiar with what’s involved.

That said, it’s only gotten better over the years. Especially if you buy hardware that is known to work well, using GNU/Linux as your main OS shouldn’t be a big deal. One of the nice things about the rise of OS X in the developer community is that they’ve come to rely on the standard GNU tools, and so much of what you’re already working with will probably translate straight over. Of course, if you work on iOS or something, this won’t be the case, but for your run-of-the-mill web developer, there isn’t a whole lot of difference.

Well, other than that whole ‘knowing exactly what’s on your machine’ thing.

Hardware: X1 Carbon

IBM’s ThinkPad series of laptops have always enjoyed fantastic Linux support. When they were purchased by Lenovo, some were worried that they’d ruin this fantastic hardware line. That didn’t really happen, though, and they’re still fantastic computers to this day. When asking my Twitter followers what they thought, everyone who owned a ThinkPad loved it, whichever model they had.

I happened to get an X1 Carbon. I was coming from a 13” MacBook Air 13“, and the Carbon is very similar. It’s 14” though, which is a really nice size; it still fits in the same pocket in my bag as the MBA did, but I get a little more screen real estate. A note for those who give presentations: the X1 Carbon uses a MiniDisplay port, so save your Apple connectors!

All of the hardware I use has Just Worked; I hear that the thumbprint reader may not, but I don’t plan on using it, so that doesn’t bother me.

If you’re not in the market for new hardware, you can generally run GNU/Linux on your Apple hardware as well, which is a starting point, at least.

Operating System: Crunchbang GNU/Linux

I’ve used a lot of different GNU/Linux distros, and have been a happy Arch Linux user for years. However, this time around, I decided to go with CrunchBang. Crunchbang is a Debian derivative, but includes OpenBox as the default window manager. The result is an OS that’s lightweight, yet very, very functional. I’ve been really happy so far with Crunchbang’s level of polish.

Okay, so actually installing Crunchbang: the X1 Carbon comes with Windows by default, and so we need to get Crunchbang on there somehow. Luckily, you can just use USB. dd is your friend here, which you can use on your Mac, of course. You can grab a copy of Crunchbang from their download page, and they have a link to making the USB stick right on that page.

After you’ve made the USB stick, just reboot your computer with it on, and press down all of your F keys. I was simply too lazy to look up which one would actually let me choose a boot device. ;) The Crunchbang installer was very straightforward, just follow the prompts and you’ll be good to go.

One thing that I did choose was the ‘encrypted lvm’ option. Basically, your entire disk is encrypted except a small boot partition. On boot, it asks you for a passphrase to decrypt the disk. It’s just a matter of choosing the right option, in practice, this has worked and worked well for me.

EDIT: One of the reasons I chose Crunchbang was because I thought that it was solely configuration on top of Debian, which includes 100% Free Software. Checking my sources.list, there is a Crunchbang-specific repository, and I haven’t evaluated it for Free-ness. So if you’re trying for 100% purity, straight-up Debian may be better.

Software

Most of the tools that I use every day are already either Free Software or open source, and so my core experience hasn’t changed: bash, vim, git, and friends aren’t any different.

I did, however, install tor. Tor is not a privacy panacea, however, it does help in many cases.

I’ve added Ghostery, NoScript, and AdBlock Plus to the built-in Iceweasel browser, which is for all intents and purposes Firefox without the Mozilla branding. These allow me to control exactly what information leaks out a little bit more. They also have the nice side effect of making pages load a bit faster, since a lot of the usual bullshit doesn’t even get loaded. (EDIT: it’s been brought to my attention that Ghostery isn’t actually free. Oops!)

I plan on setting up backups with Tarsnap later in the week, which is an excellent backup service that encrypts all of your data locally before saving it off in The Cloud. It has some pretty good (I hear) de-duplication features, and you only pay for the bytes used, so it’s pretty inexpensive. The author, Colin Percival, has said that he has some customers doing a muti-gig daily backup of their home directories, and they’re paying ~$10/month. (I should mention that Tarsnap is not Free, but it is shared source. I don’t plan on using it for anything else, so I’m okay with this tradeoff at this time.)

Most of my development work at the moment is in Ruby, and getting it set up was a breeze. I use the ruby-build tool to compile my Rubies, and the most excellent chruby to switch between them.

One thing that I have yet to explore (but I’d like to) is to use a tool like Chef to set up my personal machine, so that I can periodically wipe everything and re-build from scratch. That’s still on the to-do list, though.

I personally re-map my caps lock key to control. To do this, make a .Xmodmap file in your home directory, and put this in it:

keycode 66 = Control_L
clear Lock
add control = Control_L

I figured this out via a few simple searches. There’s a small chance that your caps lock is not code 66, and if it’s not, you can run the xev command to check it out. Just press your caps lock and make note of the number that shows up in the console.

I had to install grandr in order to share my screen to give a presentation. It’s easy enough to get: sudo apt-get install grandr.

I started giving presentations with rabbit, which is what many Japanese Rubyists use. It’s pretty fun; you write plain text files and it makes your presentation for you, as cairo/pango application. You can also use it to generate PDFs of slides. It’s not for everyone, but I’m enjoying it so far.

Oh, and incredibly important: I typed sudo apt-get install ttf-ancient-fonts to get Emoji support. They’re not as pretty as the Apple ones, but they’re not empty boxes either.

Conclusion

I’m really, really happy with this setup. Very little of my day-to-day has changed, but I have a bit more privacy, and can feel good that I know exactly what software is on my machine. No NSA backdoors here!

I still have improvements to make, but what I have works well so far. Please let me know what you think, and suggest ways to make this setup even better.

Ditching Google Chat with XMPP

Thu, 16 May 2013 00:00:00 GMT

Last week, I decided that it was time to migrate one of my services away from Google: XMPP for IM. The nice thing about XMPP is that you don’t have to totally drop support for your old identity: since Google still uses it (for now), you can get online with both.

Here’s how:

I set mine up with Digital Ocean, because my friends told me it was rad. It was pretty damn easy.

Once you have a server, you need to set up all the usual stuff: a non-privledged user account, yadda, yadda. I’m assuming you can do that.

Install ejabberd

I picked ejabberd for my server software because it’s used by a lot of people.

EDIT: I just got a tweet from @jamesgolick, who I trust to know stuff about stuff. He says:

ejabberd is buggy. Use the erlangsolutions fork instead. - @jamesgolick

So, take that for what it’s worth.

Anyway, since I use Arch Linux, installing was as easy as

$ sudo pacman -S ejabberd

After that was over, time to edit the config. ejabberd is written in Erlang, and so it uses Erlang for its config. Don’t worry! It’s easy. Just copy and paste these bits in, or change them if there’s something like them in your config:

{hosts, ["localhost", "xmpp.steveklabnik.com"]}.{acl, admin, {user, "steve", "xmpp.steveklabnik.com"}}.

These change my host to xmpp.steveklabnik.com as well as localhost and set my admin user to steve@xmpp.steveklabnik.com.

Next up, we boot the node:

sudo ejabberdctl start

And then register a way to administer things via the web interface:

sudo ejabberdctl register steve xmpp.steveklabnik.com $password

Of course, change those to be your values, and change $password to your password. Now that you’ve typed a password into the command line, it’s good to clean out your bash history. When I did this, typing history showed this:

 35  sudo ejabberdctl register ....

So, do this:

$ history -d 35$ history -w

Or, to nuke everything:

$ history -c$ history -w

Anywho, now that that’s over, you need to go to your domain registrar and insert an A record pointing at your server. I use NameCheap.

Now that that’s done, hit this page in your browser:

http://xmpp.steveklabnik.com:5280/admin/

The trailing slash is important, and obviously, you need to change it to be your server. Given that the DNS isn’t cached wrong, you should get a popup asking you to log in. Do so with the credentials you typed in on the command line up there. If you can log in, great! Everything should be working.

Let’s turn off registration, though. Go back to your ejabberd config, /etc/ejabberd/ejabberd.cfg:

{access, register, [{deny, all}]}.

You probably have one that says allow, so just change it. Check in your admin panel on the site that it’s the same.

Bam! That’s it. This whole process took me about an hour, including writing this blog post. I do have some experience with VPSes, though.

If I can help free you of Google in any way, ping me at steve@xmpp.steveklabnik.com.

Node

Wed, 23 Jan 2013 00:00:00 GMT

Soooo there’s this thing: http://harthur.wordpress.com/2013/01/24/771/

First of all, unequivocally, I said it before, and I’ll say it again: I’m sorry, and feel terrible that I made someone feel terrible. @harthur, I felt bad yesterday, and I’m feeling bad again. Not that it helps you that I feel bad. But I don’t like making other people feel bad. I am sorry.

tl;dr

TL;DR: I am sorry. I feel terrible that I made someone else feel terrible. I feel even worse because I didn’t mean to say that they were terrible, but accidentally did.

the why

Sooooooo I wrote up why I said what it was that I said, because what I was trying to say was totally unrelated to replace. I was trying to explain why I don’t like Node, and it basically came out wrong. But, explaining what I said had the affect of feeling like I was trying not to apologize. So, I’ll just cut to the chase and say this, cut straight from the chat log:

SK: i dont want people to think i'm saying "i got caught being an asshole so i'm saying sorry"
SK: i want to say "I was accidentally being an asshole so i'm sorry"

So, I’ll just leave it at that. Twitter makes it so hard not to accidentally be an asshole.

_why is a role, not a person

Wed, 16 Jan 2013 00:00:00 GMT

http://whytheluckystiff.net/ has re-ignited a discussion about _why, and while I have tons and tons and tons of things to say on the subject, this is all I can give you for now: _why is a role, not a person.

For a long time, I was talking about a separation between _why and the person who played him. I still have not yet figured out how to refer to that person, and so for now, I’ve settled on Subject. So I’ll be using that description for the rest of this post.

All the world’s a stage

All the world’s a stage, And all the men and women merely players:Shakespeare, “As You Like It,” Act II Scene VII

One of the things I’ve learned as I’ve gotten a little older (21 days till 27!) is that I’m a different person in different situations. This may seem painfully obvious, but let’s expand on this a bit: when I’m on stage at a Ruby conference, I’m one person, when I’m by myself at home, I’m a different person, online, I’m a different person, and when I’m spending time with someone I care about, I’m someone else. Each of these contexts produces different behavior, mannerisms, and vocabulary. When I’m speaking with someone who’s read French Postmodernism, I use very different terms than when I’m speaking with my Republican ’Murican relatives.

There’s absolutely nothing wrong with this, but it really suggests that ‘essentialism’ is not a thing that can properly model the world. “Essentialism” is a position that states that there is some set of attributes that one can possess to ‘be’ something. So, for example, let’s take a Zebra: some essential attributes may be:

Four legs
Black with White Stripes

http://www.flickr.com/photos/kliski/4816615762/

Shit.

Double shit.

So what makes a zebra a zebra then? I don’t want to get into it, but basically, I take the position that you can’t define something by what it is, you have to define it by what it does. I’m punting on why this is, again, I want to talk about _why, so please give me a charitable reading and take this as axiomatic for now.

If we’re defined by what we do, then it makes sense that we can contextualize what we do: we can take on different roles at different times. When I program in Ruby, I am a Rubyist; when I ride a bike, I am a cyclist, when I have a child, I am a father, when I love and am loved, I am a lover.

When you treat programming as an art, when you treat software as an artistic medium, when you spread joy and whimsy with what you do, you are _why.

It just so happens that Subject was the person who created this particular role, as was the one who best played it. That doesn’t mean that we can’t also try to take that role on from time to time, try out out, see if it fits, wiggle our toes a bit.

_why vs. Subject

This is one of the reasons I think the concept of _why is important: Subject was trying to teach us something. They were creating an abstraction, or, maybe, an indirection. By keeping themselves private, they put an emphasis on the distance between _why-as-role and _why-as-person, since the second doesn’t exist. The focus was, as it should be, on the role.

This is also why it’s wrong to ‘track _why down.’ This doesn’t make any sense. Subject is not _why, they just played them on TV. I also think that’s why it’s not wrong to keep talking about _why, as long as we make sure that we’re actually talking about _why, and not about Subject. Subject deserves their privacy.

I also don’t mean that I want someone to literally ‘take up the mask’ of _why and publish under that name. What I mean is that if we want to take on the lessons that Subject taught us through their persona, we should periodically have becoming-_why moments.

On Sloppiness and blog posts written in haste

@steveklabnik hand write it, take a picture of it. leave the mistakes in.

— tef (@tef) January 6, 2013

An early Christmas present for you

Wed, 19 Dec 2012 00:00:00 GMT

Normally, I’m a bit of a grinch. I’ll admit it, I really hate Christmas. It reminds me of all of the worst parts of America: You know, how the baby Jesus commands us to literally shop until we kill each other…

Anyway, today, I was teaching a bunch of brand new Rails devs, and I had to explain the mysterious hidden input that form_for generates:

<input name="utf8" type="hidden" value="✔">

That, combined with the impending release of Rails 4, made me think about the charming little Snowman that Rails originally had: ☃.

So?

And what happened, then? Well, in Whoville they say - that @steveklabnik’s small heart grew three sizes that day. So he decided to make you a present:

gem install bring_back_snowman

Just do it, and your forms will submit _snowman=☃ to ensure that your data is encoded in UTF-8.

It is configurable, if you prefer something else. Add this to your application.rb:

config.snowman = {:emoji => "😢"}

You may also need to add the ‘magic comment’ at the top of the file:

# encoding: UTF-8

Please remember that if you don’t use something that’s UTF-8, this won’t actually work to fix the bug.

You can find the source of bring_back_snowman on GitHub, of course.

Merry Christmas!

Narratives

Sun, 18 Nov 2012 00:00:00 GMT

Occasionally in this big wide world, things happen. When things happen, we like to tell each other about it. When you take a bunch of connected events and tell someone about it, you’ve formed a narrative. Narratives are important, because they tell the story! The details of the account that one presents can really shape how others view those events later.

Due to our lived experiences, we often have our own ‘internal narratives.’ You can think of them as pre-packaged ways that we tend to think about a given series of events. Human brains are wonderful pattern-matching machines, and so we see similarities in situations, and then apply the same narrative.

Recently, I was playing around with Unicode and Ruby, and was trying to make some funny class names. I ended up figuring some stuff out, and tweeted this, which at the time of this writing, it has 157 retweets and 48 favorites:

This is valid Ruby:class ಠ_ಠendಠ_ಠ.new

— Steve Klabnik (@steveklabnik) November 17, 2012

This tweet is factually incorrect. Yet, only two people pointed this out to me. Why?

Ruby is kind of a crazy language. That’s what I love about it; it’s got lots of wacky little corners, nooks, and crannies. I’ve also been doing Ruby for a long time, and tweet about it often. So here’s the internal narrative when you see this tweet:

Steve knows a lot about Ruby.
Ruby is kinda weird.
That’s really funny.

This particular situation fits perfectly into this internal narrative. Hence, a retweet. No checking. Just tweeting.

These narratives can play off of each other really easily. Now that we’ve seen one more way in which Ruby is kinda crazy, we assume that it’s even more crazy in the future. This tweet has reinforced our internal narrative about Ruby’s wackiness.

So #britruby.

I’m sure you’re expecting me to say something specific about this situation, but I’m not going to give you what you want. What I do want to do is analyze the narrative of this situation, and see how it compares to actual events.

Josh Susser tweets this:

Nice speaker lineup for @britruby. Except for the 100% white guys part.

— Josh Susser (@joshsusser) November 17, 2012

and James Rosen tweets this:

By what metric is @britruby “one of Europe’s… most diverse Ruby conferences”? Every speaker is a white man.

— James A Rosen (@jamesarosen) November 17, 2012

That’s it. If you examine those Twitter pages, you can see some conversation; both were conducted pretty cordially. Then, BritRuby decides to cancel their conference.

Okay. That’s the situation. Now let’s look at two versions of the narrative. First up, the cancellation announcement. I’m going to excerpt the most important parts, with my commentary afterwards.

We started this conference to build a community within the United Kingdom. Our mission statement was to encourage Ruby developers to unite and create a community, which would allow such to network, exchange ideas and provoke innovation for the future of Ruby. We wanted to encourage jobs and allow companies the opportunity to meet the community and primarily boost the UK developer industry.

We wanted to build a great conference that would help UK Rubyists tremendously.

Our selection process was the content and nothing more. Not the individuals gender, race, age or nationality.

This is trivially disproven, as everyone has biases. But the message is “We only accepted the best speakers, regardless of who they are.”

It’s about community. It’s about helping one another to strive for success and drive budding developers to do the same. We contacted universities in a hope that we would be able to empower young minds and show them what a great language Ruby really is. They are the future.

We are team players.

The Ruby community has been battling with issues of race and gender equality. We at Brit Ruby were well aware of this fundamental and important issue. This was one of the reasons why we encouraged everyone to submit a speaker proposal. Sadly, BritRuby was used as the arena to air these issues on Twitter and this has fundamentally destroyed any chance we had of addressing these issues.

BritRuby was ‘used to air these issues,’ meaning, ‘others had an agenda and took advantage of us to promote it.’

Instead the community should have worked together and allowed us to bring these issues to light at the conference.

These individuals are divisive, and we are inclusive.

How can the community address these issues if every time someone tries they are shot down and accused of such awful things?

The British Ruby team although very small consisted of a diverse mix of nationality and gender.

We cannot be tainted by accusations of monoculture, as we are diverse ourselves.

This was a non-profit conference being run by simple developers.

We were not even trying to make money.

The team has been working so hard in their own time to bring a unique conference to you all. Thank you to those of you that dedicated your time, skills and to their families for putting up with the long hours.

We, nay, you and us together all worked really hard, so you should cut us some slack.

Anyway, so there you go. Same events, different story: We are a collective that was trying to do our best. Outsiders took advantage of us to promote their agenda and divide all of us. Therefore, we’re taking our ball and going home.

Here’s the funniest thing about the cancellation: They don’t give an actual reason. The closest they get is ‘was used as an arena to air these issues’ but they don’t explain why that made them cancel their conference. This narrative attempts to shift the blame from the organizers to the critics. “We wanted to give you this beautiful thing, and we worked hard, but these other people ruined it.”

This particular narrative plays off of a well-known trope when discussing diversity issues: a persecution complex by the majority. Majorities fear losing their power, and often will develop internal narratives that they’re the ones being persecuted. Some good examples of this are the Mens Rights movement and the White Power movement. This is a milder form of that same privilege: the power of a single tweet by the ‘politically correct’ is enough to bring the most powerful of plans grinding to a halt.

Let’s examine a few narratives involved in this gist:

And here it is, brought down by careless words.

“These people don’t actually care about us or our conference.”

Yes, gender equality and racial equality are important. But the team’s motives were to get the best speakers who were able to make it to Manchester.

First of all, “but” is an amazing word. You can’t even imagine the things I’ve heard said after “I’m not racist, but.” This sentence implies two things: an absolution from blame on the part of the conference because hey, they care about equality. Their second statement implies that the best speakers all happened to be white men. The converse of this is that non-whites and/or women are not the best speakers, and the overall message is that including them would degrade the quality of the conference.

Adding a token minority speaker is offensive to that speaker, it says “You’re here because you tick a box - not because you’re skilled.”

Anyone who seeks out women or minorities is only doing it to ‘tick boxes,’ because, as we’ve already established, they clearly are not the best speakers.

Oh, and you’re actually the sexist/racist one if you try to encourage them, because it’s patronizing.

It doesn’t matter who speaks at a conference, as long as they’re capable, interesting and relevant. That’s what matters: content, not style.

We cannot be at fault because we don’t see race or gender. Again, basic psychology trivially disproves this statement.

There are a ton of other micro-narratives that occur in all of these discussions as well. Most of these are a derailing tactic, and straight-up distract from the issue. “Egalitarianism is anti-free speech,” “Feminists and their allies are irrational and hysterical,” “I don’t see race or gender,” “If they were calmer about this, it would have worked out,” “This would be better handled in private,” and many, many more. I’ve seen so many. It’s exhausting. These internal narratives get reinforced by a particular interpretation of a given situation, without examining what actually happened.

Yet we still have conferences that are mostly full of white dudes.

I lost a lot of respect for a lot of people today.

How to squash commits in a GitHub pull request

Thu, 08 Nov 2012 00:00:00 GMT

So you’ve contributed some code to an open source project, say, Rails. And they’d like you to squash all of the commits in your pull request. But you’re not a git wizard; how do you make this happen?

Normally, you’d do something like this. I’m assuming upstream is a git remote that is pointing at the official project repository, and that your changes are in your ‘omgpull’ branch:

$ git fetch upstream
$ git checkout omgpull 
$ git rebase -i upstream/master

< choose squash for all of your commits, except the first one >
< Edit the commit message to make sense, and describe all your changes >

$ git push origin omgpull -f

GitHub will then automatically update your pull request with the new commits. Super easy!

Theory and practice

Tue, 06 Nov 2012 00:00:00 GMT

purity

There is always a tension between Theory and Practice. These two separate realms are connected through a process of abstraction and application. To explain this relationship by way of theory, Theory deterritorializes Practice, and Practice reterritorializes Theory: a Theory which is a becoming-Practice and a Practice which is a becoming-Theory. To explain this relationship by way of practice, Theory is abstracted Practice, and Practice is applied Theory.

There’s an age-old problem with this particular relationship: those who specialize in Practice often claim that those who specialize in Theory are detached from the ‘real world,’ ie, the world of Practice. Those who specialize in Theory often claim that those who specialize in Practice have no fundamental understanding of what they do, and this leads to contradictory, incongruous practices.

There’s a third kind of person, though: one that embodies the becoming, the abstraction/application process. These people are a conduit, fundamentally bridging the two worlds. There’s a certain art to explaining just the core of Theory in the words of someone who Practices, and there’s a certain art to combining the essences of Practices and presenting it to those who Theorize. Building this bridge is an act of creation, of building, an opening of space.

Some people are great at Ivory Tower intellectual stuff, and others just don’t really care. It sucks, because those who are doing could be way better at it if they just knew some theory, and those who love to philosophize all day might be more understandable if they’d just actually do something sometimes.

The only way you can get these two camps to talk to each other is to figure out what the theory says that provides value to those who practice. Practice-ers are ruthlessly focused on value, so to get through to them, you have to speak their language. On the flip side, theorists understand that practicers don’t care too much about the theory, but love seeing their thoughts to go good use, and can appreciate when practicers stumble across flaws in their thought. So demonstrating how practicers produce contradictions in the theory can be really useful to theorists.

If you’re this kind of person, accept that you’re in many ways a jack of all trades, but a master of none. Of sorts, anyway. Theorists will sometimes hate you for not grokking every last detail and reference, and practical people will argue that you don’t do enough useful things. Don’t listen to either of them; you know that you’re part of neither camp, so it makes sense that they both find ‘flaws.’ You’re awesome because you know a bit of both, and can facilitate communication which makes both better. A force multiplier.

You have to remember that while you’re building things, there’s an underlying set of rules that you’re implicitly following, but it’s more important to act than it is to memorize a bunch of rules, and try to analyze what you’re doing according to them. If all you do is think all day, you’ll never get anything done. Things may go wrong, but you can always fix it later. Other people can sit around and theorize about what you’re doing, leave them to it.

Mental masturbation is fun and all, but when all is said and done, developing a culture of shipping is one of the most important things you can do. Those who can’t do, teach. History is written by the victors.

On occasion, you’ll run into someone who can actually explain complicated theory stuff to you in an accessible way. If you find someone like this, make sure to hold onto them closely, as they’re really rare. But they can help provide you with some insight that will really boost your productivity, without having to invest all the time in figuring out all that wankery that the priests of theory love.

“In theory, there is no difference between theory and practice. But, in practice, there is.” - Yogi Berra

How do you find the time?

Thu, 27 Sep 2012 00:00:00 GMT

I’m often asked the question, “How do you find the time?” Mostly, it’s around open source, sometimes it’s about books, and occasionally, on other things. I’ve generally deflected the question with something like “I don’t know, man, I just do it.” But that answer isn’t very helpful. And I’ve been wondering if I can do even better. So I’ve done some reflection this week, and here’s how I find the time:

Min/max

player’s handbook

I have this personality type that is an asset to any competitive gamer: Dungeons and Dragons nerds call it “min/maxing”. The basic idea is this: you try to minimize the things that are bad, and maximize those that are good. This sounds super obvious, but there are many times when maximizing ‘winning’ isn’t what you’re going for. One great example is ‘pretending to be a fantasy adventurer.’ If you’re not shooting for strictly winning, then min/maxing doesn’t make any sense. But if you’re looking to compete, it’s incredibly important.

Speaking of competition, you should really read this book:

playing to win

“Playing to Win” is an excellent book by one of the best Street Fighter players in the world. It is a must-read for anyone who wishes to compete in any way.

In it, he develops the theory of “Players” and “Scrubs”:

A scrub is a player who is handicapped by self-imposed rules that the game knows nothing about.

A great tell of someone being a scrub is when they mention ‘fairness.’ A game is an impartial, closed system. There are rules to the system that constrains the players. Simply, the more rules, the harder it is for you to achieve your win condition. If you invent extra rules, you handicap yourself. Games are not ‘fair.’ Players don’t handicap themselves in this way:

A player should use any tournament legal move available to him that maximizes his chances of winning the game.

If a move is legal, then it’s fair game. Good games are written in such a way that bad moves don’t exist. Use all the tools at your disposal, and use them ruthlessly.

Here’s the thing about the scrub mindset: it’s easy to do accidentally. Extra rules can be self-imposed, but they can also happen due to a poor analysis of what the rules actually are. For example, the biggest time I scrubbed out in my entire life was a choice that I made almost a decade ago: going to college. Every adult in my life told me that going to college was a good idea. They all said it was needed to get a job as a programmer. I didn’t bother to challenge them, and went along with it. Now I have a useless piece of paper and $70,000 in debt. College wasn’t all bad, but it certainly wasn’t worth the debt. I cherish the friendships I made, but not much else. This mistake was made because of my apathy towards the systems governing my life, and it’s one I try not to repeat.

Anyway, these two principles are the backdrop for everything else: properly analyze your situation, maximize the outcomes you want, and minimize the ones you don’t.

Work

“So what do you want out of life, Steve?”

“I want to teach, I want to work on open source, and I want to travel.”

“I think we can work something out.”

If you work a 9-5 job, you spend 40 hours a week at work. Let’s add a one hour commute in there too, which puts you up to 50 hours. And as Ruby will tell you…

1.9.3-p194 :001 > 24 * 7
 => 168 
1.9.3-p194 :002 > 50 / 168.0
 => 0.297619047619048

30% of your life. Let’s analyze this like a pro gamer, shall we? Here’s the situation we’re trying to maximize: “I want to live a full life.” In order to maximize this, we need to discover the constraints. The biggest one, of course, is your time on this Earth. For now, at least, we all will die someday. The primary resource that we all have to work with is time. You should guard your time more closely than any other thing you have. You can’t ever get it back. Every single thing you do consumes this most precious of resources.

When you enter into an employment agreement, you are literally selling your most precious thing in order to acquire other resources. Don’t forget that.

So, 9-5 job. 30% of your life. Is it worth it? Maybe! I can’t tell you. But in order to maximize your outcomes, you need proper analysis. Employment is a tricky thing, in that regard. Capitalism forces us to interact with the market in order to gain our subsistence, so we have to enter into this transaction somehow. What to do?

Quick, startup crowd: what’s the first rule of pricing? If you say “do it value-based, not cost-based,” then you win a patio11 sticker. You’ll never get anywhere selling yourself on an hourly basis, you have to sell yourself on value. That’s another topic for another time. Point is, don’t just assume the rules: truly analyze your situation.

Anyway, that’s neither here nor there. On a day-to-day basis, the only reason that I can do what I do is because Jeff is a stand-up guy, and working with Jumpstart Lab is awesome. We do the best Ruby and Rails classes in the world, and I mean that. I don’t want to get into the details of my employment, but basically, teaching classes, contributing to open source, and speaking at conferences are all part of my job. You can see how that directly maps to my statement above.

So, if you appreciate

then you should consider hiring us over at Jumpstart. If I had to do normal consulting to pay my bills, I would have much less time for all of this stuff.

But I don’t work at Jumpstart!

So, you’re not me. You work a 9-5 job. So you’re screwed, right?

No way! Analyze your situation. Figure out how to maximize it. Before I did software development, I did this. About 30 hours a week while at high school, and 40 hours a week through college. I still managed to put in open source time. How?

The thing is, working on open source is something I truly love to do. For me, it’s not ‘work.’ That does not mean that it’s always roses. Sometimes it’s hard. Sometimes, it really, really sucks. Hard. However, I truly love it, so I stick it out through the hard times, and it ends up being good again. Once, I suffered a really bad break-up. My mom told me this maxim that her grandmother always told her:

What bone you were meant to have, no dog will drag away.

Grandmas. They’re so opaque! The point is, contributions to Open Source is something that’s a part of me, no matter what. So I’ll always find a way to do it. I don’t give up on it easily. When I first took over Shoes, it took me six months to figure out how to compile it. Six months. No compilation. That sucked. But I stuck it out.

Eighty percent of success is showing up.Woody Allen

Relationships

Having a full life means interacting with other people. Don’t let some productivity fetish negatively affect relationships. I mean that in a broad sense, both romantic and bromantic. ;) Make time for stuff that’s not ‘ZOMG MUST GET WORK DONE’ in your schedule too. I make sure to talk to significant others at least once or twice a day, for example. Assuming they want me to. Tonight, I’m gonna go for a bike ride on the beach. I still play StarCraft from time to time. Every Saturday, my friends and I used to get together in the computer lab and hack.

If you spend all of your time working, and none of your life living, you’ve failed at life. Open source is fun, but go outside, dammit. Or read a book. Or something. But if you only program computers, you’ve failed at being a human.

Music

Listen to awesome music while you work. I first discovered this while mowing my parent’s’ lawn: when I listened to Metallica, that grass got cut way quicker. So, while I work, I listen to shitty pop music. Here’s my Pandora station. I find that upbeat, happy music makes me upbeat and happy. I’m over being embarrassed about my love for Ke$ha. I just bob my head and write more code.

Parallelize

I actually cannot be productive without doing six different things at once. This is related to “Structured Procrastination”. I do a little bit of work on a bunch of different things, and cycle between them quickly. One day, it’s Rails, another, it’s Resque, another is training, then back to Rails, then Hackety…

lift.do

I’ve been trying out Lift lately, and I really like it. You can build habits with some light social interaction. You might find it useful.

Cut distractions.

That’s a bunch of things I do to maximize good outcomes, but what about minimizing bad ones? In the interest of minimalization, I won’t dwell on them too much, but here are some things that I’ve cut down or cut out in some way:

TV
Reddit/HN
Cell phones
Video Games

I still indulge in some of these from time to time, but I’ve cut them out of the ‘every day’ loop. My life is better off for it. Figure out what you don’t actually need, and stop it.

;)

Seriously: numbers: use them!

Thu, 27 Sep 2012 00:00:00 GMT

Turbolinks test from Steve Klabnik on Vimeo.

If you don’t feel like watching…

https://github.com/steveklabnik/turbolinks_test

Results with 1000 pages:

Blank Rails app:

$ rspec
       user     system      total        real
 no turbolinks 11.170000   0.980000  12.460000 (138.656728)
yes turbolinks 10.800000   0.870000  11.670000 ( 80.436286)

With Basecamp Next’s CSS file:

$ rspec
       user     system      total        real
 no turbolinks 14.470000   1.540000  16.320000 (235.404727)
yes turbolinks 10.730000   0.870000  11.600000 ( 82.176967)

With Basecamp Next’s JS file:

$ rspec
       user     system      total        real
 no turbolinks 15.300000   1.700000  17.270000 (433.880904)
yes turbolinks 10.540000   0.890000  11.430000 (170.545663)

With Both:

$ rspec
       user     system      total        real
 no turbolinks 21.990000   2.890000  25.150000 (581.822206)
yes turbolinks 10.970000   0.910000  11.880000 (196.481247)

TL;DR

Turbolinks seems to speed up apps.

Protocol and language

Mon, 03 Sep 2012 00:00:00 GMT

This is the second part of my series on protocol. The first part contained a lot of background information, but now we’re ready to get into what Protocol actually is.

I live in a pretty unique place, the Farmhouse. It’s a psuedo-public space in which we live, others work, and everybody has a good time. We do have some rules, though:

They’re written up on the wall in chalk. Since we have lots of visitors to the house, and these are hard and fast rules, it’s important that we communicate them in a clear way. These rules form the protocol of our house. Phrased another way, they’re the rules that govern the way that we interact with each other and the house itself while at the house.

Ultimately, this is what protocol is: a language. To Galloway, protocol is a very specific kind of language:

Protocol is a language that regulates flow, directs netspace, codes relationships, and connects life-forms.

Some of these properties are contradictory. Stated more accurately, there exists a tension between opposing forces within protocol itself. Let’s expand on these meanings.

‘regulates flow’

If you’ve ever read 1984, you’re familiar with linguistic relativity, more commonly known as the Sapir-Whorf hypothesis.

How could you have a slogan like “freedom is slavery” when the concept of freedom has been abolished?

What’s truly interesting is that Orwell himself actually advocated for simplifying language in Politics and the English Language, but specifically because he felt more complicated language was obfuscatory, not illuminating.

In any case, language enumerates the possible, and while some languages can have really long words, ultimately, any language will have some things that are not expressible. This often happens with translations.

If you are the proud recipient of a computer science degree, you may have run into the Chomsky hierarchy in your compilers class. If you weren’t required to take a compilers class, please call up your alma mater and demand a refund. Here is the hierarchy:

unrestricted grammars
context-sensitive grammars
context-free grammars
regular grammars

As you go up the hierarchy, each language type is enclosed by the type above it. All context-sensitive grammars are unrestricted grammars, and all context-free grammars are context-sensitive.

Regular expressions are one area where a language was forced to move up the hierarchy in order to express more things. Strictly speaking, a regular expression cannot match nested parenthesis. If you are interested in the math, the Pumping Lemma demonstrates why. However, this kind of thing is useful, and so extensions were developed that move them up the chain. Now you can write this:

qr/(<(?:[^<>]++|(?1))*>)/

Maybe Orwell was on to something.

Others have speculated that linguistic relativity would also apply to software. Two notable examples are Kenneth E. Iverson, who created APL, and Paul Graham, Lisp enthusiast and investor extraordinaire. Iverson wrote a paper titled “Notation as a tool of thought”, which begins:

The importance of nomenclature, notation, and language as tools of thought has long been recognized.

Graham authored an essay called “Beating the Averages” in which describes the ‘Blub paradox’. The paradox relies on linguistic relativity:

You can’t trust the opinions of the others, because of the Blub paradox: they’re satisfied with whatever language they happen to use, because it dictates the way they think about programs.

I don’t mean to rain on the relativity parade, but there’s one small detail that we haven’t considered: Turing completeness. At the end of the day, most programming languages are Turing complete. This means that COBOL can do everything Lisp can, and PHP can do everything that Ruby can.

So are Graham and Iverson wrong?

Turns out there are two variations of linguistic relativity: strong relativity states that language determines thought, and weak relativity that states language influences thought. This was coined later, as it would seem that current research has found little evidence to support strong relativity, but some to support the notion of weak relativity. Graham notes this, and confines the paradox to the realm of weak:

All languages are equally powerful in the sense of being Turing equivalent, but that’s not the sense of the word programmers care about. (No one wants to program a Turing machine.)

To put it another way, language features create friction that encourages or discourages certain modes of thinking.

Python and Ruby are two very similar languages, yet they engender very different styles. One area in which this division is sharp is first-class functions. Both languages have them, but Python’s are simply expressions rather than fully-featured . Guido van Rossum likes them this way:

But such solutions often lack “Pythonicity” – that elusive trait of a good Python feature. It’s impossible to express Pythonicity as a hard constraint. Even the Zen of Python doesn’t translate into a simple test of Pythonicity.

Guido specifically wants to limit the scope of language in order to encourage a particular style of thinking about problems: a Pythonic one.

Galloway is very explicit about this regulation.

Protocol is synonymous with possibility. From the perspective of protocol, if you can do it, it can’t be bad, because if it were bad, then it would have been outlawed years ago by protocol.

‘directs netspace’

After that massive first part, I’m basically going to punt on the second. I don’t find it very interesting or worth elaborating on: languages and protocols direct the direction of the Internet. I don’t think anyone disputes this.

‘codes relationships’

Communication has an intimate connection to relationships. The act of communicating is what creates the relationship in the first place. And language is the encoding of that particular relationship.

In “A user’s guide to Capitalism and Schizophrenia: Deviations from Deleuze and Guattari,” Brian Massumi discusses an example of marriage vows:

A particular man and a particular woman say “I do.” Their words undoubtedly have personal meaning for them in their heart of hearts. But their personal intention is not in itself responsible for the magical transformation that has touched their lives. What has brought them to say these words and what makes those words effectively transformative is too big to fit into a single mid. It is a complex interplay of laws, customs, social pressure, and tax law.

Marriage is a social protocol that encodes a particular relationship. Once that relationship has been encoded, it’s made legible to others who weren’t privy to the details of that relationship. A wedding ring is only one form of relationship encoding; a couple holding hands is a similar example, whose bond is weaker both in symbol as well as in reality.

In the software world, HTTP encodes relationships as well. The client/server relationship is integral to the protocol’s design, as Roy Fielding describes in section 3 of his dissertation:

A client is a triggering process; a server is a reactive process. Clients make requests that trigger reactions from servers. Thus, a client initiates activity at times of its choosing; it often then delays until its request has been serviced. On the other hand, a server waits for requests to be made and then reacts to them. A server is usually a non-terminating process and often provides service to more than one client.

He further goes on to describe why this relationship is important:

Separation of concerns is the principle behind the client-server constraints. A proper separation of functionality should simplify the server component in order to improve scalability. This simplification usually takes the form of moving all of the user interface functionality into the client component. The separation also allows the two types of components to evolve independently, provided that the interface doesn’t change.

In other words, the protocol encodes the relationship so that others know what a client should do, and what a server should do.

If you remember our good friend Saussure, you’ll remember that he made this encoding explicit, and in fact central to language itself. While Saussure’s work isn’t directly useful to linguists in the present, much of his work is foundational to the ways that we approach language today.

One of the most fundamental concepts in structural linguistics is the ‘sign,’ and its component parts, the ‘signifier’ and the ‘signified.’ The signified is a particular concept, and the signifier is the way the signified is expressed. A sign is a combination of a signifier and a signified. Seems pretty simple, but Saussure also posited that a sign can only gain meaning and value within their relationship to other signs, which means that you cannot consider an idea outside of the way in which the idea is expressed.

The study of signs is called semiotics. In Semiotics, a particular set of conventions used to convey meaning are called codes. Hence ‘encoding’ can also refer to this kind of code, as well.

Hence, if we combine Fielding, Galloway, and Saussure, the word ‘client’ is a signifier that holds a relationship to the signified concept ‘a triggering process’ (among other things), that forms the sign ‘client.’ This is encoded by the protocol HTTP.

‘connects life-forms’

From the Massumi marriage example:

Say “I do,” and your life will never be the same. Your legal, social, and familial status instantly changes, along with your entire sexual, psychological, and financial economy. You have been pronounced man and wife. You may file a joint tax return.

This connection is related to the previous coded relationship. In this case, ‘connects’ is an active verb rather than a passive noun. It not only creates a connection, but it ‘connects’. It not only relates two objects, it is the relation.

We spent most of the last section discussing connections, though, so let’s get to the interesting part: life. Galloway is very explicit about the role protocol plays with life:

…protocol is an affective, aesthetic force that has control over “life itself.”

In the Exploit, Galloway discusses the relationship between life and protocol as well:

While the first type of network (Internet protocols) is silicon based and may use biological concepts (intelligent agents, artificial life, genetic algorithms), the second (DNA algorithms) is fully biological and yet recodes itself in computational terms (biology as computation, as opposed to evolution)

The relationship between life and software is an interesting one.

In Conway’s game of life, one builds cellular automata and lets them go. There are a few simple rules that govern their reproduction, death, and life. This protocol governs all that happens within the game. Someone playing the game is an impotent god, as there is no room in the protocol for communicating with creations. Likewise, creations have no way of perceiving an ‘outside’ of the game.

Galloway references this quote by Frederick Cohen from A Short Course on Computer Viruses:

I personally believe that reproducing programs are living beings in the information environment.

Viruses, Conway cells, Sims, and all other kinds of ‘artificial life’ exist inside of computers. They’re native to protocol. But what about actual, real life?

Galloway reaches to Norbert Wiener, a mathematician who was the originator of cybernetics, for this:

if one views the world in terms of information, then there is little instrumental difference between man and machine since both are able to affect dynamic systems via feedback loops. In this way the cybernetic system of man and machine are born. Its virtues are balance, self-regulation, circularity, and control. In a word, protocol.

Deleuze also forms a connection between life and information in “Postscript on the Societies of Control”:

The numerical language of control is made of codes that mark access to information, or reject it. We no longer find ourselves dealing with the mass/individual pair. Individuals have become “dividuals,” and masses, samples, data, markets, or “banks.”

A dividual is a play on the process of individuation. Galloway elaborates:

Deleuze’s neologism comes from the word ‘individuate.’ Dividuation would thus be the opposite: the dissolving of the individual entity into distributed networks of information.

Let’s use a personal example to demonstrate life submitting itself to dividuation in order to interface with protocol: me. This is my personal website. In it, my public face on the Internet, I’ve split myself up into five broad categories, and link out to all of the various sub-components. Examining the last category, you can see how if you just follow me on twitter ;), you will not get the whole picture. You could also check out my code on GitHub, but that wouldn’t be me, either. Even the sum of these two components would not tell you everything about me. Yet I can’t possibly put me on the network. I must break ‘me’ into a form that conforms with protocol.

Finally, there’s an important relationship with Foucault’s concept of ‘biopolitics.’

Foucault defines biopolitics as “the endeavor, begun in the eighteenth century, to rationalize the problems presented to governmental practice by the phenomena characteristic of a group of living human beings constituted as a population: health, sanitation, birthrate, longevity, race.”

Obviously, dividuating a population into informational components would be relevant to protocol. And, in fact, life provides a path to resist protocol. Deleuze in his book Foucault (because that’s not confusing…):

When power becomes bio-power resistance becomes the power of life, a vital power that cannot be confined within species, environment or the paths of a particular diagram.

Galloway expands:

Is life resistance a way of engaging with distributed forms of protological management? Part III of this book, “Protocol Futures,” answers yes. While the new networked technologies have forced an ever more reticent public to adapt to the control structures of global capital, there has emerged a new set of social practices that inflects or otherwise diverts these protological flows toward the goal of a utopian form of unalienated social life.

To conclude

Protocol has an intimate and complex relationship with language. In many ways, it is a particular language, with all of the expressive power and limiting abilities that that brings.

The implications of this language, though, will have to wait for another time.

DayZ

Wed, 25 Jul 2012 00:00:00 GMT

I’ve recently come across one of the most addicting, interesting experiences I’ve ever had online: DayZ. I hesitate to call it a ‘video game,’ because while it is a mod for a first person shooter game, it basically ignores all common wisdom about what makes for a good game. I think that the broader implications are also interesting across other domains: ‘common wisdom’ isn’t always correct. Sometimes communities reach local maximums, and something drastic needs to be done to break out of the rut. So what is DayZ, anyway?

Essentially, DayZ is a simulation of a zombie apocalypse.

You wake up on a beach with essentially nothing in your pockets. This means that you have no way to defend yourself, and no supplies. Check out the bottom right of that screenshot: See that canteen and knife and fork? Those are water and food indicators, and if you don’t eat or drink often enough, you’ll faint from fatigue, and even lose blood (which is a proxy for health, and is the middle indicator). So you have to sneak around until you can find a hatchet, food, and water. See those two things on the upper right? That’s how much audio and visual ‘noise’ you give off. Right now, this player is basically invisible, so they’re safe.

The game takes realism very seriously. There’s the food and water bit, but this also isn’t some Left 4 Dead run and gun bullshit: if you take damage, you’ll start to bleed, and will need to bandage yourself or bleed out. If you get shot, you’re basically dead. You might be able to take two bullets from small arms. I already mentioned the food and water. You can break bones, which seriously limits mobility, and will probably cause the shakes. Better take some painkillers! There are a few vehicles, but they’re all in disrepair, so find some parts, fix them up, and then stop and put some gas in them! Voice chat only works for a 40 yard radius around where you stand, so you can’t talk to people that are far away. You don’t have a map at first, so you won’t know where the hell you are until you find one. Of course, you can read the street signs, so obviously, you’ll know when you’re in черногорск. ;). (That’s ‘Cherno’ as it’s affectionately called by players, because ‘Chernogorsk’ is too long to say.) Oh, and when you die, you die. You lose all your stuff. This also adds to the crazy tenseness of the game. There’s actually 24 hours on each server, so if you play on one with your GMT offset, it’ll be light when it’s light and dark when it’s dark outside.

The biggest thing, though, is that this is a true sandbox: there are no ‘levels’ or ‘experience points’ or ‘missions’ or ‘objectives.’ You just play. Average survival time is about 45 minutes, and as you live longer, the tension goes way, way up. Because there are no missions, you develop a real attachment to the ‘emergent stories’ that come out of the game, which is the big, big draw for me. Here’s an example:

I spawned just outside of Электрозаводск (‘Electro’), and one of my roommates had been playing for about twenty minutes, and was in Пута (“Puta”), just north. He had found a barn with two hatchets in it, and obviously, as a new player, that’s a big deal. Электро and Пута are about two kilometers away from each other, so he decided to wait for me. I ran up over that hill, and as I got closer, I didn’t pay close enough attention, and a zombie started chasing me. I decided to sprint to the barn, so I’d at least be able to defend myself, and Kyle decided to brace himself: my running would obviously draw more zombies, and the barn had three doors: we couldn’t cover them all. If we weren’t careful, this could be the end.

As I sprint through the door, Kyle steps into it, and swings his hatchet. “Holy shit dude, more coming, hurry up!” I quickly located the other one, as Kyle took out the second undead trying to eat us. I stepped up too, and covered the other door. Thirty seconds of frantic hatchet-ing later, we were safe. But Kyle was starting to get thirsty, and Электро is really dangerous. We searched the rest of the settlement, but no luck. No water to be found. We decided that rather than risk the big costal town, it’d be best to make the two klick journey north to Могилевка. While trying to locate the road, Kyle attracted the attention of another zombie, and after dispatching it, found an empty watter bottle! Luckily, Пута has a public well, so he could fill it and drink up. Problem solved.

Now, if this scenario had been a scripted part of some zombie RPG, I wouldn’t have cared about it at all. I’ve now told this story three times in person, and it doesn’t get old. This tense five minutes emerged naturally out of putting players in the world and letting them go. As a result, since they’re your stories, you care about them a lot more. People end up sharing these stories, too: Check out this Jeep theft on YouTube. He spends the first 3 minutes or so telling the story.

Ignoring video game tropes

So what really makes DayZ special? There are lots of sandbox games, and there are lots of zombie games. Well, for me, it’s that DayZ ignores much about what makes for conventional games.

See, I’ve been playing video games for most of my life. I started with an NES long ago, and spent many, many, many hours with various games across genres. I play fewer games these days, and actually took a pretty big break from them over the last two years or so. I would go to GameStop, look at what was offered, shrug, and walk out. It’s not that I’m growing up (the average gamer is 37), it’s that I’ve played all these games before. I spent over a hundred hours with Diablo II, so why bother with Diablo III?

When there’s a game that breaks out of these standard genres and franchises, I get excited. I really got into music games until Activision drove Guitar Hero into the ground, Minecraft was incredibly addicting, and now DayZ with its emergent gameplay, focus on realism, and tense atmosphere.

Games need to be accessible

DayZ does not have mass appeal. Its uncompromising approach to realism means that it will alienate many people. You will die often, and it will feel random much of the time. You will get killed by other players. You’ll break a bone miles away from the nearest hospital with no morphine, and have to crawl for an hour or die.

Games should be polished

Frankly, DayZ is buggy as shit. And not trivial ones, either:

Dropping Items can cause them to disappear

Honestly, in many ways, it makes the game even more tense. Bad things can happen to you at any time. The world is scary.

The interface sucks. This is more of an ARMAII issue, but one of my roommates was trying to get something out of another roommate’s backpack, and since you both click to manipulate the UI and click to shoot, accidentally shot him in the stomach. This totally could happen in a zombie apocalypse: fumble around with the safety off… the interface can be confusing, seemingly arbitrary, and complicated. Still totally worth it.

In many ways, this is the worse is better principle.

More is better

Many games are ‘just like this other game, but with more stuff.’ DayZ has a really pleasant minimalism in many ways: the HUD is basically nonexistent, the core gameplay is pretty simple, and while there are a lot of details, most of them are not too complicated: break a leg, use some morphine. Hungry? Eat some food.

But these simple components come together in a way that gives the game a large amount of depth. But that depth is not from some big crazy feature that took weeks to build.

In fact, the author of the mod has removed entire features. There was a ‘humanity’ system at one time, and ‘bandits,’ which were players that killed other players and had their skin changed. That’s been taken out. I didn’t play the game with that particular feature in, but the author isn’t afraid to cut stuff out if it isn’t a good idea, even if he’s spent lots of time building the feature.

A good story matters

Most simulation games have some sort of goal. Even many ‘sandbox’ games, like Grand Theft Auto, have some sort of points, goals, or missions, and you’re free to ignore them, but they’re there. Minecraft is a great example of a true sandbox game, and I’m not suggesting that DayZ has a monopoly here. But it’s a genre that doesn’t have that many games in it, so another entry is very welcome; the genre isn’t as tired as many others.

Story is often used as a reason for progression; as you go through the story, you get bigger, better, and generally more powerful. DayZ manages to have progression without a story; there’s sort of five broad stages to the game: pre-hatchet, hatchet, gun, tent, PvP. This is hard to do! A story provides nice justification. But DayZ doesn’t need it.

Also, as I mentioned earlier, you sort of invent stories as you play. They emerge. And therefore, you care about your DayZ story more than the one that would have been concocted if this game was not a sandbox.

A player should be quite powerful

Often, the player is a total badass. Video games are often dripping in machismo, and so they are THE ULTIMATE DESTROYER OF WORLDS. Maybe, after killing hundreds of enemy soldiers, getting worn down over time, one gets a lucky shot and you go down.

DayZ is different. You are weak. The environment is dangerous. Even though you can kill zombies, often, the noise attracts others, and they get dangerous in packs. And you can always run out of ammo…

Also, other players are just as desperate and dangerous as you are. Sometimes, you’ll be friendly, but more often, you’re gonna end up getting shot. After all, if you’re not friendly, I could get shot, so maybe I’d better take you out first…

DayZ is awesome

Seriously, you should give it a try. ARMAII goes on sale on Steam often, I was able to pick it up for $17 the other day. If you’re bored with the mainstream of gaming, it just might be enough of a shake-up to be interesting to you.

Sparrow

Sat, 21 Jul 2012 00:00:00 GMT

I purchased Sparrow back when it was $9, and didn’t find it particularly amazing. I did like it, but ended up using the web interface more. I certainly got my $9 of value out of it, though, even though I used it for a short time.

However.

There’s been a lot of complaining about Sparrow’s acquisition, and a lot of complaining about complaining. Most of the complaining about the complaining argues that those who complain should feel like me: You bought it, you probably got a ton of value out of it. You can’t (and shouldn’t) complain.

But.

Theoretically, a market works through feedback mechanisms. That’s the whole ‘free market’ argument, right? People that participate in the market come together, the magic happens, and everything gets allocated correctly. Vote with your dollars. Shop somewhere else, and poof! A store that’s too expensive won’t be able to keep its doors open. A triumph for ‘free commerce.’

So why does everyone get all huffy when certain people provide the market with feedback about the way that business has been conducted?

Mixins: a refactoring anti-pattern

Mon, 07 May 2012 00:00:00 GMT

I spend an unusually large amount of time thinking about interactions between what I call ‘past me’ and ‘future me.’ It seems that my life changes significantly every few years, and I like to ground myself by imagining how odd it would be if ‘current me’ could tell ‘past me’ things like ‘Someday, you’ll be speaking at OSCON.’

It’s not always general life stuff, though, it’s often a trick or technique with code. How many times have you learned something new, and then had that terrible urge to go back and re-write all of that terrible, terrible code you’ve written in the past? This happens to me constantly.

So here, let me show you something I realized that I used to do wrong all the time.

Let’s do it via a quiz. Of course, because I’m framing it this way, you’ll get the answer, but seriously, just play along.

Two factorizations

Which code is better factored?

# app/models/post.rb
class Post
  include Behaviors::PostBehavior
end

# app/models/behaviors/post_behavior.rb
module Behaviors
  module PostBehavior
    attr_accessor :blog, :title, :body

    def initialize(attrs={})
      attrs.each do |k,v| send("#{k}=",v) end 
    end

    def publish
      blog.add_entry(self)
    end

    # ... twenty more methods go here
  end
end

class Post < ActiveRecord::Base
  def publish
    blog.add_entry(self)  
  end

  # ... twenty more methods go here
end

One line of reasoning asserts that the first example is better. Here’s why: all of the behavior is in one place, and the persistence is in another. A clear win.

Another line of reasoning asserts that the second is better. Here’s why: it’s significantly simpler. A clear win.

So which is right?

They’re both wrong

Both of these justifications are wrong. As far as I’m concerned, these two classes are equally complex, but because there is a lot more ceremony in version one, version two is preferable. Assertion one is wrong because both are still in the same place, they’re just in two different files. Assertion two is wrong because it’s not simpler than version one, it’s equivalently complex.

Measures of complexity

Whenever we refactor, we have to consider what we’re using to evaluate that our refactoring has been successful. For me, the default is complexity. That is, any refactoring I’m doing is trying to reduce complexity. So how do I measure complexity?

One good way that I think about complexity on an individual object level comes from security practices: the ‘attack surface.’ We call this ‘encapsulation’ in object oriented software design. Consider these two objects:

class Foo
  def bar
  end

  def baz
  end
end

class Bar
  def quxx
  end
end

Bar has one less way into the object than Foo does. Hence the idea of a ‘surface’ that an attacker would have access to. A bigger surface needs more defense, more attention, and is harder to lock down.

Note that I said ‘one less.’ If you said “Foo has a surface of two, and Bar has a surface of one,” you’d be wrong. Luckily, Ruby provides us with a simple way to calculate attack surface. Just use this monkey patch: (I’m not suggesting that you actually use this monkey patch.)

class BasicObject
  def attack_surface
    methods.count
  end
end

This gives us

1.9.2p318 > Foo.new.attack_surface
 => 59 
1.9.2p318 > Bar.new.attack_surface
 => 58

Oooooh yeah. Object.new.methods.count is 57… we forgot about those. This is why mixins do not really reduce the complexity of your objects. Both versions of Post above have the same number of methods. How many, you ask?

> Post.new.methods.count
 => 337
 > Post.new.methods.count - Object.new.methods.count
 => 236

In this case, my Post is an object that inherits from ActiveRecord::Base and defines nothing else. ActiveRecord adds in 236 methods to our object. That is a pretty huge increase.

Reducing complexity through encapsulation

Several people have been dancing around this issue for a while by saying something like ‘Consider ActiveRecord a private interface, and never use it from outside of your class.’ They feel this exact pain, but enforce it through social means rather than reducing the complexity of objects. Now, there are good reasons to do that, so I don’t think it’s categorically wrong, but it certainly is a compromise.

An implementation of DataMapper would be able to solve this problem,

class Post
  def initialize(title, body)
    @title = title
    @body = body
  end
end

class PostMapper
  def save(post)
    @database.insert(:posts, [post.title, post.body])
  end
end

PostMapper.new.save(Post.new("Hello DataMapper", "DM rocks"))

or the repository pattern:

class PostRepository
  def first
    title, body = @database.select(:posts, "LIMIT 1")
    Post.new(title, body)
  end
end

All incredibly theoretical and pretend. But note that the Post in both instances has no methods defined on it that deal with persistence whatsoever. We’ve used encapsulation to hide all of the details of the database in the database, the details of the post logic in the Post, and we use the Mapper or Repository to mediate between the two.

Using Mixins the right way

Mixins are awesome, though, and can be used to reduce duplication. They’re for cross-cutting concerns:

class Post
  def summary
    "#{author}: #{body[0,200]}"
  end
end

class Comment
  def summary
    "#{author}: #{body[0,200]}"
  end
end

moves to

class Post
  include Summarizer
end

class Comment
  include Summarizer
end

module Summarizer
  def summary
    "#{author}: #{body[0,200]}"
  end
end

This example isn’t fantastic, but whatever. We’ve eliminated some duplication, even if we didn’t reduce our method surface. This is a win along a different axis, we just shouldn’t fool ourselves that we’ve made our classes ‘simpler.’

This isn’t the only metric

Now, I don’t mean to imply that method count is the be-all, end-all metric of complexity. Real software development is too complicated to wrap up in one tiny number. However, I do think there’s a strong correlation between small, focused classes and easy to understand software systems. Aggregates can be used to make it easier to understand the use of a cluster of smaller objects. But that’s another post.

On the eve of #m1gs

Mon, 30 Apr 2012 00:00:00 GMT

Walking down the street, a woman cries for help. She’s blind, and the curb is large, and she’s unsure of herself. Knowing it’s not safe, she seeks the help of a stranger to guide her on her way. Three or four people pass by, but then someone stops, gives her their arm, and guides her safely to her destination.

HOOOONK!!! A large truck is blocked from going down the street. A Lexus with out-of-state-plates is in the way, and the driver is standing outside of the car, looking agitated. A passer-by stops to ask if they can help, and the man frantically tries to explain. His English is poor, but not that poor, but due to the stress of the situation, he’s losing his words. Luckily, the stranger speaks a little of his native tongue. The astute observer would notice that this tiny bit of familiarity causes a massive change in demeanor in this chaotic moment. 311 is called, a translator is found, and the man will be able to move his car soon.

These moments, these tiny acts of selflessness and mutual aid, are what revolutions are truly made of.

About antisocialne.ws

Tue, 03 Apr 2012 00:00:00 GMT

About Antisocialne.ws from Steve Klabnik on Vimeo.

Book review: New Programmer's Survival Manual

Mon, 28 Nov 2011 00:00:00 GMT

Disclaimer: I do have a small credit in this book. A long time ago, the author mentioned that he was looking for people to help out, so I stepped up. It was only participating in discussions like “What do you think about version control’s use in industry?”; I didn’t help with any actual writing, and yesterday was my first time reading virtually all the final content. No monetary interest, of course.

Lots of new books lately

I’ve been doing a lot of reading over the last few weeks, some of it technical, some of it non. I’ve been meaning to type up a few book reviews, and yesterday I finished reading my copy of Josh Carter’s “New Programmer’s Survival Manual,” published by PragProg. Since that’s a bit lighter material than Spinoza, I figured I’d write it up first.

The Survival Manual is intended to be the book you should buy after you graduate from college, but before you get a job. It’s a bunch of tips, arranged by topic, that give you a taste of what ‘real world’ programming is like, as well as what working life is about. I think that it accomplishes this goal wonderfully, but if you’d like to hear specifics, read on.

The Good

One of my roommates is not particularly technical. She picked the book up, and read the introduction and the first tip. She then said something that, after reading all of it, I agree with 100%. “I like this guy’s writing style. It’s very… readable.” Josh’s prose feels… comfortable. Like you’re talking to a down-to-earth friend, or co-worker. Which is great, given the topic. I really enjoyed the actual reading of this book. Especially in technical books, this is a great quality to have, but I guess this book isn’t exactly technical; it’s much more about soft skills than hard.

I also really enjoyed the cookbook-style organization. Basically, the book is one big ball of tips that you can read individually. They’re grouped together with three or four others that are on the same topic, but each stands alone, and they cross-reference each other when appropriate. While I sat down and read it front to back, I’d imagine it’s pretty easy to cherry-pick certain things and read it in a more non-linear fashion.

I also feel that ‘pragmatic’ is a good description of most of the content. In particular, as much as we all love DVCS, the book frankly states that most places still use centralized version control, which is totally true. He makes sure to carefully not be particularly dogmatic about the specifics of anything, focusing on strategy more than tactics. For example, “Get good with your editor, make sure it can work effectively without using the mouse, for speed,” not “zomg emacs r teh suck, use vim.”

Finally, Josh links to copious other works, and provides references for all of it. Some stuff is just general internet culture, but the bibliography alone would assist any programmer in finding some useful primary materials to read. Here’s the first six: SICP, Getting Things Done, Extreme Programming Explained, TDD By Example, The Mythical Man-Month, and Foundations of Object Oriented Languages: Types and Semantics. Oh, and just because there’s some XP/Agile stuff in those particular books, don’t think it’s slanted towards that: waterfall is explained as well. Just so happens those books were first.

The Bad

There’s not that much bad that I can say about this book, really. There are a few small things, though:

There’s a ‘white belt brown belt black belt’ metaphor that’s supposed to indicate a difficulty of each tip. Given that there’s only two black belt tips, I didn’t feel that three levels were really needed. I also just thought that there wasn’t a huge difference between white and brown, either, so I’m not sure that it really adds any utility.

Because this book is fairly high-level, it might be hard to actually apply these lessons. This might be more of a problem with readers than with the actual book, but I can’t count how many times I (and others) have said “Of course, write descriptive variable names!!1” and then typed x = 5. It’s just the way things are. And since this book is essentially a collection of advice at that high level, I can see someone reading this book, and then just simply not following the advice. I’m not sure if this is something Josh could have fixed, or if it’s inherent in this kind of book. Also, it won’t affect a diligent reader. It is something to think about, though.

Summary: If you just graduated, buy this book

This book is totally worth it. The style reminds me of the seminal Pragmatic Programmers. If you’re a freshly minted graduate, put this book at the top of your reading list, then also grab a copy of all the books in the bibliography to read later.

Your startup is not a platform

Tue, 25 Oct 2011 00:00:00 GMT

No, seriously. This is a public service announcment to all MBAs who want to do startups: ‘platform’ does not mean ‘web site.’

I know that ‘platform’ is a hot thing right now. But seriously, the term has become basically useless. It’s almost to the point that I just pretend you don’t even say it.

A platform is primarily a B2B play. It can be B2C too, maybe, but largely, to have a platform, you need to enable other people to build something. Usually, that’s another business. But if your website does not enable others to build something, or you’re not white labeling it in some way, you’re not a platform.

A platform basically needs an API. APIs are the primary way that two pieces of software connect to one another. If your web service doesn’t have an API, it’s not a platform. White labeling being an exception.

I’d make this rant longer, but I really need a good platform that addresses my market segment. Ideally, it’ll enable some win-win synergies with other channel partners and grow the pie for both of us. Until then, I really won’t find a good solution for this real pain I feel as a content producer.

GitHub is anarchy for programmers

Sat, 01 Oct 2011 00:00:00 GMT

I finally got to see Zach Holman give his talk, How GitHub Uses GitHub to Build GitHub. It was awesome. Zach is a great speaker, and the slides don’t do it justice. I highly recommend catching it on video or in person sometime.

Anyway, there’s something that I found really interesting about GitHub’s ‘process’: it’s basically anarchism, applied to software development.

If you’re not familliar with anarchism, check out “What I believe in”. It’s a pretty decent introduction to a complicated topic. Ultimately, anarchism is fairly simple, but it requires dismantling a lot of beliefs that Americans in particular hold about society, power, and control. We’re quite happy in this country to be slaves to the rich, but that’s another blog post. ;)

Another excellent post on that blog is “A Digression On What It Means to Be An Anarchist. I’m going to use it as a framework to point out my comparison to GitHub. The first sentence is stolen from TFA, but the second sentence is my GitHub comparison.

What anarchists oppose

Hierarchy – anarchists oppose domination of one person or group of people by another as detrimental to human society. GitHub has built a very flat process, where “some days the CEO ships more code than I do.”
Authority – all forms of authority must bear a heavy burden of proof in order to demonstrate their legitimacy and necessity. GitHub has hired people that have built awesome things, and have shown themselves to be a trusted authority in whatever kind of software that they build. Nobody needs to question if anyone is in the position to deploy or review some code.
The state – centralised rule of a set geographical area (country) or people (nation) by a government of elites is inherently illegitimate. Okay, this has nothing to do with GitHub all together. All abstractions are leaky here. :) The same goes for Capitalism and State Socialism too.
Nationalism and fascism – these are but the worst forms of the state, gaining the loyalty of the people with strong, often brutal discipline and by developing an almost religious, fevered love of the state and the rulers in the form of patriotism. GitHub actively uses multiple programming languages in production, and while it came out of the Ruby world, they don’t really run around pitching “Ruby is awesome and the only thing you should ever use.”
Discrimination – nobody should be excluded or discriminated against based on nothing more than their gender, ethnicity, sexuality, background, or beliefs. I don’t know everyone at GitHub, but it’s a pretty diverse crew overall. This is hard for any tech company, but they’re doing pretty well on the race/sex thing.

What anarchists stand for

Liberty – all people should be free to live their life as they see fit, without rules and laws passed from above that serve no purpose other than control and domination, as long as they are not infringing the right of anybody else to the same. See Homan’s talk. That’s pretty much how they work.
Equality – as stated above, nobody should face discrimination because of their gender, ethnicity, sexuality, background, or beliefs. See above comment about GitHub’s varied devs.
Community – human civilisation evolved, from its primitive roots, through the priciple of Mutual Aid. GitHub has a lot of elements of mutal aid in their process, where they mention some people in some pull requests, work together on features, and generally high five until the code is running in production.
Solidarity – humanity is divided only between the rulers and the ruled. Other divisions, those which bring about sexism, racism, heterosexism, and other bigotries, are promoted by the ruling class in order to divide their subjects and keep them under control. GitHub holds tons of events, buys everyone lots of beer, and just generally encourages lots of friendly interactions between everyone. CodeConf was a great group of diverse people hanging out and having a good time.

How anarchy would work

Self-management – groups, such as workforces or local communities, would be free to operate and govern themselves free of any higher authority. GitHub has no deadlines, no prioritization meetings, and 100% “do what you want” time.
Free association – all individuals would be free to live where they wanted and associate with who they chose. Zach mentioned in his talk that different people work on different parts of the site, and it’s really based on what people are interested in and what they’ve done in the past.
Voluntary federation – instead of the state, where indivudal communities and groups of people are bound together by the coercive force of a central authority, local communities and workers collectives can choose for themselves which other communities or collectives to associate with. Sort of like the free association comment, people get together and work on what they want to, and these groups are organic.
Direct democracy – unlike in parliamentary democracy, these spokespeople would be just that, elected not to a position of authority but to voice decisions that remain in the hands of the people, as in trade union and workers council structures. Anybody can comment on a pull request, add some commits, put in their two cents, and be heard.
Mutual Aid – in participatory communities and workers collectives, Mutual Aid is a central principle. Easily summed up with the phrase “from each according to his ability, to each according to his need,” this boils down to voluntary cooperation, fair distribution of resources, and help and support to those who need it within a local community. As mentioned before, everybody helps each other out at GitHub to get things done.
Free, fair trade – for the sharing of resources between different communities and individuals who opt out of Mutual Aid, anarchy would see the emergence of a truly free market. This doesn’t really fit with software, I think. Same with ‘individual liberty.’
Collective defence – this is not to say that anarchist society will contain “perfect people,” and there will certainly be acts of aggression, oppression, and violence – albeit on a lesser scale than is commonplace in today’s world. I’d liken this to things like “the site is down.” Everyone at GitHub cares about GitHub being up, and so they don’t need someone who’s responsible for ‘defense,’ as if shit hits the fan, everyone chips in and takes care of it.
Justice, not vengeance – courts would be elected for each individual case, rather than appointed and given unnecessary authority, with the aim to establish guilt or innocence, negotiate reparations, and organise rehabilitation rather than to support the oppressive prison systems which only make matters worse by serving as little more than universities of crime. I’m not sure this fits well here.

All abstractions are leaky

Just like when ‘lean’ was a concept that applied manufacturing techniques to software development, I don’t think that ‘anarchism’ is a 100% fit with GitHub’s process. I do think that it’s close enough to hold a lot of value, however, even to the point that Zach had to write a blog post about scaling, which is something that always gets leveled at people talking about anarchism. I see GitHub in the same way that I see anarchist Catalonia or the Free Territory in the Ukraine. It’s a place where ideals are actually working in practice. Let’s just hope that the communists don’t betray them when the fascists show up.

Damn leaky abstractions.

Extracting domain models: a practical example

Thu, 22 Sep 2011 00:00:00 GMT

Hey everyone! We’ve been doing a lot of refactoring on rstat.us lately, and I wanted to share with you a refactoring that I did. It’s a real-world example of doing the domain models concept that I’ve been talking about lately.

Step one: check the tests

I don’t know how much more emphasized step 1 of refactoring could be: don’t touch anything that doesn’t have coverage. Otherwise, you’re not refactoring; you’re just changing shit. - Hamlet D’Arcy

One of the best reasons to extract extra domain models is that they’re often much easier to test. They have less dependencies, less code, and are much simpler than the Mega Models that happen if you have a 1-1 model to table ratio.

Let’s check out the code. It’s in our User model:

# Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
  f = Feed.first :id => to_feed_id
  u = Update.first :id => update_id

  base_uri = "http://#{author.domain}/"
  salmon = OStatus::Salmon.new(u.to_atom(base_uri))

  envelope = salmon.to_xml self.to_rsa_keypair

  # Send envelope to Author's Salmon endpoint
  uri = URI.parse(f.author.salmon_url)
  http = Net::HTTP.new(uri.host, uri.port)
  res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
end

You’re probably saying “this is pretty reasonable.” Prepare to get shocked…

This code has very little to do with the user, other than using some of the User’s attributes. This really could be factored out into a domain object whose job it is to push Salmon notifications. But first, let’s see if we can get some tests in place so we know everything isn’t broken. Examining this method, we need two things: an update_id and a feed_id. Here’s my first stab at an integration test:

describe "salmon update" do
  it "integration regression test" do
    feed = Feed.create
    update = Update.create

    salmon = double(:to_xml => "<xml></xml>")
    uri = double(:host => "localhost", :port => "9001", :path => "/")
    Ostatus::Salmon.should_receive(:new).and_return(salmon)
    Uri.should_receive(:parse).and_return(uri)
    Net::HTTP.should_receive(:new).and_return(mock(:post => true))

    user = User.create
    user.send_mention_notification(update.id, feed.id)
  end
end

This is an integration test, we’re obviously testing stuff with a ton of models. After running ruby test/models/notify_via_salmon.rb about a zillion times, I ended up with this running test:

require 'minitest/autorun'
require 'rspec/mocks'

require 'mongo_mapper'
require 'whatlanguage'
MongoMapper.connection = Mongo::Connection.new("localhost")
MongoMapper.database = 'rstatus-test'

require_relative '../../app/models/feed'
require_relative '../../app/models/update'
require_relative '../../app/models/author'
require_relative '../../app/models/authorization'

$:.unshift("lib")
require_relative '../../app/models/user'

require 'ostatus'

describe "salmon update" do
  before :each do
    User.all.each(&:destroy)
    RSpec::Mocks::setup(self)
  end

  it "integration regression test" do
    author = Author.create
    user = User.create!(:username => "steve", :author => author)

    feed = user.feed
    update = Update.create!(:feed_id => feed.id, :author_id => author.id, :text => "hello world")

    salmon = double(:to_xml => "<xml></xml>")
    uri = double(:host => "localhost", :port => "9001", :path => "/")
    OStatus::Salmon.should_receive(:new).and_return(salmon)
    URI.should_receive(:parse).and_return(uri)
    Net::HTTP.should_receive(:new).and_return(mock(:post => true))

    user.send_mention_notification(update.id, feed.id)
  end
end

Holy. Fuck. Seriously. Let me just quote Avdi Grimm:

Mocks, and to some degree RSpec, are like the Hydrogen Peroxide of programming: they fizz up where they encounter subtle technical debt.

This test class is insane. It runs faster than many people’s integration tests, clocking in at about 0.4 seconds. But that’s still an order of magnitude slower than I’d like. A suite with 200 tests would still take over a minute. Also, look at all this junk that we have to do for setup.

All of this pain and effort doesn’t mean that testing sucks: it means that our implementation is terrible. So let’s use that to guide us. We’ll try to fix this code by eliminating all of this junk.

Step two: simple extraction

First, let’s extract this out to a domain model. Here’s the new code:

# Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
  NotifyViaSalmon.mention(update_id, to_feed_id)
end

module NotifyViaSalmon
  extend self

  def mention(update_id, to_feed_id)
    f = Feed.first :id => to_feed_id
    u = Update.first :id => update_id

    base_uri = "http://#{author.domain}/"
    salmon = OStatus::Salmon.new(u.to_atom(base_uri))

    envelope = salmon.to_xml self.to_rsa_keypair

    # Send envelope to Author's Salmon endpoint
    uri = URI.parse(f.author.salmon_url)
    http = Net::HTTP.new(uri.host, uri.port)
    res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
  end
end

Now, when we run the test (via ruby test/models/notify_via_salmon_test.rb) we see an error:

1) Error:
test_0001_integration_regression_test(SalmonUpdateSpec):
NameError: undefined local variable or method `author' for NotifyViaSalmon:Module
    /Users/steveklabnik/src/rstat.us/app/models/notify_via_salmon.rb:8:in `mention'

This is a form of Lean On The Compiler. This test is now failing because it’s relying on stuff that used to be internal to the User, and we don’t have that stuff now. After doing this a few times, we’re left with this:

# Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
  NotifyViaSalmon.mention(self, update_id, to_feed_id)
end

module NotifyViaSalmon
  extend self

  def mention(user, update_id, to_feed_id)
    f = Feed.first :id => to_feed_id
    u = Update.first :id => update_id

    base_uri = "http://#{user.author.domain}/"
    salmon = OStatus::Salmon.new(u.to_atom(base_uri))

    envelope = salmon.to_xml user.to_rsa_keypair

    # Send envelope to Author's Salmon endpoint
    uri = URI.parse(f.author.salmon_url)
    http = Net::HTTP.new(uri.host, uri.port)
    res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
  end
end

Okay. Now we’re cooking. We have a test that’s isolated, yet still does what we thought it did before.

Step three: break dependencies

Next step: let’s break the hard dependency this method has on both Feed and Update. We can do this by moving the finds up into the User method instead of keeping them in the mention method:

# Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
  feed = Feed.first :id => to_feed_id
  update = Update.first :id => update_id

  NotifyViaSalmon.mention(self, update, feed)
end

module NotifyViaSalmon
  extend self

  def mention(user, update, feed)
    base_uri = "http://#{user.author.domain}/"
    salmon = OStatus::Salmon.new(update.to_atom(base_uri))

    envelope = salmon.to_xml user.to_rsa_keypair

    # Send envelope to Author's Salmon endpoint
    uri = URI.parse(feed.author.salmon_url)
    http = Net::HTTP.new(uri.host, uri.port)
    res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
  end
end

Okay. Tests passing. Sweet. Now we can try testing just the mention method. Let’s try it by killing most of that crap that was in our test:

require 'minitest/autorun'
require 'rspec/mocks'

require_relative '../../app/models/notify_via_salmon'

describe "salmon update" do
  before :each do
    RSpec::Mocks::setup(self)
  end

  it "integration regression test" do
    NotifyViaSalmon.mention(stub, stub, stub)
  end
end

Let’s try running this and seeing what happens:

$ ruby test/models/notify_via_salmon_test.rb
Loaded suite test/models/notify_via_salmon_test
Started
E
Finished in 0.000992 seconds.

  1) Error:
test_0001_integration_regression_test(SalmonUpdateSpec):
RSpec::Mocks::MockExpectationError: Stub received unexpected message :author with (no args)

First of all, daaaaamn. 0.001 seconds. Nice. Secondly, okay, we are getting some messages sent to our stubs. Let’s flesh them out to make things pass:

require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'

require_relative '../../app/models/notify_via_salmon'

describe "salmon update" do
  before :each do
    RSpec::Mocks::setup(self)
  end

  it "integration regression test" do
    user = stub(:author => stub(:domain => "foo"), :to_rsa_keypair => stub)
    update = stub(:to_atom => "")
    feed = stub(:author => stub(:salmon_url => ""))

    salmon = stub(:to_xml => "")
    OStatus::Salmon.should_receive(:new).and_return(salmon)

    uri = stub(:host => "", :port => "", :path => "")
    URI.should_receive(:parse).and_return(uri)
    Net::HTTP.should_receive(:new).and_return(stub(:post => ""))

    NotifyViaSalmon.mention(user, update, feed)
  end
end

Okay! Note how similar this looks to the previous test. We’re still mocking a bunch of ‘external’ stuff. But other than that, our test is pretty simple: we make three stubs, and we call our mention method.

Step four: repeat

We could do some more work on this method. There are a few things that are a bit smelly: the nested stub of User is not great. The fact that we’re stubbing out three external dependencies isn’t great. But before we get to that, let’s check out another method that looks similar: send_{follow,unfollow}_notification. Here’s some code:

# Send Salmon notification so that the remote user
# knows this user is following them
def send_follow_notification to_feed_id
  f = Feed.first :id => to_feed_id

  salmon = OStatus::Salmon.from_follow(author.to_atom, f.author.to_atom)

  envelope = salmon.to_xml self.to_rsa_keypair

  # Send envelope to Author's Salmon endpoint
  uri = URI.parse(f.author.salmon_url)
  http = Net::HTTP.new(uri.host, uri.port)
  res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
end

# Send Salmon notification so that the remote user
# knows this user has stopped following them
def send_unfollow_notification to_feed_id
  f = Feed.first :id => to_feed_id

  salmon = OStatus::Salmon.from_unfollow(author.to_atom, f.author.to_atom)

  envelope = salmon.to_xml self.to_rsa_keypair

  # Send envelope to Author's Salmon endpoint
  uri = URI.parse(f.author.salmon_url)
  http = Net::HTTP.new(uri.host, uri.port)
  res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
end

Look familliar? Yep! 90% the same code! We’ll do the same process to these methods, just like the other one. Check out this code and test:

# Send Salmon notification so that the remote user
# knows this user is following them
def send_follow_notification to_feed_id
  feed = Feed.first :id => to_feed_id

  NotifyViaSalmon.follow(self, feed)
end

# Send Salmon notification so that the remote user
# knows this user has stopped following them
def send_unfollow_notification to_feed_id
  feed = Feed.first :id => to_feed_id

  NotifyViaSalmon.unfollow(self, feed)
end

# Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
  feed = Feed.first :id => to_feed_id
  update = Update.first :id => update_id

  NotifyViaSalmon.mention(self, update, feed)
end

module NotifyViaSalmon
  extend self

  def mention(user, update, feed)
    base_uri = "http://#{user.author.domain}/"
    salmon = OStatus::Salmon.new(update.to_atom(base_uri))

    envelope = salmon.to_xml user.to_rsa_keypair

    # Send envelope to Author's Salmon endpoint
    uri = URI.parse(feed.author.salmon_url)
    http = Net::HTTP.new(uri.host, uri.port)
    res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
  end

  def follow(user, feed)
    salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)

    envelope = salmon.to_xml user.to_rsa_keypair

    # Send envelope to Author's Salmon endpoint
    uri = URI.parse(feed.author.salmon_url)
    http = Net::HTTP.new(uri.host, uri.port)
    res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
  end

  def unfollow(user, feed)
    salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)

    envelope = salmon.to_xml user.to_rsa_keypair

    # Send envelope to Author's Salmon endpoint
    uri = URI.parse(feed.author.salmon_url)
    http = Net::HTTP.new(uri.host, uri.port)
    res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
  end
end

require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'

require_relative '../../app/models/notify_via_salmon'

describe NotifyViaSalmon do
  before :each do
    RSpec::Mocks::setup(self)
  end

  it ".mention" do
    user = stub(:author => stub(:domain => "foo"), :to_rsa_keypair => stub)
    update = stub(:to_atom => "")
    feed = stub(:author => stub(:salmon_url => ""))

    salmon = stub(:to_xml => "")
    OStatus::Salmon.should_receive(:new).and_return(salmon)

    uri = stub(:host => "", :port => "", :path => "")
    URI.should_receive(:parse).and_return(uri)
    Net::HTTP.should_receive(:new).and_return(stub(:post => ""))

    NotifyViaSalmon.mention(user, update, feed)
  end

  it ".follow" do
    user = stub(:author => stub(:domain => "foo", :to_atom => ""), :to_rsa_keypair => stub)
    feed = stub(:author => stub(:salmon_url => "", :to_atom => ""))

    salmon = stub(:to_xml => "")
    OStatus::Salmon.should_receive(:from_follow).and_return(salmon)

    uri = stub(:host => "", :port => "", :path => "")
    URI.should_receive(:parse).and_return(uri)
    Net::HTTP.should_receive(:new).and_return(stub(:post => ""))

    NotifyViaSalmon.follow(user, feed)
  end

  it ".unfollow" do
    user = stub(:author => stub(:domain => "foo", :to_atom => ""), :to_rsa_keypair => stub)
    feed = stub(:author => stub(:salmon_url => "", :to_atom => ""))

    salmon = stub(:to_xml => "")
    OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)

    uri = stub(:host => "", :port => "", :path => "")
    URI.should_receive(:parse).and_return(uri)
    Net::HTTP.should_receive(:new).and_return(stub(:post => ""))

    NotifyViaSalmon.unfollow(user, feed)
  end
end

Now we can see some of these patterns start to emerge, eh? All of this stuff is starting to come together. Let’s get rid of the URI and Net::HTTP stuff, as it’s just straight up identical in all three. Pretty basic Extract Method:

module NotifyViaSalmon
  extend self

  def mention(user, update, feed)
    base_uri = "http://#{user.author.domain}/"
    salmon = OStatus::Salmon.new(update.to_atom(base_uri))

    send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
  end

  def follow(user, feed)
    salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)

    send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
  end

  def unfollow(user, feed)
    salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)

    send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
  end

  protected

  def send_to_salmon_endpoint(salmon, keypair, uri)
    envelope = salmon.to_xml keypair

    # Send envelope to Author's Salmon endpoint
    uri = URI.parse(uri)
    http = Net::HTTP.new(uri.host, uri.port)
    res = http.post(uri.path, envelope, {"Content-Type" => "application/magic-envelope+xml"})
  end
end

We run our tests, it still all works. Now we can just mock out the endpoint method, and all of our tests on the individual methods become much, much simpler:

require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'

require_relative '../../app/models/notify_via_salmon'

describe NotifyViaSalmon do
  before :each do
    RSpec::Mocks::setup(self)
  end

  it ".mention" do
    user = stub(:author => stub(:domain => "foo"), :to_rsa_keypair => stub)
    update = stub(:to_atom => "")
    feed = stub(:author => stub(:salmon_url => ""))

    NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)

    NotifyViaSalmon.mention(user, update, feed)
  end

  it ".follow" do
    user = stub(:author => stub(:domain => "foo", :to_atom => ""), :to_rsa_keypair => stub)
    feed = stub(:author => stub(:salmon_url => "", :to_atom => ""))

    salmon = stub(:to_xml => "")
    OStatus::Salmon.should_receive(:from_follow).and_return(salmon)

    NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)

    NotifyViaSalmon.follow(user, feed)
  end

  it ".unfollow" do
    user = stub(:author => stub(:domain => "foo", :to_atom => ""), :to_rsa_keypair => stub)
    feed = stub(:author => stub(:salmon_url => "", :to_atom => ""))

    salmon = stub(:to_xml => "")
    OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)

    NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)

    NotifyViaSalmon.unfollow(user, feed)
  end
end

We’ve managed to get rid of all of the mocking of the URI stuff. It’s all details now. We also have great information about what is actually needed for this test: we know exactly what our objects expect the caller to pass. We’re only depending on ourselves and some external libraries. Our tests are fast. Overall, I’m feeling much better about this code than I was before.

In conclusion

I hope you’ll take a few things away from this post.

Start with tests first! If you don’t do this, it’s very easy to introduce simple errors. Don’t be that guy. You know, the one that has to make pull requests like this… I suck.

Don’t be afraid to change the tests! As soon as you’ve verified that you’ve transcribed the code correctly, don’t be afraid to just nuke things and start again. Especially if you have integration level tests that confirm that your features actually work, your unit tests are expendable. If they’re not useful, kill them!

Mock like a motherfucker. When done correctly, mocks allow you to help design your code better, make your tests fast, and truly isolate the unit under test. It’s pretty much all upside, unless you suck at them.

Extract, extract, extract! Adele Goldberg once said, “In Smalltalk, everything happens somewhere else.” Make tons of little methods. They help to provide seams that are useful for testing. Smaller methods are also easier to understand.

Refactoring is a marathon. Even with all this work, this code isn’t the best. The tests could even use a little TLC. But it’s still better than it was before. Those who ship, win. Make things a little better, git commit && git push, repeat.

Twitter is too important to be owned by Twitter

Sun, 24 Jul 2011 00:00:00 GMT

When I spoke on a panel at RTLM NY ’11, I got a little rise out of the crowd when I said “Twitter is too important to be owned by Twitter.” This is still absolutely true, and yesterday was a great example of that: the #fuckyouwashington debacle.

If you haven’t heard, Jeff Jarvis started something on Twitter last night: #fuckyouwashington.

As of right this second, when I search for #fuckyouwashington on Twitter, they’re coming in at two or three a second. It’s crazy. And it’s really interesting to read what people have to say, pro and con, picking their own issues, all of that stuff. Yet, it’s been removed from the trending topics.

Now, maybe this particular trend was simply blocked because of the profanity. Sure, whatever. I don’t like that, but some might. And Twitter certainly allows all kinds of trends to stick around that would also be ‘offensive,’ for example, #babymakingsong, #turnoff, and #itsalwaystheuglypeople are currently trending. Don’t even get me started on trends like #whatblackgirlslike #whatwhitegirls like and the other racially charged topics. Sure, fine, it’s their site, they can do whatever they want.

That, however, is the root of the issue: they can do whatever they want. And we’re all beholden to them. Nothing we can do. Maybe I’m not sensitive to four letter words, and I do want to hear about things like this. Too bad, Twitter says no. Maybe I don’t want promoted tweets added to my stream. Too bad, Twitter says yes.

And that’s why I’m feeling worse and worse about Twitter every day. It sucks, because I really like Twiter, the website. But Twitter, the company, really bums me out. They promised all of us nerds long ago that they wouldn’t do this kind of thing, and they’re not keeping that promise. Which means I’m not really getting emotionally invested in Twitter any more. It’s still valuable enough to me to keep using, but I can’t care about it the way I used to.

This is why we’re still hacking away at rstat.us. If you haven’t heard about it before, you can read my release announcement. If you’re lazier than that, here’s the summary: rstat.us is a distributed, open-source Twitter. You can use our public node at the website, or get your own copy of the code and run your own, make whatever customizations you’d like, and you’re still on equal footing with everyone else in the network. We’re on the cusp of a 1.0 announcement, and I’d love to have been able to make that in this post, but we’re still working out some last kinks. If you’ve seen it before, you won’t find much differences right now, but we have 100% standards compliance, a complete visual refresh, and some other fun goodies waiting in the wings, ready to come out shortly.

Just remember: if you use a service like Twitter that’s controlled by a company like Twitter, you have to play by their rules. And sometimes, they’ll change the rules, and it’ll be time to move on.

Single text, many masters

Sat, 21 May 2011 00:00:00 GMT

Hey everyone. Here’s a draft of an essay I’ve been working on. I’d love to hear your feedback.

The word ‘engineering’ has a deep connection to the word ‘trade-offs’ in my mind. Most engineering decisions come down to evaluating a few differing alternatives, and often multiple factors end up being negatively correlated. You can make something stronger, but then it will be heavier. It can be made faster, but then it’s significantly more expensive. A good engineer is able to take all of these factors into account, and design a system such that it maximizes its effectiveness across the sum of all of the relevant constraints. No matter if you consider the act of writing software an art, science, or engineering, its indisputable that designing complex software systems is identical in this respect. There are dozens of different metrics that system architects take into consideration while crafting a plan of attack, but but there’s a deeper question of balance here that’s significantly different than these more traditional engineering issues.

Text, in the form of source code, presents unique challenges of composition. These difficulties all stem from the same root: source code is a singular text, but must be intelligible to multiple, simultaneous audiences. More traditional forms of authorship still take audience into consideration, of course, but the decision to engage a diverse group of people is the choice of the writer. While mass appeal may be something that certain authors strive to attain, it’s not an inherent property of their chosen form of expression. Source code, while text, inhabits a multiplicity of forms, and software developers are confronted with this inherent multi-faceted complexity when composing any particular software work. Some of these forms suit certain audiences better than others, and so it falls to the programmer to manage which form they are currently working in, consider which audience they are attempting to write for, and arrange all of these forms amongst one another in such a way that any particular audience is able to navigate and extract the proper information from the source without confusion.

In this post, I’ll expand on the concept of audiences for code, and in a future post, I’ll explore the simultaneous forms that code takes.

The multitude of audiences

The default audience: the computer

Science is what we understand well enough to explain to a computer. Art is everything else we do.Don Knuth

This may seem obvious, but the when considering the question of “Who are programs written for?”, many would say “The computer. Duh.” In many ways, a computer is the primary reader of a particular piece of source code. The code that’s given to a computer will be executed billions of times per second, repeated, recalculated, and re-interpreted over and over and over again.

The computer’s native understanding of software comes from machine code. Machine code are the binary numbers that the CPU loads into memory and processes directly. For example, here’s a line of machine code for an x86 machine that puts the value ‘97’ into the AL register, graciously stolen from Wikipedia:

10110000 01100001

While most would consider this unintelligible, computers were actually programmed this way at one time. My uncle actually did this by flipping switches to set the binary and pushed a button to store it in memory. Unfortunately, what’s good for the computer isn’t good for the human programmer. This is why assembly language was created. Assembly language has a 1 to 1 mapping to machine code, but is much easier for humans to understand. Here’s that same line in assembly:

MOV AL, 61h       ; Load AL with 97 decimal (61 hex)

The MOV corresponds with 10110, AL maps to 000, and 61 hex is 01100001. MOV is short for ‘move,’ though, and this mnemonic is just a bit easier to understand than 10110. This is the most basic example of a compositional trade-off. It’s not a true trade-off, because they map perfectly to one another, but it illustrates the difference between composing in a language that the computer understands and one that’s more natural for the programmer. Another important concept comes into play, that of compilation. Virtually every work of composition created in software is automatically translated to another form before it is executed. We’ll address this concept more fully when we discuss form later.

If that’s where assembly stopped, it would remain a 1 to 1 mapping. However, virtually every assembly language also offers macros, and this moves the code further away from the machine and destroys the synchrony of the two forms. Here’s an example:

MOV EAX, [EBX]

The [] characters change the meaning of EBX, rather than be the value stored in that particular register, they imply that the value is a memory address, and we want to move the contents of that address to EAX. The generated machine code could now be processed into multiple valid assembly forms, and so the transformation is only perfect in one direction, even if it’s possible to ‘decompile’ it into one of those possible encodings. This is considered to be an acceptable trade-off for human readability; we very rarely want to turn machine code back into assembly.

There’s also a jump between higher level languages, as well. Here’s the assembly statements that adds 2 and 3 together:

MOV EAX, 2
ADD EAX, 3

ADD, of course, is the statement that adds a number to the register that’s given. Now EAX has the value 5. Here’s the same code, but in C:

int x = 2;
x = x + 3;

Pretty simple. You can see how the C is much easier to understand; we say what type x is (an integer), and it’s a bit more explicit. x is equal to x + three. However, since the C is divorced from the machine, and is written for the person, we can change our compiler to make assembly code that works on a different kind of processor architecture. If we were to compile the above C for x86_64, a 64 bit version of x86, we might get some assembly that’d look like this:

MOVL RAX, 2
ADDL RAX, 3

While this code looks similar to the above, it is quite different. This uses the native 64 bit types, rather than the 32 bit types above. The other important thing is that by writing code that’s divorced from the machine, and written for people, we’re able to translate it into the code for multiple machines. If we had written the assembly above, when moving to another architecture, it would have required a total re-write. And while this particular sample looks very similar, a more complex piece of code will be significantly divergent, but I don’t want to go into the details of two kinds of assembly code. Because we can define the languages for humans, and the language of computers is somewhat beholden to the physical machine itself, it’s significantly easier to do the translation from C to the two kinds of machines, rather than trying to translate from one machine to another. What we lose in this kind of translation, though, is efficiency. Code that was hand-crafted for each machine would be more performant, and better represent each individual platform.

Even though we may choose to use a language that’s more understandable to people, it still has to be understood by the computer in some form. This translation will introduce some amount of penalty, and so it’s important that this gets taken into consideration. Sometimes, code must be written in a way that’s not easy for a person to read, because it’s easier for the computer to be efficient with a more opaque implementation.

The reflexive audience: the programmer himself

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.Brian Kernighan

Everyone who writes code has experienced this at some time or another. You write a whole bunch of code, and then a few months goes by, and you take a look at it, and it’s absolutely unintelligible. This happens because at the time of inception, the author of a particular piece of code has an incredibly close relationship to it. As it was just written, the code is obvious to the author. They’re in the proper mindset to understand the intention that was drawn upon to necessitate bringing those lines into the world, and so no extra explanation is necessary. As time goes on, however, the author becomes more close to the third audience, other programmers. It’s important for coders to recognize this fact, and take preventative steps to ameliorate this confusion.

Even though the author will approach the position of the other audience eventually, this audience is distinct because there is a certain level of explanation that sits between undocumented, inexpressive code and code that’s well explained, and this is the position most code is in. An explanation that’s helpful to those who understand the code, but not to those who don’t is better than nothing. This sort of code may be overly contextual, and could use some added information to improve its clarity.

The other audience: colleagues and coworkers

Always code as if the guy who ends up maintaining your code is a violent psychopath who knows where you live.Martin Golding

As I touched on earlier, there’s a similarity between the ‘other’ audience and the reflexive. The primary distinction is drawn around the proximity to the source. The other does not have the advantage of having authored the code, and therefore doesn’t have the native’s understanding of the underlying logical organization of the source. This disadvantage can be overcome via composing in such a manner that the meaning is emergent from the design. Even if it’s too complex to be obvious, good documentation can address this particular deficiency.

Ultimately, much of software design is about modeling. Software that solves a particular problem should emulate the nature of the challenge it’s attempting to address. If this can be achieved, it’s significantly easier for those who understand the problem to figure out how the software works. Therefore, good design can help improve the effectiveness of a given piece of source to communicate its intent. Along a similar vector, if the design is similar to code that solves a particular issue, it’s easier to understand. As an example, a friend recently asked for feedback about an interface that he’d designed. It loaded a save game file for StarCraft 2. This is what he came up with:

replay_file = File.new("spec/fixtures/1v1-game1.sc2replay")
@replay = SC2Refinery::Parser.parse(replay_file)

However, Ruby already has several kinds of code in its standard library that loads some information from disk and parses it into some kind of data structure that you can use in your program. The JSON, YAML, and Marshall classes already use a set of methods to import and export data, and they’re named load and dump, and they’re part of the class directly. Also, in this case, the user of the code shouldn’t need to deal with the creation of a file, since it’s unreasonable to assume that a game replay would come from any other source. Therefore, after some discussion, he adopted the following interface instead:

@replay = SC2Refinery.load("spec/fixtures/1v1-game1.sc2replay")

This is much nicer to use, and is much simpler. While it may not seem like a whole lot, when rules like this are applied across an entire codebase, they can significantly increase understanding. Multiple reductions of mental overhead add up quickly.

My new favorite trick for adding a little bit of modeling that significantly reduces overhead for the user is the Presenter Pattern. Jeff Casimir demonstrated this very clearly in his presentation at RailsConf 2011, “Fat Models Aren’t Enough”. Here’s a slightly modified example. Imagine that we have a system that manages students, and we’d like to display a report card for them. We might start with some code that looks like this:

student = Student.find(options[:student_id])
term = Term.find(options[:term_id])
report_type = ReportType.find(options[:report_type])

puts "#{student.last_name}, #{student.first_name}"
puts "#{report_type.report_title} for #{term.start_date} to #{term.end_date}"
student.courses.each do |course|
  course_report = student.report_data_for_course(course)
  puts course_report.to_s
end

Honestly, until this past week, this is the exact code that I would have written. But it turns out that we can do better. Basically, we’re displaying some information that comes from a combination of three different objects. If we think about it some more, we’re really trying to display a report card. So let’s make an object that represents the card, and delegates to its sub-objects. It will then know how to display itself.

class ReportCard
  delegate :start_date, :end_date, :to => :term
  delegate :first_name, :last_name, :courses, :report_data_for_course, :to => :student
  delegate :report_title, :to => :report_type

  def initialize(params)
    @student = Student.find params[:student_id]
    @term = Term.find params[:term_id]
    @report_type = ReportType.find params[:report_type_id]
  end

  def student_name
    [last_name, first_name].join(", ")
  end

  def title
    "#{report_title} for #{start_date} to #{end_date}"
  end

  def course_reports
    out = ""
    courses.each do |course|
      out += report_data_for_course(course)
    end
    out
  end
end

Now, this is a lot of code. However, as you can see, it’s all focused on composing the information and exposing an interface that makes sense for a report card. Using it is super easy:

report = ReportCard.new(options)
puts report.student_name
puts report.title
puts report.course_reports

Bam! It’s incredibly obvious. This code is much more clear than before. We’ll see if I’m still as hot on this pattern as I am now in a few months, but I feel the extra object adds a significant amount of clarity.

If the model is too hard to create, or if additional clarity is needed, documentation in the form of comments can also help to improve the understanding of the ‘other’ audience. Comments can be a difficult form of prose to write, because they need to be written at the correct level of abstraction. If they simply repeat what the code does, they’re useless, and if they’re too high-level, certain details and semantics may not be made clear.

Individual bits of code can also be made more clear by developing a narrative within any particular method that’s being written. Telling a story with code may not be something you’ve considered before, but it’s really about maintaining a proper flow in the actions your code is taking. For example, if there’s a bunch of error handling strewn about inside of a method, it’s less clear than bunching all of the error handling near the end. Most code should be an act in three parts: input, processing, and output. If these three parts are mixed together, it can appear much more complicated.

The forgotten audience: end-users

If I asked my customers what they wanted, they’d have told me, “A faster horse.”Henry Ford

In the end, all software is used by someone. Use-value is the driving force of virtually all code. Code that doesn’t do anything may be making some kind of important philosophical statement, but isn’t really the sort that I’m talking about.

The introduction of a user imposes significant restrictions upon the way that code is composed. End-users do not need to understand the code itself, but they do need to be able to understand its external interfaces. These needs place an imposition on the way that the code needs to be written, because it must address this issue of interface. Sometimes, interface requirements can create a burden on the internal implementation. Needing to support certain behaviors and forms can create complexity for an implementor.

Documentation created for end users must be completely different than that which is created for those inspecting the code itself. Most end-users will not be literate in the arts of software development, and so approach the software object in an entirely different way than those who write code do. Yet, the same semantics must be passed on to them, but at a higher level. There’s a strong movement within the community to start designing software with this kind of end-user documentation in mind, called README driven development. There are advantages to thinking on this level when beginning, but a nice effect of doing it first is that you can ensure it gets done. A surprising amount of software has poor documentation for its users, because it’s created after the software is finished, and at that time there’s intense pressure to ship it out the door. Writing down information for the end user first ensures that it’s done properly, that all development works in accordance with the documentation, and that all of the use-cases for an end-user have been thought of and are being addressed.

Trolling is a art

Fri, 24 Sep 2010 00:00:00 GMT

I find it incredibly hard not to be judgmental.

I’m not sure what part of my personality makes this happen, but even when I try to curb tearing down other people, I end up doing it anyway. I’m not sure if it’s just me, but I think part of the problem is that these kinds of things are rewarded.

One of the reasons that I use my real name everywhere is to try to improve the civility of my discourse. When you’re anonymous, it’s really easy to be callous. And when I used a handle, I was even worse. But even now, I can think of three times when I accidentally was mean for basically no reason: That tweet, the Diaspora incident, and the only Hacker News comment I’ve ever deleted.

In all three of these instances, it’s not so much that I was wrong. It’s that I ended up presenting my criticisms poorly. I think I did a decent job with the Diaspora one, but people still took it the wrong way, so I failed to communicate. I’m certainly not infallible. I guarantee you there are n00b errors and security holes in my own codebases. So I really shouldn’t be so quick to judge. That’s all I have to say about that. I’ll just have to do better going forward.

80% of success is showing up

Tue, 21 Sep 2010 00:00:00 GMT

It’s funny how true this quote is.

There are three people that I really admire: _why, Zed Shaw, and Yehuda Katz. And to simplify my tenses a bit, I’ll speak of _why in the present tense. What do these three guys have in common? They all churn out badass software like nobody’s business. _why started so many projects that it just adds to his mythical qualities; Zed has been cranking out software and blogs left and right, and Yehuda’s “I’m quitting Engine Yard” post basically said “I’m pretty much done rebuilding the server side of software in my own image, now it’s time for me to focus on the client side of the equation.” Not with that much hubris, of course, but it’s pretty much true.

Now, I don’t mean to imply that they’ve ‘just shown up,’ but it seems to me that productivity is directly related to just getting started on things. I’ve had people tell me that they think I’m superhumanly productive, which is funny to me: I’m almost constantly fretting about how much time I waste every day. And while I do have a ton of projects, unlike all those guys, I tend to not fully finish them. It’s something that I’ve been trying to work on lately, because I have so many things that I want to accomplish.

For me, the biggest hold on cranking out even more stuff is just simply starting to work. Often, if i slack off for half an hour, I slack off all evening. Now that there’s a TV with Netflix in my house again, it’s difficult to not watch all of the amazing documentaries that are on there. But if I manage to start working on stuff, I find that I keep doing it. It’s like I get to pick one or the other: projects or slacking. I can’t quite seem to do both. And so I’ve been starting some minor experimentation to see if I can’t work on projects more often than not.

One thing that I’ve sort of half-tried is the Pomodoro Technique. It’s pretty sweet: basically, you set a timer for 25 minutes, and start working. You can take a break when it goes off. Surely you can wait to refresh Hacker News for 25 minutes, right? That’s something else I’ve done: on Friday, I put Reddit and HN in my /etc/hosts file. They’re totally blocked, for now. After last week’s situation, I spent waaaay too much freaking time on HN, even for me. I love it, and I’m learning a ton every day, but it’s becoming an issue for my productivity. So it’s gone for a bit. It’s been rough, but I’ve been getting lots done, and I think when I start reading it again, I’ll be able to keep it at reasonable levels.

Anyway, so that’s what I’m doing to keep productive. Anyone else have any thoughts on this? How do you keep working on side projects? What works for you?

Trouble with Diaspora

Fri, 17 Sep 2010 00:00:00 GMT

So, Wednesday, Diaspora was released.

If you’re not familiar, a few months ago everyone was up in arms about the latest Facebook privacy change. So four college kids started a Kickstarter project with a dream: let’s make a distributed, private Facebook. They asked for $10,000. They got about $200,000.

They worked on the code all summer, in secret. Wednesday, a ‘developer preview’ came out. They’d promised something by the end of the summer, and the 15th of September is the last day. So they put it up on GitHub.

Oh boy.

Basically, the code is really, really bad. I don’t mean to rain on anyone’s parade, but there are really, really bad security holes. And they’re there due to things that any professional programmer would never dream of leaving out of their code. I don’t want to disclose too many details, but you can see the code yourself on GitHub.

At first, I found one. So I tried to patch it. And I did, and it got accepted into master. Awesome. But then, the more I read, the more bad things I found. They’re going to need a complete overhaul to fix this. Go over every last piece of code. And don’t even get me started on their encryption code.

But basically, please heed their own warning from the announcement:

Feel free to try to get it running on your machines and use it, but we give no guarantees. We know there are security holes and bugs, and your data is not yet fully exportable. If you do find something, be sure to log it in our bugtracker, and we would love screenshots and browser info.

If you find one of the many, many nodes that people are publicly posting, please don’t use your real passwords, or information.

And if you’re a developer, consider helping out. It’s going to be rough going: the mailing list and issue trackers are full of people that have no idea what they’re doing. Be prepared to wade through tons of crap. But they could really, really use the help. I’ll be submitting another patch or two, but it needs much, much more than I can give.

EDIT: Please check out my follow up post, where I talk about being misquoted regarding this post.

EDIT 2: Patrick has a post up describing the exploits, now that there have been some patches applied. If you’re a technically inclined individual, you might be interested. You can find it on his blog.

A case study in being excellent: Divvy

Wed, 30 Jun 2010 00:00:00 GMT

The images in this post have been lost to time. Oh well. Original post below, images removed.

The other day I wrote a post about being excellent to each other. Today, I’ve got a great example of that for you: Divvy.

What would you do if you received an email like this?

Here’s what happened:

Here’s a link to the thread. Check out some of these comments:

… okay, maybe that last one is a bit much. But still, they’ve obviously been repaid many times over for something that didn’t even cost them money. This comment on HN really says it all:

If you’d also like to support a company for being awesome, here’s a link to Divvy.

EDIT: They’ve posted the sales afterward on Reddit here.

Productivity Waves

Tue, 08 Jun 2010 00:00:00 GMT

Right now, I’m having a bit of a ‘darkness before the dawn’ kind of moment. I feel like I might soon start to become productive again.

It’s sort of strange, how these things travel in waves. Just a month or two ago, I was super-ultra-crazy productive. I was working on Hackety, coding Bindlr, writing three days a week on the blog, and more. This doesn’t even count the more-than-a-full-time CloudFab. But lately, all I’ve wanted to do was hang out and play video games.

It seems like this is a cyclic kind of thing. I think 80 hour weeks, even of things that you love, is kind of unsustainable. It was lots of fun, and I was feeling pretty good while I was doing it, but now that I’m off that horse, it’s really hard to get back on.

In any case, here’s the status of some of the stuff I’ve been up to, if you’ve been wondering:

Blog: I’ve got a bunch of stuff half-written. I’ll be starting to finish those off and getting them up here. I’ve had lots to say, just no motivation to say it.
Watch.steve: I want a better design, so I’m not doing anything with the site ’till that’s done. I don’t want to mess with Ryan’s look and feel. But I’ve got episodes written up, they just need to be recorded.
Hackety Hack: Fela is making great progress, and I’m going to start reaching out to Mac development communities to see if I can get some help on the issue that’s ruining my ability to fix things.
Bindlr: Up, running, and working. Needs more marketing/outreach.
Twitter: started using it again, you read that post.
Selling my stuff: most of it is in boxes or thrown away. Time to start selling the things that were worth money.
Maria: currently missing her.
Friends: started actually making it back to XOmB meetings on Saturdays, and actually going out with some people I haven’t talked to in a while. I was really really bad at this in some cases.

This post is kind of random, and I apologize. I’ve got a good mix of technical things, book reviews, and social commentary waiting in the wings.

I’m excited for what June will bring.

A Declaration of the Independence of Cyberspace

Sat, 01 May 2010 00:00:00 GMT

I just saw this today. It kind of goes along with my “Move to the Internet” post from a few days ago.

Enjoy.

A declaration of the Independence of Cyberspace

Living in the cloud

Sat, 24 Apr 2010 00:00:00 GMT

I’ve been joking for a while that I’d like to “move to the Internet.” Generally I cite my familiarity and love with online culture as opposed to American culture. I don’t really care what’s going on in “the real world,” as I feel it’s kind of played out. The whole cycle of “fight wars, reduce freedoms, sit in front of your TV” has gotten really boring quickly. I find Internet culture to be more complex, entertaining, and useful. But I’m getting way off topic, here.

As a programmer, one of the key skills I rely on daily is the ability to recognize abstractions. Writing great code requires the ability to see similarities in things, even when they may not be immediately apparent. Lately, I’ve been giving a lot of thought to the idea of “living in the cloud.” Primarily, cloud computing is about providing services on-demand, and in the amount you need. Dynamic allocation. It’s generally great stuff, and even though “the cloud” is a bit buzzwordy, it’s an awesome concept.

So what happens when we apply that to meatspace? What does “cloud living” look like? “Cloud working”?

Living in the cloud would look a lot like the world Tim Ferriss encourages with his whole lifestyle design concept, or at least the lifestyle he’s designed for himself. Move around from place to place, live where you find things interesting, reallocate yourself to a better place when the feeling dictates. The concept of being a “digital nomad” certainly makes me think of “living in the cloud.” Doing this right means giving up most material possessions, as they impede freedom of movement.

Another important aspect of cloud living would be finding a way to make a living while being location independent. There was a great article making the rounds a while back called “Jobs don’t scale.” It’s only tangentially related, but his point about working for other people certainly relates to freedom of movement. If you have to convince your boss to let you work remotely, it’s much harder than just doing it yourself.

In any case, I’m very attracted to all of this as an idea. I’d love to travel the world, working from a different place every day, seeing new sights and exploring new places. It requires re-thinking a lot of things about what it means to work, how relationships work, friendships. Can you have kids if you keep moving around from place to place? How can you stay connected to friends if you never see them, because you’re halfway across the globe? Can your significant other do the same thing? Can you get month to month leases easily?

I’m still thinking about this as a concept, but I thought I’d share my thoughts so far. I’m not moving around the world yet, but I am trying to increase my own mobility as much as possible. I think I’m finally at a place where I don’t care much for most of my possessions anymore…

Book review: Rework

Wed, 10 Mar 2010 00:00:00 GMT

I’ve been waiting for this book for a while. “Rework” is the new book by Jason Fried and David Heinemeier Hansson from 37signals. It hit stores on Tuesday. Here’s a (non-affiliate) link to Rework on Amazon.

For those of you who don’t know, I’m down in Houston, Texas. I’m here for a few days before heading over to Austin for SXSW. There’s a lot of really cool start up stuff, as well as 3D people, and a friend of mine. It’s been a good trip so far. But even with waking up at 3am, connecting flights, and a few hours of driving, I still managed to find a spare moment to head over to a Borders and grab a copy of Rework. And even though I’m running all over town, Nick is driving, so I’ve been able to read all of Rework in between lunches and networking events.

Rework is interesting. I described it earlier today as a philosophy text, and I feel that description is absolutely apt. It’s 37signals in its purest, most potent form. If you’ve read Getting Real, this territory will be familiar. In fact, a lot of it is basically the same. I’d be lying if I told you otherwise. The real truth is that it doesn’t matter. People who don’t already know and love 37signals won’t have read Getting Real, and so this stuff will be novel to them. People who do won’t mind re-reading this information again, as they’ve bought into the philosophy. And an update isn’t a bad thing, either. What makes Rework interesting is how it’s different from Getting Real, not what’s the same.

I thought it’d be most interesting to talk about Rework in the context of it’s own philosophy. I think there are three points in particular in the book itself that point out just why this book is so good. The first is an old 37signals standby, the other two are new.

Build half a product, not a half-assed product

This got a section devoted to it in Getting Real. Here’s the core idea:

Throw in every decent idea that comes along and you’ll just wind up with a half-assed version of your product. What you really want to do is build half a product that kicks ass.

They mention that Rework used to be twice as big. The next to last draft was 57,000 words, and the final draft was 27,000. This is the biggest difference between the two books. It’s the most pure dosage of Kool-Aid I’ve ever read. Each section feels finely honed. They’re all either one, two, or three pages, and an accompanying picture. This book is about what’s worked for the company so far over its lifetime, and this refinement process is clearly demonstrated here.

It’s always easy to ask for more. I’m really curious about the things that were cut. Were there more sections? Was each section twice as long? A little of both? At the same time, this exactly exemplifies the thinking this section is railing against. If the book was twice as long, would I have learned twice as much? Probably not. YAGNI.

Decommoditize your product

Make you part of your product or service. Inject what’s unique about the way you think into what you sell.

This is something that these guys do really well. It’s part of having such strong opinions, and sharing them with the world. Everyone knows who 37signals is and what they stand for. If I wrote each chapter of Rework into a blog post, you’d still recognize it as their philosophy. It also comes through in the writing. They mention in the notes that Matthew Linderman helped them pull their distinct styles of writing into a more cohesive whole. He did a good job, and didn’t let the voice get lost in the editing.

Out-teach your competition

Teaching is something individuals and small companies can do that big companies can’t.

Teaching is a topic I’ve been getting more and more into lately. Hackety Hack is about teaching; when I speak, I’m teaching; this blog is about teaching. Rework is about teaching the lessons 37signals have learned about business to the world. A lot of Signal vs. Noise is about teaching. It’s a great way to get people to recognize you, and a great way to give back. The world can always use more great teachers.

Wrapping up

There’s a reason I’m up late, writing this review. I couldn’t put Rework down. I enjoyed revisiting the old topics; the new ones are still tumbling around in my brain. I’m sure this book is going to spawn a bunch of posts on this blog in the future, as I add my own thoughts to the stuff I’ve learned. I’d recommend this book to anyone who’s working in the business world or doing a startup, even if they hate that term. It’ll give you a lot of interesting thoughts to chew on.

Create a more compelling experience for your users through game mechanics

Mon, 08 Mar 2010 00:00:00 GMT

Ever wonder why some websites are so addictive? Certain sites always keep you going back, time after time after time. Well, I can’t speak for all of them, but there’s a subtle reason that some sites draw your attention on such a repeated basis: They’re actually games.

Wait, games?

Try a little thought experiment: If I say, “Yeah, he’s a ______ addict,” what are the first few things that pop into your mind? For me, top two are “heroin” and “World of Warcraft.” I’m not sure what that says about me as a person, but ignore that for now. What makes these two things so addicting? Why are they basically synonymous with the word “addict”? Lots of people smoke pot. Lots of people play Call of Duty. Lots do both, and in copious amounts. So why don’t they get the same label?

Heroin: it’s a hell of a drug

Yeah, that reference is to cocaine, another famously addictive substance. Oh well.

Heroin is the poster child for addiction because it’s got a built-in viral loop. That sentence sounds ridiculous, but it’s true. It’s very easy to start out with, as it’s snorted. No scary needles or anything. You get high really quickly, due to its chemical properties combined with the fact that your nose is already close to your brain. It gives a really intense high that is also fairly short. As you do it, you develop both a psychological addiction as well as a tolerance. You simultaneously develop a deep desire for more of the drug as you need a larger quantity of the drug to get the same high. Eventually, it becomes more and more difficult, but you’re so addicted that you get over your fear of needles and start mainlining.

World of Warcraft works the same way. It’s easy to try, as there are mechanisms to invite your friends, and the system requirements are fairly low for a video game. The first few quests are super easy, and so you hit that quick reward. You get addicted to “Ding!” but it takes longer and longer every time you do it. Eventually, you max out on levels and have to start doing other things to get your fix. It may sound funny, but it’s absolutely true. People talk about “relapsing.” They speak of “craving.” That’s why WoW has so many subscribers.

How to replicate this success

I can’t guarantee that you’ll be able to make your site as addictive as heroin is, but many sites use the same basic psychology to keep you coming back. Game mechanics are one of the tools they use to develop that psychological addiction. This is something we’ve been seeing more and more of lately, but it isn’t really being talked about explicitly as a major trend. I really think that this stuff is really important and useful.

There are a couple of different mechanisms that web sites can incorporate that fall under the realm of “game mechanics:”

Collectibles: Any sort of item you can accumulate. Sometimes comes in “sets,” which are finite lists.
Points: A concrete number that lets you compare two people.
Levels: A target number of points, you gain the “level” when you go over that number.
Trophies: A special kind of level that’s unrelated to points. You get it for some other arbitrary reason.
Rankings: A place where you can go to see how many points, levels, and trophies others have
Tournaments: A competition between people.

We’ve all heard these terms used in games. But in web sites? Okay, let’s try those things again:

Collectibles: Gowalla items. Facebook “Gifts”
Points: Twitter followers. Facebook friends. Number of feedbacks. Reddit Karma.
Levels: eBay “Power Sellers.” Foursquare “Super Users.”
Trophies: Badges, of any kind. “Achievements”
Rankings: FourSquare’s Leaderboard. Klout. Listorious. Hacker News’ top list.
Tournaments: I actually can’t come up with a good example of this. Thoughts?

The same feedback loop happens on these websites. You say something interesting on Twitter, you gain another follower or two. You say something else, another follower. You check in, oh look, you’re the mayor! You sell an extra hundred things and get your Power Seller discount.

That’s the hard stuff. It’ll get you hooked, and coming back for more.

Where’s all of this going?

This is the current stuff that’s being done with game mechanics. But where could we go, in the future?

A while back, there was a huge debacle over ReadWriteWeb and Facebook connect. To give you the basic idea, ReadWriteWeb is a blog that talks about everything Web2.0. They wrote an article entitled “Facebook Wants to be your One True Login.” Read the comments. Notice something funny? Due to some Google magic, if you were to Google “Facebook login” the day that was posted, that article would appear at the top under the “Google News” results. Now, RWW uses Facebook Connect for their commenting system, and a ton of people apparently don’t know how to use the Internet. So when they said, “Hey, I think I’ll go to Facebook today,” they Googled “facebook login,” clicked the news story, and went to RWW. They then ignored that RWW is a blog completely covered in red that looks nothing like Facebook, scrolled until they found the Facebook icon, clicked it, logged in, and then said “wtf, this isn’t my facebook? Why’d they change the interface again???” This happened a week after a middle-sized interface upgrade on Facebook, for extra hilarity.

Now, I won’t comment on those people or that situation directly. But one of my favorite Hacker News posters, patio11, posted a really interesting comment about the situation. I’m linking to the person he’s responding to, for context:

Pyre: Facebook can’t improve their interface to make users not type “facebook login” into Google as a way of accessing their site.

patio11: That is a failure of the imagination. They certainly could – whether it is worth doing or not is another question, but hey, that is what God gave us A/B testing to figure out.

“Hey user, it looks like you came to us today from Google searching for [Facebook login]. Did you know that there is a better way? Type facebook.com into [blah blah blah]. Try it now and we’ll give you 5 free credits for [without loss of generality: FarmVille]!”

Great job! You should do that every time. If you do that to log into Facebook the next five days you use the service, we’ll award you a Facebook Diploma and give you another 10 free credits for [without loss of generality: FarmVille]!”

On the back end, you show the above prompts to N% of your users who you detect coming to the login page from Google search results (this is trivial – check the referer). You then compare any user metric you want for the “Was Shown Facebook Login Course” population and “Complete Facebook Login Course” population with the population at large. Kill the test if it hurts your metrics, deploy it sitewide if it helps them.

How cool would that be? Now the game mechanics aren’t being used just to increase engagement, but to actually teach people how to use your site or service. It’s classical conditioning; reward people for doing the right thing, and they’ll keep doing the right thing.

Game mechanics are your MVP

So how’s this stuff relevant to your startup? Well, I think this idea ties in really well with the concept of a Minimum Viable Product. Here’s the idea: Build your MVP, and then build game mechanics in. Unlock new features based on game mechanics. This gives you a few advantages:

Your new users only get the most basic experience, which is still useful. It’s a simplified, streamlined experience.
Users only get the new features added that are relevant to how they use the site itself.
You can “fake it till you make it” by implementing the features that are most useful to your users. Is everyone getting Badge A and not Badge B? Implement Feature A Level 2 first!

I think that this makes for a really strong experience, if done right. Foursquare kind of does this already in a crude way with their Super User features. But I think it could be taken to a whole new level.

Think about this: Facebook, where you can only friend people, update your profile, and send messages at first. Soon you unlock the ability to use applications. Then the ability to create pages and groups. The interface slowly unfolds in front of you. What about Reddit, where posting comments is all you can do at first? A hundred upvotes gives you the ability to downvote. Ten comments lets you post stories. (Hacker News sort of does this already, with a minimum karma before downvoting is enabled.)

If you could pull it off, I think it’d make for a really compelling user experience. It does bring one extra design skill that many people may not have, though: balance. Game designers are used to this already, but your potential “Power Users” might not like having to wait to get more advanced features. Then again, this might also solve some issues, like spam. If you had to have 100 positively moderated comments before posting a story on Digg, it’d be much harder to just sign up for spam accounts to submit bogus stories.

This idea can be taken in a lot of different directions. I’m sure I’m only barely scratching the surface with this idea, but I think it’ll go really far. What do you think? Any interesting mechanics I’ve missed? Any really interesting thoughts for how services can incorporate game mechanics? I’ve decided to re-open comments, but if nobody uses them, I’ll just shut them off again. Let me know what you think.

This post has been featured on the Startup Pittsburgh blog, here.

The Little Coder's Predicament

Mon, 28 Dec 2009 00:00:00 GMT

For those of you that don’t know, I’ve taken over a project by someone named _why. He dropped off of the face of the Internet back in March, and the community has picked up his works. Because I’ll be writing about the project in the future, and because I feel these words are still relevant, I’m reprinting the two of his essays that led up to the creation of the project, Hackety Hack, here on my blog.

Enjoy.

The Little Coder’s Predicament

Okay, then, children of the modern age (where we live in a world so tied together with wires that Pangaea ain’t goin’ nowhere!), you tell me if this is a predicament or not.

In the 1980s, you could look up from your Commodore 64, hours after purchasing it, with a glossy feeling of empowerment, achieved by the pattern of notes spewing from the speaker grille in an endless loop. You were part of the movement to help machines sing! You were a programmer! The Atari 800 people had BASIC. They know what I’m talking about. And the TI-994A guys don’t need to say a word, because the TI could say it for them!

The old machines don’t compare to the desktops of today, or to the consoles of today. But, sadly, current versions of Windows have no immediately accessible programming languages. And what’s a kid going to do with Visual Basic? Build a modal dialog? Forget coding for XBox. Requires registration in the XBox Developer Program. Otherwise, you gotta crack the sucker open. GameCube? GameBoy? Playstation 2?

Coding Just Isn’t Accessible

Yes, there are burgeoning free SDKs for many of these platforms. But they are obscure and most children have no means of actually deploying or executing the code on their own hardware! This is obvious to us all and likely doesn’t seem such a big deal. But ask yourself what might have happened had you not had access to a programming language on an Atari 800 or a Commodore. You tell me if this is a predicament.

It turns out, most of the kids in my neighborhood are exposed to coding through the TI calculator. A handful of languages are available on the TI and its processor is interesting enough to evoke some curiousity. But this hasn’t spread to its PDA big brothers, where young people could have more exposure to programming. And undoubtedly the utility of a language on the Palm, Pocket PC and others would be useful to many.

So what’s the problem here? We have no shortage of new languages, but they become increasingly distanced from the populace. Are the companies behind these platforms weary of placing the power of a programming language in the hands of users? Is there not a demand any longer? It’s got to be some kind of greed, power, money thing, right?

Perhaps this is just another reason to push Linux and BSD on consumer systems. Still, are scripting languages easily accessible to beginners on those systems? OSX has made several scripting languages available (including Ruby and Python), but most users are unaware of their presence.

I should mention that Windows is equipped with its own scripting host for developing in JScript and VBScript. But the use of the scripting host is (I believe) under-documented and limited for beginners. Try doing something useful in a script without using Server.CreateObject. Let’s not let kids touch the COM objects, please!

The Christmas List

I’m thinking a toy language for consoles and desktops alike could be monumental. I’m ot saying it needs to be cross-platform. A language for GameCube that took advantage of platform-specific features could be more appealing to GameCube users than a language that used a reduced featureset, but could execute on a handheld. Really, we live in a world where both choices should be available.

As for essential features:

1. Transportable code.

On my TI-994A, I could make a little, animated Optimus Prime from pixels. Insert cassette. Record. Pass around to friends. Receive high fives from friends. Put on wraparound shades. Thank you, TI! Thank you, Optimus Prime!

A little language for the consoles could be wildly popular if combined with the good ature of sharing code. This could be done by trading memory cards, but would be more effective if code could be easily obtained and posted on the Web. Learning would accelerate and collaborative development could take place.

A suitable language should give coders access to I/O devices, to allow experimentation with network devices and the ability to enhance one’s connectivity with others. For the consoles, games could provide hooks for user mods. This has long proven a successful staple of the desktop gaming world.

2. Simplicity.

You’ve got to be able to write a single line of code and see a result. We need some instant results to give absolute beginners confidence. Simple methods for sending an e-mail, reading a web page, playing music. Demonstrable in a one- liner.

Admittedly, as our systems have grown complex, it is difficult to balance simplicity and capability. Most users will be unimpressed by code that emits beeps and bloops from a PlayStation 2. If Ruby were available on the PS2, then I would hope that I could hear rich symphonic sounds from a wee bit of code.

Orchestra.play( "A:2", "C:4", "E:1", "G:1" )

Access to the graphic engine might require more complex code. But simple drawing methods could be provided for beginners. Or images could be stored alongside code and accessed programmatically.

ImageLibrary.load( "GolfingOldMan" ).drawAt( 12, 10 )

The trick would be to uncover what small applications might entice novices and still provide the ability to write large applications that would drive developers to master the language and not limit their growth.

3. Sensible environment.

Considering that many won’t want to purchase a keyboard for their gaming unit, let’s make sure that a reasonable environment is provided for entry of text. Controllers could be worked like the Twiddler. Or code could be transferred via IR, TCP/IP. (Dare I say cassette? :D)

4. Give it away!

It used to be that programming was practically an inalienable right for users. Include a language with the system, situated in a friendly spot. Each of the game consoles I’ve mentioned has launchers. (With the exception of Game Boy and its successors.) Provide a development prompt from the launcher. From desktop software, provide shortcuts for both the command prompt and a development prompt.

Remember, we’re looking for a language that requires no system hacks. No obscure links. No warranty violation. We’ve become so used to these techniques that it seems to be an essential part of getting our way.

And in many ways it is essential. Tinkering with hardware is learning. Lobotomizing and renovating is meaningful, magical. On behalf of those who prefer to code, I make these wishes. Not to take away jobs from the Phillips screwdriver.

The Ultimatum

My challenge is to Sony, Nintendo, Microsoft, Apple, and to those who manufacture and develop our interactive technology. Let us interact with these machines more deeply. Provide us a channel for having a dialogue with the entertainment boxes we nurture and care for. I swear to you, the relationship between the public and your product will assuredly blossom. That box will become more of a chest for our personal works.

In addition, if your developers start putting out crap, then you have a whole world of people to pick up the slack.

My challenge is for you to bundle a useful programming language with your product. Ruby, Squeak, REBOL, Python. Take your pick. It will be inexpensive to add any of these languages to your systems. And people will seriously pray to you. You know how geeks get when they pledge allegiance to something. But, yes, Ruby is preferable.

About Crows

Wed, 02 Dec 2009 00:00:00 GMT

A short time ago, the G20 came right here to Pittsburgh. With an event of such worldwide importance happening in our backyards, several people decided to document it via social media. The Mattress Factory did a site called “MyG20” and it did pretty well. Some smaller groups did their own thing. But IndyMedia did one better: http://indypgh.org/g20/

The New York Times called it “One of the best all-purpose sites with updates from the streets.” It got quite a lot of attention and hits from all over. And it was put together by a good friend of mine, @edfilo I helped a teeny tiny bit, but he (with the help of LibSyn) really did an awesome job.

So why should this matter to you? The G20 is old news by now.

He’s releasing the code for the site, making it available to anyone who wants to cover events. It’s called “Crows,” and it lives at http://crowsne.st/ . With Crows, you can quickly and easily build a site, pulling in content from Flickr, Twitter, and elsewhere, and posting it all on a geotagged Google Map.

It’s still in its rough infancy, but after the awesome role that it played during the G20, several other organizations had contacted IndyMedia about getting access to the code, and so it’s already gaining traction in the space. It’s undergoing rapid development, with new features being added quite fast. Ed really wants to make this useful for people, and I’m going to be contributing to the project in what little spare time I can as well. If you’re a coder too, you can fork crows on GitHub to contribute back to the project.

Check it out, and you can follow @crowsource on Twitter for updates, too.

Steve Klabnik — Technology

I see a future in jj

Why I chose Rust

Enter jj

The future of jj

Putting my money where my mouth is

Choosing Languages

Against Names

Nameless Branches

Utility CSS

Using the Oxide Console

You're probably learning a technology in its seventh season, not its pilot

"C is how the computer works" is a dangerous mindset for C programmers

"C is not how the computer works" can lead to inefficient code

Why is Clojure so stable?

Individuals and their greatness

Clojure is a Lisp

Clojure is dynamically typed

Clojure’s development is pretty slow

Borrow checking, escape analysis, and the generational hypothesis

Should you learn C to "learn how the computer works"?

The C abstract machine

Aside: why do people get this wrong?

What do people actually mean?

Learn C to learn more about how computers work

User-Agent: moz://a

Ouroboros

Update GitHub Pages with Travis CI

SF

Is npm worth 26MM?

How VC investment works

Open Source and $$$

Infrastructure investment

npm?

How Dogecoin changed my perspective on cryptocurrency

git, history modification, and libuv

Abstract vs concrete

New Beginnings

The CLOSURE companion

Be Skeptical

What CLOSURE is

Identity

Impermanance

The Software Industry

Some final thoughts

Just the regularly scheduled apocalypse

Emoji licensing

A history

The font

Typing extended characters

Custom fonts in browsers

Apple Color Emoji

Extensions

What’s this mean for me?

Phantom Open Emoji

Returning to Free Software, a guide

My GNU/Linux Background

Hardware: X1 Carbon

Operating System: Crunchbang GNU/Linux

Software

Conclusion

Ditching Google Chat with XMPP

Install ejabberd

Node

tl;dr

the why

_why is a role, not a person

All the world’s a stage

_why vs. Subject

On Sloppiness and blog posts written in haste

Further reading

An early Christmas present for you

So?

Narratives

How to squash commits in a GitHub pull request

Theory and practice

How do you find the time?

Min/max

Work

But I don’t work at Jumpstart!