Posted 2019-12-17
Twitter's Jack Dorsey, in (of course) a Twitter thread:
Twitter is funding a small independent team of up to five open source architects, engineers, and designers to develop an open and decentralized standard for social media. The goal is for Twitter to ultimately be a client of this standard.
This is interesting on several levels. I'll mostly talk about the protocol design part of this, but before I do that: when has a business that's pretty much in a monopolist position ever voluntarily given up that position? They must really be feeling the pushback against "content and conversation that sparks controversy and outrage", and see this as a way out.
The monopoly aspect is even more noteworthy when considering what happened with instant messaging. Back in the day, we all used IRC and then ICQ. Then there were Yahoo, AIM and MSN (and a few more, I'm sure). At some point, the IETF came up with an open standard that allows everyone to run their own server, similar to how everyone can run their own mail or web server: Jabber or XMPP. But as quickly as networks merged and interconnected, new closed ones such as Whatsapp and Apple's iMessage. Unless I'm mistaken, iMessage, as well as Facetime, actually run on XMPP but don't interconnect with other networks. So we're still left with walled gardens owned by huge corporations.
Very relevant here is Metcalfe's Law:
a network's impact is the square of the number of nodes in the network.
In a network with ten people, there's (roughly) 100 possible conversations that can happen. In a network with 100 people, that's 10,000 conversations. With instant messaging, the network effect is very strong. Then again, most people predominately IM with people in their own country. The effect is that in each country, there's one clear winner in the IM space:
For "microblogging" such as Twitter it's even worse. For IM it's just about possible to use different apps to talk to different people, but the point about a service like Twitter is that everyone can react to everyone else. So you really need one unified network for it to work well.
So the challenge for the bluesky initiative is to come up with a way where everyone can run their own service independently, but they all talk to each other, similar to how you can run your own mail server and still mail everyone else who has email. (Although in practice, most people use Gmail or Outlook webmail.)
At first glance, this looks a lot like the existing XMPP protocol, that works exactly this way. However, XMPP solves a much simpler problem: one-to-one communication. In networking we call this unicast. But when you post something to Twitter, it doesn't just go to one recipient, but to all your followers—possibly millions of them. So what we have here is multicast communication: one sender, multiple receivers.
In principle, this is a relatively simple problem to solve. We have multicast routing protocols that work relatively well, so just bolt that logic onto XMPP and call it a day. However, the big problem will be to make it scale. On the one hand, you have people with millions of followers. That's a lot of work, but it's simple: Donald Trump's messages pretty much go anywhere. But what about the hundreds of millions of users who are followed by a handful of others and only post a message once a month? Just keeping track of all these relationships so the messages can flow in real time creates enormous overhead.
And then there's the issue of spam and unwanted content. How do you control those issues across domains running on servers that belong to different organizations with different use policies?
An interesting problem to think about, and I'm looking forward to seeing what bluesky comes up with.