Multi-user & real-time systems

Presence and synchronisation. Videoconferencing. All of it, at scale.

Technologies: WebSockets, WebRTC, XMPP, Twilio, Vonage

A single-user 3D scene is a rendering problem. Adding a second user makes it a synchronisation problem. Supporting thousands makes it an infrastructure problem.

Spirit Fencer Arena (2013)

Spirit Fencer Arena was the first project where I used real-time communication in production. Back then, Unity developers didn't have access to the networking libraries available today, so I created my own. I used a server-centric coordinator (written in C#, using WebSockets) which would handle connections, matchmake the players and act as authority on events. I also created daily tournaments where players could join and battle each other to gain points.

I wanted bots to populate the online arenas early after launch so the first players could experience multiplayer combat and tournaments. For that I deployed a headless version of the game where the player would be controlled by one of the enemy AIs. I had replicas running 24/7.

This project taught me a great deal about (amongst many other things) modularity. Spending the time doing good architecture really pays off when adding new features.

Hyperspace (2014)

Our custom signalling system now supports up to 16,000 concurrent participants. That has been the result of years of incremental improvements and redesigns. The first multiplayer-ready version could barely support a hundred, and it topped out at around 250 before the big redesign.

My first approach to multi-user was to build on the chat system we had just implemented, ejabberd, running over XMPP. I encoded position updates and events like waving hello into the chat messages themselves.

As the platform became richer in features and events got bigger, we had to find another solution for presence. We added higher-frequency signals once participants could join VR meetings and see each other in VR. We had to transmit lipsync, head and hand movements. I also added collaborative scene authoring and whiteboards. So we needed a new signalling system. We tested a few libraries but it soon became clear our needs were too specific for anything off the shelf.

We designed a new signalling system running in Node.js and using Socket.io. It allowed us to scale to around a thousand at first. Many months of work (on both the client and the server) pushed that number to 16,000.

The platform also has audio and video calls. For audio calls I integrated Twilio, which allowed participants to join via the 3D experience or dial in by phone.

For videoconferencing, I integrated Twilio Video and Vonage, both WebRTC under the hood. Participants can pick Mediaboards to share their cameras or their screen. This has been one of the most effective use cases for corporate clients.

In large events with hundreds or thousands of participants, when communication is one-way (a few hosts presenting to many), the platform uses adaptive video broadcasting through services like Wowza, so thousands can receive video without trouble.

Where I've used it

View as Markdown