The Undiscovered Country

Posted by

Contrary to what you might expect, this post isn’t about the Star Trek movie. Rather uncomfortably, the title is a metaphor for my long running project, Glyffe.

Glyffe is a world of texts long and short, static and moving, interactive and not. It’s a world of worlds, all created by players. Or rather, that’s what it was supposed to be.

In practice, I’m the only one who’s ever created any worlds beyond just testing things out. Two of those worlds I entered into relevant competitions. The first one, The Answer, didn’t make the shortlist for the Digital Fiction Writing Competition. The second one, Linear Love, ended dead last in the Interactive Fiction Competition (not linking to the results page here because I don’t want it to get more standing with the Google Index than it already has).

So it’s probably safe to say that Glyffe, both as a world and as a generator of worlds, is dead. I just don’t seem to have gotten around to pulling the plug yet.

I’m not sure exactly why not. Maybe it feels too unceremonious to just pull the plug. Like a burial without a service. Or even without a few words said.

So I’ll say a few words. Glyffe was exhilarating to work on. I remember the night when the server was all but coded and I found out that Node.js was single-threaded. Which meant that every connection would have to be handled by the same processor core, making for a pretty unscalable setup.

In hindsight, that setup would’ve been fine since there have never been all that many simultaneous connections anyway. But that night, dreaming still of massively multiplayer, I was groping around for solutions.

At the time, Node.js came with a module that allowed for duplication of the server on other threads and other cores. But those duplicates were all pretty much self-contained and unaware of each other. Which is fine for a web server that doesn’t need to know which page is being served to one client in order to serve a page to another client.

But it’s not ideal for a server that needs to keep track of where those clients are relative to each other and to other stuff in the game world. That’s when I stumbled upon a third-party module called node-webworker-threads.

While that module doesn’t offer to duplicate the entire server, it does provide for offloading parts of it to other threads and other cores. Which parts? Well, if the goal is to take some load off the main server thread, a good starting point would be the CPU intensive parts.

Now in the textual world of Glyffe, you don’t interact with the world through a parser (like in early text adventures such as Zork) or by pointing-and-clicking (like in later graphic adventures such as The Secret of Monkey Island). Rather, you interact by steering (WASD or arrow keys) your textual avatar into other textual objects.

From that it follows that the server should support collision detection. Every time an avatar moves, the server should check if they’ve collided with another object and if they have, determine the outcome of the interaction. Every check, in turn, should entail comparing the avatar’s new position and size to the positions and sizes of all the other objects in order to see if there’s any overlap. I’d say we’ve identified a CPU intensive part, wouldn’t you?

Okay, so we use node-webworker-threads to offload that part to another thread, and when we need to check for a collision, we just send all the positions to the other thread, let it do the heavy lifting, and get the result back. But wait, how can we send the positions? On my main thread, everything is JSON, the hip JavaScript format that’s human-readable, while node-webworker-threads only accepts String and binary to be sent along. Heck, even my database (the now defunct Facebook Parse) stores everything as JSON for persistence across sessions. So, should we convert everything on the fly from JSON to String and back?

Turns out on-the-fly conversion between data formats would defeat the purpose of using node-webworker-threads, since the conversions are about as CPU intensive as the collision detections. The only way forward, glimpsed through many a sleepless night, would seem to be to switch everything to String from the get-go.

That means replacing all the methods that operate on JSON with methods that operate on String and swapping out the JSON database for a native String data store such as Redis. An added benefit of Redis is that it comes with Lua as a scripting language, which enables me to move some of the operations to the database layer, thereby further alleviating server load.

With this setup, I could start out with just one CPU core for the server (about $20 per month with Google) and just add cores as more clients come online, requiring more threads to handle the collision detections. Except that, as I mentioned earlier, they never did come online in those kinds of numbers.

But putting the numbers aside for a moment, I hope I’ve managed to convey to you, however imperfectly, why Glyffe was so exhilarating to work on for me.

There, I’ve said a few words. I wonder if I’ll be able to pull the plug now.

Leave a Reply

Your email address will not be published. Required fields are marked *