Porting my game to the web

I have been working on a game written in Zig for the past few months, and I wanted to get some playtesters. I first put out a windows build, and the response was a bit underwhelming, with some mentioning that they don't have access to a Windows PC. So I decided that I would try and port the game over to the web.

A bit about the game; it is called Konkan Coast Pirate Solutions. It is a puzzle game where you place commands over a grid, and watch as ships navigate around and interact with each other.

A few relevant points about the game:

1. The game is built in Zig using SDL and OpenGL.
2. The game is controlled mostly only by mouse.
3. A lot of data is loaded in from files. Writing to files is only for saving progress.
4. The game currently has no sound.

Before beginning the port, I had a few things in mind;

1. The changes in the code should be minimal, preferably only in the main function, and parts of the renderer. This was mostly because the web build was meant to only be for this specific version. I do not intend to maintain the web build moving forward, so I didn't want to make large sweeping changes across the codebase. I would rather handle any extra complications in the web specific parts of the code.
2. I didn't want to spend more than a week working on this. I have no idea how easy or hard it would be, and if at the end of a week, I wasn't almost done, I would let it go.
3. Since this was just for playtesting, I just needed the game to run correctly. Any additional options / polish was not necessary.

With that in mind, I started going through, and seeing what my options were. I evaluated a few options, and ended up deciding to compile the game to wasm, and then integrate that and get it working on the web.

I found an old project, WASM tetris that was a great starting off point. It had several similarities to my project. It was a little old, but it served as the perfect jumping off point for me.

wasm basics for a game engine

The most common way to set up a game engine's main loop looks something like this:

pub fn main() !void {
    var game = Game.init();
    var renderer = Renderer.init(); // this could be within the game, but I have it separate.
    while (!game.quit) {
        for (inputs) |input| game.handle_input(input);
        game.update();
        renderer.render(game);
    }
}

This exact structure doesn't really work in the browser. It has a different way of going about things. There is no real concept of a main function. So there is no "game loop" as such. Instead, we work with a lot of callbacks and handlers. So you would have to break up the main function into a few different parts.

var game: Game = undefined;
var renderer: Renderer = undefined;

export fn init() void {
    var game = Game.init();
    var renderer = Renderer.init();
}

export fn handle_input(input: Input) void {
    game.handle_input(input);
}

export fn update_and_render() void {
    game.update();
    renderer.render(game);
}

These functions would then get exposed to the wasm module, and you can call them

WebAssembly.instantiate(...) {
    instance.exports.init();
    document.addEventListener(event, e => instance.exports.handle_input(e));

    function render() {
        instance.exports.update_and_render();
        window.requestAnimationFrame(render); // this results in a loop that calls itself, like the main while(!game.quit) loop.
    }
    window.requestAnimationFrame(render);
}

This is just the scaffolding however. The devil is in the detail, and we now have to get everything to work together.

Sweeping changes

Before actually starting off, we first need to remove all the aspects of the code that implicitly assume that it is not a web build. Unfortunately, the zig compiler does not make that very easy. It generally just likes to throw up a couple of errors:

...\lib\std\os.zig:182:28: error: container 'system' has no member called 'timespec'
pub const timespec = system.timespec;

or

...\lib\std\os.zig:147:24: error: container 'system' has no member called 'fd_t'
pub const fd_t = system.fd_t;

The errors point to the standard library, but unfortunately they don't point to the source code, and we can't see the full call stack.

I ended up just adding a lot of if (false) blocks all over the place, and that helped me narrow it down to the source of the problem. timespec was mostly due to the std.time.milliTimestamp() calls, and fd_t was std.debug.print() or calls to std.fs.cwd() and other file related calls.

In both these cases, I had to go and replace all the calls made to these functions, with a handler function that had a web flag on it. I will go into further detail about file reading and writing later in this article.

Now that we had solved the surface level problems, we had to build a wasm file.

Building a wasm file

Actually building a wasm file turned out to take a lot of effort. The bleeding edge nature of zig meant that a lot of the examples in blogs etc. online is out of date. Here is the command that I used that worked for me:

zig build-lib -target wasm32-freestanding src\main.zig --name pirates -isystem src -dynamic

or

const mode = b.standardReleaseOptions();
const target = std.zig.CrossTarget.parse(.{ .arch_os_abi = "wasm32-freestanding" }) catch unreachable;
const exe = b.addSharedLibrary("pirates", "src/main.zig", .unversioned);
exe.setTarget(target);
exe.setBuildMode(mode);
exe.addSystemIncludeDir("src");
exe.install();

Finding this was mostly a lot of experimentation, flailing around with the compiler until something stuck. It was a relief to get it done as it meant that we could now get into the meat of the implementation.

Convincing WebGL to behave like OpenGL

webgl2 #version 300 es is similar enough to opengl #version 330 core. Or atleast, my usage of opengl is simple enough that it is possible to make the two behave identically.

There are two main differences;

1. OpenGL does some amount of state management while WebGL does not.
2. OpenGL is a C api and WebGL is a javascript API.

The first point is easy enough to deal with. It just meant that we had to do the state management within the javascript wrapper calls. OpenGL expects you to deal with objects like shaders, buffers, programs etc. as handles that are unsigned ints. So whenever you want to use a particular buffer, you would refer to it with the handle that OpenGL provided. It's fairly trivial to "reimplement" this functionality.

var glBuffers = [];
const glCreateBuffer = () => {
  glBuffers.push(webgl.createBuffer());
  // we use the index as a unique handle.
  // this must be kept in mind while deleting handles
  // (or in my case, I just didn't bother deleting any handle...)
  return glBuffers.length - 1;
}

The second point is really simple again, but it represents a whole lot of work that needs to be done. The main difference is that javascript allows a certain amount of function overloading due to its dynamically typed nature.

This mostly means sitting with two tabs open, the OpenGL reference on one side, and the WebGL reference on the other. Then check each of the openGL calls our program makes, and check which version of the webgl call is appropriate.

Also, we would have to create the extern call signatures to all the API that we used. So if you're uncomfortable with the type interaction between zig and C, this is a great opportunity to face that discomfort.

It was fairly slow and tedious work, but the subset we were using was fairly small, so it wasn't all that bad.

Similarly, the shaders are almost identical between the two, with the only changes being in the #version and webgl mandating the specification of float precision.

wasm, c and zig strings

One of the things that took a lot of time to figure out was how to pass strings around. C strings are null terminated. Zig behaves well enough with this in simple cases, but for more advanced cases, we need to take things into our own hands.

For example, if we want to format a string using std.fmt.allocPrint, the result is not a null terminated string, but a const u8 slice. So when this is passed to C (or WASM), issues can arise.

A lot of implementations I looked through decided to forgo this issue, and would always pass the length of the string along with the pointer. But I didn't want to do that. OpenGL makes use of strings mostly for getting the location of uniforms. But we also needed to pass in strings for other purposes, from debugging, to generating file paths.

I dug in a little further, and ended up figuring out a way to deal with this issue. Firstly, in the wasm module, we had to take the string and find the null terminator;

const wasmString = (ptr) => {
  const bytes = new Uint8Array(memory.buffer, ptr, memory.buffer.byteLength - ptr);
  let str = '';
  for (let i = 0; ; i++) {
    const c = String.fromCharCode(bytes[i]);
    if (c == '\0') break;
    str += c;
  }
  return str;
}

On the zig side, I ended up manually adding the null terminator wherever needed before passing it to the JS.

// create null terminated path string
var path_str = try allocator.alloc(u8, path.len + 1);
defer allocator.free(path_str);
std.mem.copy(u8, path_str[0..path.len], path);
path_str[path.len] = 0;

Now we can easily pass strings to wasm from zig.

Passing strings from wasm to zig is pretty much the reverse of this. However, we don't know how long the string is, and thus we don't know how much memory we have to alloc in zig. To get around this, I ended up creating an additional function that just gives the length of the string, which we use to allocate memory, and then call the function again with a pointer to enough memory.

const len = c.get_string_len(id);
var data = try allocator.alloc(u8, len);
_ = c.get_string(id, data.ptr);
return data;

There is a more fleshed out example of this in the Writable Files section.

It might be possible to send an allocator to JS, and then do the allocation on that side, but I thought this way was simpler, and went ahead with it.

User inputs

The next major step was to get the users inputs, and get them to impersonate SDL events.

I spent some time trying to do this, with using translate-c to get the requisite structs and then figure out how make the data match, but it ended up being far simpler to just rewrite a small struct that handles all the events that we care about.

So, our inputs basically now have two branches:

pub fn handle_input(self: *Self, event: c.SDL_Event) void {}
pub fn handle_web_input(self: *Self, event: WebEvent) void {}

There is a bit of code duplication here, but its on the order of 40-50 lines of code, not too much to worry about.

For this game, we only care about the mousemove, mousedown, mouseup, keydown and keyup events, and those are plugged in using document.addEventListener.

File I/O

The game loads a lot of data from files. Each levels data is stored in a separate file. Also the game has its own version of what is essentially vector graphics, and each "sprite" is saved in a separate file.

We also write to a savefile whenever we need to save data etc.

So the first thing that we had to do was differentiate between read-only files and read-write files. This was a part of the sweeping changes we had done earlier. These handler functions would change their behaviour depending on the WEB_BUILD flag.

Initially the read-only files were served as data from the server, and we would call to js to load each file separately. By default, XMLHttpRequest uses callbacks to load files. I forced the files to load synchronously because I wasn't able to get it working otherwise.

const getFileText = (path) => {
    let request = new XMLHttpRequest();
    // TODO (12 May 2022 sam): This is being deprecated... How can we do sync otherwise?
    request.open('GET', path, false); // false flag forces synchronous requests.
    request.send(null);
    if (request.status !== 200) return false;
    return request.responseText;
}

While this ran fine on my local system, once it was uploaded online, the first load of the data took upwards of 10 seconds, and there would just be a black screen for that time. While I could have put up some kind of loading screen, that felt like it would take too many changes. The native version loads up in ~50 millis, and I didn't want to make such a big change to the engine. One of the reasons for this could be the way that I was passing data from wasm to zig. In this case, we would have to send the XMLHttpRequest twice. Once to get the length of the string, and second to load the string itself. This might have been another reason for the slow reads.

To speed it up, I instead ended up packaging all the data into a single json file, and used @embedFile to make it a part of the compilation process itself. This json would then be parsed once, and would return the data. This brought down the load time to less than 1 second, and I was happy enough with that.

const STATIC_DUMP = if (constants.WEB_BUILD) @embedFile(constants.STATIC_DATA_PATH) else void;
var static_data: ?std.json.ValueTree = undefined;

pub fn read_file_contents(path: []const u8, allocator: std.mem.Allocator) ![]const u8 {
    if (!constants.WEB_BUILD) {
        const file = try std.fs.cwd().openFile(path, .{});
        defer file.close();
        const file_size = try file.getEndPos();
        const data = try file.readToEndAlloc(allocator, file_size);
        return data;
    } else {
        if (static_data == null) {
            var parser = std.json.Parser.init(allocator, false);
            static_data = parser.parse(STATIC_DUMP) catch unreachable;
        }
        if (static_data.?.root.Object.get(path)) |val| {
            return val.String;
        } else {
            return error.FileNotFound;
        }
    }
}

Writable Files

If we want to save progress, we also need to be able to write to files. I ended up opting with using HTML5 storage for this. Writing this data is fairly straightforward overall.

const writeStorageFile = (path, text) => {
    path = wasmString(path);
    text = wasmString(text);
    try {
      localStorage.setItem(path, text);
    } catch {
      return false;
    }
    return true;
}

To read the file, we have to pass string data from wasm to C.

// get file size, alloc data, and then read into it.
const raw_size = c.getStorageFileSize(path_str.ptr);
if (raw_size < 0) return error.FileNotFound;
const size = @intCast(usize, raw_size);
var data = try allocator.alloc(u8, size);
const success = c.readStorageFile(path.ptr, data.ptr, size);
if (!success) return error.FileReadFailed;
return data;

const getStorageText = (path) => {
    try {
      const text = localStorage.getItem(path);
      if (text === null) return false;
      return text;
    } catch {
      return false;
    }
}

const getStorageFileSize = (path) => {
  path = wasmString(path);
  const text = getStorageText(path);
  if (text === false) return -1;
  return text.length;
}

const readStorageFile = (path, ptr, len) => {
    path = wasmString(path);
    // read text from URL location
    const text = getStorageText(path);
    if (text === false) return false;
    if (text.length != len) {
      console.log("file length does not match requested length", path, len);
      return false;
    }
    const fileContents = new Uint8Array(memory.buffer, ptr, len);
    for (let i=0; i<len; i++) {
      fileContents[i] = text.charCodeAt(i);
    }
    return true;    
}

Porting C libraries

The game uses two C libraries. stb_perlin to create perlin noise, and stb_truetype, to load fonts.

I used c-translate to easily convert the perlin noise library. It had a few math.h dependencies, and I manually changed all of those to use the zig standard library methods instead. That was now pure zig code, and was easily integrated into the project.

stb_truetype was a lot harder. It uses a lot more of the libc, and after spending a lot of time trying to manually replace all the calls, I decided not to bother porting it over.

Instead, I created a binary dump of the data that the library generates, and @embedFiled that into the project again. For this specific usecase, that works out fine. If the C library is more integral, then this solution may not work, and more work would have to be put in to get it working.

Final Tweaks

I had to make some final tweaks to the code before I could publish the web version. The native version has options to change window size, and toggle full screen. Since these are not supported, I had to remove those options from the Options menu.

Theoretically, I had to remove the Save and Exit option from the main menu as well, but it threw off the balance of the screen, and I opted to keep it in place.

Completion

And with that, the port was done. It took about a week of work, so I hit my deadline.

Assorted

When working on reading wasm memory, I faced a lot of bugs. I imagine a lot of it was reading unsafe memory, reading beyond bounds etc. In this respect, Firefox behaved really well, while Chrome kept crashing on me.

However, in the final release build, the performance on Chrome is much better, and the game runs much smoother compared to Firefox.

I just thought that was an interesting point to note.

Further Work

While I don't currently intend to put in any more work on this, there are still some things that I have not yet worked on.

1. Sound.
   The game currently doesn't have sound, so I don't know how easy or hard that would be.

2. Full Screen
   The native version has this, but I didn't really investigate into how it would work on web.

Possible improvements to zig compiler

The main complaint I have with the compiler was the lack of call-stack in the errors, as mentioned above. While it was easy for me to work around this, I wasn't able to figure out why the call-stack wasn't visible.

Another small gripe I had was the lack of #ifdefs. Since there are no ifdef macros, defining constants looks a little ugly and hard to read.

const VERTEX_BASE_FILE: [:0]const u8 = if (constants.WEB_BUILD) @embedFile("shaders/web_vertex.glsl") else @embedFile("shaders/vertex.glsl");
const FRAGMENT_ALPHA_FILE: [:0]const u8 = if (constants.WEB_BUILD) @embedFile("shaders/web_fragment_texalpha.glsl") else @embedFile("shaders/fragment_texalpha.glsl");

as opposed to a C style

#ifdef (constants.WEB_BUILD) {
const VERTEX_BASE_FILE: [:0]const u8 = @embedFile("shaders/web_vertex.glsl") 
const FRAGMENT_ALPHA_FILE: [:0]const u8 =  @embedFile("shaders/web_fragment_texalpha.glsl") 
}
# else {
const VERTEX_BASE_FILE: [:0]const u8 = @embedFile("shaders/vertex.glsl");
const FRAGMENT_ALPHA_FILE: [:0]const u8 = @embedFile("shaders/fragment_texalpha.glsl");
}
#endif

With so many constants that need to be changed, it feels a little clunky. The only way I know around this is to use multiple files to store the data and then something like

pub usingnamespace if (constants.WEB_BUILD) @import("web.zig") else @import("native.zig");

which would result in a lot of extra files. But that's really more of a nitpick, and I don't have a good solution to the issue.

Conclusion

Overall, porting this game over was quite a fun experience. Zig proved to be a great tool for the job. Considering the stage of growth that the project is at, I was really happy with how things got done.

The game is currently closed-source, but a lot of the work that I did was in my zig_sdl_base repo, in which you can find further implementation details about all the things that I mentioned here.

If I made any wrong assumptions in this post, or missed any obvious simpler solutions please let me know. I have the habit of mostly just getting things done the way I know how, and not bothering about other ways to solve the problem. I would love to expand my knowledge of zig, building for wasm, and wasm in general.