play with new comptime var rule of zig 0.12.0

zig 0.12.0 is released, WoW! But as it is still on road to 1.0, there are plenty of breaking changes. One of them I want to explore more here is Comptime Memory Changes. Or simply I call it new comptime var rule :) It is a change will help zig going to better next phase of incremental compiling while eliminating some tricky bugs according to the release note.

First what has changed in simple words?

I encourage you to read the official release note, but if you are hurry, simply below code will not compile in 0.12.0 (but it will compile in 0.11.0)

pub fn main() !u8 {
    comptime var x: u32 = 123; // mark 1
    var ptr: *const u32 = undefined;
    ptr = &x;
    if (ptr.* != 123) return error.TestFailed;
    return 0;
}

The reason for this will not compile because ptr is a runtime known item, while x is a comptime know variable, and make ptr point to x now is not allowed. Further reason is, if allowing this happening, because x is variable, so zig will not know when to stop comptime execution, so that zig will not know when can start generating runtime code for ptr becuase it depends on x's comptime.

The solution is let us break the tie, tell zig exactly when x is finalized, like something below (see mark 1 changes). A small note is that when write a comptime const, we have to move the comptime keyword to right hand value side and manually specify the type.

pub fn main() !u8 {
    const x = comptime @as(u32, 123); // mark 1    var ptr: *const u32 = undefined;
    ptr = &x; // mark 1
    if (ptr.* != 123) return error.TestFailed;
    return 0;
}

A more practical example and the pattern

Dealing with integer can illustrate the problem but will never the case in real life problem solving. So let us look at below example

const std = @import("std");

fn formatName(writer: anytype, name: anytype) !void {
    const fmt = comptime brk: {
        var fmt_buf: [32 * 4]u8 = undefined;
        var fmt_len: usize = 0;
        for (0..name.len) |i| {
            if (i != 0) {
                fmt_buf[fmt_len] = ' ';
                fmt_len += 1;
            }
            @memcpy(fmt_buf[fmt_len .. fmt_len + 3], "{s}");
            fmt_len += 3;
        }
        break :brk fmt_buf[0..fmt_len];
    };

    try writer.print(fmt, name);
}

pub fn main() !u8 {
    var buf = std.ArrayList(u8).init(std.heap.page_allocator);
    defer buf.deinit();
    try formatName(buf.writer(), .{ "Samus", "Aran" });
    std.debug.print("{s}\n", .{buf.items});

    return 0;
}

The code should be simple enough to understand. It formats a name tuple by constructing a format string based on comptime known name tuple. But if we compie this, we will see

/home/yli/.asdf/installs/zig/0.12.0/lib/std/io.zig:324:48: error: runtime value contains reference to comptime var
            return @errorCast(self.any().print(format, args));
                                               ^~~~~~
/home/yli/.asdf/installs/zig/0.12.0/lib/std/io.zig:324:48: note: comptime var pointers are not available at runtime
example3.zig:19:21: note: called from here
    try writer.print(fmt, name);
        ~~~~~~~~~~~~^~~~~~~~~~~
referenced by:
    main: example3.zig:45:19
    callMain: /home/yli/.asdf/installs/zig/0.12.0/lib/std/start.zig:511:32
    remaining reference traces hidden; use '-freference-trace' to see all reference traces

The last trace zig gives us is that fmt is referencing comptime var, but which var, it does not say. So we have to look back into the comptime block generating fmt, to figure out that it should be problem of fmt_buf.

The how to fix this, seems not that obvious, because fmt_buf is an array with capacity larger than its content, so though looks like stupid, copy again fmt_buf will work. So fix above code like below

const std = @import("std");

fn formatName(writer: anytype, name: anytype) !void {
     const fmt = comptime brk: {
         var fmt_buf: [32 * 4]u8 = undefined;
         var fmt_len: usize = 0;
         for (0..name.len) |i| {
             if (i != 0) {
                 fmt_buf[fmt_len] = ' ';
                 fmt_len += 1;
             }
             @memcpy(fmt_buf[fmt_len .. fmt_len + 3], "{s}");
             fmt_len += 3;
         }
         var fmt_final: [fmt_len]u8 = undefined; // mark 1
         for (0..fmt_len) |i| fmt_final[i] = fmt_buf[i];
         break :brk fmt_final;
     };

     try writer.print(&fmt, name);
}

pub fn main() !u8 {
    var buf = std.ArrayList(u8).init(std.heap.page_allocator);
    defer buf.deinit();
    try formatName(buf.writer(), .{ "Samus", "Aran" });
    std.debug.print("{s}\n", .{buf.items});

    return 0;
}

The extra copy to fmt_final code start from mark 1, will help zig to realize that it can throw fmt_buf and finalized with fmt_final as it is the only thing returned outside that block. after that everything is const known in comptime so it will be happy to continue.

With this example, we may have observed a pattern: if we need to compute something dynamically in comptime (so that we will use var), there will need a finalizing phase like what we did above: copy the dynamic var to const. Release notes also gave good explanation of this pattern, but hopefully with above example, it helps us all to understand in practical scenario.

Next let us look at an even tricky example

example when zig will not point us to the obvious spot of problem

const std = @import("std");

fn getFirstName() []const u8 {
    comptime var buf: [5]u8 = undefined;
    @memcpy(&buf, "hello");
    return &buf;
}

fn getLastName() []const u8 {
    comptime var buf: [5]u8 = undefined;
    @memcpy(&buf, "world");
    const final_name = buf;
    return &final_name;
}

const Name = struct {
    first: []const u8,
    last: []const u8,
};

fn getName() []const []const u8 {
    comptime {
        var names: [2][]const u8 = undefined;
        names[0] = getFirstName();
        names[1] = getLastName();
        const names_const = names;
        return &names_const;
    }
}

pub fn main() !u8 {
    std.debug.print("{s} {s}\n", .{ getName()[0], getName()[1] });
    return 0;
}

The final example is another simple program try to generate a Name struct in compile time and then print it. You may ask why it makes sense? It is a simplified version of program generating complex structs in comptime, like genearting database table definition info from struct which will later be used for helping generating SQL. But with that scenario there is unnecessary noises so I created this example.

Compile above example we will get:

example2.zig:27:9: error: function called at runtime cannot return value at comptime
        return &names_const;
        ^~~~~~~~~~~~~~~~~~~
referenced by:
    main: example2.zig:32:51
    callMain: /home/yli/.asdf/installs/zig/0.12.0/lib/std/start.zig:511:32
    remaining reference traces hidden; use '-freference-trace' to see all reference traces

This time, zig has pointed names_const referencing comptime variables, but what we did is following the above pattern: we create an array, copy the values then assign to a const and return. So we have to follow the trace to find out var names is the problem, and then follow to names[0] which points to getFirstName and then inside getFirstName find actually must be the buf we returned violated the comptime var rule. And then we can fix the problem like below

// only show getFirstName here
fn getFirstName() []const u8 {
    comptime var buf: [5]u8 = undefined;
    @memcpy(&buf, "hello");
    const final_name = buf;
    return &final_name;
}

zig is not pointing us to the real location of where is wrong probably because when we use the array to store the values in comptime, it has lost the stack (because of array). I do not know whether this should be fixed (seems not very necessary), but it is confusing when we see above compile errors as it has not gave us the correct location. Hopefully with this example, this kind of error can be more easily addressed as we now know how to trace.

That's all for now in playing with zig's new comptime var rule. Hopefully this will help you!