Want to create a TUI application? - The Basics of "Uncooked" Terminal IO

(this has only been tested on Linux, but should work on other operating systems with a POSIX-esque terminal in theory as well)

Do you want to create one of those fancy pseudo-graphical terminal based user interfaces, but don't know where to start? I recently found myself in that position and here is what I had to learn to achieve my goal.

Of course you could probably just use ncurses, the common and well known library meant exactly for this. But its pretty massive, requires linking against libc, includes some questionable design decisions such as intentional memory leaks and its multi-threading support is pretty sad.

There are newer and objectively better libraries. But I find if you just want to create a simple interface driven by simple logic, using one of those needlessly makes your application more complex than it has to be.

So in this post I will demonstrate the basics of raw - "uncooked" - terminal IO, so if you want to implement your UI manually, you'll have a starting point.

"Uncooked"?

Terminals on POSIX compliant systems do a lot of preprocessing for both things they send to programs as well as things they receive from programs. This is called "canonical" or "cooked" mode. To have more control over the output and receive usable input, a TUI program tells the terminal to disable all this preprocessing and enter raw mode.

The program does this using two methods: Some settings are adjusted with the termios interface, others by dumping various magic spells escape sequences to the terminal.

However before we can do anything like that, we need to open the terminal interface.

Don't worry about copying any code to your project yet, this post includes the complete code for two test/example applications.

var tty: fs.File = try fs.cwd().openFile("/dev/tty", .{ .read = true, .write = true });
defer tty.close();

Now we can "uncook" the terminal. Let's start with the termios thing.

const original_termios = try os.tcgetattr(tty.handle);
var raw = original_termios;

A well-behaved program returns the terminal to its initial state when exiting (yes, that does indeed not happen automatically), so keep the original termios around. It may contains settings that we do not want to overwrite, so it will also serve a starting point for building our own.

The termios struct contains multiple entries you need modify. Most of this is legacy cruft and if you do not want to lose faith in UNIX, it's best to just accept the following code without doing any further thinking. For the interested I have however added an explanation of all flags in the comments.

//   ECHO: Stop the terminal from displaying pressed keys.
// ICANON: Disable canonical ("cooked") input mode. Allows us to read inputs
//         byte-wise instead of line-wise.
//   ISIG: Disable signals for Ctrl-C (SIGINT) and Ctrl-Z (SIGTSTP), so we
//         can handle them as "normal" escape sequences.
// IEXTEN: Disable input preprocessing. This allows us to handle Ctrl-V,
//         which would otherwise be intercepted by some terminals.
raw.lflag &= ~@as(
    os.system.tcflag_t,
    os.system.ECHO | os.system.ICANON | os.system.ISIG | os.system.IEXTEN,
);

//   IXON: Disable software control flow. This allows us to handle Ctrl-S
//         and Ctrl-Q.
//  ICRNL: Disable converting carriage returns to newlines. Allows us to
//         handle Ctrl-J and Ctrl-M.
// BRKINT: Disable converting sending SIGINT on break conditions. Likely has
//         no effect on anything remotely modern.
//  INPCK: Disable parity checking. Likely has no effect on anything
//         remotely modern.
// ISTRIP: Disable stripping the 8th bit of characters. Likely has no effect
//         on anything remotely modern.
raw.iflag &= ~@as(
    os.system.tcflag_t,
    os.system.IXON | os.system.ICRNL | os.system.BRKINT | os.system.INPCK | os.system.ISTRIP,
);

// Disable output processing. Common output processing includes prefixing
// newline with a carriage return.
raw.oflag &= ~@as(os.system.tcflag_t, os.system.OPOST);

// Set the character size to 8 bits per byte. Likely has no efffect on
// anything remotely modern.
raw.cflag |= os.system.CS8;

Two fields of the termios structs cc member deserve special attention, vtime and vmin. These are used to control the read syscall when getting input from the terminal interface. The first sets the timeout, after which the syscall will return, the second the minimum amount of bytes it reads before returning. Note that timeout takes precedence over the minimum byte count.

raw.cc[os.system.V.TIME] = 0;
raw.cc[os.system.V.MIN] = 1;

How you need to configure this depends on your application. If you want to drive your application using poll() (or one of its many friends), you should probably set both to 0 (try to read, but return immediately, even if no bytes where read). If you want things to be driven by read, either in simple applications like this example code or for event generator threads, disable the timeout and let it wait until at least one byte can be read.

Don't forget to commit your changes.

try os.tcsetattr(tty.handle, .FLUSH, raw);

Wait, .FLUSH? Yeah, fun fact: tcsetattr() can operate in different modes. The two you need to know about are .FLUSH, which will cause the output to be flushed and any unread input to be discarded before applying the new termios struct, and .NOW, which will cause it to be applied immediately. Remember this for later.

We are almost done uncooking the terminal, we just need to send it some escape sequences. There are a lot of possible escape sequences to configure a ton of different terminal behaviour, but here are just the ones you actually need.

Basically you want to hide the cursor (careful: people using screenreaders or braille displays need the cursor, so this should be configurable in your application) and save it's position. Also save all screen contents and then enter the alternative buffer.

Thanks to the alternative buffer, you can draw all kinds of crazy and fancy things but always return to the content before. It is the reason you can read man ls in your terminal and when you quit the pager, it disappears without leaving a trace and the previous text in your terminal returns.

try writer.writeAll("\x1B[?25l"); // Hide the cursor.
try writer.writeAll("\x1B[s"); // Save cursor position.
try writer.writeAll("\x1B[?47h"); // Save screen.
try writer.writeAll("\x1B[?1049h"); // Enable alternative buffer.

If you are buffering your writes, don't forget to flush!

\x1B is the escape character. Together with [ it forms the CSI ("Control Sequence Introducer") sequence, which prefixes a lot - but not all - escape sequences.

Not all terminals support all escape sequences. You could use termcap or termio to look up your terminals capabilities based on the $TERM variable. But if you stick to ANSI codes, you can safely assume that any sane and modern terminal will support the important ones.

Un-"uncooking" the Terminal

As I already mentioned, at exit your application must clean up its mess.

Unlike the OS cleaning up un-freed memory at exit, the terminal will not reset to the state prior to your application launching. The user can restore the terminal with the reset command, but cleaning up yourself is probably a better idea.

try os.tcsetattr(tty.handle, .FLUSH, original_termios);
try writer.writeAll("\x1B[?1049l"); // Disable alternative buffer.
try writer.writeAll("\x1B[?47l"); // Restore screen.
try writer.writeAll("\x1B[u"); // Restore cursor position.

Input

Now that you know how to uncook and cook your terminal, we can get to the interesting bits.

Let's start with getting and interpretting input.

var buffer: [1]u8 = undefined;
_ = try tty.read(&buffer);

In this example, our code is driven by read. We want to respond to each key press immediately, so we only read a single byte. As such, we are not interested in read returning the amount of bytes read (careful: if you drive your application differently, you might need to care about this).

Using everything you've learned so far, you can write a simple example program that just prints what input it receives. Note that for the sake of brevity I only configured the input related settings. You can exit this program by pressing q.

const std = @import("std");
const debug = std.debug;
const fs = std.fs;
const io = std.io;
const mem = std.mem;
const os = std.os;

pub fn main() !void {
    var tty = try fs.cwd().openFile("/dev/tty", .{ .read = true, .write = true });
    defer tty.close();

    const original = try os.tcgetattr(tty.handle);
    var raw = original;
    raw.lflag &= ~@as(
        os.linux.tcflag_t,
        os.linux.ECHO | os.linux.ICANON | os.linux.ISIG | os.linux.IEXTEN,
    );
    raw.iflag &= ~@as(
        os.linux.tcflag_t,
        os.linux.IXON | os.linux.ICRNL | os.linux.BRKINT | os.linux.INPCK | os.linux.ISTRIP,
    );
    raw.cc[os.system.V.TIME] = 0;
    raw.cc[os.system.V.MIN] = 1;
    try os.tcsetattr(tty.handle, .FLUSH, raw);

    while (true) {
        var buffer: [1]u8 = undefined;
        _ = try tty.read(&buffer);
        if (buffer[0] == 'q') {
            try os.tcsetattr(tty.handle, .FLUSH, original);
            return;
        } else if (buffer[0] == '\x1B') {
            debug.print("input: escape\r\n", .{});
        } else if (buffer[0] == '\n' or buffer[0] == '\r') {
            debug.print("input: return\r\n", .{});
        } else {
            debug.print("input: {} {s}\r\n", .{ buffer[0], buffer });
        }
    }
}

Play around with this a bit. Press some alpha numerical keys. Try pressing escape. Try pressing the arrow keys or insert or delete. Try pressing a letter while holding either Alt or Control or both. Now the horror should dawn on you.

Yep, that's right. Pressing the escape key will cause your program to receive a single \x1B. Pressing one of the special keys will cause your program to receive \x1B followed by an arbitrary sequence.

Our little input loop is currently incapable of handling this situation. First we need to expand it in a way that if \x1B is read, it tries to read again to get the rest of the sequence. But how to recognize the escape key, since it sends no such sequence? Well, remember when I told you to remember something about termios? This is where we need it in all its grotesque glory. And this time we'll obviously need to use .NOW.

After reading \x1B and before trying to read the rest of the sequence, let's re-configure the termios, so that read will timeout and return even if no sequence has been read.

// ...
while (true) {
    var buffer: [1]u8 = undefined;
    _ = try tty.read(&buffer);

    if (buffer[0] == 'q') {
        try os.tcsetattr(tty.handle, .FLUSH, original);
        return;
    } else if (buffer[0] == '\x1B') {
        raw.cc[os.system.V.TIME] = 1;
        raw.cc[os.system.V.MIN] = 0;
        try os.tcsetattr(tty.handle, .NOW, raw);

        var esc_buffer: [8]u8 = undefined;
        const esc_read = try tty.read(&esc_buffer);

        raw.cc[os.system.V.TIME] = 0;
        raw.cc[os.system.V.MIN] = 1;
        try os.tcsetattr(tty.handle, .NOW, raw);

        if (esc_read == 0) {
            debug.print("input: escape\r\n", .{});
        } else if (mem.eql(u8, esc_buffer[0..esc_read], "[A")) {
            debug.print("input: arrow up\r\n", .{});
        } else if (mem.eql(u8, esc_buffer[0..esc_read], "[B")) {
            debug.print("input: arrow down\r\n", .{});
        } else if (mem.eql(u8, esc_buffer[0..esc_read], "a")) {
            debug.print("input: Alt-a\r\n", .{});
        } else {
            debug.print("input: unknown escape sequence\r\n", .{});
        }
    } else if (buffer[0] == '\n' or buffer[0] == '\r') {
        debug.print("input: return\r\n", .{});
    } else {
        debug.print("input: {} {s}\r\n", .{ buffer[0], buffer });
    }
}
/// ...

(Implementing all other escape codes is left as an exercise to the reader. You can copy my match map if you like.)

If you think that this is super hacky, than you are right, it is. And to further increase the hackyness, the minimum timeout you can configure is 100ms, which is still long enough that a fast typist will easiely run into the problem of "swallowed" key presses when presing any key after escape. Even if no keypresses are swallowed, it still takes some time before pressing the escape key will register.

A much better way would be to only read once, into a large buffer, and then parse that. However, that would require a dedicated parser that can handle a buffer containing multiple separate input events and - trust me - you don't want to write that (luckily I did that for you already, stay tuned!). Also this would still have the swallowed key-press problem.

There are basically two things you can do against swallowed key pressed, both of which are pretty simple.

If you can not recognize an escape sequence and based on some simple heuristic (like for example it not containing [) you suspect it being not an escape sequence but rather swallowed individual key presses, then just try to handle all of the bytes of the sequence individually. Do not forget that you have an implicit \x1B in your sequence!

(For matching escape sequences, I recommend using std.ComptimeStringMap.)

The other method is using the Kitty Keyboard Protocol, which improves a lot on how you receive input. For example pressing the escape key will result in an escape sequence being written, so there will be no issues with the timeout. It is not widely supported, so I will not cover it here, but it is not hard to implement if you already understand everything covered so far. It can integrate into the existing input loop I have shown, you just need to write one additional escape sequence when uncooking and add a few additional escape sequences to your match map.

Now that escape sequences are covered, all that's left regarding input is the Control modifier. Ctrl+[a-z] does not result in an escape sequence, instead it sends an ASCII control character. They all have various different historical meanings, which unfortunately results in us being unable to use Ctrl-m and Ctrl-j, since they generate \n and \r. The other keys are easiely matched however.

// ...
outer: while (true) {
    // ...
    } else {
        const chars = [_]u8{ 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'j', 'k', 'l', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w' };
        for (chars) |char| {
            if (buffer[0] == char & '\x1F') {
                debug.print("input: Ctrl-{c}\r\n", .{char});
                continue :outer;
            }
        }
        debug.print("input: {} {s}\r\n", .{ buffer[0], buffer });
    }
}
// ...

This pretty much covers how to do input.

If you are interested in getting more (meta-)data, like for example key up and key down events, want to support more modifiers than just Alt and Control, or just want to reduce the hackyness of your input code, I again strongly recommend checking out the kitty keyboard protocol and switching to a terminal which implements it.

Output

As with input, output involves a lot of spells escape sequences. Just this time you are the caster sender. While I do recommend looking them all up, only few are actually needed to create your UI.

The single most important escape sequence you should know is the one for moving the cursor around. Because unlike programming graphical user interfaces, you can not just put any character at any arbitrary position. If you want to draw text, you need to move the cursor first. And careful: The cursor automatically moves one character to the right for every character you write.

The coordinate system of the cursor position has its origin in the upper left corner, just like with "normal" graphics programming. However unlike "normal" graphics programming, it is 1-based. I recommend working with 0-based cursor positions and doing a conversion right before writing the escape sequence.

fn moveCursor(writer: anytype, row: usize, col: usize) !void {
    _ = try writer.print("\x1B[{};{}H", .{ row + 1, col + 1 });
}

Now you are able to write text anywhere onto the terminal. That is only practical if you know the size of the terminal.

const Size = struct{ width: usize, height: usize };

pub fn resize(self: *Self) !Size {
    var size = mem.zeroes(os.system.winsize);
    const err = os.system.ioctl(self.tty.handle, os.system.T.IOCGWINSZ, @ptrToInt(&size));
    if (os.errno(err) != .SUCCESS) {
        return os.unexpectedErrno(@intToEnum(os.system.E, err));
    }
    return Size{
         .height = size.ws_row,
         .width = size.ws_col,
    };
}

Now only one piece is missing before you can start writing your application: Recognizing when the terminal has been resized. Someone once unironically thought that a signal is the best way to communicate this: SIGWINCH. (This brings us up to 5 communication channels your program needs to implement: Writing escape sequences, reading escape sequences, termios, ioctl, signals. No sane person can look at this and still honestly believe backwards compatability and long-term API stability are actually good and smart things. Send help)

But wait, there is more! While you now have everything to write a functional application, it will look plain and boring. You need colours! Unsurprisingly, colouring your text is also done using escape sequences: You write some magic string to the terminal, and henceforth all text you write will be stylised accordingly. Make sure to always clear the style before drawing your interface, there might be a "leftover" style that will mess up your text.

This leads us, (actually just you, because I won't cover it here) to another rabbit hole. First you need to decide between using your terminals "native" 16 colours, use 256-colours mode or use RGB colours. I recommend sticking with native colours, as it's the most simple and is pretty much guaranteed to integrate well with your terminals colour scheme. A pretty good list of these sequences can be found on Wikipedia (or you can just copy my text attribute code). One quirk of using colours is that some foreground-background-combinations do not work as expected. In those cases, I recommend trying to swap the colours and using the inverse style.

Ok, now let's use everything covered so far to create a simple TUI menu, that hopefully will server as a good starting point for your terminal applications.

const std = @import("std");
const debug = std.debug;
const fs = std.fs;
const io = std.io;
const mem = std.mem;
const os = std.os;
const math = std.math;

var i: usize = 0;
var size: Size = undefined;
var cooked_termios: os.termios = undefined;
var raw: os.termios = undefined;
var tty: fs.File = undefined;

pub fn main() !void {
    tty = try fs.cwd().openFile("/dev/tty", .{ .read = true, .write = true });
    defer tty.close();

    try uncook();
    defer cook() catch {};

    size = try getSize();

    os.sigaction(os.SIG.WINCH, &os.Sigaction{
        .handler = .{ .handler = handleSigWinch },
        .mask = os.empty_sigset,
        .flags = 0,
    }, null);

    while (true) {
        try render();

        var buffer: [1]u8 = undefined;
        _ = try tty.read(&buffer);

        if (buffer[0] == 'q') {
            return;
        } else if (buffer[0] == '\x1B') {
            raw.cc[os.system.V.TIME] = 1;
            raw.cc[os.system.V.MIN] = 0;
            try os.tcsetattr(tty.handle, .NOW, raw);

            var esc_buffer: [8]u8 = undefined;
            const esc_read = try tty.read(&esc_buffer);

            raw.cc[os.system.V.TIME] = 0;
            raw.cc[os.system.V.MIN] = 1;
            try os.tcsetattr(tty.handle, .NOW, raw);

            if (mem.eql(u8, esc_buffer[0..esc_read], "[A")) {
                i -|= 1;
            } else if (mem.eql(u8, esc_buffer[0..esc_read], "[B")) {
                i = math.min(i + 1, 3);
            }
        }
    }
}

fn handleSigWinch(_: c_int) callconv(.C) void {
    size = getSize() catch return;
    render() catch return;
}

fn render() !void {
    const writer = tty.writer();
    try writeLine(writer, "foo", 0, size.width, i == 0);
    try writeLine(writer, "bar", 1, size.width, i == 1);
    try writeLine(writer, "baz", 2, size.width, i == 2);
    try writeLine(writer, "xyzzy", 3, size.width, i == 3);
}

fn writeLine(writer: anytype, txt: []const u8, y: usize, width: usize, selected: bool) !void {
    if (selected) {
        try blueBackground(writer);
    } else {
        try attributeReset(writer);
    }
    try moveCursor(writer, y, 0);
    try writer.writeAll(txt);
    try writer.writeByteNTimes(' ', width - txt.len);
}

fn uncook() !void {
    const writer = tty.writer();
    cooked_termios = try os.tcgetattr(tty.handle);
    errdefer cook() catch {};

    raw = cooked_termios;
    raw.lflag &= ~@as(
        os.system.tcflag_t,
        os.system.ECHO | os.system.ICANON | os.system.ISIG | os.system.IEXTEN,
    );
    raw.iflag &= ~@as(
        os.system.tcflag_t,
        os.system.IXON | os.system.ICRNL | os.system.BRKINT | os.system.INPCK | os.system.ISTRIP,
    );
    raw.oflag &= ~@as(os.system.tcflag_t, os.system.OPOST);
    raw.cflag |= os.system.CS8;
    raw.cc[os.system.V.TIME] = 0;
    raw.cc[os.system.V.MIN] = 1;
    try os.tcsetattr(tty.handle, .FLUSH, raw);

    try hideCursor(writer);
    try enterAlt(writer);
    try clear(writer);
}

fn cook() !void {
    const writer = tty.writer();
    try clear(writer);
    try leaveAlt(writer);
    try showCursor(writer);
    try attributeReset(writer);
    try os.tcsetattr(tty.handle, .FLUSH, cooked_termios);
}

fn moveCursor(writer: anytype, row: usize, col: usize) !void {
    _ = try writer.print("\x1B[{};{}H", .{ row + 1, col + 1 });
}

fn enterAlt(writer: anytype) !void {
    try writer.writeAll("\x1B[s"); // Save cursor position.
    try writer.writeAll("\x1B[?47h"); // Save screen.
    try writer.writeAll("\x1B[?1049h"); // Enable alternative buffer.
}

fn leaveAlt(writer: anytype) !void {
    try writer.writeAll("\x1B[?1049l"); // Disable alternative buffer.
    try writer.writeAll("\x1B[?47l"); // Restore screen.
    try writer.writeAll("\x1B[u"); // Restore cursor position.
}

fn hideCursor(writer: anytype) !void {
    try writer.writeAll("\x1B[?25l");
}

fn showCursor(writer: anytype) !void {
    try writer.writeAll("\x1B[?25h");
}

fn attributeReset(writer: anytype) !void {
    try writer.writeAll("\x1B[0m");
}

fn blueBackground(writer: anytype) !void {
    try writer.writeAll("\x1B[44m");
}

fn clear(writer: anytype) !void {
    try writer.writeAll("\x1B[2J");
}

const Size = struct { width: usize, height: usize };

fn getSize() !Size {
    var win_size = mem.zeroes(os.system.winsize);
    const err = os.system.ioctl(tty.handle, os.system.T.IOCGWINSZ, @ptrToInt(&win_size));
    if (os.errno(err) != .SUCCESS) {
        return os.unexpectedErrno(@intToEnum(os.system.E, err));
    }
    return Size{
        .height = win_size.ws_row,
        .width = win_size.ws_col,
    };
}

If you run this code, you'll be faced with a little menu where you can selected "foo", "bar", "baz", and "xyzzy", with the arrow keys and exit with q.

This program is super naive. For example it always renders everything. That is probably fine for this little content, but you might want to make sure you only update what has actually changed. Some libraries use a front-buffer and back-buffer, comparing them and only writing the differences, but I find that if you do the rendering manually, you already know what has changed since last render and do not need to waste cycles calculating that again. Also take care to not write more escape sequences than absolutely necessary, because some terminal emulators use a lot of resources parsing them.

Also it probably makes sense to reduce the amount of write syscalls by buffering your writes, for example using io.BufferedWriter.

Anyway, I hope this little info-dump was helpful, or at least entertaining. There is nothing quite like the eldritch horrors you find when digging into legacy POSIX / UNIX stuff; And this is all still pretty tame compared to other areas...