Zig NEWS

Cover image for Building SQLite with CGo for (almost) every OS
Loris Cro
Loris Cro

Posted on • Updated on

Building SQLite with CGo for (almost) every OS

A recent post on the front page of HN made a comparison between different SQLite packages for Go. Some bundled the official C implementation, while others offered a pure Go implementation.

The C implementation was twice as fast as the Go one, but it saves you from having to deal with CGo. Is the tradeoff worth it? I would say no.

Firstly, SQLite is famous for being tested extremely thoroughly and, while the Go version is the result of machine translation of the original code, I would trust the original code over any reimplementation.

Secondly, dealing with CGo is actually not that bad nowadays.

In this post I'm going to show all the good and bad of compiling for different OSs mattn/go-sqlite3, the package that relies on CGo to compile the original SQLite source file.

The setup

I'm going to create the following builds on a aarch64 linux machine:

  • aarch64 linux
  • aarch64 macos
  • x86_64 windows

Each build will consist of the test suite that the package comes with. Normally you would just run go test, but in my case I'll add -c to make Go produce the test executable without running it, allowing me to move the executable on the correct machine.

Of course I will be using zig cc as the C compiler. If you want to follow along, get yourself a copy of Zig (v0.9 or above), a copy of Go (v1.18 or above) and clone the repo in question.

Linux aarch64

This is the native target and it's going to be super easy.

$ CGO_ENABLED=1 CC="zig cc" CXX="zig cc" go test -c
$ ./go-sqlite3.test
PASS
Enter fullscreen mode Exit fullscreen mode

This is so easy that I want to do something extra

$ ldd go-sqlite.test
    linux-vdso.so.1 (0x0000ffffa4840000)
    libpthread.so.0 => /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/libpthread.so.0 (0x0000ffffa47e0000)
    libc.so.6 => /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/libc.so.6 (0x0000ffffa466d000)
    libdl.so.2 => /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/libdl.so.2 (0x0000ffffa4659000)
    /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/ld-linux-aarch64.so.1 (0x0000ffffa480e000)
Enter fullscreen mode Exit fullscreen mode

As you can see:

  1. It's a dynamic executable
  2. I'm using NixOS btw

Let's make this a static executable

$ CGO_ENABLED=1 CC="zig cc -target native-native-musl" CXX="zig cc -target native-native-musl" go test -c
$ ./go-sqlite3.test
PASS
$ ldd go-sqlite3.test
    statically linked
Enter fullscreen mode Exit fullscreen mode

Nice.

MacOS aarch64

This is going to be a bit more involved. I will post all the intermediate steps that result in an error, scroll to the end of the section if you just want the working command.

$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 CC="zig cc -target aarch64-macos" CXX="zig cc -target aarch64-macos" go test -c
# runtime/cgo
warning(link): framework not found for '-framework CoreFoundation'
warning(link): Framework search paths:
error: FrameworkNotFound
Enter fullscreen mode Exit fullscreen mode

Apparently SQLite Go needs CoreFoundation on mac, so we need to provide it. In my case, the linux machine is a VM running inside a physical Mac Mini, so I can just find where the framework libs are on the host and mount them into the VM.

# MacOS
$ echo $(xcrun --show-sdk-path)
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
# Inside that path there's System/Library/Frameworks, which I then mount to /host/Frameworks
Enter fullscreen mode Exit fullscreen mode
# Linux
$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 CC="zig cc -target aarch64-macos -F/host/Frameworks" CXX="zig cc -target aarch64-macos -F/host/Frameworks" go test -c
# github.com/mattn/go-sqlite3.test
warning: unsupported linker arg: -headerpad
warning: unsupported linker arg: 1144
warning(link): unable to resolve dependency /usr/lib/libobjc.A.dylib
/home/kristoff/golang/go/pkg/tool/linux_arm64/link: /home/kristoff/golang/go/pkg/tool/linux_arm64/link: running dsymutil failed: exec: "dsymutil": executable file not found in $PATH
Enter fullscreen mode Exit fullscreen mode

The warnings are fine, but the final message is a hard error: Go is expecting to be able to run dsymutil, which I don't have on my system. I tried to get dsymutil by installing LLVM (nix-shell -p llvm) but compilation would still fail as it apparently didn't like the executable produced by Go.

One way forward is to ask Go to not add debug info to the executable by passing -ldflags="-s -w", which is annoying, but will help us move forward.

$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 CC="zig cc -target aarch64-macos -F/host/Frameworks" CXX="zig cc -target aarch64-macos -F/host/Frameworks" go test -c -ldflags="-s -w"
# github.com/mattn/go-sqlite3.test
warning(link): unable to resolve dependency /usr/lib/libobjc.A.dylib
Enter fullscreen mode Exit fullscreen mode

I then ran the executable on the host and it passed all tests.

tests passing on mac

Windows x86_64

This one was surprisingly smooth.

$ CGO_ENABLED=1 GOOS=windows GOARCH=amd64 CC="zig cc -target x86_64-windows" CXX="zig cc -target x86_64-windows" go test -c 
Enter fullscreen mode Exit fullscreen mode

Running the tests on windows show a couple failed tests, but they have to do with my locales, so everything went well also in this case.

tests failed successfully on windows

Conclusion

Not the smoothest experience, but not that bad either. Go could stop depending on dsymtool, the libobjc warnings could be silenced by providing the actual file, like we did with CoreFoundation (I didn't out of lazyness since everything succeeds anyway), and everything would be pretty much seamless.

Pretty good for a programming language and toolchain that is not yet v1.0 :^)

Want cross-compilation to improve even more?
Consider sponsoring the Zig Software Foundation.

Art by kprotty.

Oldest comments (1)

Collapse
 
yanwenjiepy profile image
花大喵

Very good combination, and finally compiles a pure static executable