A recent post on the front page of HN made a comparison between different SQLite packages for Go. Some bundled the official C implementation, while others offered a pure Go implementation.
The C implementation was twice as fast as the Go one, but it saves you from having to deal with CGo. Is the tradeoff worth it? I would say no.
Firstly, SQLite is famous for being tested extremely thoroughly and, while the Go version is the result of machine translation of the original code, I would trust the original code over any reimplementation.
Secondly, dealing with CGo is actually not that bad nowadays.
In this post I'm going to show all the good and bad of compiling for different OSs
mattn/go-sqlite3, the package that relies on CGo to compile the original SQLite source file.
I'm going to create the following builds on a aarch64 linux machine:
- aarch64 linux
- aarch64 macos
- x86_64 windows
Each build will consist of the test suite that the package comes with. Normally you would just run
go test, but in my case I'll add
-c to make Go produce the test executable without running it, allowing me to move the executable on the correct machine.
Of course I will be using
zig cc as the C compiler. If you want to follow along, get yourself a copy of Zig (v0.9 or above), a copy of Go (v1.18 or above) and clone the repo in question.
This is the native target and it's going to be super easy.
$ CGO_ENABLED=1 CC="zig cc" CXX="zig cc" go test -c $ ./go-sqlite3.test PASS
This is so easy that I want to do something extra
$ ldd go-sqlite.test linux-vdso.so.1 (0x0000ffffa4840000) libpthread.so.0 => /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/libpthread.so.0 (0x0000ffffa47e0000) libc.so.6 => /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/libc.so.6 (0x0000ffffa466d000) libdl.so.2 => /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/libdl.so.2 (0x0000ffffa4659000) /nix/store/34k9b4lsmr7mcmykvbmwjazydwnfkckk-glibc-2.33-50/lib/ld-linux-aarch64.so.1 (0x0000ffffa480e000)
As you can see:
- It's a dynamic executable
- I'm using NixOS btw
Let's make this a static executable
$ CGO_ENABLED=1 CC="zig cc -target native-native-musl" CXX="zig cc -target native-native-musl" go test -c $ ./go-sqlite3.test PASS $ ldd go-sqlite3.test statically linked
This is going to be a bit more involved. I will post all the intermediate steps that result in an error, scroll to the end of the section if you just want the working command.
$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 CC="zig cc -target aarch64-macos" CXX="zig cc -target aarch64-macos" go test -c # runtime/cgo warning(link): framework not found for '-framework CoreFoundation' warning(link): Framework search paths: error: FrameworkNotFound
SQLite Go needs CoreFoundation on mac, so we need to provide it. In my case, the linux machine is a VM running inside a physical Mac Mini, so I can just find where the framework libs are on the host and mount them into the VM.
# MacOS $ echo $(xcrun --show-sdk-path) /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk # Inside that path there's System/Library/Frameworks, which I then mount to /host/Frameworks
# Linux $ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 CC="zig cc -target aarch64-macos -F/host/Frameworks" CXX="zig cc -target aarch64-macos -F/host/Frameworks" go test -c # github.com/mattn/go-sqlite3.test warning: unsupported linker arg: -headerpad warning: unsupported linker arg: 1144 warning(link): unable to resolve dependency /usr/lib/libobjc.A.dylib /home/kristoff/golang/go/pkg/tool/linux_arm64/link: /home/kristoff/golang/go/pkg/tool/linux_arm64/link: running dsymutil failed: exec: "dsymutil": executable file not found in $PATH
The warnings are fine, but the final message is a hard error: Go is expecting to be able to run
dsymutil, which I don't have on my system. I tried to get
dsymutil by installing LLVM (
nix-shell -p llvm) but compilation would still fail as it apparently didn't like the executable produced by Go.
One way forward is to ask Go to not add debug info to the executable by passing
-ldflags="-s -w", which is annoying, but will help us move forward.
$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 CC="zig cc -target aarch64-macos -F/host/Frameworks" CXX="zig cc -target aarch64-macos -F/host/Frameworks" go test -c -ldflags="-s -w" # github.com/mattn/go-sqlite3.test warning(link): unable to resolve dependency /usr/lib/libobjc.A.dylib
I then ran the executable on the host and it passed all tests.
This one was surprisingly smooth.
$ CGO_ENABLED=1 GOOS=windows GOARCH=amd64 CC="zig cc -target x86_64-windows" CXX="zig cc -target x86_64-windows" go test -c
Running the tests on windows show a couple failed tests, but they have to do with my locales, so everything went well also in this case.
Not the smoothest experience, but not that bad either. Go could stop depending on
libobjc warnings could be silenced by providing the actual file, like we did with CoreFoundation (I didn't out of lazyness since everything succeeds anyway), and everything would be pretty much seamless.
Pretty good for a programming language and toolchain that is not yet v1.0 :^)
Want cross-compilation to improve even more?
Consider sponsoring the Zig Software Foundation.
Art by kprotty.