Cover image for Ziggy Pydust
Nicholas Gates
Nicholas Gates

Posted on • Updated on • Originally published at nickgates.com

Ziggy Pydust

Pydust is our framework for building native Python extensions in Zig.

As we all know, Zig is an excellent language! But more specifically, we believe it to be the best language for extending Python. Pydust makes heavy use of Zig’s C integration as well as comptime to provide a framework that, at least to us, feels wonderfully Pythonic 🐍

Here’s an incredibly contrived example:

const py = @import("pydust");

const Post = struct { title: []const u8, tag_counts: py.PyDict };

pub fn tag_count(args: struct { post: Post, tag: []const u8 = "news" }) !u64 {
    return try args.post.tag_counts.getItem(u64, args.tag) orelse 0;

comptime {
Enter fullscreen mode Exit fullscreen mode

And after a poetry install

>>> import example
>>> example.tag_count({'title': 'foo', 'tag_counts': {'example': 3}}, tag='example')
>>> example.tag_count({'title': 'foo', 'tag_counts': {}})
Enter fullscreen mode Exit fullscreen mode

Beyond the Zig library however, Pydust also ships with:

So, why are we building this? At Fulcrum, we are building a cloud native storage engine for Python arrays and data frames (aren’t tables boring?). The core of our engine is written in Zig which gives us tight control over a major performance killer: memory allocations. If you’re handling any sort of large high-dimensional data in Python, we’d love to chat!

Pydust is licensed under Apache 2.0 and can be found here: https://github.com/fulcrum-so/ziggy-pydust

Let us know how you get on!

Oldest comments (4)

kristoff profile image
Loris Cro

Thank you for sharing! I'm actually curious if it would make sense for Zig to generally avoid creating wheels and instead prefer doing any compilation work directly on the final machine.

I understand why this is not normally the case when depending on C/C++, but I think this is a custom worth revisiting for Zig (and even C/C++ projects that have a build.zig file) given its ability to easily create hermetic build processes.

gatesn profile image
Nicholas Gates

Absolutely. In fact, the source dists produced by Pydust will pull in the ziglang PyPi package and perform compilation at install time. This lets Zig fully optimize e.g. SIMD instructions.

lisael profile image

Is it already possible to inspect and mutate python objects at runtime and to call python functions from the zig code ?

gatesn profile image
Nicholas Gates