Zig NEWS

Zig Package Manager - WTF is Zon

Ed Yu on June 27, 2023

The power hack and complexity of Package Manager in Zig Ed Yu (@edyu on Github and @edyu on Twitter) Jun.27.2023 Introduction Zi...

Read full post

freakmangd • Oct 4 '23

this is a nice informative article. I'd suggest though that instead of putting a random hash to find the correct hash, you omit the hash field entirely:

.dependencies = .{
  .pkg = .{
    .url = "https://...",
  }
}

this gives the same effect without any potential errors.

also good to mention is that you can use .path instead of .url for local dependencies on master branch, and it's just that one line change :) (it also doesnt need a hash)

Fuzhou Chen • Jul 4 '23

Wow a built-in package manager is what I'm looking for. Thanks Ed for sharing.

Besides, I'm thinking whether it's a good idea to use an URL to package instead of searching for a tag in Git commit history. That was what I learned from go mod package management. It removes the needs to ask package provider maintain their tarballs, and gets rid of our own hash verification logic because Git has provided it. It seems Zigmod also does a similar way.

Did we get a chance to consider whether this is also a feasible approach for zon?

Ed Yu • Jul 6 '23

Hi Fuzhou,
Thanks for the comment. Andrew replied to a post I made about this blog in ziggit and he's thinking about removing the hash just for the reason you mentioned. I however wouldn't know when it would be made into the Zig system.
As for commit history, there is a way to point to the commit such as

.url = "https://github.com/beachglasslabs/duckdb.zig/[COMMIT-HASH].tar.gz"

but I haven't tested it with the latest zig. Is this what you meant?

Fuzhou Chen • Jul 9 '23 • Edited

Thanks Ed. Actually what I'm looking for is something like below.

// For stable builds - I used my own project as example
.url = "https://github.com/fuzhouch/lpath"
.version = "v0.1.0"

// or for non-stable builds 
.url = "https://github.com/fuzhouch/lpath"
.version = "4e7d4c8" // It point to https://github.com/fuzhouch/lpath/commit/4e7d4c8cb7a7948114fb1f4e738176dc1f30f6e3

My first goal is we eliminate the needs of maintaining tarballs. For source code dependency scenario, tarball does not work well on a common scenario, that we want to do close peer testing with upstream dependency on an unstable branch. Even we have strong devops support today, many small projects do not publish non-stable source code tarballs. Even for large projects, they don't always publish as we want. In this case, developers seems have no choice but fallback to checkout source code, then re-package it ourselves. Thus, forcing the use of tarballs on dependencies does not sound like a ideal idea.

Meanwhile, hash validation is necessary for sure, while I prefer we avoid developers or library users maintaining it. Git hash should be able to do this trick.

Btw, I somehow has a task to evaluate zigmod/zpm earlier. So far it seems existing non-official package managers support downloading source code from Git (just requires an additional index server). The difference is it seems they don't actively honor versions/tags/commits. And seems they don't validate (not 100% sure, to be double confirmed). I was thinking they are in a lack of complete git support in Zig, while I like their design decision of depending on source code repo.

I'm not very sure why a built-in tool, instead, depends on tarballs. This looks like we are taking one step back (sorry for saying this). It could be totally understandable if this is a temporary solution, while I'm wondering the final direction.

Or kindly suggest/correct me if I miss anything.

Ed Yu • Jul 10 '23

I actually agree. I would highly recommend that you go to the zig github page and file an issue. The core team is usually very supportive of good idea especially if you have real-world use-case that contradicts how they think about the issue. I do think it's a temporary solution as we've been battling the TLS bug (if it indeed is the problem) for a while. I still have problems getting any tarball to download properly without segfault on the first try.
My goal for this article is more of "what it is" not "what it should be" so people like you can point the project to the right direction. :)
Let me know when you do and I'll help getting Andrew to look at it if and when I can get his attention.

Talha Qamar • Jul 22 '23

Has this happened yet?

Fuzhou Chen • Jul 24 '23

Filing an issue on Github? Not yet. :(. I'm busy recently to catch up Halloween release for my game project. Thanks for @edyu 's suggestion. I will find a chance to make it happen!

Fuzhou Chen • Jul 25 '23

Hey @edyu , I'm about to create an issue on Github.com. However, it seems I can't find a proper category. For now it allows creating issues for Autodoc Issue, Bug Report, and Error Message Improvement. I don't think they fit our discussion. There are Language Proposal and Question. Unfortunately they point to Wiki page.

Could you suggest a place that we can bring our topic to community?

Ed Yu • Aug 2 '23

You can use bug report. As for discussion, you can use zigget.dev but eventually it would still go back to GitHub.

Fuzhou Chen • Aug 3 '23

Thanks! Seems the idea is already under tracking in Github. github.com/ziglang/zig/issues/14298 . Let's track from here.

Guido Schmidt • Jul 26 '23

Nice introduction to zon, thanks for sharing. I was wondering if there's already the possibilty to use a local path instead of a git url for a package? For like when you want to work on your own library in parallel to a project which is using it as a package via zon.

cryptocode • Aug 2 '23 • Edited

Here's one way:

mod2$ tar --exclude=mod2.tar.gz --exclude='./zig-cache' --exclude='./zig-out' -zcvf  mod2.tar.gz .

mod2$ python3 -m http.server 9000

mod1$ cat build.zig.zon
.{
    .name = "test-mod2",
    .version = "0.0.1",

    .dependencies = .{
        .mod2 = .{
            .url = "http://localhost:9000/mod2.tar.gz",
            .hash = "12207ba8570a2249dda4f695dac501e0a9d2b5f22276a4bceae1f5b87f7dc0a896d1",
        },
    },
}

Guido Schmidt • Aug 2 '23

Ah nice, didn't thought about spinning up a local http server for this, thx :)

cryptocode • Aug 2 '23 • Edited

Most people run zig build a first time to get the Zig-compatible sha256 hash of the tar.gz file (you get the actual hash as an error)

If you wanna do it yourself, here's quick Python script I made: gist.github.com/cryptocode/7bf963c...

Guido Schmidt • Aug 2 '23

404 unfortunatelly, might be a private gist 😇

cryptocode • Aug 2 '23

Can you try again? Others seems to be able to access it.

gist.github.com/cryptocode/7bf963c...

Guido Schmidt • Aug 3 '23

Thanks for checking again, works. Maybe I wasn't logged in or sth.

Ed Yu • Oct 19 '23

A follow-up Part 2.