We’ve released a shining^Wshiny new version of Johnny Deps, our dependency manager for Go. The new version is simpler, cleaner, and more powerful. It also has a shorter name,
jd, instead of being called
johnny_deps. It’s not backwards compatible, but it’s pretty similar.
We rewrote because we found a few shortcomings in the older version. These can be summarized as two main items:
go get. The
go gettool is great for its intended purpose, but it has no notion of package versioning. As a result, it’s sometimes at odds with versioned packages. A simple example of this is if you want to pin a package to an older version and there’s a backwards-incompatible change. If you run
go get, it will try to pull the code from its repositories, but it’ll fail because there will be a compilation error. There are other problems as well –
go getcan be a bit heavyhanded sometimes, and isn’t really something you want to run again and again after you’ve fetched your source code and set it up. It’s really better suited for fetching source for the first time in a non-versioned way.
- Not enough functionality. We ended up with
build.shscripts in all of our repositories. These glued together some tasks such as fetching the source, setting all the dependencies to the correct versions, setting our
--build-versionvariable, and the other types of things we discussed in our previous blog post on creating 100% reproducible builds. A lot of this was boilerplate that we wanted to move out of that script and combine with the dependency and version management.
The New Johnny Deps
A summary of the new Johnny Deps would need to mention the following points:
- It is named
jdso it’s clear that it’s a different tool. It’s also nicer to type.
- It is designed to support a common set of workflows in a single convenient tool. The default action is to get and build, but you can also ask it to only get, or to update the
Godepsfile. In most cases for our own development workflow, we’ll be using
jdinstead of a mixture of
goand other commands.
- It doesn’t use
go getanymore. It just uses
git. It doesn’t make much sense to wrap a wrapper around
git, so we do it directly. This gives us a lot more control and eliminates some overhead and unwanted behaviors. It’s true that it only works with Git. That’s because this tool is for our use first and foremost. We have a policy that all code must be in our own Git repositories. We do not build binaries in the ways that Go’s tools support (e.g. fetching unknown source code from potentially random and untrusted repositories scattered around the Internet). Those workflows are nice for open-source collaboration but do not suit our business needs. The
jdtool is designed for our requirements. We’ve open-sourced it because we believe lots of other people are likely to have similar needs.
- It works recursively. Whereas previously a
Godepsfile was expected to include every package and its SHA at a top-level, now each repository’s
Godepsfile only lists its immediate dependencies and their versions. Dependencies are expected to have their own
Godepsfiles. If it’s not in a
Godepsfile, the source won’t be fetched, and the program won’t compile. Dependencies are verified; if there’s a diamond-shaped dependency graph and conflicting/different dependency versions are listed in the respective
Godepsfiles, the tool will print an error and exit non-zero.
- It is flexible so you can include custom build instructions in an executable if you want.
Please take a look at the documentation. We hope you find the new Johnny Deps helpful and welcome your contributions!