Updating dependencies in a gradle project
If there is one hill I’m willing to get mildly injured on, it’s that you should keep your dependencies up to date.
Keeping my dependencies up to date is something I keep on writing about, most recently to introduce a command line tool I wrote to list out of date dependencies in Swift projects. Kotlin has replaced Swift for me these days, but the problem remained, especially as we’re building cool stuff at Sound Games and I get to make the servers go brrrrr1.
Dependabot is one of the most popular ways to keep your dependencies up to date. It started as a standalone product and is now part of GitHub’s offering. As you may know, I’ve decided to stop using GitHub, but that’s not (only) why I’m not using Dependabot on other projects. I think that updating dependencies should be a conscious decision. Just because you get a pull-request created automatically where all your tests are passing does not mean that the update is safe. It’s actually pretty easy to bite yourself in the butt by merging a green PR, going to bed and waking up to angry message from the 12 daily users of your side project about comic books2.
Now for the perfect solution: I don’t really have one. Recently at work, I created a super quick and dirty script that retrieves all the top-level dependencies we use in the project, retrieve the most recent versions available on the maven central repository and print a sorted list to compare both. I figured I would write about this.
Listing dependencies
You can list your dependencies in a Gradle project using the gradle command line tool to run the :dependencies subtask. It will print a tree with all the dependencies used for a specific configuration. Something that roughly looks like this:
| +--- io.ktor:ktor-utils-jvm:3.2.3 (c)
| +--- io.ktor:ktor-serialization-kotlinx-jvm:3.2.3 (c)
| +--- io.ktor:ktor-call-id-jvm:3.2.3 (c)
| +--- io.ktor:ktor-network-jvm:3.2.3 (c)
| \--- io.ktor:ktor-io-jvm:3.2.3 (c)
+--- io.ktor:ktor-client-core:3.2.3
| \--- io.ktor:ktor-client-core-jvm:3.2.3
| +--- org.slf4j:slf4j-api:2.0.17
| +--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.10.2
| | \--- org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm:1.10.2
| | +--- org.jetbrains:annotations:23.0.0
| | +--- org.jetbrains.kotlinx:kotlinx-coroutines-bom:1.10.2
| | | +--- org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm:1.10.2 (c)
| | | +--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.10.2 (c)
| | | \--- org.jetbrains.kotlinx:kotlinx-coroutines-slf4j:1.10.2 (c)
| | \--- org.jetbrains.kotlin:kotlin-stdlib:2.1.0 -> 2.1.21 (*)
My original approach was to run this command and parse its output to retrieve our dependencies, but I didn’t like it. No real reasons to justify this except that it didn’t feel right. I then quickly realized that all the dependencies are in a version catalog. If you’re not familiar, the version catalog is a file in the TOML format that looks like this:
[versions]
ktor-version = "3.3.0"
[libraries]
ktor-server-core = { module = "io.ktor:ktor-server-core-jvm", version.ref = "ktor-version" }
ktor-server-core-html-builder = { module = "io.ktor:ktor-server-html-builder", version.ref = "ktor-version" }
I don’t have anything that can parse TOML3 so instead I went for a naive approach where I read the file line by line, remove the comments and use a regular expression to retrieve which module we’re using and at which version. How confident am I in this approach? A solid 2 out of 10: what is easy to break is probably just as easy to fix, so that’s good enough right now.
Getting all the versions available for a module
The next step was to retrieve each modules’ available versions so we can compare with the version we’re using. After a quick search, I found out that there is a search API available for the maven central repository, so I started with that.
As the screenshot above shows using RapidAPI4, the API accepts a query (see 1) with a group (g), and an artifact name (a). Please don’t ask me what the core argument means, for I don’t remember. Using the filter’s feature (2) that accepts a jq syntax, you can see what we retrieve a list of versions (3). Good enough!
Unfortunately, I’ve found this API to be slow and unreliable, as most of my requests failed with timeouts. At this point it’s not clear to me if this search API is still actively maintained or not. I’ll try to find out because my alternative solution was a little, well, alternative.
Instead of using the search API, I’m grabbing the content of the page that lists all the versions available on a repo5 (in the screenshot above, those are the versions for the sql-gauss-db artifact in the com.easy-query group) and parse the HTML. That’s a little fragile, but it does the job for now. Plus, the chances that the format of the URLs on that page change in the foreseeable future are pretty low6.
Comparing versions
Finally, I have the versions we use and the versions that exist, so I originally started by sorting the list alphabetically and grabbed the top version as the latest. I knew this was a bad idea that I would have to fix eventually, but I didn’t expect I would have to address it immediately when my script suggested I upgrade a library from 1.12.x to 1.2.x.. As a result, I resigned into writing a proper semver parser (that doesn’t account for pre-releases versions yet) and addressed that issue.
The result is a simple script that shows the version used by a library and the latest version available upstream, in this simple format.
com.cjbooms:fabrikt : 24.0.0 -> 24.1.3
org.jetbrains.kotlinx:kotlinx-serialization-json : 1.8.0 -> 1.9.0
org.jetbrains.kotlinx:kotlinx-datetime : 0.6.1 -> 0.7.1
org.testcontainers:testcontainers : 1.19.8 -> 1.21.3
org.testcontainers:postgresql : 1.19.8 -> 1.21.3
...
Of course, there is already a bunch of things I want to improve, but this feels like a straight path toward re-implementing a bad version of dependabot, so I just merged my pull-request, selected a few dependencies to update first and moved on with my day, giving time for those improvement ideas to settle. I guess I’ll get back to it when all my dependencies are up to date?
-
That’s my official job description, if you were wondering. ↩
-
This is of course a random example ↩
-
I also think that the irony of adding a new dependency solely to retrieve the dependencies I use would have been too much to bear ↩
-
RIP Paw, which was a much better/cuter name, don’t @ me on this. ↩
-
After checking their robots.txt, which explicitly disallows uses of their data for training of AI models, but allows it for searching purposes. ↩
-
Famous last words. ↩