Losing my sanity after switching from YAML to HOCON in a Ktor project

A picture of a long haired grey cat napping on a laptop sleeve.

Ktor has become my go-to framework for all the projects I start these days. This week, I wanted to setup a simple API, so I went on the Ktor website and created a project with the extensions I needed. Except I made a mistake: I configured the project to use YAML instead of HOCON as the configuration format. It was an easy mistake to fix, but once I deployed my project… all hell broke loose and nothing made sense anymore.

TL;DR: don’t add the “ktor-server-config-yaml” dependency to your Ktor project unless you plan on using YAML configuration files.

First of all, how did I fix the mistake? As the documentation states, you can configure a Ktor server using either code, a YAML file or an HOCON file. Personally, I much prefer HOCON to YAML1 so I simply renamed my application.yaml to application.conf, adapted the content to the new format and kept working on my API. As I mentioned, it was pretty easy to fix.

As long as I ran the project locally or launched automated tests, everything went as expected. Unfortunately, once I deployed the app on my server, it would fail to start and print this error:

Exception in thread "main" java.lang.IllegalArgumentException: Neither port nor sslPort specified. Use command line options -port/-sslPort or configure connectors in application.conf

This was really strange because this is what my configuration file looked like:

ktor {
  application {
    modules = [codes.romain.ApplicationKt.module]
  }
  deployment {
    port = 8080
    port = ${?PORT}
  }
}

Of course there was a very easy solution to this, I could have just re-created a new project and moved my code into its new vessel, but I figured this was an opportunity to learn something new. Here is roughly what I did.

I host my projects on a self-hosted Dokku instance. I don’t use any of the herokuish buildpacks 2. Instead, I build docker images, publish them to a registry and ask Dokku to pull that new docker image via SSH. The first thing I wanted to do was pull the latest image and run it locally to see if the error was happening as well. Unsurprisingly, it did, so I moved on to the next test3.

Running the docker image locally shows the same error. Please enjoy the tag.
Running the docker image locally shows the same error. Please enjoy the tag.

Next, I ran the same command that I was running in my Dockerfile to create the jar on my machine. After creating the jar, I tried to run the server locally (with java -jar ...) and got the same error. I still wanted to inspect the jar, so I played around with unzip (since, you know, jar are mostly zip files) first to make sure that the configuration file was in there (it was), then to inspect its content and make sure I hadn’t somehow messed it up (I didn’t).

Showing the content of the bundled application.conf file using unzip
Showing the content of the bundled application.conf file using unzip

At this point it got really weird, fortunately it was time to call it quit for the night and have dinner, watch some TV and go to bed. It was really hard to forget about it thought, because I’ve been using this framework for a bit more than a year on a bunch of projects and this made no sense at all!

A few days later, it was time to face the problem again and try to understand what was going on, so I went back to the Ktor website, created a project using YAML configuration and replaced with a HOCON file, compiled the jar, tried to run it, and… it happened as well! At least I was sure that the issue wasn’t something I caused later in my own project. I created another project on the Ktor website, making sure that I selected HOCON as the config file this time and throw both project folders into Kaleidoscope: the only difference was the dependency on ktor-server-config-yaml. Everything else was the same.

Finally, I cloned the Ktor repository, started looking at the ApplicationConfig file, and learned a few interesting things. Ktor supports multiple configuration loaders (YAML, HOCON…) and will iterate through them until one doesn’t return null after loading the configuration file. When I looked at the YAML configuration loader, I noticed one of those weird “META-INF” folders with a service configured in there to automatically load the YAML configuration loader.

I know enough about those META-INF folders to know that I wouldn’t be able to explain how they work or what they’re used for, but I knew that was the solution to my problem.

I removed the dependency to ktor-server-config-yaml from my project and the problem went away.

What’s Next?

I’m glad I took the time to investigate this issue and not just move past it. I learned a few things along the way about the Ktor internals, so that’s cool. I’m still not sure I understand why it didn’t work, so I’m planning to dig a little deeper into those different config loaders. I’ll make sure to write about it if I find something interesting.

Honestly, this is giving me an itch to contribute to the documentation or Ktor itself at some point. We’ll see!

  1. but I’m not really willing to fight you over it. 

  2. Which mostly means I don’t build and run my project on the same server 

  3. Although to be truly thorough, I rebuilt the image locally first.