A configuration driven approach in software

I only started to hear the phrase ‘configuration driven’ in the early days of my career in the context of my code. What does it really mean? I find it hard to put into words. At its simplest, it’s about moving the drivers of logic in our code from the code itself to a file reserved only for things we can configure. Theoretically, we should be able to ‘plug in’ something to our configuration (ie. make a change to it), and we don’t have to make changes to our code and our tests. I think an example will make this easier to understand.

Upon arriving in a new dev role I’m almost immediately thrown into cleaning up the code of a guy who’s recently departed the company. One of my first jobs is to make some changes to the alerts he set up. The premise of the problem is as follows: In the codebase, for every exception that is thrown, we send an alert to an AWS SNS topic which is then picked up by a subscriber. A subscriber is just someone’s email address. This way, we know which alert was thrown without scanning the logs manually because it sends us an email. The task at hand was that we need to exclude one of the alerts from being sent because the business stakeholders don’t truly consider it to be relevant and they’re sick of seeing emails! However, it is still an exception in our code and we don’t want that to change, nor do we want to exclude it from our logs. Therefore, we need to build some kind of logic to exclude this particular alert from being sent.

The scenario

Let’s visualise this scenario.

If disabled, just exclude it?

The naive developer inside of you tells you this is a pretty easy one, right? If it’s an alert that should be disabled, just exclude it. What’s the big deal? Maybe something like the following where we don’t send the alert if the exception.errorCode doesn’t match our MISSING_DATA String:

//ExceptionHandler.kt

if (exception.errorCode != "MISSING_DATA") {
	sendToSNS(exception)
}

For now this might be fine, but it’s not extensible and it’s considered a red flag by many. Wait, what, really? Let’s dive deeper. Something like this could easily blow out of proportion. What if a new requirement comes along saying that we have another error, and then another, to be excluded? Errors aplenty, our code turns into:

//ExceptionHandler.kt

if (exception.errorCode != "MISSING_DATA" || exception.errorCode != "MISSING_DATA_OUTBOUND" || exception.errorCode != "MISSING_DATA_BIG_DATA") {
	sendToSNS(exception)
}

Ok so now we have to exclude each one individually. This is getting a bit ugly, but it’s not the end of the world. What if we have even more of them? Maybe we choose to refactor and exclude them as a list rather than as individual exceptions. In the following case, we’ll store our excluded exceptions in a list and we’ll use Kotlin’s none method on the Collection to determine that none of the items in the list match our exception:

//ExceptionHandler.kt

private val exceptionsList = arrayListOf("MISSING_DATA", "MISSING_DATA_OUTBOUND", "MISSING_DATA_BIG_DATA", "MISSING_OTHER_DATA", "MISSING_EVEN_MORE_DATA")

if (exceptionsList.none( myException -> exception.errorCode == myException )) {
	sendToSNS(exception)
}

This is getting slightly better, however, we end up with our data being hardcoded as strings in the exceptionsList. The question is whether this is really a big deal? Is it really that hard to just update your conditional statement and put a new item in the list if a new requirement to update this arises? I would argue, yes. It’s a code smell and I don’t like smells.

A smelly list!

The smell soon arises upon the sharing of exceptionsList among classes in the source code. If we suspect that the list shall be used in other places then we either have to (a) share it around or (b) we have to make a copy of it. Those are our two options. On the face of it, it doesn’t seem like that big a deal to do either one but I want to explore what could go wrong.

If we share it around (a), what might that look like? Assume there is a logging class in our codebase called Logger. In this class, we want to log the excluded exceptions that we configured every time the application starts up.

//Logger.kt

log.info("Initialised with the following exceptions alerts disabled: $exceptionsList")

The logger will just print the list and then do its thing. However, to do this there is a need to expose the list outside of the original class and make it public! private val exceptionsList will turn into val exceptionsList.

Ideally, principles of encapsulation should be followed. Exposing the data of one class to other classes breaks that imperative encapsulation principle. Classes shouldn’t know data from every other class or we’ll end up with a global spaghetti bowl of data within the source code.

If we make a copy of it (b) then we face a problem of keeping data in sync. What if we have to add or remove a new exception in future, or change an existing exception? Now the developer must remember to maintain two lists! What if that developer leaves the company and a new developer comes along, and doesn’t realise this? All of a sudden the list is updated in one class but not another. That esoteric knowledge that sat with one person didn’t transfer so easily.

Clean the smelly list

The reality is that this problem could be easily avoided by taking a configuration approach to our data. There are a few ways we could tackle this:

  • We could have a list stored in our code, perhaps inside a configuration class as an object. There is absolutely nothing wrong with this approach.
  • We could have a list stored outside our code, as a file. The great thing about this is that we don’t need to rebuild the entire application if one of those values changes. This probably isn’t a big deal in smaller applications but I’m going to proceed with this approach in this post to demonstrate the example.

Different frameworks have their own approaches but since I’m most familiar with building APIs in Spring with Java and Kotlin, I’ll go with ‘the Spring way’ for today. Regardless, we usually see that there is some kind of file on our file system where we can store our configuration properties and load it into our application. iOS has the classic .plist property list file but in the case of the Spring framework we use a .properties text file or a .yaml file that does the same thing.

I’ll add to my Spring configuration file:

//application.yaml

exceptions:
	disabled:
		- "MISSING_DATA"
		- "MISSING_DATA_OUTBOUND"
		- "MISSING_DATA_BIG_DATA"
		- "MISSING_OTHER_DATA"

I’ll now have to modify my code to pull the list in from the configuration file. Good thing we don’t have to do any manual parsing of the list (the framework handles that all for us). We can load it in with Spring’s @ConfigurationProperties annotation or the older @Value annotation. I like @ConfigurationProperties because in Kotlin we can load the entire list in with a regular data class based on the name of the list in the configuration file - more info on that here (note that we also need the @ConstructorBinding annotation from Spring as a requirement but we won’t go into detail on that).

//ExceptionsConfig.kt

@ConstructorBinding
@ConfigurationProperties(prefix = "exceptions")
data class ExceptionsConfig(
	private val disabled: List<String>
)

Okay, so what’s the difference? Let’s talk through, step by step. The Spring framework does its magic with the @ConfigurationProperties annotation, looking inside our configuration .yaml file to find the ‘exceptions’ label. Within that, it will populate a List<String> based on the ‘disabled’ label nested under the ‘exceptions’ label, as is stated in our yaml. Cool, so that maps our configuration file to our Kotlin code, populated inside a data class.

Now, in our original code we can modify it to use the list from the newly created ExceptionsConfig data class that we made by letting the framework automatically inject that dependency. Alternatively we could manually instantiate an ExceptionsConfig but we’re using Spring so let’s take advantage of its core purpose of dependency injection. In this case, I use the @Autowired annotation to tell Spring ‘hey, please give me an instance of ExceptionsConfig’. Now, in my ‘if’ statement I can modify the source of the list to use our newly imported disabledExceptions which is a List taken from configuration:

//ExceptionHandler.kt

@Autowired
private lateinit var disabledExceptions: ExceptionsConfig

if (disabledExceptions.none( myException -> exception.errorCode == myException )) {
	sendToSNS(exception)
}

We end up with code that does the same as what it did before, but instead of maintaining a list of excluded exceptions in the ExceptionHandler class itself that data is loaded in from a yaml configuration file and loaded in by the framework.

The benefits

The benefits of a configuration-driven approach are equal parts subtle and remarkable. Benefits appear as follows:

  • Our data is encapsulated within the configuration file only and we don’t ‘leak’ it to be modified in any other class.
  • There is a single source of truth for exceptions that need to be disabled.
  • If we modify that list, it is only updated in one place, and that flows throughout our codebase.
  • If new developers join the project, there is one clear origin of the data and it won’t have side effects if the data is updated, it doesn’t need to be updated in any other place. It’s clearer and more readable for that new developer, allowing that person to gain a much easier understanding of my intent.

Important considerations

Based on feedback from sharing this post, I want to add these important considerations:

  • Be sure to factor in how you will construct your tests when externalising the configuration! This is really important since if you’re ‘plugging in’ a value from a configuration file, that may have to be tested for each scenario you provide and/or consider if you’ll mock what the configuration class will provide.
  • Do you really need to use a configuration driven approach? As stated, my opinion is that it promotes encapsulation, we have a single source of truth and it improves readability. A very valid consideration is that a simple ‘if’ statement could be fine for simple configurations. Some may argue it’s whether you expect this list of ‘configurable stuff’ to grow in future, or, whether you only refactor when you need to, which is very sensible.