Composing Configuration Sources

Recently there were discussions regarding a new Java EE Configuration JSR. At JavaOne 2016 Oracle announced that they want to relaunch this JSR (first attempt was about 3 years ago, when I was working at Credit Suisse). Unfortunately  with latest reschedules and strategic decisions Oracle decided to postpone this initiative until Java EE 9. But the topic is an important cross-cutting concern, especially when running in distributed Cloud environments. Consequently discussions are ongoing, e.g. at microprofile.io. In this blog I would like to discuss my thaughts on how different configuration sources could be assembled into one uniform Configuration instance.

What is Configuration?

To keep things simple I assume Configuration to be something as simple as follows (using Java styled syntax):

public interface Configuration{
  String getProperty(String key);
  Map<String,String> getProperties();

  static Configuration getInstance(){
    ...
  }
}

Similarly I define a Configuration Source, which provides configuration entries as follows:

public interface ConfigSource{
  String getProperty(String key);
  Map<String,String> getProperties();
}

Composing Configuration from multiple Configuration Sources

In general composing of a Configuration is a function f that maps all known ConfigSources csx to a Configuration c:

f(cs1, cs2, cs3, …, csx) -> c

Most configuration frameworks currently known hereby simplify this function  by applying repeatedly a mapping function fm, which maps a initial value and a new value of property k repeatedly for pairs of configuration sources to a final value V. This can be written as follows:

V = fm(…fm(fm(fm(null, CV(cs1, k), CV(cs2, k),CV(cs3, k), …)

Despite Apache Tamaya, where fm can be configured (as so called PropertyValueCombinationPolicy), most frameworks use a fixed (overriding) mapping function, which can be defined as follows:

fm(v1, v2) ->  v2==null ? v1 : v2

There are use cases, where this strict kind of mapping function is not suitable (e.g. mapping of collection types and type safe configuration). This is worth to be discussed in a separate post. For now, given a mapping function fm, obvisously the only free dimension is the ordering of configuration sources. So the question is how the sources should be ordered before the evaluation functionality is applied. This sounds  easy, but in fact can get quite tricky, since the sources are not necessarily known at deploy time. This can be the case because:

  • The number of files in a config directory are different for different environments.
  • The configuration should be automatically composed based on what is accessible in the current classpath.
  • Configuration is evaluated from different class loaders with different configuration sources visible.

Using Numeric Values as Ordering Criteria

Many configuration systems solve this problem, by using some kind of a numeric value applied to each configuration source, called ordinal. So to a ConfigSource a method is added returning the ordinal as illsutrated below:

int getOrdinal();

Based on this ordinal value a configuration system can easily determine the order of ConfigSources. Advantages of ordinals hereby are

  • the concept is simple
  • it allows comparing of multiple property sources also from different sources/frameworks/modules
  • the ordinal value can be configured by adding config entries to owning configuration source
Nevertheless the concept has also its disadvantages, some of them are:
  • what should be done with multiple config sources that provide the same ordinal value?
  • what is the exact semanic of an ordinal. Is a config source with a ten times bigger ordinal, ten times more significant?
  • what to do if a an order based on ordinals is not sufficient for your requirements, meaning your configuration assembly policy requires a more complex ordering criteria?
  • how to add a config source between subsequent ordinals n and n+1?
  • when using builders to define your configuration, ordinals are confusing, since they are not used if a Configuration is built programmatically.

Given that let us discuss, if the concept of ordinal is really needed.

Using Meta-Configuration to define the Config Sources and their corresponding order

In the sandbox part of Apache Tamaya there is a metamodel module, which allows to define a configuration system using an XML formatted metaconfiguration file, similar to the one below (in fact there are many more features available ike immutability, property-source-level filtering, auto-refresh etc):

<configuration>
  <property-sources>
    <source type=”sys-properties” />
    <source type=”env-properties” />
    <source type=”etcd” />
 <property-sources>
</configuration>
This is very convenient, easy to understand and does not need any ordinal values at all, since the XML structure defines the ordering by the order of child elements of the property-sources element.

Using a Builder to define the Config Sources and their corresponding order

As part of Apache Tamaya core API it is possible to acquire a ConfigurationContextBuilder, which allows to configure the configuration system. The resulting ConfigurationContext can then be easily be converted into a Configuration instance:

ConfigurationContextBuilder builder = ConfigurationProvider
                                 .createConfigurationContextBuilder();
builder.addPropertySources(
   new SystemPropsPropertySource(),
   new EnvironmentPropsPropertySource(),
   new EtcdPropertySource(),
);
Configuration config = ConfigurationProvider.createConfiguration(builder.build());

Supporting Ordinals

If someone still wants to use an ordinal based approach, the ordinal can be accessed as a normal property of a ConfigSource:
String ordinal = propertySource.getProperty(“_ordinal”);

Since ordinals may not be the single and only property to define the ordering of ConfigSources, Tamaya’s ConfigurationContextBuilder supports passing a Comparator that has to be used for sorting of ConfigSources. As a consequence we must only add one line (highlighted in red) of code to emable ordinal based sorting:

ConfigurationContextBuilder builder = ConfigurationProvider
                                 .createConfigurationContextBuilder();
builder.addPropertySources(
   new SystemPropsPropertySource(),
   new EnvironmentPropsPropertySource(),
   new EtcdPropertySource(),
);
builder.sortPropertySources(
                   DefaultPropertySourceComparator.getInstance());
Configuration config = ConfigurationProvider.
                   createConfiguration(builder.build());
If we want to apply a custom ordering we simply pass our  custom Comparator instance:
ConfigurationContextBuilder builder = ConfigurationProvider
                                 .createConfigurationContextBuilder();
builder.addPropertySources(
   new SystemPropsPropertySource(),
   new EnvironmentPropsPropertySource(),
   new EtcdPropertySource(),
);
builder.sortPropertySources(MyCustomComparator.getInstance());
Configuration config = ConfigurationProvider.
                   createConfiguration(builder.build());
This way the API is simple, but still flexible and not pulluted by an ordinal concept.

Winding up…

Looking at all the disadvantages and especially at the function f, which renders a set of configuration sources into a Configuration, it is IMO not a good idea to use ordinals as part of the API of a configuration system. Instead
  • let the configuration system provide a Builder based approach to build a Configuration, which allows to define the ordering
    • freely by adding property sources programmatically in the desired order. The Builder must not change any order of the ConfigSources added.
    • by applying an arbitrary Comparator to establish the ordering of the configuration sources to be applied.
  • let the configuration system allow to define the function f, which defines the mapping of configuration sources to a final configuration. This way any kind of mapping can be implemented by users and the system does not imply concepts, which may not be appropriate for some use cases.

Feel free to add your comments below.

Advertisements

About atsticks

Advanced Java Software Engineer and Architect.
This entry was posted in Architecture, Uncategorized and tagged . Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s