Composing Configuration Sources

Recently there were discussions regarding a new Java EE Configuration JSR. At JavaOne 2016 Oracle announced that they want to relaunch this JSR (first attempt was about 3 years ago, when I was working at Credit Suisse). Unfortunately  with latest reschedules and strategic decisions Oracle decided to postpone this initiative until Java EE 9. But the topic is an important cross-cutting concern, especially when running in distributed Cloud environments. Consequently discussions are ongoing, e.g. at microprofile.io. In this blog I would like to discuss my thaughts on how different configuration sources could be assembled into one uniform Configuration instance.

What is Configuration?

To keep things simple I assume Configuration to be something as simple as follows (using Java styled syntax):

public interface Configuration{
  String getProperty(String key);
  Map<String,String> getProperties();

  static Configuration getInstance(){
    ...
  }
}

Similarly I define a Configuration Source, which provides configuration entries as follows:

public interface ConfigSource{
  String getProperty(String key);
  Map<String,String> getProperties();
}

Composing Configuration from multiple Configuration Sources

In general composing of a Configuration is a function f that maps all known ConfigSources csx to a Configuration c:

f(cs1, cs2, cs3, …, csx) -> c

Most configuration frameworks currently known hereby simplify this function  by applying repeatedly a mapping function fm, which maps a initial value and a new value of property k repeatedly for pairs of configuration sources to a final value V. This can be written as follows:

V = fm(…fm(fm(fm(null, CV(cs1, k), CV(cs2, k),CV(cs3, k), …)

Despite Apache Tamaya, where fm can be configured (as so called PropertyValueCombinationPolicy), most frameworks use a fixed (overriding) mapping function, which can be defined as follows:

fm(v1, v2) ->  v2==null ? v1 : v2

There are use cases, where this strict kind of mapping function is not suitable (e.g. mapping of collection types and type safe configuration). This is worth to be discussed in a separate post. For now, given a mapping function fm, obvisously the only free dimension is the ordering of configuration sources. So the question is how the sources should be ordered before the evaluation functionality is applied. This sounds  easy, but in fact can get quite tricky, since the sources are not necessarily known at deploy time. This can be the case because:

  • The number of files in a config directory are different for different environments.
  • The configuration should be automatically composed based on what is accessible in the current classpath.
  • Configuration is evaluated from different class loaders with different configuration sources visible.

Using Numeric Values as Ordering Criteria

Many configuration systems solve this problem, by using some kind of a numeric value applied to each configuration source, called ordinal. So to a ConfigSource a method is added returning the ordinal as illsutrated below:

int getOrdinal();

Based on this ordinal value a configuration system can easily determine the order of ConfigSources. Advantages of ordinals hereby are

  • the concept is simple
  • it allows comparing of multiple property sources also from different sources/frameworks/modules
  • the ordinal value can be configured by adding config entries to owning configuration source
Nevertheless the concept has also its disadvantages, some of them are:
  • what should be done with multiple config sources that provide the same ordinal value?
  • what is the exact semanic of an ordinal. Is a config source with a ten times bigger ordinal, ten times more significant?
  • what to do if a an order based on ordinals is not sufficient for your requirements, meaning your configuration assembly policy requires a more complex ordering criteria?
  • how to add a config source between subsequent ordinals n and n+1?
  • when using builders to define your configuration, ordinals are confusing, since they are not used if a Configuration is built programmatically.

Given that let us discuss, if the concept of ordinal is really needed.

Using Meta-Configuration to define the Config Sources and their corresponding order

In the sandbox part of Apache Tamaya there is a metamodel module, which allows to define a configuration system using an XML formatted metaconfiguration file, similar to the one below (in fact there are many more features available ike immutability, property-source-level filtering, auto-refresh etc):

<configuration>
  <property-sources>
    <source type=”sys-properties” />
    <source type=”env-properties” />
    <source type=”etcd” />
 <property-sources>
</configuration>
This is very convenient, easy to understand and does not need any ordinal values at all, since the XML structure defines the ordering by the order of child elements of the property-sources element.

Using a Builder to define the Config Sources and their corresponding order

As part of Apache Tamaya core API it is possible to acquire a ConfigurationContextBuilder, which allows to configure the configuration system. The resulting ConfigurationContext can then be easily be converted into a Configuration instance:

ConfigurationContextBuilder builder = ConfigurationProvider
                                 .createConfigurationContextBuilder();
builder.addPropertySources(
   new SystemPropsPropertySource(),
   new EnvironmentPropsPropertySource(),
   new EtcdPropertySource(),
);
Configuration config = ConfigurationProvider.createConfiguration(builder.build());

Supporting Ordinals

If someone still wants to use an ordinal based approach, the ordinal can be accessed as a normal property of a ConfigSource:
String ordinal = propertySource.getProperty(“_ordinal”);

Since ordinals may not be the single and only property to define the ordering of ConfigSources, Tamaya’s ConfigurationContextBuilder supports passing a Comparator that has to be used for sorting of ConfigSources. As a consequence we must only add one line (highlighted in red) of code to emable ordinal based sorting:

ConfigurationContextBuilder builder = ConfigurationProvider
                                 .createConfigurationContextBuilder();
builder.addPropertySources(
   new SystemPropsPropertySource(),
   new EnvironmentPropsPropertySource(),
   new EtcdPropertySource(),
);
builder.sortPropertySources(
                   DefaultPropertySourceComparator.getInstance());
Configuration config = ConfigurationProvider.
                   createConfiguration(builder.build());
If we want to apply a custom ordering we simply pass our  custom Comparator instance:
ConfigurationContextBuilder builder = ConfigurationProvider
                                 .createConfigurationContextBuilder();
builder.addPropertySources(
   new SystemPropsPropertySource(),
   new EnvironmentPropsPropertySource(),
   new EtcdPropertySource(),
);
builder.sortPropertySources(MyCustomComparator.getInstance());
Configuration config = ConfigurationProvider.
                   createConfiguration(builder.build());
This way the API is simple, but still flexible and not pulluted by an ordinal concept.

Winding up…

Looking at all the disadvantages and especially at the function f, which renders a set of configuration sources into a Configuration, it is IMO not a good idea to use ordinals as part of the API of a configuration system. Instead
  • let the configuration system provide a Builder based approach to build a Configuration, which allows to define the ordering
    • freely by adding property sources programmatically in the desired order. The Builder must not change any order of the ConfigSources added.
    • by applying an arbitrary Comparator to establish the ordering of the configuration sources to be applied.
  • let the configuration system allow to define the function f, which defines the mapping of configuration sources to a final configuration. This way any kind of mapping can be implemented by users and the system does not imply concepts, which may not be appropriate for some use cases.

Feel free to add your comments below.

Posted in Architecture, Uncategorized | Tagged | Leave a comment

Embracing Microservices…

There are quite some discussions around micoservices and monoliths. Some people think microservices (MS) are a new version of SOA, others think MS are the only way to build modern software systems at all. What microservices are all about? Especially when looking at the reality in daily business there is in most cases no simple way to build your IT systems around microservices. But microservice can give you hints how can improve your IT landscape’s architecture in the future and I think many of the ideas are worth being embraced. But before doing so let us have a quick look at what microservices are. One definition I found quite handy is the one from  Martin Fowler , which I will try to shortly summarize here:

  • Componentization via Services: hereby a component can be defined as a unit of software that is independently replaceable and upgradeable. Services do more target the fact that we talk about out-of-process communications (in contrary to in-process communications as with libraries). Services also imply a clear API boundery because accessing internal state is basically not possible. This is basically very similar to what SOA tried to achieve, but in case of MS connectivity is achieved mainly using lean REST based communcation protocols or asynchronous messaging. As a result microservices enforece clear module bounderies, allowing us to indivually develop, deploy and scale things. We can have different teams and also easily different programming languages, libraries and even runtime containers. Nevertheless API design is still a crucial discipline. Badly designed APIs still leak into components.
  • Organized around Business Capabilities: IMO this is one of the most disrupting aspects when thinking of microservices. Basically microservices are designed around business capabilities. This is especially challenging when looking at traditional IT organizations, where UI development, middleware platforms, database and OS/network operations are separated into different teams (or even organizational units). Also think on Conway’s law, which says:

    Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.
    — Melvyn Conway, 1967

    As a consequence, for succeeding with microservices,  you might have to restructure your enterprise IT’s organization as well as parts of your business teams correspondingly.

  • Products not Projects: Also this is not matching traditional IT organizations. Their used to organize everything in projects, which are completed within some time and then abandoned. Teams leave the project when finished and operations are passed to some operations teams. Microservices, in contrary, are managed as products, typically with a long term lifecycle. As long as the corresponding business capability is required, there is a dedicated team dealing with it.
  • Smart endpoints and dumb pipes: Complexity should be located at the endpoints of communication. Communication between components is done directly without any intermediates. So there is no complex routing, no intermediary protocol and data conversion or similar happening between. Consequently there is no enterprise service bus middleware only dump pipes and queues.
  • Decentralized Governance: Many enterprise architects tend to standardize everything. Their idea is to avoid complexity. Unfortunately they achieve sometimes exactly the contrary. This is because Human beings are not robots, they think. And if standards do not match the problem, people will find ways around it. They will think inevitably on alternatives and they will implement them (but they will not tell you 😉 ). Also it is well known that influence is naturally getting weaker with incresed organizational and/or physical distance. So simply said centralized governance is not really matching with microservices (to some extend). With microservices you offer choice. Each microservice team can basically decide, which programming language, which database and which runtime platform they want to use.
  • Decentralized Data Management: As well as decentralizing governance and tooling, microservices also decentralize data models and storage decisions. Monolithic applications prefer typically a single logical database for persistant data.
  • Infrastructure Automation: Since with microservices we split systems along business capabilities, we end up with much more runtime components that interact with each other. Hereby components can hopefully, also due to its reduced complexity, be automatically tested, deployed and scaled. Fortunately experience of companies such as Google, Netflix and Amazon evolved into some common technologies that are implemeting the mechanisms needed. These technologies are preconditions before you should think on jumping the microservice train.
  • Design for failure: When splitting up systems into multiple independent services, we introduce network communications. Whereas in-process calls within a monolith rarely fail (excluded cases where the whole system goes down in one), network calls may fail, timeout, be slow, incomplete or even not returning at any time! These means system failures will inevitably occur! But this is nothing microservice specific. All modern distributed software systems nowadays are affected. Therefore every architect should really have a deep look into resilient system design and learn how to embrace failures (something I will blog probably in another post).
  • Evolutionary Design: the idea here is that the enforced coupling through services and APIs allows for easier evolution of components. The reduced complexity allows to deliver new features fast and in a reliable way.

Summarizing with microservices you get some awsome advantages:

  • you have a clear and enforceable component model
  • you have a reduced complexity because of clear component bounderies
  • your services are simpler, since they are dealing typically with only one concern
  • you have a clear mapping of business capabilities, services and teams
  • you can use the persistence technologies that best match your requirements
  • you can have independently operating teams (time-wise, geographic, organizational unit etc)
  • you can react quickly on changes and easily introduce new features and business capabilities

Looking at the majority of features above these are not new. These is what you get if you do componentization and isolation right. This is a good thing, because this means that you can achieve these things also without going the full microservice way.

But there are more features, which may not that easy to achieve:

  • you can independently and fine grained deploy, update and add new features (or fix bugs) to microservices
  • you can autoscale as needed, for each business capability
  • you have clear responsibilities for a products stability, you must cope with a real DevOps attitude
  • you get a system that (possibly) embraces failures

But also here looking at the bullets in detail you don’t have to write that logic completely on your own. Transparent service location is implemented by multiple OSS solutions ready for production. But my recommendation is to go one step further and use a common cloud PaaS platform that provides you automatic service location, scalability, isolation and DevOps functionality, such as automatic deployments, component monitoring etc.

Given that we may then as the ultimate step also consider aspects from resilient system design. Many of these aspects are also targeted by microservices. Nevertheless for your microservice landscape to be effectively resilient, you must also decouple communications. But also this is relatively easily achievable. You can use a platform that out-of-the-box comes with a resilient design, such as Typesafe’s Akka platform. Or you can add in and out message queues for your microservices as default offering into your PaaS solution and rewrite your services to use asynchronous communications instead of the synchronous one.

So this is the solution we are looking for since decades. Maybe. But…

  • your current IT and business organization may look completely different
  • development and operations are separated in different business organizations with competing objectives
  • your monoliths are lacking good encapsulation and API design
  • your technical dept is overwhelming
  • you probably lack of IT know how for dealing with MS, PaaS, IaaS etc.
  • your business capabilities are not well defined or distributed
  • you have huge databases with lots of business code operating on it
  • you do not have an efficient IT tooling in place
  • you have complex processes, budgeting issues or no high level management support for introducing microservices
  • you run lots of legacy host systems

The list could be continued. Summarizing I tend to say that in reality many (if not most) companies are simply not microservices-ready. Nevertheless, and this is my point, I think it is unevitable to adopt the ideas and principals of microservices in the long term:

  1. Educate your people how components should be designed. Enable them to define simple and comprehensive APIs (regardless of the language or technology). Teach them about dependencies, cohesion, coupling and reuse.
  2. Enable your developers to implement modular software components. Support them to use the platforms that matches best their problems. Don’t force them to use all the same technology, let them tailor their solutions with the requirements they must implement (I don’t say here they should not align with architecture).
  3. Enable IaaS and PaaS cloud services (separate discussions if you want to run your cloud in public, privately or hybrid). This also implies support for different runtime containers, e.g. in the Java World Jetty, Tomcat, a full blown application server, OSGI containers, Spring Boot, Wildfly Swarm, Typesafe Akka all are legit solutions (decide on a good selection to be offered). And most important, don’t try to implement all this yourself. Evaluate cloud solutions or container products and integrate it with your enterprise infrastructure (if your infrastructure lacks some needed capabilities, improve your infrastructure instead of reducing the service level of your IaaS, PaaS offering).
  4. Educate people to embrace failures. Start discussions also with your business stakeholders, where useful, how to deal with failures.

For IT departments thinking on how to improve their quality level and speed based on a cloud architecture is the first step to be taken. Cloud solutions mostly also ship with working deployment pipelines and other useful services, which were to be managed by companies individually for long times. All this comes with a high quality. This solutions enables your development departments to regain efficiency and to deliver new features at scale fast and in a unified way. And it allows you to improve your IT skills for building systems using resilient system design principles. But good news is you don’t need a big-bang approach. Add this kind of infrastructure to your IT portfolio and start gaining experience by migrating projects on it.

A working PaaS layer and automated build ans deployment processes are IMO preconditions to be met before you should start moving towards a product/business-capability/MS oriented operational model. And for sure it is a long term effort, requiring good planning, high management attention and long term budgeting. But looking at the technological evolution currently happening I think there is no way around because the efficiency gains of competitors following this path will sooner or later erase your business competitiveness, so start adopting things now!

If you have any kind of remarks, criticism or interesting links to add, use the comment function to share it with the community.

Posted in Architecture | Tagged , , , , , , , | 1 Comment