Sunday, September 28, 2014

Configuration for the Java Platform

Configuration for the Java Platform

Outcome from the last EC Meeting

Last week the EC meeting @Twitter HQ discussed a proposal for a new JSR for Configuration targeting application configuration. Since this topic affects not only Java EE, it was proposed as a SE JSR. Discussions were open and constructive. I try to summarize the outcome here:
  • One idea was, that we should do an OSS project out of it. I do not think this makes sense, there are already a few of them, all with reasonable maturity. So it should be finally time to standardize how things are configured in the Java eco-system.
  • Others said we should have a look at what was already done in the ME area. But also after having a look at JSR 36 (CDC 1.0) noch JSR 218 (CDC 1.1), I don't see anything relevant there. Perhaps someone can give me a hint here ;)
  • Others thought we should create an OpenJDK project. I tend also to disagree with that due to several reasons:
    • Developing a JSR that is targeting the JDK directly is much more complicated and my experience from JSR 354 shows that without having Oracle heavily pushing things it might be a high risk to go that way.
    • I think community involvement is much easier to achieve doing a "normal" JSR. We can setup a GitHub repo and start working on it. The OpenJDK tool chain is not that easy to use and know how about is not as widely available. Most of all it is much more difficult to get OpenJDK commit rights, than join a OSS project.
    • With Jigsaw coming hopefully in Java 9, I expect we get a module system. So it would be pretty easy to add configuration as a Java module once Jigsaw is there. This makes it much more easy again, to focus on the configuration usability aspects and realize Java integration as Jogsaw module later.
    • As well waiting for Jigsaw means that no SE JSR can be done for several year. IMO this makes no sense. Nevertheless I agree that the configuration mechanisms proposed must be compatible with Jigsaw.
    • Finally this JSR tries to focus on application configuration. This also includes relatively early configuration. But we definitively do not want to be part of the Java early loading code. We saw it quite good, how bad design can get and what kind of unwanted side effects can occur, when infrastructure required by low level Java boot processes is messed up with common user functionality. As an example try to load a class using Class.forName(String) during the loading process of the java.util.logging.LogManager (you have to subclass the LogManager and configure it as a system property).
  • Also Redhat, SouJava, LJC, iJUG and others were not seen to be sufficient that we should file the JSR already. There is some probability that it could be voted down, especially, when Oracle is not on board. We definitively have to talk with Oracle people here during Java One!
  • Finally feedback was that our JSR proposal should be more precise though no exact details were given, what should be fixed. Nevertheless have a look yourself at XXXX and help us to improve the filing.
So summarizing, adding configuration to the Java eco-system is definitively a Hercules task. OK, it was expected that it will not be easy ;) But good news is, we still we have a chance to get it on track, but we definitively need more support and work to be done. So if you are interested in the topic, write about it, tweet it, advertise it and help to improve our proposal. Communicate your interest, especially to EC members, that we have a unique chance here to improve the overall Java platform and we should not miss it. Especially support by more companies, including Oracle itself, in the EC would be important...
As always feedback is welcome and comments will not be restricted, also when you don't agree on my thoughts here.

Sunday, September 14, 2014

Integrating Configuration with CDI 2.0

Integrating Configuration with CDI

Most of you are aware that the original initiative of standardizing of configuration for Java EE deployment aspects was stopped lately. But looking at the topic of configuration we have multiple aspects:

  1. Deployment configuration in Java EE configuring a deployment into a Java EE server, including things like configuration of EJBs, transactions, datasources, servlets and filters etc. You can basically see this part as a replacement or improvement over the scattered file configurations as of today (web.xml, ejb.jar.xml, ...).
  2. Configuration of aspects that basically are not related to the Java EE platform, but related to the use cases and logic implemented. We call it application configuration. Since dependencies from a configuration mechanism should be minimized, IMO inversion of control is my preferred design concept to be used to "configure" a component, so CDI is the one that matches best.
  3. If we leave Java EE and imagine our solution built up on Spring, JavaFX, OSGI or other technologies, configuration gets even more generic. In many cases, though the existence of JSR 330, we can not rely on IoC/Dependency Injection to be present, so we need also to provide some kind of SE API, e.g. as outlined in previous blogs here.
Now given the current situation, I suggest point 1 is not in scope anymore for Java EE 8. Nevertheless there is still much we can do:
  • Leverage CDI so it is configuration aware
  • Standardize configuration as a global and unified concept in a SE JSR.
In this blog I would like to focus on what would be possible extensions to CDI to support application configuration effectively. 

Usage Perspective: Injecting Configured Values

As a starting point lets just inject String properties, e.g. as follows

public class MyBean {

  @Configured
  private String myProperty;

}

This is the simplest case, it will request for a configuration entry, with the same name as the field, so it looks for "myProperty". Now in many cases we probably want to have a different configuration key to be used for the lookup:

 @Configured("com.apple.myconfig.myProperty")
 private String myProperty;

Now what should happen, if configuration is not present? I suggest, without any default value, it should be managed like a deployment error. So we also want to be able to pass a default value here:

 @Configured(value="com.apple.myconfig.myProperty",
                     defaultValue="256")
 private String myProperty;

Obviously the configured value above is a number. So I would expect the runtime system is capable of injecting it also as number:

 @Configured(value="com.apple.myconfig.myProperty", 
                     defaultValue="256")
 private int myProperty;

Fine so far. Looking at Spring and other frameworks, you may also want to configure more complex things, e.g. injecting a Collection type:

 @Configured(value="com.apple.myconfig.myList", 
                     defaultValue="['a','b','c']")
 private List<String> MyList;

Since configuration is basically data added to a system externally it must also  be validatable, so we could add bean validation annotations on it:

  @Configured(value="com.apple.myconfig.myProperty", 
                     defaultValue="256")
 @Min(5)
 @Max(200)
 private int myProperty;

Basically looking at the code above it looks quite the same as what you can do with CDI already as of now. This makes sens since basically you simply do nothing different than injecting values. Similarly you want to separate different configurations in a system, may by qualifying them:

  @Configured(value="com.apple.myconfig.myProperty", 
                     defaultValue="256")
 @SystemConfig // Qualifier annotation
 private int myProperty

Finally default values also could support EL expressions:

 @Configured(value="com.myComp.stage", 
                     defaultValue="${environment.stage")
 @NotNull
 private Stage stage

Advanced Use Cases

Multiple Configuration Keys
In some cases there is a need to read out information from several configuration locations, hereby defining an order of precedence between several keys. That can be easily achieved by allowing an array of configuration keys to be defined within the annotation:

 public class MyBean {
  @Configured("myProperty", my.myconfig.myProperty")
  private int myProperty
  ...
 }

Configuration Root Keys
When organizing configuration in a hierarchical tree, a component typically reads several keys out of one sub-path, often called configuration area. Instead of having to retype the area all the time (which is error prone), it can be added as a default on class level, so the following example would lookup configuration with keys a.b.c.myProperty, a.b.c.num:

  @ConfigArea("a.b.c")
 public class MyBean {
  @Configured
  private int myProperty;
  @Configured("num")
  private int myProperty2;
  ...
 }

Configuration Filtering
Similarly you want to add sometimes a filter (operator) converting the configured String value to another String value (e.g. decrypting an encrypted password):

 @ConfiguredProperty(value={"users.admin.myPwd"},                      filter=foo.bar.MyPwdConverter.class)
 private String password;


Custom Adapter
Or you want to override/provide the converter to be used for type conversion from String to the required target type:

 @Configuredured(value={"users.admin.myAmount"},                      converter=foo.bar.MyAmountConverter.class)
 private MonetaryAmount amount;


Listening to Configuration Changes
Several times we want to be aware if configuration values change. Basically we could simply listen for PropertyChangeEvent instances: 

public class MyBean {

  @Configured
  private String myProperty;


  /**
   * Event sent, whenever a configured property is
   * changed (reinjected).
   * @param evt the event describing the changed property
   */
  void configChanged(@ConfigChange PropertyChangeEvent evt){
    ...
  }
}

This would allow to get informed on configuration changes easily, but would not imply any internal know how, how configuration is managed.

Alternate: Constraint Injected Types to simple Type only

The above though clearly separated has a couple of intersection with CDI. So it might be an option to reduce the configurable types to only the following:
  1. All primitive types such as boolean, byte, char, short, int, long, float, double.
  2. The corresponding wrapper types such as Boolean, Character, Byte, Short, Integer, Long, Float, Double, but also Number.
  3. Additionally: String, BigDecimal, BigInteger.
This would result in a fairly simple configuration mechanism. Additional features can then be added on top of that using CDI mechanisms, e.g. factory or producer classes that provide corresponding more complex instances that can be injected in other places:

 public class NameListProvider{
   @Configured
   private String configuredList;

   @Produces @NamedList @Dependent
   public List<String> getNamedList(){
      List<String> result = new ArrayList<>();
      parseList(result);
      return result;
   }
   
   private void parseList(List<String> list){ ... }
 }

A possible disadvantage of this solution is that the conversions are not inherently reusable and therefore similar conversions may be duplicated multiple times on a system. 

CDI Perspective: How can configuration features be realized

Basic Mechanism

As a starting point let us elaborate the runtime requirements configured beans/properties should have. I would suggest something like the following:
  1. Configured properties are based on String values only and managed by the configuration system only and not related to CDI.
  2. They can be converted to any non String based type, but conversion will be performed by the configuration subsystem. Especially configured objects do not support injection of any dependencies.
  3. Configured properties can be qualified. 
  4. Configured values are injected during the initialization of the CDI bean. It is the responsibility of the configuration system to determine the correct value, e.g. depenending on the current runtime context, or whatever is required to determine the correct configured value. The CDI container only provides the glue mechanism so configuration can be injected at the right locations.
  5. Any configured values are always injected with @Dependent scope. Adding any other scope annotation is handled as a deployment error.

Realization Variants

When we look at the possible implementation requirements and the possible way, how this can be implemented using CDI, there are different possible options, how concerns can be separated between CDI and a configuration subsystem:

Extending CDI
In this scenario CDI would be extended. meaning, that CDI itself defines the required annotations such as @Configured, @ConfigChange and also provides an SPI interface that can be registered (e.g. with java.util.ServiceLoader) for accessing configuration values. One important key aspect is that CDI should not know how configuration is managed. Basically configuration can be as simple as a .properties file in the classpath. In other cases it may be multilayered, dynamic system that even may run external to the current VM. So we need some very general abstraction to decouple CDI from this complexities. A minimalistic variant of such an SPI interface could look as follows:

 public interface ConfigAccessor{
   <T> T getConfiguredValue(Field f, Object instance,
                            Class<T> type,
                            Annotation... qualifiers,

                            ConfigChangeListener l);
   void addConfigChangeListener(ConfigChangeListener l);
   void removeConfigChangeListener(ConfigChangeListener l);
 }
Nevertheless this variant would have significant drawbacks:
  • It is questionable, if such an SPI should really be part of CDI.
  • Also CDI would probably not bring a configuration implementation with it (or only a very minimalistic one). 
An advantage of doing so, would be that the basic mapping between configuration and the CDI injection mechanisms is well defined. Nevertheless let's further see if we have other options.

Generalizing Producers
Looking at the examples we could also think about if the mechanisms shown could be generalized. Basically configuration injection is similar to satisfying dependencies using an external bean provider. Basically this what producers are built for. So basically we could also let CDI to the injection work...

 public class MyBean {

  @Inject @Configured
  private String myProperty;
  ...
 }

...and add an according producer bean for injecting configuration and hereby defining @Configured as CDI qualifier:


 @ApplicationScoped
 public class ConfigProducer{
   @Produces @Configured @Dependent
   public String getStringConfig(){...}
   @Produces @Configured @Dependent
   public Integer getStringConfig(){...}
   @Produces @Configured @Dependent
   public Boolean getStringConfig(){...}
   ...
 }
Nevertheless, if we could define that a producer will provide instances for any kind of injection targets with a certain qualifier, this would definitively allow us to inject all kind of configured values. To achieve this I added an additional flag (ProducerType) to the @Produces annotation to tell the CDI container that this producer should be used to delegate object production to any kind of target types qualified with @Configured :

 @ApplicationScoped
 public class ConfigProducer{

   @Produces(ProducerType.ALL@Configured @Dependent
   public Object getConfiguredValue(InjectionTarget<?> tgt){...}
 }

Clearly also this variant has drawbacks: 

  • It is not type-safe.
  • Producers may easily have intersections with other producers and more easily lead to deployment errors.

Using Portable Extensions
Finally simply implementing a portable extension is not only enough, it provides also the most portable and powerful mechanism. Even the current CDI 1.1 is fully sufficient to implement a configuration feature seamlessly:
  • The extension listens to the ProcessInjectionTarget event. This will be called for every bean.
  • The extension could then check for @Configured (or @ConfigChange) annotations and could adapt the InjectionTarget, so it can intercept the injection process, when the bean is initalized. This allows to populate the bean created with the configuration values as needed (as dependent scope).
  • Similarly the extension manages the injected beans (using weak references), which want to listen for configuration changes. Similarly if the underlying configuration changes, it can reinject the according properties and call according methods on the bean annotated with @ConfigChange to inform the bean about the configuration change.
  • If the configuration subsystem wants to publish further CDI events should be solely in the responsibility of the configuration system used.
As an additional advantage also CDI itself could be configured using the same extension mechanism (see below).

Configuring CDI

Going down the layers CDI itself should be made externally configurable. Hereby one may say we already have beans.xml and  annotations. The problem is that both of these concepts are bound to compile or package time mechanisms. A bundled ear or war archive cannot be changed without opening it, changing it and recreating it with changes included. But this is a thing configuration exactly wants to prevent. Configuration therefore may be provided in a way and with mechanisms that is not controlled by the component to be configured. As a consequence we need a dynamic way for connecting configuration logic with the CDI subsystem that is capable of
  • be available and accessible during deployment/initialization time of the CDI container.
  • be dynamic, so depending on the current valid configuration for a given setup the right values are returned.
Basically this can be achieved most easily by an service provider interface (SPI) defined by CDI. E.g. CDI could add a CDIConfiguration interface as illustrated below that can be configured using the java.util.ServiceLoader.

 public interface CDIConfiguration{
   Collection<Class<? extends Extension>> getExtensions();
   Collection<Class<?>> getAlternativeBeans();
   Collection<Class<?>> getAlternativeStereotypes();
   List<Class<?>> getInterceptors();
   List<Class<?>> getDecorators();
   Collection<String> getScanExcludes();
 }

The problem with this proposal is that adding/enabling things is easy, but disabling things is more complex. But wait! Fortunately CDI comes with a much more powerful concept: CDI extensions, which gives you full access for modifying the CDI metamodel. With that we can
  • Register beans or deadactivate (veto) beans and alternatives.
  • Add, Adapt, Remove injection points
  • Adding or removing interceptors, decorators.
  • Add/remove interceptors, decorators on individual beans
  • ...
So basically CDI extensions already enables us to control CDI. The only thing missing is to define a configuration model, that reflects the different things we want to made configurable. Since we are using a simple Map<String,String> based model, we could e.g. define the following:

// Adding/defining things
javax.enterpise.inject.beans.alternativeClasses=class1,class2,class3
javax.enterpise.inject.beans.alternativeStereotypes=class1,class2,class3
javax.enterpise.inject.beans.interceptors=class1,class2,class3
javax.enterpise.inject.beans.decorators=class1,class2,class3
javax.enterpise.inject.beans=class1,class2,class3
javax.enterpise.inject.beans.scanExcludes=path1;path2
javax.enterpise.inject.extensions=class1,class2,class3

// Removing/deactivating things
javax.enterpise.inject.beans.alternativeClasses.vetoed=class1,class2,class3
javax.enterpise.inject.beans.alternativeStereotypes.vetoed=class1,class2,class3
javax.enterpise.inject.beans.interceptors.vetoed=class1,class2,class3
javax.enterpise.inject.beans.decorators.vetoed=class1,class2,class3
javax.enterpise.inject.beans.vetoed=class1,class2,class3
javax.enterpise.inject.beans.scanExcludes.vetoed=path1;path2
javax.enterpise.inject.extensions.vetoed=class1,class2,class3

This configuration basically would enable to configure everything you can do with beans.xml. But we can also think of additional features, e.g.:

// Adding interceptors to a bean
javax.enterpise.inject.bean:myBeanClass.intercetors=class1

// Adding interceptors to a method on a bean
javax.enterpise.inject.bean:myBeanClass#myMethodName(class).intercetors=class1

// Remove an interceptors from a bean
javax.enterpise.inject.bean:myBeanClass.intercetors.vetoed=class1

// Remove an interceptor from a method on a bean
javax.enterpise.inject.bean:myBeanClass#myMethodName(class).intercetors.vetoed=class1

...

Outlook

The most important outcome for me is that adding application configuration support to CDI does not require CDI to be adapted in any way. CDI already as of now is flexible enough so it can fully support configuration injection, including advanced use cases. This is basically a consequence of the very powerful SPI provided by CDI that allows to adapt the metamodel in a very flexible way. Unfortunately, given the EE configuration initiative seems to be stuck there is only little probability that also all other EE JSRs follow this example and provide similar SPIs, so they can be easily configured without the deployment of xml deployment descriptors. Nevertheless if they do (or would be accordingly enhanced during EE8 work, we could add configuration support in wide areas nevertheless.

As a consequence a standardization initiative as SE standalone JSR would probably be a good variant for starting defining configuration aspects in Java and therefore make corresponding code more interoperable. Such a SE JSR could start very small in a first release, so it could be finished as well within the EE8 timeline. That would enable applications not only to benefit from EE8 in a couple of years, but also having a configuration mechanism in place that is powerful enough to cover many of the aspects in daily live, except those of deployment configuration of the application servers. Such a JSR could define a simple APIs covering the following
  • an API that models configuration and the current runtime environment (util.preferences is not sufficient!)
  • listeners for listening to config changes
  • accessors to access configuration
  • utilities to work buildup configurations
  • optionally also adapting of Strings to objects
  • some functional extension points (operators, queries)
  • an SE based SPI
Such a configuration implementation then could be easily hooked into CDI and all other JSRs as they provide configurable meta-models. thus covering most of the configuration requirements.
More advanced concepts like configuration meta-modelling, building configurations from multiple trees, merging, filtering, security, views and templates would probably be postponed to a later version of such a spec. Nevertheless the reference implementation could provide some of these features, even as pluggable modules, and best practice will show, which of them will qualify to be further standardized at a later point.

Everybody is welcome to comment on this blog. Especially comments and ideas on how specification of configuration aspects should evolve would be very useful.

Wednesday, September 3, 2014

No Java EE Configuration JSR for EE 8

No Java EE Configuration for EE8

Dear all

I apologize to have really disappointing news about the configuration JSR. Though we had basically setup everything, and I was really working hard to get the config JSR up and running, no suitable agreement on the long term support for the TCK/RI could finally be found. Also my last trials to find alternative ways/partners was not successful. Since Credit Suisse is not able/willing to lead such an important JSR on its own, I must inform you that, if not something significant happens, there will be NO CONFIGURATION JSR. 

I am personally very disappointed on that and also think that a good chance to improve our Java EE eco-system will be missed.

I will still try to push forward the topic, but since my current employer does not support OSS projects outside of the JCP, I will not be able to contribute any code in the next time.

If you have any questions you still may get in contact with me, or talk to me at JavaOne.

Cheers,
Anatole