Published 2014-03-21.
Last modified 2018-04-09.
Time to read: 9 minutes.
HOCON configuration is a flexible and standards-compliant way to provide configuration data to any Java or Scala application. This lecture will be of interest if you intend to work with any JVM-based application that has configuration data.
The sample code for this lecture can be found in
courseNotes/.
The HOCON Config utility
is a flexible and standards-compliant way to provide configuration data to any Java or Scala application.
The utility classes are written in Java and the
JavaDoc is here.
This Java-based configuration utility is commonly used for specifying and parsing Scala configuration data.
Config can read Java properties, YAML, JSON, and a human-friendly JSON superset called HOCON.
Config does not currently have a standardized Scala wrapper, although some people have
created their own.
From the point of view of your application, the file format of your configuration data makes no difference. What’s more, if a combination of data file formats is used, they can easily be merged without regard to their format on disk.
We will use the sbt console command below instead of the Scala REPL because we want to
be able to access the configuration data files on the application classpath.
Working with Java Properties Files
A Java properties file called
courseNotes/
has been provided in the courseNotes project.
It looks like this:
string1=Hello from demo.properties int1=42 double1=123.45 elapsedTime=1 day bytes1=2K bytes2=3G
We can use Config from the SBT console to obtain strongly typed configuration data from this file.
First we start the sbt console and import ConfigFactory:
$ sbt console Loading /usr/share/sbt/bin/sbt-launch-lib.bash [info] Loading global plugins from /home/mslinn/.sbt/0.13/plugins [info] Loading project definition from /home/mslinn/work/course_scala_intermediate_code/courseNotes/project [info] Updating {file:/home/mslinn/work/course_scala_intermediate_code/courseNotes/project/}coursenotes-build... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Set current project to IntermediateScalaCourse (in build file:/home/mslinn/work/course_scala_intermediate_code/courseNotes/) import java.io.File import java.net.URL import scala.sys.process._ Welcome to Scala version 2.11.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_60). Type in expressions to have them evaluated. Type :help for more information.
scala> import com.typesafe.config.ConfigFactory import com.typesafe.config.ConfigFactory
Next we load demo.properties from the classpath and obtain a Java
com. object; this object will contain all the values stored in the file.
Notice that I followed the invocation of parseResources with
resolve.
You should get in the habit of doing that because the resolve method performs variable evaluation;
if you do not call resolve, and there are variables to resolve in the configuration, the values returned will be incorrect.
scala> val confDemo = ConfigFactory.parseResources( "demo.properties ") .resolve confDemo: com.typesafe.config.Config = Config(SimpleConfigObject({ "bytes1 ": "2K ", "bytes2 ": "3G ", "double1 ": "123.45 ", "elapsedTime ": "1 day ", "int1 ": "42 ", "string1 ": "Hello from demo.properties "}))
Typed Conversions
You can obtain a property’s value after converting to a desired type using the appropriate getter. An exception is thrown if the conversion fails.
scala> val string1 = confDemo.getString( "string1 ") string1: String = Hello from demo.properties
scala> val int1 = confDemo.getInt( "int1 ") int1: Int = 42
scala> val string = confDemo.getString( "int1 ") string: String = 42
scala> val double1 = confDemo.getDouble( "double1 ") double1: Double = 123.45
Durations
Config can also perform time conversions while parsing using
java.util.concurrent.TimeUnit.
scala> import java.util.concurrent.TimeUnit"._ import java.util.concurrent.TimeUnit._
scala> val elapsedDays = confDemo.getDuration( "elapsedTime ", DAYS) elapsedDays: Long = 1
scala> val elapsedHours = confDemo.getDuration( "elapsedTime ", HOURS) elapsedHours: Long = 24
scala> val elapsedSeconds = confDemo.getDuration( "elapsedTime ", SECONDS) elapsedSeconds: Long = 86400
Size Conversions
Byte conversions are also supported from various size abbreviations, including K (kilo),
M (mega), G (giga), T(tera), P (peta), E (exa),
Z (zetta) and Y (yotta).
scala> val bytes1 = confDemo.getBytes( "bytes1 ") bytes1: Long = 2048
scala> val bytes2 = confDemo.getBytes( "bytes2 ") bytes2: Long = 3221225472
An exception is thrown if you use an abbreviation that results in Long overflow,
such as P (peta), E (exa), Z (zetta) or Y (yotta).
Stacking Properties Files Using withFallback
Multiple withFallback invocations can be chained together so the Config files are stacked, or layered.
Config will load them all from right to left, overwriting keys with values as they are encountered.
Although I originally drew this diagram to describe Akka configuration, it can equally well apply to your program.
The conventions used by Akka and Play are the same: a file called reference.conf embedded in the
framework to provide default values, and you can provide a file called application.conf to override selected values.
You could take this convention a couple of steps further, whereby each library you write provides its own library.conf
to set default configuration values for that library, and you also override specific values in test suites with a
Config set from a String.
One of the exercises in this lecture will provide you with a practical example of stacking Configs.
I provided a second Java properties file,
courseNotes/.
This file contains some duplicate keys that are also found in demo.properties.
string1=Hello from demo2.properties int1=13 double2=999.99
Config can overlay the two files by chaining parseResourses.withFallback.
This causes demo2.properties to define the key/value pairs; any keys defined in demo.properties that are not also specified in demo2.properties will be made available.
Again, I call resolve to ensure that any variables have values substituted for references.
scala> val confDemo2 = ConfigFactory.parseResources( "demo2.properties ") .withFallback(confDemo) .resolve confDemo2: com.typesafe.config.Config = Config(SimpleConfigObject({ "bytes1 ": "2K ", "bytes2 ": "3G ", "double1 ": "123.45 ", "double2 ": "999.99 ", "elapsedTime ": "1 day ", "int1 ": "13 ", "string1 ": "Hello from demo2.properties "}))
scala> val string1b = confDemo2.getString( "string1 ") string1b: String = Hello from demo2.properties
scala> val int1b = confDemo2.getInt( "int1 ") int1b: Int = 13
scala> val double1b = confDemo2.getDouble( "double1 ") double1b: Double = 123.45
scala> val double2 = confDemo2.getDouble( "double2 ") double2: Double = 999.99
Reading JSON Files
The JSON file
courseNotes/
has been provided and looks like this:
{
"firstName ": "Jane ",
"lastName ": "Smith ",
"isAlive ": true,
"age ": 25,
"height_cm ": 147.64,
"address ": {
"streetAddress ": "21 2nd Street ",
"city ": "New York ",
"state ": "NY ",
"postalCode ": "10021-3100 "
},
"phoneNumbers ": [
{ "type ": "home ", "number ": "212 555-1234 " },
{ "type ": "office ", "number ": "646 555-4567 " }
]
}
Config can read this file using the same methods as was used for Java properties.
Notice that demo.json is parsed into a Config object,
and the combined contents of the two Java properties files are also added to the resulting Config object.
scala> val confDemo3 = ConfigFactory.parseResources( "demo.json ") .withFallback(confDemo2) confDemo3: com.typesafe.config.Config = Config(SimpleConfigObject({ "address ":{ "city ": "New York ", "postalCode ": "10021-3100 ", "state ": "NY ", "streetAddress ": "21 2nd Street "}, "age ":25, "bytes1 ": "2K ", "bytes2 ": "3G ", "double1 ": "123.45 ", "double2 ": "999.99 ", "elapsedTime ": "1 day ", "firstName ": "Jane ", "height_cm ":147.64, "int1 ": "13 ", "isAlive ":true, "lastName ": "Smith ", "phoneNumbers ":[{ "number ": "212 555-1234 ", "type ": "home "},{ "number ": "646 555-4567 ", "type ": "office "}], "string1 ": "Hello from demo2.properties "}))
We can see that all of the key/value pairs from the two Java properties files are available.
scala> val string1c = confDemo3.getString( "string1 ") string1c: String = Hello from demo2.properties
scala> val int1c = confDemo3.getInt( "int1 ") int1c: Int = 13
scala> val double1c = confDemo3.getDouble( "double1 ") double1c: Double = 123.45
scala> val double2b = confDemo3.getDouble( "double2 ") double2b: Double = 999.99
We can also access key/value pairs from the JSON file.
scala> val firstName = confDemo3.getString( "firstName ") firstName: String = Jane
scala> val isAlive = confDemo3.getString( "isAlive ") isAlive: String = true
scala> val height_cm = confDemo3.getString( "height_cm ") height_cm: String = 147.64
JSON is a more flexible format than Java properties files because it can store hierarchical data.
Config can access the hierarchy passing a path to a Config object.
Here we see the paths address. and address.
used to extract the value of the streetAddress and city keys from the address JSON object.
scala> val streetAddress = confDemo3.getString( "address.streetAddress ") streetAddress: String = 21 2nd Street
scala> val city = confDemo3.getString( "address.city ") city: String = New York
JSON supports lists and objects, which are not possible with Java properties files.
Config can handle these types easily.
Config. can parse a JSON list, and returns a ConfigList,
and key/value pairs in a ConfigList can be parsed using ConfigList..
scala> val phoneNumbers: ConfigList = confDemo3.getList( "phoneNumbers ") phoneNumbers: com.typesafe.config.ConfigList = SimpleConfigList([{ "number ": "212 555-1234 ", "type ": "home "},{ "number ": "646 555-4567 ", "type ": "office "}])
scala> val phoneType = phoneNumbers.atKey( "type ") phoneType: com.typesafe.config.Config = Config(SimpleConfigObject({ "type ":[{ "number ": "212 555-1234 ", "type ": "home "},{ "number ": "646 555-4567 ", "type ": "office "}]}))
scala> val phoneNumber = phoneNumbers.atKey( "number ") phoneNumber: com.typesafe.config.Config = Config(SimpleConfigObject({ "number ":[{ "number ": "212 555-1234 ", "type ": "home "},{ "number ": "646 555-4567 ", "type ": "office "}]}))
Config.getObject can parse a JSON object, and returns a ConfigObject.
ConfigObject. converts the ConfigObject to a Config object,
from which key/value pairs can be extracted as previously shown.
scala> val addresses: Config = confDemo3.getObject( "address ").toConfig addresses: com.typesafe.config.Config = Config({ "city ": "New York ", "postalCode ": "10021-3100 ", "state ": "NY ", "streetAddress ": "21 2nd Street "})
scala> val streetAddress2 = addresses.getString( "streetAddress ") streetAddress2: String = 21 2nd Street
scala> val city2 = addresses.getString( "city ") city2: String = New York
Config does a pretty good job of parsing JSON, but because it is Java-based it cannot convert JSON objects into Scala case classes.
If you want to do that, you should consider more capable JSON parsers for Scala such as
circe,
JSON4S and
others.
Generating JSON
Lets convert the configuration data stored in demo.properties into JSON.
First I’ll load just that file into a new Config instance.
scala> import com.typesafe.config._ import com.typesafe.config._
scala> val confDemo: Config = ConfigFactory.parseResources( "demo.properties ") confDemo: com.typesafe.config.Config = Config(SimpleConfigObject({ "bytes1 ": "2K ", "bytes2 ": "3G ", "double1 ": "123.45 ", "elapsedTime ": "1 day ", "int1 ": "42 ", "string1 ": "Hello from demo.properties "}))
ConfigValue is a Java interface implemented by
ConfigList and
ConfigObject..
A ConfigValue instance can be obtained from a suitable Config object’s root property as follows:
scala> confDemo.root res0: com.typesafe.config.ConfigObject = SimpleConfigObject({ "bytes1 ": "2K ", "bytes2 ": "3G ", "double1 ": "123.45 ", "elapsedTime ": "1 day ", "int1 ": "42 ", "string1 ": "Hello from demo.properties "})
ConfigValue instances can be converted into JSON by invoking their render method.
Before we do that, however, we need to specify the correct
ConfigRenderOptions so the ConfigValue.render method generates the desired output.
Those options can be set by chaining method calls.
Typical options for generating JSON are
concise
(which removes whitespace and comments),
setJson(true) and
setFormatted(true).
In this example, I’m storing the options in a variable called options.
scala> val options = ConfigRenderOptions.concise .setJson(true) .setFormatted(true) options: com.typesafe.config.ConfigRenderOptions = ConfigRenderOptions(formatted,json)
We can generate formatted JSON from the configuration object:
scala> confDemo.root .render(options) res1: String = "{ "bytes1 " : "2K ", "bytes2 " : "3G ", "double1 " : "123.45 ", "elapsedTime " : "1 day ", "int1 " : "42 ", "string1 " : "Hello from demo.properties " } "
Unformatted JSON can be generated by calling setFormatted(false).
scala> val options = ConfigRenderOptions.concise.setJson(true).setFormatted(false) options: com.typesafe.config.ConfigRenderOptions = ConfigRenderOptions(json)
scala> confDemo.root.render(options) res2: String = { "bytes1 ": "2K ", "bytes2 ": "3G ", "double1 ": "123.45 ", "elapsedTime ": "1 day ", "int1 ": "42 ", "string1 ": "Hello from demo.properties "}
Working with HOCON Files
The HOCON format is just an incremental improvement on JSON. It is also the standard configuration file format for Akka and Play. The HOCON syntax is more flexible and included files are supported.
So far we have used Config.parseResources to load the contents of a configuration file from the classpath.
The Config.load method works just the same, and also provides default values for standard Akka and Play configuration parameters.
By default, Config.load looks for a file called application.conf on the classpath.
The courseNotes project contains a file called
courseNotes/, which looks like this:
include "custom.conf "
aws { accessKey = "applicationAccessKey " accessKey = ${?ACCESS_KEY}
secretKey = "applicationSecretKey " secretKey = ${?SECRET_KEY} }
courseNotes/
is included by application.conf, and it looks like this:
keyName1 : "Value 1 "
keyName2 = true
nested.key.name3 : 5
nested {
key {
name4 = {
"one " : 10,
"two " : 12
}
}
}
blarg = "default blarg "
blarg = ${?BLARG}
Given the above configuration files, we could read the values they contain as follows.
$ sbt console Loading /usr/share/sbt/bin/sbt-launch-lib.bash [info] Loading global plugins from /home/mslinn/.sbt/0.13/plugins [info] Loading project definition from /home/mslinn/work/course_scala_intermediate_code/courseNotes/project [info] Set current project to IntermediateScalaCourse (in build file:/home/mslinn/work/course_scala_intermediate_code/courseNotes/) import java.io.File import java.net.URL import scala.sys.process._ Welcome to Scala version 2.11.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_60). Type in expressions to have them evaluated. Type :help for more information.
scala> import com.typesafe.config.ConfigFactory import com.typesafe.config.ConfigFactory
scala> val conf = ConfigFactory.load ...
Notice that ConfigFactory.load pulls in a lot more information than was provided in the configuration file;
the extra information is the default Akka configuration data I just mentioned.
If you are not working with Akka or Play Framework, then you probably should use
ConfigFactory.parseResources as shown above instead.
Now we can extract strongly typed values from the configuration as before.
scala> val value1 = conf.getString( "keyName1 ") value1: String = Value 1
scala> val value2 = conf.getBoolean( "keyName2 ") value2: Boolean = true
Again, we can specify the value of an inner object property using dotted syntax.
scala> val value3 = conf.getInt( "nested.key.name3 ") value3: Int = 5
We can also obtain a subset of the configuration data using getConfig,
and then refer to inner properties without having to qualify them using dotted syntax.
scala> val confNested = conf.getConfig( "nested.key.name4 ") confNested: com.typesafe.config.Config = Config(SimpleConfigObject({ "one ":10, "two ":12}))
scala> val value5 = confNested.getInt( "one ") value5: Int = 10
scala> val value6 = confNested.getInt( "two ") value6: Int = 12
Creating Config Objects from Strings
ConfigFactory.parseString can create a Config object by reading configuration values from
a String.
This is useful for specifying defaults and for testing.
The contents of custom.conf could be expressed as a String this way.
scala> val string = " " " keyName1 : "Value 1 " keyName2 = true nested.key.name3 : 5 nested { key { name4 = { "one " : 10, "two " : 12 } } } array : [ 1, 2, 3] " " " string: String = " keyName1 : "Value 1 " keyName2 = true nested.key.name3 : 5 nested { key { name4 = { "one " : 10, "two " : 12 } } } array : [ 1, 2, 3] "
The values can be extracted from the resulting Config object just as if they had originated from a .conf file.
scala> val conf2 = ConfigFactory.parseString(string) conf2: com.typesafe.config.Config = Config(SimpleConfigObject({ "array ":[1,2,3], "keyName1 ": "Value 1 ", "keyName2 ":true, "nested ":{ "key ":{ "name3 ":5, "name4 ":{ "one ":10, "two ":12}}}}))
scala> val value1b = conf2.getString( "keyName1 ") value1b: String = Value 1
scala> val value2b = conf2.getBoolean( "keyName2 ") value2b: Boolean = true
scala> val value3b = conf2.getInt( "nested.key.name3 ") value3b: Int = 5
scala> val confNested2 = conf2.getConfig( "nested.key.name4 ") confNested2: com.typesafe.config.Config = Config(SimpleConfigObject({ "one ":10, "two ":12}))
scala> val value5b = confNested2.getInt( "one ") value5b: Int = 10
scala> val value6b = confNested2.getInt( "two ") value6b: Int = 12
scala> val array = conf2.getIntList( "array ") array: java.util.List[Integer] = [1, 2, 3]
Arrays of Properties
There is a special syntax for concatenating arrays in a .conf file.
This example is taken from the Cadenza Play Framework webapp’s unit testing setup
(Cadenza powered the original ScalaCourses webapp).
In that unit test setup, a .config file sets an array of values called play.,
then includes another .config file.
Here is an excerpt from the first .config file.
play {
modules {
enabled = ${?play.modules.enabled} [
"modules.cadenza.CadenzaAuthViewsModule ",
"modules.cadenza.CadenzaModule "
]
disabled = []
}
}
include "silhouette.application.conf "
Notice that the assignments to play. starts with the incantation
${?play.modules.enabled}.
This concatenates the previous value of play. with the value in square brackets.
Because there was no previous definition for play., just the value provided in square brackets is assigned to play..
The included .config file (silhouette.application.conf) looks like this:
play {
modules {
enabled = ${?play.modules.enabled} [
"com.mohiva.play.silhouette.api.actions.SecuredActionModule ",
"com.mohiva.play.silhouette.api.actions.UnsecuredActionModule ",
"com.mohiva.play.silhouette.api.actions.UserAwareActionModule ",
"modules.silhouette.BaseModule ",
"modules.silhouette.SilhouetteModule ",
"play.api.libs.openid.OpenIDModule ",
"play.api.libs.ws.ahc.AhcWSModule ",
"play.api.libs.mailer.MailerModule ",
"play.api.libs.mailer.SMTPConfigurationModule ",
"play.filters.csrf.CSRFModule "
]
}
}
Now that play.modules.enabled has a value,
that value is concatenated with the value within square brackets.
Notice that the reference to the previous value had to be fully qualified:
${?play.modules.enabled}; merely specifying ${?enabled}
does not work.
The resulting value for the play. object becomes.
play {
modules {
enabled = [
"modules.cadenza.CadenzaAuthViewsModule ",
"modules.cadenza.CadenzaModule ",
"com.mohiva.play.silhouette.api.actions.SecuredActionModule ",
"com.mohiva.play.silhouette.api.actions.UnsecuredActionModule ",
"com.mohiva.play.silhouette.api.actions.UserAwareActionModule ",
"modules.silhouette.BaseModule ",
"modules.silhouette.SilhouetteModule ",
"play.api.libs.openid.OpenIDModule ",
"play.api.libs.ws.ahc.AhcWSModule ",
"play.api.libs.mailer.MailerModule ",
"play.api.libs.mailer.SMTPConfigurationModule ",
"play.filters.csrf.CSRFModule "
]
disabled = []
}
}
Overriding with Environment and Java System Variables
HOCON files can allow for overrides from environment variables so sensitive data can be provided at runtime instead
being committed to a code base.
The syntax is peculiar, but works quite well.
The file src/main/resources/override.conf looks like this.
blarg = "default blarg "
blarg = ${?BLARG}
The value of blarg will depend on whether an environment variable or Java System variable called BLARG is defined.
If not, the value default blarg will be used.
Before you can access the values from your code you must call resolve.
scala> sbt console ...usual output not shown ... scala> import com.typesafe.config.{ConfigObject, ConfigList, ConfigFactory} import com.typesafe.config.{ConfigObject, ConfigList, ConfigFactory}
scala> val confOverride = ConfigFactory.parseResources( "override.conf ").resolve confOverride: com.typesafe.config.Config = Config(SimpleConfigObject({ "blarg ": "default blarg "}))
scala> val blarg = confOverride.getString( "blarg ") blarg: String = default blarg
Now lets set some environment variables and restart sbt console.
scala> export BLARG= "blarg set from environment variable " scala> sbt console ...usual output not shown... scala> import com.typesafe.config.{ConfigObject, ConfigList, ConfigFactory} import com.typesafe.config.{ConfigObject, ConfigList, ConfigFactory}
scala> val confOverride = ConfigFactory.parseResources( "override.conf ").resolve confOverride: com.typesafe.config.Config = Config(SimpleConfigObject({ "blarg ": "blarg set from environment variable ", "item ": "Item set from environment variable "}))
scala> val blarg = confOverride.getString( "blarg ") blarg: String = blarg set from environment variable
The Config documentation also states that Java system variables can be used to pass configuration data. This is true, however if your program uses SBT or Play to launch you will likely have a very difficult time. I recommend you avoid doing this.
Scala Compatibility
Because Config was written in Java, the collections it provides don’t have the handy Scala behavior.
As we learned in the To Converters lecture,
you can enhance the behavior by importing JavaConverters and using the asScala method.
scala> import com.typesafe.config.{ConfigObject, ConfigList, ConfigFactory} import com.typesafe.config.{ConfigObject, ConfigList, ConfigFactory}
scala> import collection.JavaConverters._ import collection.JavaConverters._
scala> val conf2 = ConfigFactory.parseResources( "custom.conf ") conf2: com.typesafe.config.Config = Config(SimpleConfigObject({ "keyName1 ": "Value 1 ", "keyName2 ":true, "nested ":{ "key ":{ "name3 ":5, "name4 ":{ "one ":10, "two ":12}}}}))
scala> val keys = conf2.entrySet.asScala.map(_.getKey) keys: scala.collection.mutable.Set[String] = Set(array, nested.key.name4.one, keyName1, keyName2, nested.key.name4.two, nested.key.name3)
scala> val keyName1 = conf2.getString( "keyName1 ") keyName1: String = Value 1
scala> val confNested3 = conf2.getConfig( "nested ") configNested3: com.typesafe.config.Config = Config(SimpleConfigObject({ "key ":{ "name3 ":5, "name4 ":{ "two ":12, "one ":10}}}))
scala> confNested3.entrySet.asScala.map(_.getKey) res9: scala.collection.mutable.Set[String] = Set(key.name4.two, key.name4.one, key.name3)
scala> confNested3.getString( "key.name3 ") res10: String = 5
PureConfig
Several Scala wrappers for Config exist, including PureConfig,
ScalaConfig, StaticConfig,
ValidatedConfig,
CediConfig and
kxbmap.
I like PureConfig best, and we’ll discuss a short demo program now.
The PureConfig documentation is terse, so I’ll try to give more information. There is also some useful documentation hidden in the docs directory of the project on GitHub.
The dependency is declared in build.sbt like this (we discussed how to declare dependencies in the
SBT Project Setup lecture of the
Introduction to Scala course).
"com.github.pureconfig" %% "pureconfig" % "0.7.2" withSources()
PureConfig requires that a case class be declared that matches the content of the HOCON configuration file.
This allows the case class to automatically be populated by the information from the configuration.
I like this idea, because it ends up simplifying your code and makes it much easier to maintain.
This also means you don’t need to write a lot of low-level code that calls getXXX
methods to read values from the underlying Config object.
What’s more, PureConfig has an internal cache, so it can be repeatedly queried efficiently.
The cache is cleared before each call to pureconfig.load and pureconfig.loadOrThrow.
PureConfig also works best if the case class is strongly typed; in other words,
instead of declaring the properties to be String and Int, etc,
define highly specific types.
Use value classes whenever possible, because they can provide type safety without runtime overhead.
Value classes were discussed in the Implicit Values lecture.
Each value class will need an implicit converter, and they are simple to write.
More recent versions of PureConfig don’t work well with value classes, so some experimentation might be required to see what works.
PureConfig works well with properties that themselves are case classes. This lets you easily define a complex structure and populate it from a HOCON configuration file.
Here is an example of a HOCON file, provided as
courseNotes/.
ew {
console {
enabled = true
enabled = ${?EW_CONSOLE_ENABLED}
}
feed {
port = 9090
port = ${?EW_FEED_PORT}
}
repl {
home = "~ "
home = ${?EW_REPL_HOME}
}
speciesDefaults {
attributeMinimum = 0
attributeMinimum = ${?EW_ATTRIBUTE_MINIMUM}
attributeMaximum = 100
attributeMaximum = ${?EW_ATTRIBUTE_MAXIMUM}
eventQLength = 20
eventQLength = ${?EW_EVENT_QUEUE_LENGTH}
historyLength = 20
historyLength = ${?EW_HISTORY_LENGTH}
}
sshServer {
address = localhost
address = ${?EW_SSH_SERVER_ADDRESS}
ammoniteHome = ~
ammoniteHome = ${?EW_SSH_SERVER_HOME}
enabled = true
enabled = ${?EW_SSH_SERVER_ENABLED}
hostKeyFile = ${?EW_SSH_SERVER_HOST_KEY_FILE}
password = " "
password = ${?EW_SSH_SERVER_PASSWORD}
port = 1101
port = ${?EW_SSH_SERVER_PORT}
userName = "repl "
userName = ${?EW_SSH_SERVER_USER_NAME}
}
}
The case class that will be populated by pure.config is
courseNotes/
Notice that all of its properties are strongly typed, and that default values are provided.
import java.nio.file.{Path, Paths}
import PureConfigFun._
case class PureConfigFun(
console: ConsoleConfig = defaultConsoleConfig,
feed: FeedConfig = defaultFeedConfig,
repl: ReplConfig = defaultReplConfig,
speciesDefaults: SpeciesDefaults = defaultSpeciesConfig,
sshServer: SshServer = defaultSshServerConfig
)
The companion object has several moving parts, which we’ll look at individually. The general outline is.
object PureConfigFun {
import pureconfig.{CamelCase, ConfigConvert, ConfigFieldMapping, ProductHint}
import pureconfig.error.{CannotConvert, ConfigReaderFailures, ConfigValueLocation}
import pureconfig.ConfigConvert._
import com.typesafe.config.{ConfigValue, ConfigValueFactory, ConfigValueType}
val defaultConsoleConfig = ConsoleConfig()
val defaultFeedConfig = FeedConfig()
val defaultReplConfig = ReplConfig()
val defaultSpeciesConfig = SpeciesDefaults()
val defaultSshServerConfig = SshServer()
// TODO add implicits that control how PureConfig works here
lazy val confPath: Path = new java.io.File(getClass.getClassLoader.getResource( "pure.conf ").getPath).toPath
def load: Either[ConfigReaderFailures, PureConfigFun] = pureconfig.loadConfig[PureConfigFun](confPath, "ew ")
def loadOrThrow: PureConfigFun = pureconfig.loadConfigOrThrow[PureConfigFun](confPath, "ew ")
def apply: PureConfigFun = loadOrThrow
}
Notice that the default values are defined in the companion object, and the load method uses the file called
pure.conf (which should be in the classpath, normally by placing it in the src/main/resources/ directory).
The load method as written only looks at the ew key’s value, which PureConf calls the ew namespace.
I also defined a method called loadOrThrow which either loads the contents of pure.conf,
or throws a ConfigReaderException if any problem was enountered while parsing the configuration file.
If you want to parse the entire configuration file, don’t supply a namespace when invoking pureconfig.loadConfig, like this:
def load: Either[ConfigReaderFailures, PureConfigFun] = pureconfig.loadConfig[PureConfigFun](confPath)
def loadOrThrow: PureConfigFun = pureconfig.loadConfigOrThrow[PureConfigFun](confPath)
A Closer Look
The typesafe ConfigFactory.load method loads the given file but also adds overrides and fallbacks, including values taken from the system environment variables.
If you don’t give provide namespace to the pureconfig.loadConfig methods, PureConfig will return an unknown key failures for every system property not specified as a property in your case class.
To debug this, load a raw ConfigValue to see what’s going on and then invoke render so you can see from where each configuration value was loaded from.
scala> pureconfig.loadConfigOrThrow[ConfigValue](confPath) .render .lines foreach println [...] "user " : { # system properties "country " : "US ", # system properties "dir " : "/home/foo/prog/pure-config-test ", # system properties "home " : "/home/foo ", # system properties "language " : "en ", # system properties "name " : "foo ", # system properties "timezone " : "Europe/Bar " } [...]
The solution to this problem is to either specify a
ProductHint with the value allowUnknownKeys = true
or to always use a namespace that you know won’t contain anything else except the configuration you need.
Here are the classes that will automatically be instantiated and populated when the configuration file is read. I defined as many of them as possible as value classes.
case class FeedConfig(port: Port = Port(1100))
case class ConsoleConfig(enabled: Boolean = true) extends AnyVal
case class Port(value: Int) extends AnyVal
case class ReplConfig( home: Path = Paths.get(System.getProperty( "user.home ")) )
case class SpeciesDefaults( attributeMinimum: Int = 0, attributeMaximum: Int = 100, eventQLength: Int = 25, historyLength: Int = 20 )
case class SshServer( address: String = "localhost ", ammoniteHome: Path = Paths.get(System.getProperty( "user.home ") + "/.ammonite "), enabled: Boolean = true, hostKeyFile: Option[Path] = None, //Some(Paths.get(System.getProperty( "user.home ") + "/.ssh/id_rsa ")), password: String = " ", port: Port = Port(1101), userName: String = "repl " )
We need a way to convert an Int into a Port when a configuration file is read.
PureConfig’s ConfigConvert type can be extended to do that.
Define the subclass before the configuration is loaded so the implicit conversion is in scope.
implicit val readPort = new ConfigConvert[Port] {
override def from(config: ConfigValue): Either[ConfigReaderFailures, Port] = {
config.valueType match {
case ConfigValueType.NUMBER =>
Right(Port(config.unwrapped.asInstanceOf[Int]))
case _ =>
fail(CannotConvert(config.render, "Port ", s "A port should be a number, but ${ config.valueType } was found ", ConfigValueLocation(config)))
}
}
override def to(port: Port): ConfigValue = ConfigValueFactory.fromAnyRef(port.value)
}
It would be nice to allow tilde (~) to be used to specify the user’s home directory.
To do this, we need to define special handling for java.nio.file.Path conversions, like this.
val expandTilde: String => Path = (string: String) => Paths.get(string.replace( "~ ", sys.props( "user.home ")))
import pureconfig.ConvertHelpers._ implicit val overridePathReader: ConfigReader[Path] = ConfigReader.fromString[Path](catchReadError(expandTilde))
It would be useful to have PureConfig throw an error if an unknown key is found.
To do that, define an implicit
ProductHint, before PureConfig loads the configuration.
implicit val hint: ProductHint[PureConfigFun] = ProductHint[PureConfigFun]( allowUnknownKeys = false )
The default fieldMapping is
KebabCase.
Because the keys in the configuration file are CamelCase, we need to add a fieldMapping hint to the implicit ProductHint.
implicit val hint: ProductHint[PureConfigFun] = ProductHint[PureConfigFun]( allowUnknownKeys = false, fieldMapping = ConfigFieldMapping(CamelCase, CamelCase) )
Now for a short test application.
object PureConfigTest extends App {
val pureConfigFun = PureConfigFun.load
println(pureConfigFun)
}
You can run it by typing:
$ sbt "runMain PureConfigTest" Lots of output... Right(PureConfigFun(ConsoleConfig(true),FeedConfig(Port(9090)),ReplConfig(~),SpeciesDefaults(0,100,25,20),SshServer(localhost,C:\Users\mslin_000\.ammonite,true,None,,Port(1101),repl)))
I’m not comfortable with this program.
If the config file is missing, and that should be considered as an error, the program silently uses the default value.
Instead, I prefer an exception to be raised.
This can be done by calling pureconfig.loadConfigOrThrow instead of pureconfig.load.
object PureConfigTest2 extends App {
val pureConfigFun = PureConfigFun.loadOrThrow
println(pureConfigFun)
}
You can run this program by typing.
$ sbt "runMain PureConfigTest2" Lots of output...
PureConfigFun(ConsoleConfig(true),FeedConfig(Port(9090)),ReplConfig(~),SpeciesDefaults(0,100,25,20),SshServer(localhost,C:\Users\mslin_000\.ammonite,true,None,,Port(1101),repl))
© Copyright 1994-2024 Michael Slinn. All rights reserved.
If you would like to request to use this copyright-protected work in any manner,
please send an email.
This website was made using Jekyll and Mike Slinn’s Jekyll Plugins.