In his article Why I chose OCaml as my primary language, my friend Xavier Van de Woestyne presents, in the section Dependency injection and inversion, two approaches to implementing dependency injection: one using user-defined effects and one using modules as first-class values. Even though I’m quite convinced that both approaches are legit, I find them sometimes a bit overkill and showing fairly obvious pitfalls when applied to real software. The goal of this article is therefore to briefly highlight the ergonomic weaknesses of both approaches, and then propose a new encoding of inversion and dependency injection that I find more comfortable (in many cases). In addition, this gives an example of using objects in OCaml, which are often overlooked, even though in my view OCaml’s object model is very interesting and offers a lot of practical convenience.
This approach is by no means novel and is largely the result of several experiments shared with Xavier Van de Woestyne during multiple pair-programming sessions. However, a precursor of this encoding can be found in the first version of YOCaml (using a Left Kan Extension/Freer Monad rather than a Reader). It's also interesting we can find a similar approach in the recent work around Kyo, an effect system for the Scala language, which uses a similar trick based on subtyping relationships to type an environment.
Why use dependency injection?
There are plenty of documents that describe (sometimes a bit aggressively) all the benefits of dependency injection, which are sometimes extended into fairly formalized software architectures (such as hexagonal architecture). For my part, I find that dependency injection makes unit testing a program trivial, which I think is reason enough to care about it (For example, the time-tracker I use at work, Kohai, uses Emacs as its interface and the file system as its database. Thanks to inversion and dependency injection, we were able to achieve high test coverage fairly easily).
Effect system and dependency injection
There are many different ways to describe an effect handler, so in my view it is difficult to give a precise definition of what an Effect system is. However, in our modern interpretation, the goal is more to suspend a computation so it can be interpreted by the runtime (the famous IO monad), and an Effect system is often described as a systematic way to separate the denotational description of a program, where propagated effects are operational “holes” that are given meaning via a handler, usually providing the ability to control the program’s execution flow (its continuation), unlocking the possibility to describe, for example, concurrent programs. In this article, I focus on dependency injection rather than on building an effect system because that would be very pretentious (my article does not address performance or runtime concerns). Similarly to how exception handling can be seen as a special case of effect propagation and interpretation (where the captured continuation is never resumed), I also see dependency injection as a special case, this time, where the continuation is always resumed. It’s quite amusing to see that dependency injection and exception capturing can be considered two special cases of effect abstraction, differing only in how the continuation is handled.
The Drawbacks of Modules and User-Defined Effects
As I mentioned in the introduction, I believe that both approaches proposed by Xavier are perfectly legitimate. However, after using both approaches in real-world software, I noticed several small annoyances that I will try to share with you.
Using modules
Using modules (through functors or by passing them as values) seems to be the ideal approach for this kind of task. However, the module language in OCaml has a different type system, in which type inference is severely limited. From my point of view, these limitations can lead to a lot of verbosity whenever I want to imagine that my dependencies come from multiple different sources. For example, consider these two signatures:
The first one provides basic manipulation of the file system (very simple, with no handling of permissions or errors):
module type FS = sig
val read_file : path:string -> string option
val write_file : path:string -> content:string -> unit
end
The second one simply allows logging to standard output (also very basic, without support for log levels):
module type CONSOLE = sig
val log : string -> unit
end
The first way to depend on both signatures is to introduce a new signature that describes their combination:
module type FS_CONSOLE = sig
include FS
include CONSOLE
end
let show_content (module H : FS_CONSOLE) =
match H.read_file ~path:"/foo/bar" with
| None -> H.log "Nothing"
| Some x -> H.log x
Or simply take the dependencies as two separate parameters:
let show_content (module F : FS) (module C : CONSOLE) =
match F.read_file ~path:"/foo/bar" with
| None -> C.log "Nothing"
| Some x -> C.log x
Indeed, constructing the module on the fly, directly in the function definition, is not possible. Although very verbose, this expression is rejected by the compiler:
# let show_content (module H : sig
include FS
include CONSOLE
end) =
match H.read_file ~path:"/foo/bar" with
| None -> H.log "Nothing"
| Some x -> H.log x ;;
Lines 1-4, characters 30-10:
Error: Syntax error: invalid package type: only module type identifier and with type constraints are supported
Moreover, beyond the syntactic heaviness, I find the loss of type inference quite restrictive. While in situations where you don’t need to separate and diversify dependency types this isn’t a big deal, I find that this approach sometimes forces unnecessary groupings. I often ended up in situations where, for all functions that had dependencies, I had to provide a single module containing them all, which occasionally forced me, in testing scenarios, to create dummy functions. A very frustrating experience!
Using User-Defined-Effects
I proudly claimed that dependency injection is a special case of using an Effect System, so one might wonder: could using OCaml’s effect system be a good idea? From my understanding, the integration of effects was primarily intended to describe interactions with OCaml’s new multi-core runtime. In practice, the lack of a type system tracking effects makes, in my view, their use for dependency injection rather cumbersome. Indeed, without a type system, it becomes, once again in my view, difficult to mentally keep track of which effects have been properly handled. In YOCaml, we recorded effects in a module called Eff that encodes programs capable of propagating effects in a monad (a kind of IO monad). This allows us to handle programs a posteriori (and thus inject dependencies, of course) but restricts us in terms of handler modularity. Indeed, it assumes that in all cases, all effects will be interpreted. And, in the specific case of YOCaml, we usually only want to either continue the program or discard the continuation (which can be done trivially using an exception). Control-flow management is therefore a very powerful tool, for which we have very little use.
In practice, there are scenarios where using OCaml’s effects seems perfectly legitimate (and I think I have fairly clear ideas about why introducing a type system for effects is far from trivial, particularly in terms of user experience):
- When you want to perform backtracking
- When you want to express concurrency libraries, schedulers, etc., which makes a lot of sense in libraries like Eio, Miou, or Picos
So, in the specific case of dependency injection, I have the intuition that using OCaml’s effects gives too much power, while putting significant pressure on tracking effects without compiler assistance, making them not entirely suitable.
Using objects
It’s not very original to use objects to encode a pattern typically associated with object-oriented programming. However, in functional programming, it’s quite common to encounter encodings of dependency injection sometimes referred to as Functional Core, Imperative Shell. Although relatively little used and sometimes unfairly criticized, OCaml’s object model is actually very pleasant to work with (its theoretical foundation is even extensively praised in Xavier’s article, in the section Closely related to research). To my knowledge, OCaml is one of the rare mainstream languages that draws a very clear separation between objects, classes, and types. Objects are values, classes are definitions for constructing objects, and objects have object types, which are regular types, whereas classes also have types that are not regular, since classes are not regular expressions but rather expressions of a small class language.
In order to integrate coherently into OCaml and with type inference, the object model relies on four ingredients: structural object types (object types are structural, their structure is transparent), row variables, equi-recursive types, and type abbreviations.
In practice, an object type is made up of the row of visible members (associated with their types) and may end with a row variable (to characterize closed/open object types). This row variable can serve as the subject of unification as objects are used together with their types.
A quick way to become aware of the presence of the row variable is to
simply write a function that takes an object as an argument and sends
it a message (in OCaml, sending a message is written with the syntax
obj # message):
# let f obj = (obj # foo) + 10 ;;
val f : < foo : int; .. > -> int = <fun>
In the return type, the row variable indicating that the object type
is open is represented by <..>. It is thanks to this variable that we
can perform dependency injection, finely tracked by the type system
and guided by inference.
Simple example
To keep things simple, let’s just revisit the classic teletype example, which we could write in direct style like this:
# let teletype_example () =
print_endline "Hello, World!";
print_endline "What is your name?";
let name = read_line () in
print_endline ("Hello " ^ name)
val teletype_example : unit -> unit = <fun>
Let’s rewrite our example by taking an object as an argument, which will serve as the handler:
# let teletype_example handler =
handler#print_endline "Hello, World";
handler#print_endline "What is your name?";
let name = handler#read_line () in
handler#print_endline ("Hello " ^ name)
val teletype_example :
< print_endline : string -> 'a; read_line : unit -> string; .. > -> 'a =
<fun>
To convince ourselves of the compositionality of our dependency injection, let’s imagine the following function, whose role is to simply log traces of the execution:
# let log handler ~level message =
handler#do_log level message
val log : < do_log : 'a -> 'b -> 'c; .. > -> level:'a -> 'b -> 'c = <fun>
By using the teletype_example and log functions, we can directly
observe the precision of the elision, showing that our handler
object was open each time:
# let using_both handler =
let () = log handler ~level:"debug" "Start teletype example" in
teletype_example handler
val using_both :
< do_log : string -> string -> unit; print_endline : string -> 'a;
read_line : unit -> string; .. > ->
'a = <fun>
All the requirements of our using_both function are (logically)
correctly tracked. We can now implement a dummy handler very simply, using
immediate object syntax:
# using_both object
method do_log level message =
print_endline ("[" ^ level ^ "] " ^ message)
method print_endline value = print_endline value
method read_line () =
(* Here I should read the line but hey,
it is not interactive, let's just return my name. *)
"Pierre"
end;;
[debug] Start teletype example
Hello, World
What is your name?
Hello Pierre
- : unit = ()
This approach is extremely similar to using polymorphic variants for composable error handling, which share many features with objects (structural subtyping, rows).
Typing, sealing and reusing
In our examples, we were guided by type inference. However, in OCaml, it is common to restrict the generality of inference using explicit signatures. As we have seen, some of our inferred signatures are too general and arguably not very pleasant to write:
< do_log : string -> string -> unit;
print_endline : string -> 'a;
read_line : unit -> string;
..
>
Fortunately, type abbreviations allow us to simplify this notation. By using class types and certain abbreviations, we can simplify the type expressions of our functions that require injection:
First, we will describe our types using dedicated class types, let's
start with console:
class type console = object
method print_endline : string -> unit
method read_line : unit -> string
end
Now let's write our loggable interface:
class type loggable = object
method do_log : level:string -> string -> unit
end
Now, let's rewrite our three functions inside a submodule (to make their signatures explicit):
module F : sig
val teletype_example : #console -> unit
val log : #loggable -> level:string -> string -> unit
end = struct
let teletype_example handler =
handler#print_endline "Hello, World";
handler#print_endline "What is your name?";
let name = handler#read_line () in
handler#print_endline ("Hello " ^ name)
let log handler ~level message =
handler#do_log ~level message
end
And now, we can describe our using_both function as taking an object
that is the conjunction of loggable and console, like this:
module G : sig
val using_both : <console; loggable; ..> -> unit
end = struct
let using_both handler =
let () = F.log handler ~level:"debug" "Start teletype example" in
F.teletype_example handler
end
At the implementation level, the separation between inheritance (as a
syntactic action) and subtyping (as a semantic action) allows us to
benefit from this kind of mutualization directly at the call
site. For example, let's implement a handler for console and
loggable:
class a_console = object
method print_endline value = print_endline value
method read_line () =
(* Here I should read the line but hey,
it is not interactive, let's just return my name. *)
"Pierre"
end
class a_logger = object
method do_log ~level message =
print_endline ("[" ^ level ^ "] " ^ message)
end
Even though it would be possible to constrain our classes by the
interfaces they implement (using class x = object (_ : #interface_)), it is not necessary because structural subtyping will
handle the rest (and it also allows us to potentially introduce
intermediate states easily). We can now use inheritance to share
functionality between our two handlers:
# G.using_both object
inherit a_console
inherit a_logger
end ;;
[debug] Start teletype example
Hello, World
What is your name?
Hello Pierre
- : unit = ()
One could argue that it’s still a bit verbose, but in my view, this approach offers much more convenience. We know statically the capabilities that need to be implemented, we retain type inference, and we have very simple composition tools. At this point, I have, subjectively, identified scenarios for using the different approaches discussed in this article:
-
When you need to control the program’s continuation (essentially for backtracking or concurrency), it’s preferable to use effects (hoping that one day we’ll be able to type them ergonomically).
-
When you want to introduce types (without parametric polymorphism) into dependencies, first-class modules work very well.
-
When you want to introduce types that can have type parameters (for example, to express the normal forms of our dependencies via a runtime value, e.g., through a monad), functors are suitable.
-
When you want to do simple dependency injection, guided by type inference and requiring the ability to fragment or share handlers, objects are perfectly suitable.
However, the approach based on first-class modules or objects can heavily pollute a codebase, whereas effect interpretation and functor instantiation can remain at the edges of the program. Let’s look at the final part of this article, which explains how to reduce the aggressive propagation of handlers.
Injected dependencies as part of the environment
Currently, our approach forces us to pass our handler explicitly from call to call, which can drastically bloat business logic code. What we would like is the ability to only worry about the presence of dependencies when it is actually necessary. To achieve this, we’ll use a module whose role will be to pass our set of dependencies in an ad-hoc manner:
module Env : sig
type ('normal_form, 'rows) t
val run : 'rows -> ('normal_form, 'rows) t -> 'normal_form
val perform : ('rows -> 'normal_form) -> ('normal_form, 'rows) t
val return : 'normal_form -> ('normal_form, 'rows) t
val ( let* ) : ('a, 'rows) t -> ('a -> ('b, 'rows) t) -> ('b, 'rows) t
end = struct
type ('normal_form, 'rows) t = 'rows -> 'normal_form
let run env comp = comp env
let perform f = f
let return x _ = x
let ( let* ) r f x = (fun y -> (f y) x) (r x)
end
Our module allows us to separate the description of our program from
the passing of the environment. The only subtlety lies in the (let*)
operator, a binding
operator that lets us
simplify the writing of programs. Let's define some helpers based on
our previous interfaces:
module U : sig
val print_endline : string -> (unit, #console) Env.t
val read_line : unit -> (string, #console) Env.t
val log : level:string -> string -> (unit, #loggable) Env.t
end = struct
let print_endline str =
Env.perform (fun h -> h#print_endline str)
let read_line () =
Env.perform (fun h -> h#read_line ())
let log ~level message =
Env.perform (fun h -> h#do_log ~level message)
end
And now, we can describe our previous programs using our let* syntax
shortcut:
let comp =
let open Env in
let* () = U.log ~level:"Debug" "Start teletype example" in
let* () = U.print_endline "Hello, World!" in
let* () = U.print_endline "What is your name?" in
let* name = U.read_line () in
U.print_endline ("Hello " ^ name)
And we can handle it using Env.run:
# Env.run object
inherit a_console
inherit a_logger
end comp ;;
[Debug] Start teletype example
Hello, World!
What is your name?
Hello Pierre
- : unit = ()
Some prefer to pass handlers explicitly to avoid relying on binding operators. Personally, I really like that the operators make it explicit whether I’m dealing with a pure expression or one that requires dependencies, and I find that binding operators make the code readable and easy to reason about.
Under the hood
Looking at the signature of the Env module, many will notice that it
is a Reader
Monad
(a specialization of ReaderT transformed with the Identity monad),
which is sometimes also called a Kleisli. In practice, this allows
us to be parametric over the normal form of our expressions. For
simplicity in the examples, I’ve minimized indirection, but it is
entirely possible to project our results into a more refined monad,
defining a richer runtime (and potentially supporting continuation
control, effect interleaving, etc.).
To conclude
Some might be surprised—indeed, I’ve fairly closely applied the
Boring
Haskell
philosophy. I literally used objects for dependency injection (the
extra-classical approach to DI) and rely on a ReaderT to access
my environment on demand, which is a very common approach in the
Haskell community.
The only idea here, which isn’t particularly novel, is to use subtyping relationships to statically track the required dependencies, and rely on inheritance to arbitrarily compose handlers. From my point of view, this drastically reduces the need to stack transformers while still keeping modularity and extensibility. By relying on a different normal form than the identity monad, it’s possible to achieve results that are surprisingly pleasant to use and safe, as demonstrated by Kyo in the Scala world. In OCaml, the richness of the object model (and its structural capabilities) is a real advantage for this kind of encoding, allowing a drastic reduction of the boilerplate needed to manage multiple sets of dependencies.
In practice, I’ve found this approach effective for both personal and professional projects (and, importantly, very easy to explain). One limitation is the inability to eliminate dependencies when they are partially interpreted in ways other than by partial function application. Still, the encoding remains neat for at least three reasons:
- It allows us to statically track dependencies in the context of dependency injection.
- It makes it easy to write testable programs by providing handlers adapted for unit tests.
- It provides another example showing that OCaml’s objects are really powerful and can offer fun solutions to well-worn problems.
In the not-so-distant future, we might even imagine providing handlers more lightly using Modular Implicits.
Thank you for reading (if you made it this far), and a special thanks to Xavier Van de Woestyne for his article that inspired me to write this one, and to Jonathan Winandy for showing me Kyo and helping me rephrase some sentences.