Extensibility in Elixir Using Behaviours

The last post dug into protocols and how they can be used to define implementations for new or different data types. This post will cover another option Elixir provides for extensibility, behaviours.

When to use Behaviours

You might look to use Behaviours when there can be different implementations to achieve the same overall goal using the same function inputs and outputs that can be checked at compile time. Another helpful way to think Behaviours might be pluggable implementations that use the same type of data to fulfil a contract. Behaviours are handy both for library authors so that consumers can provide custom implementations for contracts defined in those libraries, but also within an app that may choose or configure different behaviours depending on various conditions. Often times you can get quite a lot of payoff by defining some key contracts / callbacks that lots of functions can utilize to achieve an overall goal in an extensible way.

Some Behaviours that you may come across, and may inspire you, include:

• GenSever
• Plug
• Ecto.Type
• Ecto.Adapter

Behaviours are contracts

To quote the guide:

At their core behaviours are a specification or contract. Any Module can define a contract that can be adopted as a @behaviour by specifying one or more @callback or @macrocallback Module attributes. These callbacks can then be adopted as a behaviour by any other Module. It is important to note that you don't mark a Module as a behaviour, merely defining one or more callback attributes is enough. Callbacks are defined by passing a specification to the callback Module attribute. These are identical to how you would define a specification using the @spec Module attribute. As explained in the docs a callback specification is made up of:

• the callback name
• the arguments that the callback must accept
• the expected type of the callback return value

Lets take a look at a contrived example of a simple contract or callback that can print Strings. How and where the Strings are printed depends on the adopter of the Behaviour:

defmodule Printer do
  @moduledoc """
  Defines the contract to print to output "devices"
  """

  @doc """
  Prints the given string to an output device.

  Returns `:ok` if successful or `{:error, msg}` if it fails
  """
  @callback print(text :: String.t()) :: :ok | {:error, String.t()}
end

The example above highlights that you can document the callback the same as would for any named function or macro. I personally think it is worth providing docs that explain the callback, whether you are authoring a library that expects consumers to adopt the contract, or defining and using the contracts in a single app.

Note: The Printer Module above will have no functions defined on it. If you try to invoke print/1 on Printer you will get an error:

iex> Printer.print("hello")
** (UndefinedFunctionError) function Printer.print/1 is undefined or private

You can get all of the callbacks defined on a Module with the behaviour_info/1 function:

iex> Printer.behaviour_info(:callbacks)
[print: 1]

The result of behaviour_info is a keyword list of all callbacks and their arity specified on the Module. If a callback is a macro callback the key will be prefixed with MACRO-. The function behaviour_info will only be exported (defined) for a Module that defines callbacks.

Adopting behaviour

The first step when adopting a behaviour is to specify the Module name that defines the callback(s) as the value of a @behaviour Module attribute.

defmodule StdOutPrinter do
  @behaviour Printer
end

Compiling the above gives us a warning:

warning: function print/1 required by behaviour Printer is not implemented (in Module StdOutPrinter)

To get rid of the warning we have to implement the print callback defined in the Printer Module.

defmodule StdOutPrinter do
  @behaviour Printer

  @impl Printer
  def print(text) when is_binary(text) do
    IO.puts(text)
  end
end

The @impl attribute above that was introduced in Elixir 1.5. It is optional, but again I think it is worthwhile to always specify when define functions or macros that are implementing a callback. It both gives the complier a hint about the intent of the definition, and improves the readability of the Module. From the docs:

You may pass either false, true, or a specific behaviour to @impl.

I tend to always pass the behaviour Module name, but this is just personal preference. If a Module implements callbacks from more than one behaviour I would argue you should definitely specify the Module name. Another interesting side-effect of providing an explicit @impl is that @doc will be (implicitly) given a value of false. This makes sense, because it should really be Module that defines that callback that documents it. The docs define the @docs Module attribute as:

Accepts a string (often a heredoc) or false where @doc false will make the function/macro invisible to documentation extraction tools like ExDoc

If for some reason it makes sense to document the adopter of the callback simply specify a value for @doc on the implemented callback.

Note: If an attempt is made to adopt two behaviours in the same Module that specify a callback with the same name and arity you will get a warning warning: conflicting behaviours found. This should be avoided because it may not be possible to satisfactorily implement both of the conflicting behaviours being adopted.

Type checking behaviours

If we were to change the above implementation to erroneously check for is_map in the guard:

defmodule StdOutPrinter do
  @behaviour Printer

  @impl Printer
  def print(text) when is_map(text) do
    IO.puts(text)
  end
end

It would still compile with no errors or warnings, because the Elixir compiler will only look at callback name and arity. It would be great to be notified when an implementation of a callback was incorrectly typed in an obvious way. Dialyxir does a great job of this, in fact if you add any typescpecs to your project dialyxir is an invaluable tool for picking up type errors. Running mix dialyzer on the above code will produce a warning

The inferred type for the 1st argument of print/1 (map()) is not a supertype of binary(), which is expected type for this argument in the callback of the 'Elixir.Printer' behaviour.

That is much more helpful for catching erroneous implementations.

Note: I have also heard good things about Dialyzex but I haven't tried it myself yet.

Optional callbacks

Modules support an @optional_callbacks Module attribute. It takes the name and arity of a @callback or a @macrocallback attribute. Let's update our contrived behaviour to have an optional callback:

defmodule Printer do
  @moduledoc """
  Defines the contract to print to output "devices"
  """

  @doc """
  Prints the given string to an output device.

  Returns `:ok` if successful or `{:error, msg}` if it fails
  """
  @callback print(text :: String.t()) :: :ok | {:error, String.t()}

  @doc """
  Optionally restrict the maximum length of the string that can be printed

  Returns the maximum length of the string supported by the `Printer`
  """
  @callback max_length() :: pos_integer()

  @optional_callbacks max_length: 0
end

behaviour_info can also report the optional callbacks in a Module.

iex> Printer.behaviour_info(:callbacks)
[print: 1, max_length: 0]
iex> Printer.behaviour_info(:optional_callbacks)
[max_length: 0]

StdOutPrinter would still compile without the warning of a missing callback because we have marked the max_length callback as optional. We could similarly update StdOutPrinter to implement the max_length callback:

defmodule StdOutPrinter do
  @behaviour Printer

  @impl Printer
  def print(text) when is_binary(text) do
    IO.puts(text)
  end

  @impl Printer
  def max_length() do
    1024
  end
end

To determine if an adopting Module defines an optional callback one simple option is to use the Kernel.function_exported?/3 or Kernel.macro_exported?/3 functions. If we were to use function_exported? on the last example of StdOutPrinter above it would be true.

iex> function_exported?(StdOutPrinter, :max_length, 0)
true

Alternatively identifying whether optional callbacks can be part of config

Another neat example of the advantages of using Dialyzer is that it will pickup if we return an integer that is not positive because of the return type of our specification being pos_integer() even if we use a Module attribute. So if we changed the above to:

defmodule StdOutPrinter do
  @behaviour Printer
  @max_length -1

  @impl Printer
  def print(text) when is_binary(text) do
    IO.puts(text)
  end

  @impl Printer
  def max_length() do
    @max_length
  end
end

Running mix dialyzer would generate a warning The inferred return type of max_length/0 (-1) has nothing in common with pos_integer(), which is the expected return type for the callback of the 'Elixir.Printer' behaviour

Choosing an adopter

There is no simple way to discover all the adopters of a given behaviour. Instead the app or library needs to be told in some way which adopter to use, and there are a few common ways of doing this.

Calling the function directly

If you know the Module in advance you can always just call it directly:

StdOutPrinter.print("some text")

Although the above print/1 is implementing a callback there is nothing special about it at runtime, it is just a function on the StdOutPrinter Module.

Note: If you are interested in registering adopting modules via a macro PlugBuilder is an example of this approach.

Passing the Module

Elixir supports dynamic dispatch so it is relatively common to find functions that take the adopter of a behaviour as an argument. We can make a print_up_to_max/2 function that is suitable for any Module adopting the Printer behaviour.

  @doc """
  Print as much of the text as possible given the supplied Printer
  """
  @spec print_up_to_max(module(), String.t()) :: :ok | {:error, String.t()}
  def print_up_to_max(printer, text)
      when is_atom(printer) and is_binary(text) do
    max_length =
      if function_exported?(printer, :max_length, 0),
        do: printer.max_length(),
        else: :infinite

    print_up_to_max(printer, text, max_length)
  end

  defp print_up_to_max(printer, text, :infinite), do: printer.print(text)

  defp print_up_to_max(printer, text, max_length) do
    text
    |> String.split_at(max_length)
    |> elem(0)
    |> printer.print()
  end

Specifying the Adopting Module in Configuration

Another common approach is to specify the adopting Module's name in an Application's config.exs. This approach is often used when you can provide a Behaviour to a dependency that defines the callbacks. Imagine there is a HTTP library that defined callbacks for serializing/deserialzing JSON. It could support configuration to specify an adopting Module that did the serialization work with a JSON serializer of choice. Let's again imagine that there is a library that wanted to support printing text and defined the Printer callbacks above. There could be configuration that supported us specifying StdOutPrinter as the adopting Module.

# In config.exs
config :dummy_text_printer, printer: StdOutPrinter

This configuration could even support specifying whether there is a max_length/0 function defined:

# In config.exs
config :dummy_text_printer, printer: StdOutPrinter, max_length: true

Then the dummy_text_printer dependency could look up the specified Printer where needed:

printer = Application.get_env(:dummy_text_printer, :printer)
printer.print("Some text to print")

This approach is particularly useful when the implementation changes when testing. José's blog post on mocks and explicit contracts provides some guidance regarding this approach.

Choosing an implementation based on a predicate

Predicates can be built in to the specifications so that the implementation may be chosen at runtime. A contract that centers around deserialization can provide a function to determine whether an implementation is applicable based on a file's extension extension_supported?/1. The app could cycle through all the known implementations and use the first that it finds that supports an extension. The docs have an example that eludes to this approach.

Mixing in functions

A Module that defines callbacks can also define other functions as we have seen above. This can be a good place to put functions that build upon the implemented callbacks, as we saw with the print_up_to_max/2 function. It is possible to take this further and support being able to define those functions on the adopting Module with the __using__/1 macro

defmodule Printer
  defmacro __using__(_) do
    quote do
      @behaviour Printer

      @spec print_up_to_max(String.t()) :: :ok | {:error, String.t()}
      def print_up_to_max(text) when is_binary(text) do
        Printer.print_up_to_max(__MODULE__, text)
      end

      defoverridable print_up_to_max: 1
    end
  end

  # The rest of the Module
end

StdOutModule could then adopt the behaviour by "using" the Printer Module:

defmodule StdOutPrinter do
  use Printer

  @impl Printer
  def print(text) when is_binary(text) do
    IO.puts(text)
  end
end

The use Printer expression will call the __using__/1 macro in the Printer Module. The macro defines the print_up_to_max/1 function that calls the Printer.print_up_to_max/2 function with the StdOutPrinter module. The print_up_to_max/1 function can now be called on the StdOutPrinter Module directly.

Printer.printer_up_to_max("Some text to print")

It is even possible to mark the "mixed in" function as overridable using defoverridable. In the above example print_up_to_max/1 is marked as overridable using the expression defoverridable print_up_to_max: 1. This allows any Module that adopts the Printer Module via the __using__/1 macro to define it's own implementation of print_up_to_max/1 if it needs to do something specific.

Wrapping up

Behaviours are a great way to get compile time checking for contracts that can differ in implementation. Not only that, they be a great source of documentation for both your future self, your team, or consumers of a library. Obviously most implementations don't need to be "pluggable" so you probably won't end up with a large number of callbacks. Identifying those that do, however, can pay dividends for maintainability and extensibility. There is currently no way to discover adopters of a Behaviour so deciding on how an adopting Module is specified or determined at run-time is important.