The essence of OTP

August 3, 2016

What is the most important section in Joe Armstrong‘s Erlang book? The Road to the Generic Server. This post breaks down the first step along that road, illuminating the main idea behind OTP behaviors.

Single bright, old-fashioned lightbulb against black background. class=
© 2016 Milind Kaduskar for Unsplash

This is the most important section in the entire book, so read it once, read it twice, read it a hundred times—just make sure the message sinks in.

A plug for the Programming Erlang book.

Before I dig into the code, a word or two about the book Programming Erlang. I really like this book. I can jump from section to section, and it works; each section stands on it’s own. The best part is the examples; they are super creative and simple at the same time—a rare combination and one that shows off the power and beauty of the Erlang language.

For example, in the chapter I am going over in this blog entry, he builds from a very simple generic server (the one I cover), and then layers in transaction management and hot code swapping in an easy-to-follow and a ridiculously small number of lines of code. (Turns out this final server example is Joe Armstrong’s favorite Erlang program.)

Highly recommended.

Start the name server.

The idea is to divide the code for a process in a generic part (a behaviour module) and a specific part (a callback module).

The server in this example is a name server. It is a typical setup, where you have many clients and a single server that is synchronizing access to a shared resource; in this case, that resource is an in-memory hash-table that maps names to locations.

I’ll show the code below, but the way I really understood the code was to break it down into a message sequence chart, How to read a message sequence chart: The boxes represent processes and the solid arrows are messages. The dotted arrows are responses. A global clock is assumed, and time runs from top to bottom. The distances between the messages represent time order but not scale. The borders represent the outside environment; for example, the Erlang shell. so I’ll start with that:

Messages sent when starting up a name server.

The key here is to look at responsibilities of the application-specific code: ns.erl

Add a name to the name server.

Now that the name server is running, we add a name.

Messages sent when starting add a name to name server.

Again, focus on ns.erl. Here we have the basic idiom that is used over and over by OTP:

  1. the environment makes a method call to our application-specific code
  2. our app sends that message to the server’s mailbox (queue)
  3. the server reads messages off the queue (sequentially)
  4. the server calls our app code to handle the message
  5. the return value cascades back to the caller.

At first glance, I thought this was very complex. But once you get it (and it took me a few days of thinking hard to really grok it), the benefits are pretty sweet:

The generic and specific modules.

ns.erl (the specific code)

-module(ns).

-export([add/2, find/1, handle/2, init/0]).

-import(server1, [rpc/2]).

%% client routines

add(Name, Place) ->
    rpc(ns, {add, Name, Place}).

find(Name) -> rpc(ns, {find, Name}).

%% callback routines

init() -> dict:new().

handle({add, Name, Place}, Dict) ->
    {ok, dict:store(Name, Place, Dict)};
handle({find, Name}, Dict) ->
    {dict:find(Name, Dict), Dict}.

server1.erl (the generic code)

-module(server1).

-export([rpc/2, start/2]).

start(Name, Mod) ->
    Pid = spawn(fun () -> loop(Name, Mod, Mod:init()) end),
    io:format("~p~n", [Pid]),
    register(Name, Pid).

rpc(Name, Request) ->
    Name ! {self(), Request},
    receive {Name, Response} -> Response end.

loop(Name, Mod, State) ->
    receive
      {From, Request} ->
          {Response, State1} = Mod:handle(Request, State),
          From ! {Name, Response},
          loop(Name, Mod, State1)
    end.

How does this all relate to OTP behaviors?

The pattern in this simple example shows how most The gen_event OTP behavior is a different beast. OTP behaviors work.

The specific code defines the public api.

-export([add/2, find/1, …]).

The public api forwards the message to the generic server.

This example:

add(Name, Place) ->
    rpc(ns, {add, Name, Place}).

find(Name) -> rpc(ns, {find, Name}).

With a gen_server: In ns.erl, no “server1:” prefix is needed for the rpc calls because of the import statement “import(server1, [rpc/2]).” When you specify a function in an import statement, you can refer to it as it is were a local function.

add(Name, Place) ->
    gen_server:call(ns, {add, Name, Place}).

find(Name) -> gen_server:call(ns, {find, Name}).

The specific code exports callbacks used by generic server.

This example:

-export([…, handle/2, init/0]).

With gen_server:

-export([…, handle_call/3, init/1]).

The specific code implements the callbacks.

This example:

init() -> dict:new().

handle({add, Name, Place}, Dict) ->
    {ok, dict:store(Name, Place, Dict)};
handle({find, Name}, Dict) ->
    {dict:find(Name, Dict), Dict}.

With gen_server:

init(_Args) -> {ok, dict:new()}.

handle_call({add, Name, Place}, _From, Dict) ->
    {reply, ok, dict:store(Name, Place, Dict)};
handle_call({find, Name}, _From, Dict) ->
    {reply, dict:find(Name, Dict), Dict}.

This does not cover all aspects of OTP behaviors (handling asynchronous messages, starting) but those are minor details when compared to the core pattern of api-forwarder/handler-callback described here.

Message sequence chart resources

In case you are curious about message sequence charts, I learned about them from a post by Joe Armstrong on the Erlang questions mailing list:

Erlang is all about sending messages to things, so message sequence charts (https://en.wikipedia.org/wiki/Message_sequence_chart) are brilliant for describing how Erlang programs work.

Change Log

Sep. 14, 2016

Aug. 17, 2016

Sep. 1, 2016


If you see an error or something that could be improved, please let me know. This is a blog about me learning, so I expect I will get some stuff wrong. The best way to reach me is by email: mkbucc1234@gmail.com (after deleting all the numbers).

To make a comment, check for a thread on the erlang subreddit and if there isn't one, then start one up.

Follow on Twitter: @mbucc

Back to the index.