The essence of OTP

What is the most important section in Joe Armstrong‘s Erlang book? The Road to the Generic Server. This post breaks down the first step along that road, illuminating the main idea behind OTP behaviors.

A plug for the Programming Erlang book.

Before I dig into the code, a word or two about the book Programming Erlang. I really like this book. I can jump from section to section, and it works; each section stands on it’s own. The best part is the examples; they are super creative and simple at the same time—a rare combination and one that shows off the power and beauty of the Erlang language.

For example, in the chapter I am going over in this blog entry, he builds from a very simple generic server (the one I cover), and then layers in transaction management and hot code swapping in an easy-to-follow and a ridiculously small number of lines of code. (Turns out this final server example is Joe Armstrong’s favorite Erlang program.)

Highly recommended.

Start the name server.

The idea is to divide the code for a process in a generic part (a behaviour module) and a specific part (a callback module).

The server in this example is a name server. It is a typical setup, where you have many clients and a single server that is synchronizing access to a shared resource; in this case, that resource is an in-memory hash-table that maps names to locations.

I’ll show the code below, but the way I really understood the code was to break it down into a message sequence chart⊕, How to read a message sequence chart: The boxes represent processes and the solid arrows are messages. The dotted arrows are responses. A global clock is assumed, and time runs from top to bottom. The distances between the messages represent time order but not scale. The borders represent the outside environment; for example, the Erlang shell. so I’ll start with that:

The key here is to look at responsibilities of the application-specific code: ns.erl

• only get’s one message during server startup,
• simply returns the State the server should start up with,
• has zero responsibility for details of starting the server.

Add a name to the name server.

Now that the name server is running, we add a name.

Again, focus on ns.erl. Here we have the basic idiom that is used over and over by OTP:

1. the environment makes a method call to our application-specific code
2. our app sends that message to the server’s mailbox (queue)
3. the server reads messages off the queue (sequentially)
4. the server calls our app code to handle the message
5. the return value cascades back to the caller.

At first glance, I thought this was very complex. But once you get it (and it took me a few days of thinking hard to really grok it), the benefits are pretty sweet:

• your application code is purely sequential,
• no locks and you get a guarantee of no race conditions around using your shared resource,
• you don’t have to write any of the server code.

The generic and specific modules.

ns.erl (the specific code)

    -module(ns).
    
    -export([add/2, find/1, handle/2, init/0]).
    
    -import(server1, [rpc/2]).
    
    %% client routines
    
    add(Name, Place) ->
        rpc(ns, {add, Name, Place}).
    
    find(Name) -> rpc(ns, {find, Name}).
    
    %% callback routines
    
    init() -> dict:new().
    
    handle({add, Name, Place}, Dict) ->
        {ok, dict:store(Name, Place, Dict)};
    handle({find, Name}, Dict) ->
        {dict:find(Name, Dict), Dict}.
    

server1.erl (the generic code)

    -module(server1).
    
    -export([rpc/2, start/2]).
    
    start(Name, Mod) ->
        Pid = spawn(fun () -> loop(Name, Mod, Mod:init()) end),
        io:format('~p~n', [Pid]),
        register(Name, Pid).
    
    rpc(Name, Request) ->
        Name ! {self(), Request},
        receive {Name, Response} -> Response end.
    
    loop(Name, Mod, State) ->
        receive
          {From, Request} ->
              {Response, State1} = Mod:handle(Request, State),
              From ! {Name, Response},
              loop(Name, Mod, State1)
        end.
    

How does this all relate to OTP behaviors?

The pattern in this simple example shows how most⊕ The gen_event OTP behavior is a different beast. OTP behaviors work.

The specific code defines the public api.

    -export([add/2, find/1, …]).
    

The public api forwards the message to the generic server.

This example:

    add(Name, Place) ->
        rpc(ns, {add, Name, Place}).
    
    find(Name) -> rpc(ns, {find, Name}).
    
    

With a gen_server⊕: In ns.erl, no “server1:” prefix is needed for the rpc calls because of the import statement “import(server1, [rpc/2]).” When you specify a function in an import statement, you can refer to it as it is were a local function.

    add(Name, Place) ->
        gen_server:call(ns, {add, Name, Place}).
    
    find(Name) -> gen_server:call(ns, {find, Name}).
    
    

The specific code exports callbacks used by generic server.

This example:

    -export([…, handle/2, init/0]).
    

With gen_server:

    -export([…, handle_call/3, init/1]).
    

The specific code implements the callbacks.

This example:

    init() -> dict:new().
    
    handle({add, Name, Place}, Dict) ->
        {ok, dict:store(Name, Place, Dict)};
    handle({find, Name}, Dict) ->
        {dict:find(Name, Dict), Dict}.
    

With gen_server:

    init(_Args) -> {ok, dict:new()}.
    
    handle_call({add, Name, Place}, _From, Dict) ->
        {reply, ok, dict:store(Name, Place, Dict)};
    handle_call({find, Name}, _From, Dict) ->
        {reply, dict:find(Name, Dict), Dict}.
    

This does not cover all aspects of OTP behaviors (handling asynchronous messages, starting) but those are minor details when compared to the core pattern of api-forwarder/handler-callback described here.

Message sequence chart resources

In case you are curious about message sequence charts, I learned about them from a post by Joe Armstrong on the Erlang questions mailing list:

Erlang is all about sending messages to things, so message sequence charts (https://en.wikipedia.org/wiki/Message_sequence_chart) are brilliant for describing how Erlang programs work.

• I used the LaTex macro package, documentation is here.
• A message sequence tutorial is here.
• The spec is here.

Change Log

Sep. 14, 2016

• Add link to today’s blog entry on gen_event.

Aug. 17, 2016

• Rewrote lede.
• Fixed typo in message sequence chart heading.
• Added book title and link to A plug for the Programming Erlang book. section, fixed a spelling error, and wordsmithed the text a little.
• Fixed indentation in ns.erl.
• Moved instructions for reading a message sequence chart into a margin note.
• Changed the contents of all second-level headings.
• Added How does this relate to OTP behaviors section.
• Add link to Joe Armstrong’s favorite Erlang program blog entry.

Sep. 1, 2016

• Added margin note explaining why there is no “server1:” qualifier on the rpc calls in ns.erl.