How to return JSON from an Erlang web service

January 4, 2017

In this HOWTO, we use Elli and JSX to return JSON from an Erlang web service.

A blackboard with math notation. class=
© 2016 Roman Mager for Unsplash

Tools used in this tutorial:

  1. Erlang/OTP 19
  2. Elli 1.0.5
  3. JSX 2.8.1
  4. GNU Make 3.81
  5. git 1.9.5

All of the code used in this blog can be found at https://github.com/mbucc/markbucciarelli.com/tree/master/sandbox/json.

Step 1. Create a Makefile that retrieves the required dependencies.

Erlang lets you define a library directory on your system with the environmental variable ERL_LIBS. Typically you would set this to something like /usr/local/lib/erl but in this example we put a lib directory under the current working directory.

all: ./lib/jsx-2.8.1/ebin ./lib/elli-1.0.5/ebin ./lib/elli-1.0.5/include

#==============================================================================
#
#                           D E P E N D E N C I E S
#
#  Check out Elli and JSX from github to $HOME/src, compile each and copy the
#  ebin directories and the elli include directory under ./lib.
#==============================================================================

${HOME}/src/elli/ebin:
	(cd ${HOME}/src; git clone https://github.com/knutin/elli.git)
	(cd ${HOME}/src/elli ; git checkout tags/v1.0.5 ; make)

./lib/elli-1.0.5/ebin: ${HOME}/src/elli/ebin
	mkdir -p ./lib/elli-1.0.5
	cp -r $< ./lib/elli-1.0.5

./lib/elli-1.0.5/include: ${HOME}/src/elli
	mkdir -p ./lib/elli-1.0.5
	cp -r ${HOME}/src/elli/include ./lib/elli-1.0.5

${HOME}/src/jsx/_build/default/lib/jsx/ebin:
	(cd ${HOME}/src; git clone https://github.com/talentdeficit/jsx.git)
	(cd ${HOME}/src/jsx ; git checkout tags/v2.8.1 ; rebar3 compile)

./lib/jsx-2.8.1/ebin: ${HOME}/src/jsx/_build/default/lib/jsx/ebin
	mkdir -p ./lib/jsx-2.8.1
	cp -r $< ./lib/jsx-2.8.1

Step 2. Create Elli handler.

This handler returns a JSON representation of event data for the resource /events. The event data JSX does not encode records, only proplists and maps. See the JSX README quickstart for more details. is passed in as a configuration argument, as described in the next section.

-module(json_handler).

-export([handle/2, handle_event/3]).

-include_lib("elli/include/elli.hrl").

-behaviour(elli_handler).

handle(Req, Args) ->
    handle(Req#req.method, elli_request:path(Req), Req, Args).

handle(’GET’, [<<"events">>], _Req, Args) ->
    Events = proplists:get_value(events, Args),
    {ok, [], jsx:encode(Events)};
handle(_, _, _Req, _Args) -> {404, [], <<"Not Found">>}.

handle_event(Event, Data, Args) ->
    io:format("Event=~p~n, Data=~p~n, Args=~p~n",
              [Event, Data, Args]),
    ok.

This handler also dumps each Elli event to stdout because it implements the handle_event method of the elli behavior.

Step 3. Use OTP supervisor behavior to start Elli on port 3000.

The MiddlewareConfig variable configures the json_handler to get the event data returned by the test_events() method. In a real application you would pass in the registered name of an ets store, or a database connection pool, or some other set of credentials to the data store.

-module(json_sup).

-behaviour(supervisor).

-export([start_link/0]).

-export([init/1]).

start_link() ->
    supervisor:start_link({local, ?MODULE}, ?MODULE, []).

test_events() ->
    E1 = [{id, 1}, {name, <<"test1">>}],
    E2 = [{id, 2}, {name, <<"test2">>}],
    [E1, E2].

init(_Args) ->
    Port = 3000,
    MiddlewareConfig = [{mods,
                         [{json_handler, [{events, test_events()}]}]}],
    ElliOpts = [{callback, elli_middleware},
                {callback_args, MiddlewareConfig}, {port, Port}],
    ElliSpec = {json, {elli, start_link, [ElliOpts]},
                permanent, 5000, worker, [elli]},
    io:format("starting server on port ~p~n", [Port]),
    {ok, {{one_for_one, 5, 10}, [ElliSpec]}}.

Step 4. Write the application behavior and resource file.

Implement the application behavior.

-module(json).

-behavior(application).

-export([start/0, start/2, stop/1]).

start(_Type, _Args) -> json_sup:start_link().

start() -> application:start(json).

stop(_State) -> ok.

Define the application resources.

{application, json,
  [{description, "Produce JSON with Erlang"},
   {vsn, "1.0.0"},
   {modules, [json, json_sup, json_handler]},
   {registered, []},
   {applications, [kernel, stdlib]},
   {mod, {json,[]}}]}.

Step 5. Update Makefile and create a run script.

The run script starts the server from the command line.

#! /bin/sh -e

ERL_LIBS=./lib erl -noshell -s json

Update the Makefile to compile Erlang sources.

all: json_handler.beam json_sup.beam json.beam

#==============================================================================
#
#                           D E P E N D E N C I E S
#
#  Check out Elli and JSX from github to $HOME/src, compile each and copy
#  ebin directory under ./lib.
#==============================================================================

${HOME}/src/elli/ebin:
	(cd ${HOME}/src; git clone https://github.com/knutin/elli.git)
	(cd ${HOME}/src/elli ; git checkout tags/v1.0.5 ; make)

./lib/elli-1.0.5/ebin: ${HOME}/src/elli/ebin
	mkdir -p ./lib/elli-1.0.5
	cp -r $< ./lib/elli-1.0.5

./lib/elli-1.0.5/include: ${HOME}/src/elli
	mkdir -p ./lib/elli-1.0.5
	cp -r ${HOME}/src/elli/include ./lib/elli-1.0.5

${HOME}/src/jsx/_build/default/lib/jsx/ebin:
	(cd ${HOME}/src; git clone https://github.com/talentdeficit/jsx.git)
	(cd ${HOME}/src/jsx ; git checkout tags/v2.8.1 ; rebar3 compile)

./lib/jsx-2.8.1/ebin: ${HOME}/src/jsx/_build/default/lib/jsx/ebin
	mkdir -p ./lib/jsx-2.8.1
	cp -r $< ./lib/jsx-2.8.1


#==============================================================================
#
#                      C O M P I L E   A N D   C L E A N 
#
#==============================================================================

%.beam: %.erl ./lib/elli-1.0.5/ebin ./lib/jsx-2.8.1/ebin ./lib/elli-1.0.5/include
	ERL_LIBS=./lib erlc $<

.PHONY: clean
clean:
	rm -f *.beam
	rm -rf ./lib

The Result.

Start the server.

$ ./run.sh 
starting server on port 3000
Event=elli_startup
, Data=[]
, Args=[{events,[[{id,1},{name,<<"test1">>}],[{id,2},{name,<<"test2">>}]]}]

Elli fires the elli_startup event and we are up and running.

Hit this with curl, and we get our events back in JSON.

$ curl -D- localhost:3000/events
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 49

[{"id":1,"name":"test1"},{"id":2,"name":"test2"}]$

And Ellie fires two events while processing the request.

Event=request_complete
, Data=[{req,’GET’,
            [<<"events">>],
            [],<<"/events">>,
            {1,1},
            [{<<"Accept">>,<<"*/*">>},
             {<<"User-Agent">>,<<"curl/7.51.0">>},
             {<<"Host">>,<<"localhost:3000">>}],
            <<>>,<0.69.0>,
            {plain,#Port<0.428>},
            {elli_middleware,
                [{mods,
                     [{json_handler,
                          [{events,
                               [[{id,1},{name,<<"test1">>}],
                                [{id,2},{name,<<"test2">>}]]}]}]}]}},
        200,
        [{<<"Connection">>,<<"Keep-Alive">>},{<<"Content-Length">>,49}],
        <<"[{\"id\":1,\"name\":\"test1\"},{\"id\":2,\"name\":\"test2\"}]">>,
        [{user_start,{1483,499053,904446}},
         {request_end,{1483,499053,910409}},
         {accepted,{1483,499053,904331}},
         {user_end,{1483,499053,910369}},
         {headers_end,{1483,499053,904423}},
         {body_end,{1483,499053,904445}},
         {request_start,{1483,499053,904411}}]]
, Args=[{events,[[{id,1},{name,<<"test1">>}],[{id,2},{name,<<"test2">>}]]}]
Event=request_closed
, Data=[]
, Args=[{events,[[{id,1},{name,<<"test1">>}],[{id,2},{name,<<"test2">>}]]}]

Towards the end of the request_complete event, you can see the timings that were used to produce the Prometheus metrics in A Simple Erlang Application, with Prometheus.

Performance Note

In the blog entry Use protobufs - now, techion does a nice comparison of Jiffy versus JSX for encoding JSON. Jiffy uses a C nif, and is more than five-times as fast as JSX. JSX isn’t that shabby, encoding 4,000 100-element lists every second.

You can find a conversation about this blog entry where techion shared his blog entry on here on Reddit.

Change Log

Jan. 6, 2017