Running Elixir in Docker Containers

转载

rongfengliang 2021-07-19 15:01:20

文章标签 编程 文章分类 后端开发

Running Elixir in Docker Containers

One of the wonderful things about Docker Containers is that you can easily freeze your environment and application in a Docker Image, and deploy it in production without worrying about installing dependencies on your servers. It just works.

This is also great for debugging. If something doesn’t work in production, you can run exactly the same image locally in matter of seconds.
The features and advantages of docker obviously don’t stop here.
In this article we see how to use Docker to run our development Elixir environment and to run multiple Elixir Nodes over a Docker bridge network.

In docker hub we find the official Elixir docker image. There are mainly two branches: the defacto image (1Gb) and the one based on alpine (80Mb). For production I would definitively go with alpine, which is much lighter.
But, for development, the main one brings many more tools and is based on Debian stretch. The image itself is based on erlang:21 that, in turn, is based on buildpack-deps:stretch, which is Debian stretch + a collection of common build dependencies.

Dowloading the image is easy.

$ docker image pull elixir:1.7.4

Where elixir is the image (elixir in this case) and 1.7.4 (the elixir version) is the tag of the image. To check its size

$ docker image inspect elixir:1.7.4 --format '{{ .Size}}'
1072576831

To download elixir 1.7.4 based on alpine, we just need to use a different tag

$ docker image pull elixir:1.7.4-alpine
$ docker image inspect elixir:1.7.4-alpine --format '{{ .Size}}'
86796332

Both images run iex if no command is passed.

$ docker run -it elixir:1.7.4

Erlang/OTP 21 [erts-10.1.3] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>

We use docker run here to create a new container from the image elixir:1.7.4.
The -it option allocates a pseudo-TTY connected to the container stdin, which is what gives you interaction with the iex running in the container.

Leave the terminal with iex open and take another terminal trying this command

$ docker container ls
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
bb667cecf5da        elixir              "iex"                    3 minutes ago       Up 3 minutes                            dazzling_rosalind

We see the container id and the name, dazzling_rosalind. The name, when not specified is randomly set by docker. If we quit from iex we see that the container stops.

$ docker container ls -a
...
bb667cecf5da  elixir  "iex" 12 minutes ago  Exited (0) 8 seconds ago  dazzling_rosalind

With the -a options we list all the containers, both stopped and running. Our dazzling_rosalind container stopped once we quit from iex. We can recover the same container starting it and attacching to its terminal session

$ docker container start dazzling_rosalind
dazzling_rosalind
$ docker container attach dazzling_rosalind
PRESS RETURN

iex(2)>

To stop and remove the container we just use rm with -f option, which forces the removal when the container is running.

docker container rm -f dazzling_rosalind

We can save files on a container and until this container is removed, these files are saved in its file system.
Containers should be treated as ephemeral though. It means that containers are thought to be stopped, removed and recreated without having to rely on configurations and files we save in them. To automatically remove the container when stopped we use the --rm option

$ docker container run -it --rm elixir:1.7.4

In general, when we want some files to stay persistent, we use volumes. There are different type of volumes, docker local volumes, cloud volumes connected to a docker container etc.. In this way we can destroy and recreate our containers without loosing important data.

In our case, since we want to use docker for development, the best idea to make a container see the project’s files is to use bind mounting. In this way we mount the directory of our local machine into the container filesystem. All the changes made by the container reflect in our local machine filesystem.

Bind Mounting

We use then the -v local_path:container_path option, where both paths need to be absolute. Let’s try to create a new elixir project using mix new and bind mounting a local directory.

$ docker container run --rm -v $PWD:/data -w /data elixir:1.7.4 mix new crypto
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating config
...

$ ls -l
drwxr-xr-x  9 alvise  staff   288  8 Dec 19:54 crypto

We’ve mounted the current directory into the contasiner’s /data path, using $PWD environment variable. We’ve also added a -w /data option which tells the container to start the command into the /data working directory. We can see that the project directory crypto is now in our current directory.

Let’s move inside the crypto directory and add a dependency into mix.exs

defp deps do
  [ {:poison, "~> 4.0"} ]
end

$ docker container run --rm -v $PWD:/app -w /app -it elixir:1.7.4 mix deps.get
Could not find Hex, which is needed to build dependency :poison
Shall I install Hex? (if running non-interactively, use "mix local.hex --force") [Yn] Y
* creating /root/.mix/archives/hex-0.18.2
...

After the dependency is downloaded and built, the container is destroyed and all the data in /root/.mix is lost. If we run again the same commands we will have again to install hex. To solve this, an similar issues, we can create a local volume

$ docker volume create elixir-mix
elixir-mix

To use it we need to add another option to the command -v volume-name:container_mount_point.

$ docker container run --rm -v elixir-mix:/root/.mix -v $PWD:/app -w /app -it  elixir:1.7.4 mix deps.get
Could not find Hex, which is needed to build dependency :poison
Shall I install Hex? (if running non-interactively, use "mix local.hex --force") [Yn] Y
* creating /root/.mix/archives/hex-0.18.2
...

$ docker container run --rm -v elixir-mix:/root/.mix -v $PWD:/app -w /app -it  elixir:1.7.4 mix deps.get
Resolving Hex dependencies...
Dependency resolution completed:
...

The first time, Hex needs to be installed. The second time the command is working properly because hex is found in the /root/.mix directory, where elixir-mix volume is mounted.

We can now run our iex loading our crypto project

$ docker container run --rm -v elixir-mix:/root/.mix -v $PWD:/app -w /app -it  elixir:1.7.4 iex -S mix
...
==> poison
Compiling 4 files (.ex)
Generated poison app
...
iex(1)> Poison.encode! %{hello: :world}
"{"hello":"world"}"

The dependencies are downloaded in deps and compiled in build project’s directory. Since these changes are reflected in our project directory via bind mounting, if we run the same command again we shouldn’t have any dependencies compilation.

Running Multiple Containers

Let’s see now how to use docker to run multiple containers in a virtual network and then run multiple Elixir Nodes.

We first need to create our bridge network, to which we will link our containers to.

$ docker network create elixir-net
$ docker network inspect elixir-net
...
"Config": [ {
    "Subnet": "172.18.0.0/16",
...

With inspect we see that our elixir-net network has 172.18.0.0/16 subnet.

To run a container and to make it join the network we’ve just created, we use the --network option

$ docker run -it --rm --network elixir-net elixir:1.7.4
iex>

Let’s use another terminal to see first the id of the container we started, and get then its IP inside elixir-net.

$ docker container ls
0c219f042eb8  elixir:1.7.4  ... 

$ docker container inspect 0c219f042eb8 | jq 
  '.[] .NetworkSettings .Networks ."elixir-net" .IPAddress'
"172.18.0.2"

inspect prints all the details of the container. With jq we filter them to get the IP address.

Let’s run another container in the same network, starting the session in bashinstead of iex. The main image has the ping command we can use to ping the other container.

$ docker run -it --rm --network elixir-net elixir:1.7.4 bash
root@aeb8101fbb9b:/# ping 172.18.0.2
PING 172.18.0.2 (172.18.0.2) 56(84) bytes of data.
64 bytes from 172.18.0.2: icmp_seq=1 ttl=64 time=0.095 ms

Even if it works, we mustn’t use the IPs directly: each container has its own IP and if a container is destroyed and re-created the IP could not be the same.
Hopefully we can rely on names. To make them work we need to start our container with the --name option. We also add the -h option to set the hostname (which will be useful later running multiple Elixir nodes).
Remove first all the running containers so we can make a fresh start. Then run these two containers in two seperate terminals

# in terminal 1
$ docker run -it --rm --network elixir-net --name elixir-1 
  -h elixir-1 elixir:1.7.4 bash

root@elixir-1:/#



# in terminal 2
$ docker run -it --rm --network elixir-net --name elixir-2 
  -h elixir-2 elixir:1.7.4 bash

root@elixir-2:/# ping elixir-1
PING elixir-1 (172.18.0.2) 56(84) bytes of data.
64 bytes from elixir-1.elixir-net (172.18.0.2): icmp_seq=1 ttl=64 ..

Now we have all we need to start iex in two separate containers and to connect the two nodes. To make them able to connect we need to use the --sname iex option, which assigns a short name to the node. In our case the name can be the same since we have different hostnames (-h option in docker).

Running Elixir in Docker Containers_编程_02

# Terminal 1
$ docker run -it --rm --network elixir-net --name elixir-1 
  -h elixir-1 elixir:1.7.4 iex --sname docker
iex(docker@elixir-1)1>

# Terminal 2
$ docker run -it --rm --network elixir-net --name elixir-2 
  -h elixir-2 elixir:1.7.4 iex --sname docker

iex(docker@elixir-2)1> Node.connect :"docker@elixir-1"
false

And we see an error in the terminal-1

iex(docker@elixir-1)1>
Connection attempt from disallowed node :"docker@elixir-2"

Something went wrong. What’s missing? The secret cookie needs to be the same. The best way to pass a secret in general is using environment variables. In a docker/kubernetes production environment we can safely store secrets and map them to environment variables. Right now we set the environment variable passing the plain cookie.
Note we are passing '$COOKIE' with the quotes. doing so we avoid that our shell substitutes $COOKIE with our system $COOKIE environment variable.

# Terminal 1
$ docker run -it --rm -e COOKIE=secret --network elixir-net --name elixir-1 
  -h elixir-1 elixir:1.7.4 iex --sname docker --cookie '$COOKIE'

# Terminal 2
$ docker run -it --rm -e COOKIE=secret --network elixir-net --name elixir-2 
  -h elixir-2 elixir:1.7.4 iex --sname docker --cookie '$COOKIE'

iex(docker@elixir-2)1> Node.connect :"docker@elixir-1"
true
iex(docker@elixir-2)2> Node.list
[:"docker@elixir-1"]

Fantastic! We can now have fun running distributed elixir code

# Terminal 2
iex(docker@elixir-2)3> Agent.start_link(
    fn -> {:hello, :world} end, 
    name: {:global, MyAgent})
{:ok, #PID<0.118.0>}

And in the other container, in Terminal 1, we can now access to the MyAgentprocess.

# Terminal 1
iex(docker@elixir-1)1> Agent.get({:global, MyAgent}, fn state-> state end)
{:hello, :world}

Wrap Up

We saw how easy it is to use docker containers to develop and run locally our Elixir’s projects, also distributed across multiple nodes. This is just the foundation. where to start adding other services, like a postgres container linked to our elixir-netnetwork, so we can start working with Ecto in matter of seconds.