>> Introspection

Using docker and docker-compose to share a packaged development version of an Elixir application in a team was successful. My Dockerfile and docker-compose.yml is straightforward although could be further optimized:

FROM elixir:1.9.0-alpine

RUN apk --no-cache --update add \
        git \
        erlang \
        alpine-sdk \
        gmp-dev \
        automake \
        libtool \
        inotify-tools \
        autoconf

ENV HOME="/opt/elixir" \
    MIX_HOME="/opt/elixir/mix" \
    MIX_BUILD_PATH="/opt/elixir/build" \
    HEX_HOME="/opt/elixir/hex" \
    PATH="$PATH:/opt/elixir/mix" \
    ERL_AFLAGS="-kernel shell_history enabled" \
    ELIXIR_ERL_OPTS="+C multi_time_warp"

RUN mkdir -p /opt/elixir/hex && \
    mkdir -p /opt/elixir/mix && \
    mkdir -p /opt/elixir/build && \
    mkdir -p /opt/elixir/.cache

RUN mkdir -p /opt/app && \
    mkdir -p /opt/app/deps

RUN chmod -R 777 /opt

RUN mix local.hex --force && mix local.rebar --force

WORKDIR /opt/app

EXPOSE 4000

CMD ["mix", "run", "--no-halt"]

For my development Dockerfile, I use an alpine Elixir image to minimize the download size and handpick a few packages to build dependencies. I place everything related to Elixir in /opt/elixir so that it can be cached via a docker volume and mount the project in /opt/app. I also prepare the deps folder to be locally cached which the docker-compose.yml can illustrate:

version: '3.1'
services:
  db:
    image: postgres:9.5.17-alpine
    restart: always
  app:
    build:
      context: .
      dockerfile: Dockerfile
    command: "mix phx.server"
    entrypoint: /opt/app/docker-entrypoint.sh
    volumes:
      - .:/opt/app
      - elixir:/opt/elixir
      - mix_deps:/opt/app/deps
    depends_on:
      - db
    links:
      - db:db
volumes:
  elixir:
  mix_deps:

This setup works usually for an phoenix umbrella application that requires a postgresql database. When the container runs, I have it fetch dependencies and migrations via docker-entrypoint.sh:

#!/bin/bash

set -e

echo "Updating dependencies if any uninstalled packages..."
HEX_HTTP_CONCURRENCY=1 HEX_HTTP_TIMEOUT=1200 mix deps.get

echo "Running migrations if any..."
mix do ecto.create --no-deps-check, ecto.migrate --no-deps-check

echo "Done with housecleaning"
exec "$@"

Running it is direct although with a minor chmod on mix.lock since mix.lock is updated in the container:

git pull
git checkout feature/branch

docker-compose build

sudo chmod 777 mix.lock
docker-compose up

For the most part, it works without issue; surprisingly, the most frequent issue is that hex.pm fails to fetch dependencies when rebuilding. Even with a good connection, I do run into the issue occasionally with minor annoyance. With poorer connection quality, it becomes a blocker. To mitigate this, is it possible to download all the dependencies locally and use that instead?

>> Local Mirror

In finding a solution, I came across mini_repo, a minimal hex.pm server that can mirror and serve a few remote dependencies. Sadly, it is bare, so I forked it to demonstrate how I tweaked it for docker.

Running it by default, it only mirrors decimal. To make it configurable in docker-compose, I added an space-delimited PACKAGES environment variable:

services:
  app:
    enviroment:
      PACKAGES: >-
        decimal
        phoenix

# config/dev.exs

packages = System.get_env("PACKAGES", "")
|> String.trim()
|> String.split(" ", trim: true)

config :mini_repo,
  repositories: [
    hexpm_mirror: [
      # only: ~w(decimal)
      only: packages,
    ]
  ]

# Does not fetch any package
mix run --no-halt

# Fetch one package
PACKAGES="decimal" mix run --no-halt
HEX_MIRROR=http://localhost:4000/repos/hexpm_mirror mix hex.package fetch decimal 1.8.0

# Fetch multiple packages
export HEX_MIRROR=http://localhost:4000/repos/hexpm_mirror
PACKAGES="decimal phoenix" mix run --no-halt
mix hex.package fetch phoenix 1.5.6

The mini_repo logs are promising:

17:17:21.433 [info]  GET /repos/hexpm_mirror/packages/phoenix
17:17:21.434 [info]  GET /repos/hexpm_mirror/tarballs/phoenix-1.5.6.tar
17:17:21.522 [info]  Sent 200 in 88ms
17:17:21.523 [info]  Sent 200 in 89ms

However for every package, it downloaded every available version of it:

17:16:24.717 [debug] MiniRepo.Mirror.Server fetching tarball phoenix-1.5.1.tar
17:16:25.301 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.4.17.tar"}
17:16:25.309 [debug] MiniRepo.Mirror.Server fetching tarball phoenix-1.5.2.tar
17:16:26.331 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.0-rc.0.tar"}
17:16:26.336 [debug] MiniRepo.Mirror.Server fetching tarball phoenix-1.5.3.tar
17:16:27.206 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.0.tar"}
17:16:27.228 [debug] MiniRepo.Mirror.Server fetching tarball phoenix-1.5.4.tar
17:16:28.017 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.1.tar"}
17:16:28.029 [debug] MiniRepo.Mirror.Server fetching tarball phoenix-1.5.5.tar
17:16:29.008 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.2.tar"}
17:16:29.022 [debug] MiniRepo.Mirror.Server fetching tarball phoenix-1.5.6.tar
17:16:29.706 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.3.tar"}
17:16:30.375 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.4.tar"}
17:16:30.967 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.5.tar"}
17:16:31.568 [debug] {MiniRepo.Store.Local, :put, "data/repos/hexpm_mirror/tarballs/phoenix-1.5.6.tar"}

This is problematic if the application only uses a single/fixed version of an dependency. The current application does not support a targeted version perhaps in a name-version format:

PACKAGES="decimal phoenix-1.5.6" mix run --no-halt

In this case, all of the decimal versions are downloaded while a specific version for phoenix. Hacking this behavior can be done by changing two files:

# config/dev.exs

packages = System.get_env("PACKAGES", "")
|> String.trim()
|> String.split(" ", trim: true)
|> Enum.map(fn package ->
  case String.split(package, "-", trim: true, parts: 2) do
    [package, version] ->
      {package, version}
    [package | _] ->
      {package, nil}
    [] ->
      nil
  end
end)
|> Enum.reject(&is_nil/1)

# lib/mini_repo/mirror/server.ex
defp sync(mirror) do
  # Since mirror.only is now an list of tuples instead of strings
  # We need to handle every mirror.only
  only_packages = Enum.map(mirror.only, &elem(&1, 0))

  versions = for %{name: name} = map <- versions,
            !mirror.only or name in only_packages,
            into: %{},
            do: {name, Map.delete(map, :version)}
end

defp sync_created_packages(mirror, config, diff) do
  # Before the main loop, load the pinned version map
  only_map = Enum.into(mirror.only, %{})

  {:ok, releases} = sync_package(mirror, config, name)

  # Filter out releases if a pinned version exists
  releases = if pin_version = Map.get(only_map, name, nil) do
    Enum.filter(releases, fn release -> release.version == pin_version end)
  else
    releases
  end
end

defp sync_releases(mirror, config, diff) do
  # Before the main loop, load the pinned version map
  only_map = Enum.into(mirror.only, %{})

  {:ok, releases} = sync_package(mirror, config, name)

  # Filter out releases if a pinned version exists
  releases = if pin_version = Map.get(only_map, name, nil) do
    Enum.filter(map.created, fn release -> release == pin_version end)
  else
    map.created
  end

  map = %{map | created: releases}
end

Regardless of the experimental hack, how about we test on a new minimal phoenix app:

mix archive.install hex phx_new 1.5.6
mix phx.new hello --no-html --no-webpack --no-ecto --no-gettext --no-dashboard

cd hello
HEX_MIRROR=http://localhost:4000/repos/hexpm_mirror mix deps.get

# Change port from 4000 to avoid conflict
sed -i -r s/4000/5000/ config/dev.exs

mix phx.server

Repeatedly running mix deps.get and updating PACKAGES until it succeeds:

PACKAGES="cowboy cowboy_telemetry cowlib decimal jason mime phoenix phoenix_html phoenix_pubsub plug plug_cowboy plug_crypto ranch telemetry telemetry_metrics telemetry_poller" mix run --no-halt

If I had to determine the exact version for each package without a mix.lock, it would be tedious so the previous hack is a post-setup optimization. Anyway, the test app runs so our mirror works and time to dockerize it.

>> Docker

Since this uses mix release to package the app, I copy over config/dev.exs to config/releases.exs and clean up the repo:

1. Remove the test_repo and security keys since I only need mirroring, not publishing
2. Change the package directory to data instead of priv/data
3. Set the fetch timeout to :infinity
4. Set the max concurrency to nil to use all machine cores.
5. Set the sync interval to :timer.hours(24) to sync daily instead of 5 minutes.

Since this is a released container, I use this Dockerfile instead:

FROM elixir:1.9.0-alpine AS builder

RUN mix local.hex --force && \
        mix local.rebar --force

WORKDIR /opt/app

COPY config ./config
COPY lib ./lib
COPY rel ./rel
COPY mix.exs .
COPY mix.lock .

ENV MIX_ENV=prod

RUN mix deps.get && \
        mix deps.compile && \
        mix release

FROM elixir:1.9.0-alpine AS app

WORKDIR /opt/app

COPY --from=builder /opt/app/_build .

EXPOSE 4000
ENV PORT=4000
ENV PACKAGES=""
ENV PACAKGE_DIR=/opt/app/hex

CMD ["./prod/rel/mini_repo/bin/mini_repo", "start"]

More importantly, the docker-compose.yml looks good:

version: '3.1'
services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    restart: always
    volumes:
      - ./cache:/opt/app/data
    networks:
      default:
      hexpm:
        aliases:
          - hexpm_mirror
    environment:
      PORT: "20000"
      PACKAGES: >
        cowboy
        cowboy_telemetry
        cowlib
        decimal
        jason
        mime
        phoenix
        phoenix_html
        phoenix_pubsub
        plug
        plug_cowboy
        plug_crypto
        ranch
        telemetry
        telemetry_metrics
        telemetry_poller
networks:
  hexpm:
    external: true

The important part here is the network setting which allows another container to use this. Referencing the development template, this can be done by adding the HEX_MIRROR variable and hexpm network:

version: '3.1'
services:
  app:
    networks:
      default:
      electron:
    environment:
      HEX_MIRROR: "http://hexpm_mirror:20000/repos/hexpm_mirror"
networks:
  hexpm:
    external: true

Staring the mirror and waiting for it to sync:

docker-compose build

docker network create hexpm
docker-compose up

The Elixir container should now build faster thanks to the local mirror:

docker-compose down -v

docker network create hexpm
docker-compose up

>> Conclusion

While we were able to locally cache certain Elixir dependencies in docker, it did take quite some work and it is not truly optimized. It would be nice if the package caching can be dynamic where the mirror downloads the package if it does not exist. While a static list is fine, it can be stale if a new library is introduced and is not synced which can cause the same build errors. Another lingering issue is the excess downloads which feels ironic but makes sense when the mix dependencies are not pinned to a specific version. Nonetheless, it works and time will tell if it will reduce the network fetch issue.