Set up locales in Ubuntu Docker images

The entire journey into "fixing" the locale in the Ubuntu base image on the Docker Hub started with an update to pip, the package manager for Python. The release of version 8.1.0 introduced better handling of character encodings by respecting locale settings. But why would that be a problem?

I am using the Ubuntu base image for a few of my personal projects as well as most projects at Mobify rely on it. I am also obsessed with using emoji wherever I can. Let's ignore that you might disagree with that and except the fact that some comments in our requirement.txt file used by pip contained 😞 this little fellow. Which caused pip to fail immediately after loading the file (this has been fixed and will be released as part of version 8.1.1):

pip install -r /app/requirements.txt
Exception:
Traceback (most recent call last):
  File "/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
    status = self.run(options, args)
  File "/venv/local/lib/python2.7/site-packages/pip/commands/install.py", line 287, in run
    wheel_cache
  File "/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 289, in populate_requirement_set
    wheel_cache=wheel_cache):
  File "/venv/local/lib/python2.7/site-packages/pip/req/req_file.py", line 84, in parse_requirements
    filename, comes_from=comes_from, session=session
  File "/venv/local/lib/python2.7/site-packages/pip/download.py", line 414, in get_file_content
    content = auto_decode(f.read())
  File "/venv/local/lib/python2.7/site-packages/pip/utils/encoding.py", line 23, in auto_decode
    return data.decode(locale.getpreferredencoding(False))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 4731: ordinal not in range(128)

As a result, I looked into the default encoding that is setup when you are using the ubuntu:15.10 image hosted on the Docker Hub:

$ docker run -it ubuntu:15.10 locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

You can see that the default locale is POSIX which will only work for displaying ASCII characters. So I set out to produce an updated Ubuntu image that is setup with a UTF-8 locale, in this case en_CA.UTF8.

The default locale in Ubuntu is stored in /etc/default/locale which should contain the following settings for the locale to work correctly:

# /etc/default/locale

LC_ALL=en_CA.UTF8
LANG=en_CA.UTF8
LC_CTYPE=en_CA.UTF8
LC_COLLATE=en_CA.UTF8

To use a different locale, you can replace en_CA.UTF8 with any other locale that is available. The next thing that we need to do is make sure that the locale that we are trying to setup is available on the system. To check that, just run:

$ docker run -it ubuntu:15.10 locale -a
C
C.UTF-8
POSIX

If you locale is not available, you can generate it by running locale-gen:

$ docker run -it ubuntu:15.10 bash -c "locale-gen en_CA.utf8 && locale -a"
Generating locales...
  en_CA.UTF-8... done
Generation complete.

# This is the output of 'locale -a' now
C
C.UTF-8
POSIX
en_CA.utf8

We now understand what we have to do to get a locale setup on a regular Ubuntu system. That doesn't help us in a Docker environment, unless we want to manually set this up every time a container is started.

The better way of handling it is creating a Docker image that can be used in place of the regular Ubuntu image. This seems pretty straight forward, we put the /etc/default/locale into a local file, let's call it default_locale and put together a Dockerfile that looks like this:

FROM ubuntu:15.10
MAINTAINER Mobify <ops@mobify.com>

RUN apt-get -qq update && \
    apt-get -q -y upgrade && \
    apt-get install -y sudo curl wget locales && \
    rm -rf /var/lib/apt/lists/*

# Ensure that we always use UTF-8 and with Canadian English locale
RUN locale-gen en_CA.UTF-8

COPY ./default_locale /etc/default/locale
RUN chmod 0755 /etc/default/locale

Unfortunately, that only solves half the problem. I haven't quite worked out why but when starting a now Docker container, Docker doesn't follow the same process in running scripts as when starting a login shell. Let me demonstrate with the Docker image ubuntu-with-locale created from the above Dockerfile:

$ docker run -it ubuntu-with-locale.10 bash 
root@2c29852860a3:/# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

root@2c29852860a3:/# bash --login
root@2c29852860a3:/# locale
LANG=en_CA.UTF-8
LANGUAGE=en_CA.UTF-8
LC_CTYPE="en_CA.UTF-8"
LC_NUMERIC="en_CA.UTF-8"
LC_TIME="en_CA.UTF-8"
LC_COLLATE="en_CA.UTF-8"
LC_MONETARY="en_CA.UTF-8"
LC_MESSAGES="en_CA.UTF-8"
LC_PAPER="en_CA.UTF-8"
LC_NAME="en_CA.UTF-8"
LC_ADDRESS="en_CA.UTF-8"
LC_TELEPHONE="en_CA.UTF-8"
LC_MEASUREMENT="en_CA.UTF-8"
LC_IDENTIFICATION="en_CA.UTF-8"
LC_ALL=en_CA.UTF-8

There is an easy way to fix this, although I wouldn't say it feels like the right way to do this. The Dockerfile syntax provides the ENV command (Dockerfile reference) that makes it possible to set environment variables during the build process as well as when running a container based on that image. So all we have to do is export the variables in /etc/default/locale inside of the Dockerfile. It now looks like this:

FROM ubuntu:15.10
MAINTAINER Mobify <ops@mobify.com>

RUN apt-get -qq update && \
    apt-get -q -y upgrade && \
    apt-get install -y sudo curl wget locales && \
    rm -rf /var/lib/apt/lists/*

# Ensure that we always use UTF-8 and with Canadian English locale
RUN locale-gen en_CA.UTF-8

COPY ./default_locale /etc/default/locale
RUN chmod 0755 /etc/default/locale

ENV LC_ALL=en_CA.UTF-8
ENV LANG=en_CA.UTF-8
ENV LANGUAGE=en_CA.UTF-8

We can now use the Docker image generated from this Dockerfile and be sure that we always have a UTF-8 capable shell with or without a login shell 🍻🎉. You can find a ready-made Docker image on the Hub as mobify/ubuntu:15.10:

$ docker run -it mobify/ubuntu:15.10 bash
root@02a02eb5163a:/# locale
LANG=en_CA.UTF-8
LANGUAGE=en_CA.UTF-8
LC_CTYPE="en_CA.UTF-8"
LC_NUMERIC="en_CA.UTF-8"
LC_TIME="en_CA.UTF-8"
LC_COLLATE="en_CA.UTF-8"
LC_MONETARY="en_CA.UTF-8"
LC_MESSAGES="en_CA.UTF-8"
LC_PAPER="en_CA.UTF-8"
LC_NAME="en_CA.UTF-8"
LC_ADDRESS="en_CA.UTF-8"
LC_TELEPHONE="en_CA.UTF-8"
LC_MEASUREMENT="en_CA.UTF-8"
LC_IDENTIFICATION="en_CA.UTF-8"
LC_ALL=en_CA.UTF-8

I hope this saves some of you some time and makes it safer to head into a emoji-ready future 👾.

Loading