Showing posts with label dockerfile. Show all posts
Showing posts with label dockerfile. Show all posts

Thursday, March 09, 2023

Docker container user cannot write to shared volume

Something that has been baffling me for a while was how to mount a volume in a docker container such that a specific user inside said container would have the same group ID (gid) as a given user outside the container. But, before we delve into that, let's simplify the problem a bit with a few assumptions:

Assumptions

  • The "user outside the container" is a user in the docker server. This user can run docker (is in the docker group). We shall call this user outbob, with uid = gid = 1500
  • The "user inside the container" is a user in the docker container, which is created by the Dockerfile. This user will be known as inbob, with uid = gid = 1995
  • This is a bare example, the smallest proof of concept I could come up with. I may update this article later with a link to a practical application as soon as I shove it in my github account.

The Problem

Let's create a really simple Dockerfile that shows the problem in /tmp/bob for no reason whatsoever.

outbob@dockerbox:/tmp/bob$ cat Dockerfile
# Set the base image to Ubuntu
FROM ubuntu

ENV DEVUSER inbob
ENV DEVID 1995

# Create user
RUN useradd -m --shell /bin/bash -u $DEVID $DEVUSER

USER $DEVUSER
ENV WD /home/${DEVUSER}
WORKDIR ${WD}
outbob@dockerbox:/tmp/bob$

We build the image from that Dockerfile as usual:

outbob@dockerbox:/tmp/bob$ docker build -t bob .
Sending build context to Docker daemon  2.048kB
Step 1/7 : FROM ubuntu
 ---> 27941809078c
Step 2/7 : ENV DEVUSER inbob
 ---> Running in e8501dbe8398
[...]
 ---> Running in 963671a09a5f
Removing intermediate container 963671a09a5f
 ---> a6049000bfda
Successfully built a6049000bfda
Successfully tagged bob:latest
outbob@dockerbox:/tmp/bob$

And run it, passing the same directory we used because I am not in the mood of being original. Yes, I could pass the command I wanted to execute directly from the docker run statement instead of starting bash and then running the command. Deal with it.

outbob@dockerbox:/tmp/bob$ docker run -i --rm -v /tmp/bob:/bob -t bob bash
inbob@3d7c097a38d1:~$ id
uid=1995(inbob) gid=1995(inbob) groups=1995(inbob)
inbob@3d7c097a38d1:~$ exit
exit
outbob@dockerbox:/tmp/bob$ 

So far so good. Now what happens if I try to become an user with the same uid and gid as outbob?

outbob@dockerbox:/tmp/bob$ docker run -i --rm  -u $(id -u):$(id -g) -v /tmp/bob:/bob -t bob bash
groups: cannot find name for group ID 1500
I have no name!@c4355f38747a:/home/inbob$ id
uid=1500 gid=1500 groups=1500
I have no name!@c4355f38747a:/home/inbob$ cd /bob
I have no name!@c4355f38747a:/bob$ touch nose
I have no name!@c4355f38747a:/bob$ ls -lh
total 4.0K
-rw-r--r-- 1 1500 1500 200 Feb 14 01:01 Dockerfile
-rw-r--r-- 1 1500 1500   0 Feb 14 18:31 nose
I have no name!@c4355f38747a:/bob$ id inbob
uid=1995(inbob) gid=1995(inbob) groups=1995(inbob)
I have no name!@c4355f38747a:/bob$

It works in that I can write to the volume but I am now a completely different user; a user with no name. Since I am not Clint Eastwood, I would rather be inbob. Is there a solution?

The Solution (so far)

Note: if you do not want to cut-n-paste the following excerpts, I also put this code in a repo.

We stablished I do not want to find out inbob created something in /tmb/bob as its default gid. The cleanest solution I found so far is to add inbob to the same group outbob used to create /tmb/bob, and then ensure anyone belonging to that group can write to this directory as a member of that group. So, let's do whatever I just said!

Set the directory in question to inherit the gid

We will set the setgid attribute for the directory:

outbob@dockerbox:/tmp/bob$ ls -ld /tmp/bob
drwxr----- 2 outbob outbob 4096 Mar  9 08:55 /tmp/bob/
outbob@dockerbox:/tmp/bob$
outbob@dockerbox:/tmp/bob$ chmod 2770 /tmp/bob
outbob@dockerbox:/tmp/bob$
outbob@dockerbox:/tmp/bob$ ls -ld /tmp/bob
drwxrws--- 2 outbob outbob 4096 Mar  9 08:55 /tmp/bob/
outbob@dockerbox:/tmp/bob$

The little s after the group permissions indicate we have been successful.

Configure the docker build to add user to proper group at runtime

We need to change the Dockerfile a bit and then create a docker-entrypoint.sh file:

outbob@dockerbox:/tmp/bob$ cat Dockerfile
# Set the base image to Ubuntu
FROM ubuntu

ENV DEVUSER inbob
ENV DEVUID 1995
ENV DEVGID 1995
ENV EXTGID 1999

# Create user
RUN useradd -m --shell /bin/bash -u $DEVUID $DEVUSER

# USER $DEVUSER
ENV WD /home/${DEVUSER}
WORKDIR ${WD}

# Put the entrypoint script somewhere we can find
COPY docker-entrypoint.sh /entrypoint.sh
RUN chmod 0700 /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

outbob@dockerbox:/tmp/bob$ cat docker-entrypoint.sh
#!/bin/sh
set -e

groupadd -g $EXTGID extgroup
adduser $DEVUSER extgroup

su - $DEVUSER

# And we are done here
exec "$@"
outbob@dockerbox:/tmp/bob$

Test time!

After we buld the new image we run it passing the default gid for outbob. Nothing stopping us from passing a different gid if it is what we need; docker does not care.

outbob@dockerbox:/tmp/bob$ docker run -i --rm  -e EXTGID=$(id -g) -v /tmp/bob:/bob -t bob bash
Adding user `inbob' to group `extgroup' ...
Adding user inbob to group extgroup
Done.
inbob@2a7339336f3e:~$ cd /bob
inbob@2a7339336f3e:/bob$ ls -l
total 8
-rw-r--r-- 1 5000 extgroup 389 Mar  9 13:42 Dockerfile
-rw-r--r-- 1 5000 extgroup 122 Mar  9 13:48 docker-entrypoint.sh
inbob@2a7339336f3e:/bob$ touch nose
inbob@2a7339336f3e:/bob$ ls -l
total 8
-rw-r--r-- 1    5000 extgroup 389 Mar  9 13:42 Dockerfile
-rw-r--r-- 1    5000 extgroup 122 Mar  9 13:48 docker-entrypoint.sh
-rw-rw-r-- 1 inbob extgroup   0 Mar  9 14:00 nose
inbob@2a7339336f3e:/bob$ exit
logout
root@2a7339336f3e:/home/inbob# exit
exit
outbob@dockerbox:/tmp/bob$ ls -l
total 8
-rw-r--r-- 1 outbob outbob 122 Mar  9 08:48 docker-entrypoint.sh
-rw-r--r-- 1 outbob outbob 389 Mar  9 08:42 Dockerfile
-rw-rw-r-- 1 1995 outbob   0 Mar  9 09:00 nose
outbob@dockerbox:/tmp/bob$ rm nose
outbob@dockerbox:/tmp/bob$

Other than getting out is now a two-step process, which may not be important when running a container in the background, the only telltale we were up to not good is that the uid for the file we created does not match any in our docker server. But, outbob can still delete the file.

And that is how we bring harmony in the divided worlds of bob.

Wednesday, July 27, 2016

Starting Tomcat manually in Docker

So I needed to test some settings between CentOS 6 + Apache Tomcat 6 + Java 6 and CentOS 7 + Tomcat7 + Java 8. Best way I found to do these quick tests is to build them in docker: write a quick dockerfile, spool it up, test, and blow it up.

Now, when I built and ran the tomcat 6 + CentOS 6 + Java 6 setup by connecting to the container and manually starting tomcat by typing

service tomcat6 start
It worked fine. So I then built the tomcat 7 + CentOS 7 + Java 8, and tried to start it

[root@tomcat ~]# systemctl start tomcat.service
Failed to get D-Bus connection: Operation not permitted
[root@tomcat ~]# 

Since systemd could not start it, and I could not figure out why (I do not take solace knowing I am not the only one), I tried starting it even more manually:

[root@tomcat /]# /usr/sbin/tomcat start
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory
[root@tomcat /]#

This is the time to do what everyone does at a time like this: look for answers online. All I got was someone asking the very same question. It seems if we want some answers we will need do more exploring on our own.

With that in mind, let's see those two lines we are being barked about:

[root@tomcat /]# sed -n '21p' /usr/sbin/tomcat
    . /etc/sysconfig/${NAME}
[root@tomcat /]# sed -n '39p' /usr/sbin/tomcat
  ${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
[root@tomcat /]#

Not much help here, but we will revisit that later. First, let's run the bash script again, but this time in a debugging (-x) mode:

[root@tomcat /]# bash -x /usr/sbin/tomcat start
+ '[' -r /usr/share/java-utils/java-functions ']'
+ . /usr/share/java-utils/java-functions
++ _load_java_conf
++ local IFS=:
++ local java_home_save=
++ local java_opts_save=
++ local javaconfdir
++ local conf
++ unset _javadirs
++ unset _jvmdirs
++ set -- /etc/java
++ _log 'Java config directories are:'
++ '[' -n '' ']'
++ for javaconfdir in '"$@"'
++ _log '  * /etc/java'
++ '[' -n '' ']'
++ for javaconfdir in '"$@"'
++ conf=/etc/java/java.conf
++ '[' '!' -f /etc/java/java.conf ']'
++ local IFS
++ local JAVA_LIBDIR
++ local JNI_LIBDIR
++ local JVM_ROOT
++ '[' -f /etc/java/java.conf ']'
++ _log 'Loading config file: /etc/java/java.conf'
++ '[' -n '' ']'
++ . /etc/java/java.conf
+++ JAVA_LIBDIR=/usr/share/java
+++ JNI_LIBDIR=/usr/lib/java
+++ JVM_ROOT=/usr/lib/jvm
++ _javadirs=/usr/share/java:/usr/lib/java
++ _jvmdirs=/usr/lib/jvm
++ _load_java_conf_file /root/.java/java.conf
++ local IFS
++ local JAVA_LIBDIR
++ local JNI_LIBDIR
++ local JVM_ROOT
++ '[' -f /root/.java/java.conf ']'
++ _log 'Skipping config file /root/.java/java.conf: file does not exist'
++ '[' -n '' ']'
++ _javadirs=/usr/share/java:/usr/lib/java
++ _jvmdirs=/usr/lib/jvm
++ '[' -d '' ']'
++ '[' -n '' ']'
++ '[' _ '!=' _off -a -f /usr/lib/abrt-java-connector/libabrt-java-connector.so
-a -f /var/run/abrt/abrtd.pid ']'
++ _log 'ABRT Java connector is disabled'
++ '[' -n '' ']'
+ '[' -z '' ']'
+ TOMCAT_CFG=/etc/tomcat/tomcat.conf
+ '[' -r /etc/tomcat/tomcat.conf ']'
+ . /etc/tomcat/tomcat.conf
++ TOMCAT_CFG_LOADED=1
++ TOMCATS_BASE=/var/lib/tomcats/
++ JAVA_HOME=/usr/lib/jvm/jre
++ CATALINA_HOME=/usr/share/tomcat
++ CATALINA_TMPDIR=/var/cache/tomcat/temp
++ SECURITY_MANAGER=false
+ '[' -r /etc/sysconfig/ ']'
+ . /etc/sysconfig/
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
+ set_javacmd
+ local IFS
+ local cmd
+ '[' -x '' ']'
+ set_jvm
+ local IFS=:
+ local cmd
+ local cmds
+ _set_java_home
+ local IFS=:
+ local jvmdir
+ local subdir
+ local subdirs
+ '[' -n /usr/lib/jvm/jre ']'
+ '[' -z '' ']'
++ readlink -f /usr/lib/jvm/jre/..
+ JVM_ROOT=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el7_2.x86_64
+ return
+ '[' -n /usr/lib/jvm/jre ']'
+ return
+ for cmd in jre/sh/java bin/java
+ JAVACMD=/usr/lib/jvm/jre/jre/sh/java
+ '[' -x /usr/lib/jvm/jre/jre/sh/java ']'
+ for cmd in jre/sh/java bin/java
+ JAVACMD=/usr/lib/jvm/jre/bin/java
+ '[' -x /usr/lib/jvm/jre/bin/java ']'
+ _log 'Using configured JAVACMD: /usr/lib/jvm/jre/bin/java'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ return 0
+ cd /usr/share/tomcat
+ '[' '!' -z '' ']'
+ '[' -n '' ']'
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-jul
i.jar
++ build-classpath commons-daemon
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-jul
i.jar:/usr/share/java/commons-daemon.jar
+ '[' start = start ']'
+ '[' '!' -z '' ']'
[root@tomcat /]# + /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bo
otstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon
.jar -Dcatalina.base= -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -D
java.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/conf/logg
ing.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
 org.apache.catalina.startup.Bootstrap start
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory
[root@tomcat /]#

If you never ran a bash script with the -x option, you should since it shows the steps being performed by the script, including tests, as it runs. For instance, you can see it starts by learning a lot about the current Java installation. After that, it loads some file, TOMCAT_CFG=/etc/tomcat/tomcat.conf, and then gives the first error message:

+ TOMCAT_CFG=/etc/tomcat/tomcat.conf
+ '[' -r /etc/tomcat/tomcat.conf ']'
+ . /etc/tomcat/tomcat.conf
++ TOMCAT_CFG_LOADED=1
++ TOMCATS_BASE=/var/lib/tomcats/
++ JAVA_HOME=/usr/lib/jvm/jre
++ CATALINA_HOME=/usr/share/tomcat
++ CATALINA_TMPDIR=/var/cache/tomcat/temp
++ SECURITY_MANAGER=false
+ '[' -r /etc/sysconfig/ ']'
+ . /etc/sysconfig/
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
Now, /etc/tomcat/tomcat.conf (same thing as /usr/share/tomcat/conf/tomcat.conf) defines a few global (to tomcat) to variables. The top of the file also explains it is the file where you should define variables that are custom to your system but global to all tomcat instances being run here. For instance, when I built the tomcat6 container, I had

JAVA_HOME="/usr/lib/jdk1.6.0_41"

because that was the specific java version I wanted to run. Now, if we look not only at line 21 in /usr/sbin/tomcat but also around said line, we can see it wants to load a file in /etc/sysconfig

# Get instance specific config file
if [ -r "/etc/sysconfig/${NAME}" ]; then
    . /etc/sysconfig/${NAME}
fi

If we look at /etc/sysconfig,

[root@tomcat ~]# ls /etc/sysconfig/
network  network-scripts  rdisc  tomcat
[root@tomcat ~]#

It sure makes me think that $NAME = "tomcat" and $NAME is not defined.

For the second error message we should examine the following lines

[root@tomcat /]# + /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bo
otstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon
.jar -Dcatalina.base= -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -D
java.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/conf/logg
ing.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
 org.apache.catalina.startup.Bootstrap start
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory

That really looks like it wants to write to the log file catalina.out but can't find it. So we take a look at the lines around line 39:

if [ "$1" = "start" ]; then
  ${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
    -classpath "$CLASSPATH" \
    -Dcatalina.base="$CATALINA_BASE" \
    -Dcatalina.home="$CATALINA_HOME" \
    -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
    -Djava.io.tmpdir="$CATALINA_TMPDIR" \
    -Djava.util.logging.config.file="${CATALINA_BASE}/conf/logging.properties" \
    -Djava.util.logging.manager="org.apache.juli.ClassLoaderLogManager" \
    org.apache.catalina.startup.Bootstrap start \
    >> ${CATALINA_BASE}/logs/catalina.out 2>&1 &
    if [ ! -z "$CATALINA_PID" ]; then
      echo $! > $CATALINA_PID
    fi

where we find the line

>> ${CATALINA_BASE}/logs/catalina.out 2>&1 &

That makes me think that the $CATALINA_BASE = "/usr/share/tomcat" since

[root@tomcat ~]# ls /usr/share/tomcat/logs/
catalina.out
[root@tomcat ~]#

Now, /etc/sysconfig/tomcat knows about $CATALINA_BASE even though it really does not define it (commented out):

#CATALINA_BASE="/usr/share/tomcat"

Sounds like we need to define $NAME and $CATALINA_BASE somewhere. My vote would be for
/etc/tomcat/tomcat.conf because it claims it is where we put custom stuff.

# For tomcat.service it's /etc/sysconfig/tomcat, for
# tomcat@instance it's /etc/sysconfig/tomcat@instance.

# THE TWO LINES I MENTIONED IN THE ARTICLE
NAME="tomcat"                                   
CATALINA_BASE="/usr/share/tomcat"

# This variable is used to figure out if config is loaded or not.
TOMCAT_CFG_LOADED="1"

# In new-style instances, if CATALINA_BASE isn't specified, it will
# be constructed by joining TOMCATS_BASE and NAME.
TOMCATS_BASE="/var/lib/tomcats/"

After that, I was able to start it and verify it was indeed running

[root@tomcat tomcat]# ps -ef|grep tomcat
root       352     1  8 13:09 ?        00:00:01 /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start
root       372     1  0 13:09 ?        00:00:00 grep --color=auto tomcat
[root@tomcat tomcat]#

I wrote a shorter version of this article as a reply to the question I found online and mentioned earlier in this article. I hope it will be useful to the original poster.



Tuesday, June 23, 2015

Ubuntu Docker container with multiple architectures

Here's yet another quickie: let's say you created a docker Ubuntu container to do some sort of cross compilation. You know, maybe your container is 64bit Intel and you need to spit out some static 32bit Intel stuff. Or PPC for that matter. Now your development environment needs the 32bit version of libncurses-dev, which for ubuntu 14.04 would be libncurses5-dev:i386. We do need to specify the architecture (:i386) if we are not using the default one..

So you scribble up a quick Dockerfile:

############################################################
# Dockerfile to build a ubuntu container image thingie
# Based on Ubuntu (Duh!)
############################################################

# Set the base image to Ubuntu
FROM ubuntu:14.04

################## BEGIN INSTALLATION ######################

RUN apt-get update && apt-get -y upgrade 

RUN apt-get install -y build-essential curl libc6-dev libncurses5-dev:i386

##################### INSTALLATION END #####################
And then you do some building:
ducker@boot2docker:~/docker/test$ docker build -t raubvogel/test.
Sending build context to Docker daemon 4.096 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:14.04
 ---> 6d4946999d4f
Step 1 : RUN apt-get update && apt-get -y upgrade
 ---> Running in 2d035053a431
Ign http://archive.ubuntu.com trusty InRelease
Ign http://archive.ubuntu.com trusty-updates InRelease
Ign http://archive.ubuntu.com trusty-security InRelease
Hit http://archive.ubuntu.com trusty Release.gpg
[...]
Processing triggers for ureadahead (0.100.0-16) ...
Setting up initscripts (2.88dsf-41ubuntu6.2) ...
guest environment detected: Linking /run/shm to /dev/shm
 ---> c1434c42218e
Removing intermediate container 2d035053a431
Step 2 : RUN apt-get install -y build-essential zip bzr curl libc6-dev libncurses5-dev:i386
 ---> Running in 2ac39f9430c2
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package libncurses5-dev
INFO[0017] The command [/bin/sh -c apt-get install -y build-essential zip bzr curl libc6-dev libncurses5-dev:i386] returned a non-zero code: 100
ducker@boot2docker:~/docker/test$
And then it barks at the libncurses5-dev entry. What's going on?

The Debian Multiarch HOWTO seems to have a bit of a clue. I was going to write a long winded explanation but I am bored. So here is the short version:

dpkg --add-architecture i386
apt-get update
apt-get install libncurses5-dev:i386
See the dpkg --add-architecture i386 line? It tells the host that we are can do 32bit Intell thingies. The next line is there just to feed us with the 32bit repository data, so when we look for a 32bit package we can get it. Let apply that to our Dockerfile:
############################################################
# Dockerfile to build a ubuntu container image thingie
# Based on Ubuntu (Duh!)
############################################################

# Set the base image to Ubuntu
FROM ubuntu:14.04

################## BEGIN INSTALLATION ######################

# We need i386 crap
RUN dpkg --add-architecture i386
# Business as usual
RUN apt-get update && apt-get -y upgrade 

RUN apt-get install -y build-essential curl libc6-dev libncurses5-dev:i386

##################### INSTALLATION END #####################
And build it again.
ducker@boot2docker:~/docker/test$ docker build -t raubvogel/test.
Sending build context to Docker daemon 4.608 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:14.04
 ---> 6d4946999d4f
Step 1 : RUN dpkg --add-architecture i386
 ---> Running in 89382c3c3469
 ---> f367d3357fc4
Removing intermediate container 89382c3c3469
Step 2 : RUN apt-get update && apt-get -y upgrade
 ---> Running in 6d0fc9519029
Ign http://archive.ubuntu.com trusty InRelease
Ign http://archive.ubuntu.com trusty-updates InRelease
[...]
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 build-essential : Depends: gcc (>= 4:4.4.3) but it is not going to be installed
                   Depends: g++ (>= 4:4.4.3) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
INFO[0002] The command [/bin/sh -c apt-get install -y build-essential  zip bzr curl libc6-dev libncurses5-dev:i386] returned a non-zero code: 100
ducker@boot2docker:~/docker/test$
Hmmm, that didn't work. Crap. Let's massage Dockerfile a bit:
############################################################
# Dockerfile to build a ubuntu container image thingie
# Based on Ubuntu (Duh!)
############################################################

# Set the base image to Ubuntu
FROM ubuntu:14.04

################## BEGIN INSTALLATION ######################

# We need i386 crap
RUN dpkg --add-architecture i386
# Business as usual
RUN apt-get update && apt-get -y upgrade

RUN apt-get install -y build-essential  &&\
    apt-get install -y zip bzr curl libc6-dev libncurses5-dev:i386

##################### INSTALLATION END #####################
And this time it works fine:
ducker@boot2docker:~/docker/test$ docker build -t raubvogel/test.
Sending build context to Docker daemon 4.608 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:14.04
[...]
Setting up libpam-systemd:amd64 (204-5ubuntu20.12) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
invoke-rc.d: unknown initscript, /etc/init.d/systemd-logind not found.
invoke-rc.d: policy-rc.d denied execution of start.
Processing triggers for libc-bin (2.19-0ubuntu6.6) ...
Processing triggers for ca-certificates (20141019ubuntu0.14.04.1) ...
Updating certificates in /etc/ssl/certs... 173 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ...
Processing triggers for ureadahead (0.100.0-16) ...
 ---> d86a5113c577
Removing intermediate container 32e0cc9a0b5a
Successfully built d86a5113c577
ducker@boot2docker:~/docker/test$ 

What I found out is that it does not like to have the build-essentials in the same install statement as libncurses5-dev:i386. The easiest solution is to have build-essentials on its own install statement and everyone else in the next one. And life is once again well.

Wednesday, February 04, 2015

Creating a git server in docker... with NFS and custom port and ssh key pair

I usually like to start my post describing what we will try to accomplish here, but I think I can't do any better than what the title states. So, let's say if I can come up with a convincing excuse. Well, all I can come up with right now is that I think it is wasteful to create an entire VM to run a distributed version control system. At least one that does not have helpful paperclips with eyes and other features requiring you to download crap. And, it is nice to know if the docker host (or cloud) takes a dump, we can bring this service back rather quickly. For this article we will use git; some other time we can talk about svn.

The git server I will be using is gitolite because it is reasonably simple and quick to manage and get going. What I really like on it is that the accounts for the users using git are not accounts in the host itself, so they cannot login to the machine hosting git. By default the git users login using

Since I am lazy, I will store the repositories in a NFS fileshare that is mounted into the container at runtime. We talked about how to do the mounting in a previous article.

Assumptions

  1. gitolite running off /home/git
  2. We will connect to the git server on port 2022. I chose that because I need port 22 for to ssh into the docker host. Yes, they are in different IPs (in my case completely different VLANs), but I am weird like that.
  3. We will use ssh key pair authentication. And will use a different key than the default one. Note you can authenticate against LDAP, but that would go against what I wrote in the name of this article.
  4. gitolite being run as user git
  5. I am running this on centos6.

Install

  1. I created a CNAME for the docker host, gitserver.example.com, so it looks pretty.
  2. In the NFS server, create a fileshare owned by the user git, which in this example has uid=1201.
  3. We will need to create a ssh key pair for the gitadmin. I created my pair by doing something like
    ssh-keygen -t rsa -C gitadmin -f ~/.ssh/gitadmin
    You will need to copy ~/.ssh/gitadmin.pub into the docker host by whatever means you desire.
  4. I create a directory in the docker host to put all the files (docker-entrypoint.sh and Dockerfile) related to this container. Here is the Dockerfile
    ############################################################
    # Dockerfile to build a gitolite git container image
    # Based on CentOS
    ############################################################
    
    # Set the base image to CentOS
    FROM centos:centos6
    
    # File Author / Maintainer
    MAINTAINER Mauricio Tavares "raubvogel@gmail.com"
    
    ################## BEGIN INSTALLATION ######################
    # We need epel
    RUN rpm -Uvhi http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.no
    arch.rpm && \
        sed -i -e 's/^enabled=1/enabled=0/' /etc/yum.repos.d/epel.repo
    
    # We need NFS, openssh, and git
    # And ssmtp (from Epel)
    RUN yum update -y && yum install -y \
            git \
            nfs-utils \
            openssh-server && \
        yum install -y ssmtp --enablerepo=epel
    
    # Configure NFS
    RUN sed -i -e '/^#Domain/a Domain = example.com' /etc/idmapd.conf
    
    ##################### INSTALLATION END #####################
    
    # Create git user
    RUN adduser -m -u 1201 git
    
    # Configure ssmtp
    
    # Configure sshd
    RUN sed -i -e 's/^#Port .*$/Port 2022/' \
               -e 's/^#PermitRootLogin .*$/PermitRootLogin no/' \
               /etc/ssh/sshd_config && \
        sed -i -e \
            's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.s
    o@g' \
             /etc/pam.d/sshd && \
        ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa && \
        ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
    
    # And a mountpoint for repositories
    # Note: can't NFS mount from dockerfile, so will do it in an entrypoint script
    RUN su - git -c 'mkdir repositories'
    
    
    # And now the git server
    # Gitolite admin: gitadmin (it is based on the name of the pub key file)
    RUN su - git -c 'mkdir bin' && \
        su - git -c 'git clone git://github.com/sitaramc/gitolite' && \
        su - git -c 'mkdir -m 0700 .ssh' && \
        su - git -c 'echo "ssh-rsa AAAAB3NzaC1yc2EAASLDAQCOOKIEQDehf5hxGq9//34yrsL
    [...]
    7CfSpbiP gitadmin" > .ssh/gitadmin.pub'
    # The rest will be configured in the entrypoint script
    
    # Put the entrypoint script somewhere we can find
    COPY docker-entrypoint.sh /entrypoint.sh
    ENTRYPOINT ["/entrypoint.sh"]
    
    EXPOSE 2022
    # Start service
    CMD ["/usr/sbin/sshd", "-D"]
    

    Where

    1. You will need to put the public key gitadmin.pub between the double quotes in the line beginning with su - git -c 'echo "ssh-rsa.
    2. I am running a lot of things in the Dockerfile as user git.
    3. The NFS setup was mentioned before, so I will not bother with it right now.
    4. I forgot to add the setup for ssmtp. I will think about that sometime later.
  5. The docker-entrypoint.sh file looks vaguely like this:
    #!/bin/sh
    set -e
    
    # Mount git's repositories
    mount.nfs4 fileserver.example.com:/git /home/git/repositories
    
    su - git -c 'gitolite/install -to $HOME/bin'
    # setup gitolite with yourself as the administrator
    su - git -c 'gitolite setup -pk .ssh/gitadmin.pub'
    
    # And we are out of here
    exec "$@"
  6. So far so good? Ok, so let's build the image. I will call it git.
    docker build -t git .
    If the last build message looks like
    Successfully built 6fb1ac15b47a
    chances are the build was successful and you can go to the next step. Otherwise, figure out what went boink.
  7. Now start the service. Remember you need to run in priviledge mode because of NFS. Since this is a test, I am calling the contained test-git.
    docker run --privileged=true -d -P -p 3306:3306 --name test-git git

Setup and Testing

  1. Let's start testing by seeing what repositories we can see as an admin:
    $ /usr/bin/ssh -i /home/raub/.ssh/gitadmin git@gitserver.example.com info
    hello gitadmin, this is git@docker running gitolite3 v3.6.2-12-g1c61d57 on git 1.7.1
    
     R W    gitolite-admin
     R W    testing
    
    
    FYI, testing is a repo everyone allowed to use the git server can play with. Think of it as a, as its name implies, test repo. Since that worked we can proceed to the next step.
  2. Now we should edit your .ssh/config file to access the gitadmin repository.
    cat >> ~/.ssh/config << EOF
    Host gitadmin
            Hostname        gitserver.example.com 
            User            git
            Port            2022
            identityfile    /home/raub/.ssh/gitadmin
            protocol        2
            compression     yes
    EOF
    Yes, you can ask about what to do if you have a script that needs to pull stuff out of a repo, and I will tell you to wait for the next installment. This article deals with getting it to work.
  3. Retrieving the admin repository is now much easier. So, instead of having to do something like
    git clone ssh://git@gitserver.example.com:[port]/gitolite-admin
    which would also require us to feed the key (or rename it as the default which IMHO is a bad idea), thanks to the previous step we can now lazily do
    git clone gituser:/gitolite-admin
  4. Repository config file is in gitolite-admin/conf/gitolite.conf
  5. Adding a new user (and perhaps a repository)
    1. Get user's ssh public key, say raub.pub. How did the user created the ssh key pair? Don't know, don't care.
    2. Copy raub.pub to gitolite-admin/keydir. NOTE:file must be named after username user will use to connect to the git server; it does not need to have anything to do with the user's normal/real username.
    3. Create a repository for the user. Let's give it a nice and snuggly name, like somethingawful
      cat >> conf/gitolite.conf << EOF
      
      repo somethingawful
          RW      = raub
      EOF
    4. Commit changes
      git add conf keydir
      git commit -m 'Added new user and config a repo'
      git push origin master
  6. Now, let's pretend we are the user (i.e. what the user should do/see). Which repos can you/user see? If this looks like a step we did before, it is. Just using a new user.
    $ ssh -i /home/raub/.ssh/raub git@gitserver.example.com info 
    hello raub, this is git@docker running gitolite3 v3.6.2-12-g1c61d57 on git 1.7.1
    
    
     R W    somethingawful
     R W    testing
    
    There is somethingswful!
  7. Let's grab somethingawful
    $ git clone gituser:/somethingawful
    Cloning into 'somethingawful'...
    warning: You appear to have cloned an empty repository.
    Checking connectivity... done.
  8. Edit, commit, and off you go

I will put all those guys in my github account later on. And shall sneakly update this post afterwards.

Sunday, February 01, 2015

Entrypoint scripts and data persistency: mounting a NFS fileshare inside a docker container

This is a quick article; you have been warned.

The traditional way to have persistent data in a docker container is to feed it with a volume from the docker host. You begin by locating a directory, be it in the local docker host drive, network mounted to the docker host, or a volume in another docker container. Then feed it to the container using the VOLUME statement defined in the Dockerfile or to command line. We all know that. But what some (very few we hope) of you might not have been aware of is the volume is only mounted/attached to the container when you tell docker to run the container, which might make running/configuring something that require files in those volumes a bit challenging.

At this point I would expect one of you to shout "Aha! But you can then use an entrypoint script to do all that monkeying that needs to happen after volume is mounted and before whatever service this container provides starts!" And you would be quite right! Here's a quick example in case we lost somebody: if our Dockerfile ended up like this:

# Put the entrypoint script somewhere we can find
COPY docker-entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

EXPOSE 22
# Start service
CMD ["/usr/sbin/sshd", "-D"]

we would have created a file called entrypoint.sh in the root path of the container, which is run just after the volume is created and before the, in this example, sshd service is started. Said entrypoint file could do, I don't know:

#!/bin/sh
set -e

# Do something in /home, which is the mounted volume
/do-something-at-home.sh

# And we  are done here
exec "$@"

point is that do-something-at-home.sh is called only after the persistent volume is mounted into the container.

What if you want to mount a network fileshare straight into the container? Before I have time to answer that, someone from the audience will jump and state "Why would you want to do that? You can mount the network fileshare in the docker host and then use VOLUME just like Wicked Witch of the West intended!" What if I don't want to do that? What if I can't for whatever reason mount a network fileshare on the docker host?

The answer is that it is completely doable; you just have to ask the entrypoint script to do the mounting for you. To show what I mean I will use the example of mounting a NFSv4 fileshare that is supposed to be used by some website. So we modify our little entrypoint script a bit

#!/bin/sh
set -e

# /export/www was created in the Dockerfile
mount.nfs4 fileserver.example.com:/export/www /www/some-website 

# And we are done here
exec "$@"

Since we are using NFSv4, chances are you might need to add something like

RUN sed -i -e '/^#Domain/a Domain = example.com' /etc/idmapd.conf

to your dockerfile in addition to telling it to get the nfs client packages, but the general idea should apply for, say, SMB or ZFS or whatever other network filesystem that you fancy: let your entrypoint script do the heavy lifting!