Quick notes on using git (gitolite) + NetBeans + ssh keys

Most of my posts (hopefully) talk about something that might be useful/helpful to others. Others are to help me not repeat a mistake. This is one of the latter ones; don't expect it to be very impressive. Really. Just look the other way, or at least have the decency of waiting until to laugh at my expense. Deal?

If you remember, I setup a git server using gitolite and docker not long ago. And then I found out one of the future users wanted to access it using NetBeans in Windows. As I have never used that IDE before, I looked in its website and found some info on how to make it talk to github. Well that sounded promising. First place I got stuck was creating the ssh key pair. You see, I know how to use ssh-keygen in Linux/Unix/OSX, but did not know how to do that in Windows. I could install cygwin and do it command line, and that would be great for me but not as nice for a typical Windows user. And, I would be installing a lot of crap this user did not need.

Searching around the net, I found a quick article about how to generate a ssh key on windows using putty (that is exactly how it is called). Now we are making progress; we just need to go and get puttygen, whcih can be obtained by itself, and create the keys. Following the last link, I created a key pair -- 4092 SSH2 RSA without passphrase since this is a test -- as shown in the picture on the left (you can click on it to make it bigger or something). Note the field I highlighted; I will refer to it later. For now, let's ignore that.

Here is the second place I screwed up. At first I thought the buttons to export the public and private keys were what I needed. So, I clicked on them and saved the files. The fact it wanted to give the private key the extension .ppk should have woken me up, but I dozed through that. Completely.

But, as I completely ignored that warning sign, I went back to the NetBeans Instructions and put the private key where it should be. And then I put the public key in my gitolite server. And then tried.

And it did not work.

So I decided to take a look at the keys. Here is the private one (trimmed out a bit to show the format while keeping this article small):

PuTTY-User-Key-File-2: ssh-rsa
Encryption: none
Comment: rsa-key-20150320
Public-Lines: 12
AAAAB3NzaC1yc2EAAAABJQAAAgEA1pA7YEyqbDyVAjjK+VCJpCGxfOwf8WGJ6J4Q
PaE1KMs0TBt5rEPlOpIdsJwOPMBvIUGG8hIHTyxrmOhxj221GYLPNSku7BdLCrt8
[....]
Q/G9k8opoE/0UPtFC0ykkGJg3Vhjq2XRGn3nYJaps7hP8ZUnvkiBuOmBR5SETnFi
3PrYFZ3lgYf1Nz2FgPMjg1uufIxWWv3MUb2Nya44x3JYQUkNfXCjUVmKiK/gE5sM
enufKa0=
Private-Lines: 28
AAACAFb8M79BrN/FiIRcNpxswi3IeGMTnj9DN+iuyFBWHHhSYU9JaK2e/BDTc9H6
E5wWqZfcS3bkttrofqXGBIZBO5S4fYRBINxAy2U3QG8mkBRWoBX/FuyztWBXxKug
pPIDWliV0oiTP0Q+PSXQ4LKTNcZm4UYlKg0Qk+ejR7FTpELqbv0nJQvMulvnEqVk
[...]
a6KahmtK1FFcFUtnLUFW+t7d59TYB/aB6HByzf5JDZPE9dtVpdBF33NSHe0Z4i2e
9XybtlNfYqKWEfaFzaeJqkyhRH5woG7le8GPU67BKVg/mKHYnG3Tk93NvhcOXuyx
7I2AKDeR1M0qMl0aH0ympymKoNxJHD2CBweLyJSoG3QdXY5DxoRvdn/gMeGRdLp0
mR80D0IN/tUduNT/69lB+I1oTU4N+WB/NcHCicOZyzHQRIKnmHC/raBM5ln2Q/ih
qtM=
Private-MAC: 56aa3a9bcb05a65a89110a5de990d5021cfb9273

It sure does not look like the ones I created using ssh-keygen. The public key also looked a bit different. Then it hit me: because gitolite uses openssh, it expects the key to be puttygen is exporting the keys in a different format. So, how do we make this work? Well, do you remember the field I highlighted in the first picture? That is the public key already in the ssh format; that is what gitolite needs. So, cut-n-paste that to, say, smurf.pub inside the keydir directory in the gitolite config file.

Next is to export the private key in the right format. That is done by clickign on Conversions->Exporting OpenSSH key, naming it as something helpful; I named mine smurf_rsa to remind me I happened to have created a RSA key.

Time to go back to Netbeans. The picture on the left shows the setup I used. I was in a hurry so I used the testing repo, which is a bit of a village bicycle in my server: everyone who can connect to localgit can access, read, and write to and in general monkey with it. The key was fed and then I told it to finish. This time it worked or seemed to: proceeded to want me to create a project (with all the little files and directories the IDE creates). I let it do the deed and later on was able to check out what was created.

The moral of this tale is make sure you use the right key format or things will get very interesting. Either that or only drink warm beer if the fridge was built by Lucas, the Prince of Darkness. Or something like that. On the bright side, you managed to reach the end of this tale! You now may put your seat and tray in the upright position and start to laugh.

I did.

Creating a git server in docker... with NFS and custom port and ssh key pair

I usually like to start my post describing what we will try to accomplish here, but I think I can't do any better than what the title states. So, let's say if I can come up with a convincing excuse. Well, all I can come up with right now is that I think it is wasteful to create an entire VM to run a distributed version control system. At least one that does not have helpful paperclips with eyes and other features requiring you to download crap. And, it is nice to know if the docker host (or cloud) takes a dump, we can bring this service back rather quickly. For this article we will use git; some other time we can talk about svn.

The git server I will be using is gitolite because it is reasonably simple and quick to manage and get going. What I really like on it is that the accounts for the users using git are not accounts in the host itself, so they cannot login to the machine hosting git. By default the git users login using

Since I am lazy, I will store the repositories in a NFS fileshare that is mounted into the container at runtime. We talked about how to do the mounting in a previous article.

Assumptions

1. gitolite running off /home/git
   
2. We will connect to the git server on port 2022. I chose that because I need port 22 for to ssh into the docker host. Yes, they are in different IPs (in my case completely different VLANs), but I am weird like that.
   
3. We will use ssh key pair authentication. And will use a different key than the default one. Note you can authenticate against LDAP, but that would go against what I wrote in the name of this article.
   
4. gitolite being run as user git
   
5. I am running this on centos6.
   

Install

1. I created a CNAME for the docker host, gitserver.example.com, so it looks pretty.
   
2. In the NFS server, create a fileshare owned by the user git, which in this example has uid=1201.
   
3. We will need to create a ssh key pair for the gitadmin. I created my pair by doing something like

   ssh-keygen -t rsa -C gitadmin -f ~/.ssh/gitadmin

   You will need to copy ~/.ssh/gitadmin.pub into the docker host by whatever means you desire.
   
4. I create a directory in the docker host to put all the files (docker-entrypoint.sh and Dockerfile) related to this container. Here is the Dockerfile

   ############################################################
   # Dockerfile to build a gitolite git container image
   # Based on CentOS
   ############################################################
   
   # Set the base image to CentOS
   FROM centos:centos6
   
   # File Author / Maintainer
   MAINTAINER Mauricio Tavares "raubvogel@gmail.com"
   
   ################## BEGIN INSTALLATION ######################
   # We need epel
   RUN rpm -Uvhi http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.no
   arch.rpm && \
       sed -i -e 's/^enabled=1/enabled=0/' /etc/yum.repos.d/epel.repo
   
   # We need NFS, openssh, and git
   # And ssmtp (from Epel)
   RUN yum update -y && yum install -y \
           git \
           nfs-utils \
           openssh-server && \
       yum install -y ssmtp --enablerepo=epel
   
   # Configure NFS
   RUN sed -i -e '/^#Domain/a Domain = example.com' /etc/idmapd.conf
   
   ##################### INSTALLATION END #####################
   
   # Create git user
   RUN adduser -m -u 1201 git
   
   # Configure ssmtp
   
   # Configure sshd
   RUN sed -i -e 's/^#Port .*$/Port 2022/' \
              -e 's/^#PermitRootLogin .*$/PermitRootLogin no/' \
              /etc/ssh/sshd_config && \
       sed -i -e \
           's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.s
   o@g' \
            /etc/pam.d/sshd && \
       ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa && \
       ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
   
   # And a mountpoint for repositories
   # Note: can't NFS mount from dockerfile, so will do it in an entrypoint script
   RUN su - git -c 'mkdir repositories'
   
   
   # And now the git server
   # Gitolite admin: gitadmin (it is based on the name of the pub key file)
   RUN su - git -c 'mkdir bin' && \
       su - git -c 'git clone git://github.com/sitaramc/gitolite' && \
       su - git -c 'mkdir -m 0700 .ssh' && \
       su - git -c 'echo "ssh-rsa AAAAB3NzaC1yc2EAASLDAQCOOKIEQDehf5hxGq9//34yrsL
   [...]
   7CfSpbiP gitadmin" > .ssh/gitadmin.pub'
   # The rest will be configured in the entrypoint script
   
   # Put the entrypoint script somewhere we can find
   COPY docker-entrypoint.sh /entrypoint.sh
   ENTRYPOINT ["/entrypoint.sh"]
   
   EXPOSE 2022
   # Start service
   CMD ["/usr/sbin/sshd", "-D"]
   

   Where

   1. You will need to put the public key gitadmin.pub between the double quotes in the line beginning with su - git -c 'echo "ssh-rsa.
      
   2. I am running a lot of things in the Dockerfile as user git.
      
   3. The NFS setup was mentioned before, so I will not bother with it right now.
      
   4. I forgot to add the setup for ssmtp. I will think about that sometime later.
      
5. The docker-entrypoint.sh file looks vaguely like this:

   #!/bin/sh
   set -e
   
   # Mount git's repositories
   mount.nfs4 fileserver.example.com:/git /home/git/repositories
   
   su - git -c 'gitolite/install -to $HOME/bin'
   # setup gitolite with yourself as the administrator
   su - git -c 'gitolite setup -pk .ssh/gitadmin.pub'
   
   # And we are out of here
   exec "$@"

6. So far so good? Ok, so let's build the image. I will call it git.

   docker build -t git .

   If the last build message looks like

   Successfully built 6fb1ac15b47a

   chances are the build was successful and you can go to the next step. Otherwise, figure out what went boink.
   
7. Now start the service. Remember you need to run in priviledge mode because of NFS. Since this is a test, I am calling the contained test-git.

   docker run --privileged=true -d -P -p 3306:3306 --name test-git git

Setup and Testing

1. Let's start testing by seeing what repositories we can see as an admin:
   

   $ /usr/bin/ssh -i /home/raub/.ssh/gitadmin git@gitserver.example.com info
   hello gitadmin, this is git@docker running gitolite3 v3.6.2-12-g1c61d57 on git 1.7.1
   
    R W    gitolite-admin
    R W    testing
   
   

   FYI, testing is a repo everyone allowed to use the git server can play with. Think of it as a, as its name implies, test repo. Since that worked we can proceed to the next step.
   
2. Now we should edit your .ssh/config file to access the gitadmin repository.

   cat >> ~/.ssh/config << EOF
   Host gitadmin
           Hostname        gitserver.example.com 
           User            git
           Port            2022
           identityfile    /home/raub/.ssh/gitadmin
           protocol        2
           compression     yes
   EOF

   Yes, you can ask about what to do if you have a script that needs to pull stuff out of a repo, and I will tell you to wait for the next installment. This article deals with getting it to work.
3. Retrieving the admin repository is now much easier. So, instead of having to do something like

   git clone ssh://git@gitserver.example.com:[port]/gitolite-admin

   which would also require us to feed the key (or rename it as the default which IMHO is a bad idea), thanks to the previous step we can now lazily do

   git clone gituser:/gitolite-admin

4. Repository config file is in gitolite-admin/conf/gitolite.conf
   
5. Adding a new user (and perhaps a repository)
   
   1. Get user's ssh public key, say raub.pub. How did the user created the ssh key pair? Don't know, don't care.
      
   2. Copy raub.pub to gitolite-admin/keydir. NOTE:file must be named after username user will use to connect to the git server; it does not need to have anything to do with the user's normal/real username.
      
   3. Create a repository for the user. Let's give it a nice and snuggly name, like somethingawful

      cat >> conf/gitolite.conf << EOF
      
      repo somethingawful
          RW      = raub
      EOF

   4. Commit changes

      git add conf keydir
      git commit -m 'Added new user and config a repo'
      git push origin master

6. Now, let's pretend we are the user (i.e. what the user should do/see). Which repos can you/user see? If this looks like a step we did before, it is. Just using a new user.

   $ ssh -i /home/raub/.ssh/raub git@gitserver.example.com info 
   hello raub, this is git@docker running gitolite3 v3.6.2-12-g1c61d57 on git 1.7.1
   
   
    R W    somethingawful
    R W    testing
   

   There is somethingswful!
   
7. Let's grab somethingawful

   $ git clone gituser:/somethingawful
   Cloning into 'somethingawful'...
   warning: You appear to have cloned an empty repository.
   Checking connectivity... done.

8. Edit, commit, and off you go
   

I will put all those guys in my github account later on. And shall sneakly update this post afterwards.