Converting a .ovf file to work on an older/different VMWare ESXi (maybe also player) setup

I will be using ESXi because that is what I have; I do not see why it would not work in Player or Workstation.

As you know, the way vmware likes to export/import vm guests is using a ovf format. So, let's say we are supposed to add a guest called strangeguest. We get it as a directory called strangeguest, which contains the disk (strangeguest-disk1.vmdk in our case), the config file strangeguest.ovf and a mysterious file called strangeguest.mf (.mf extension for Mysterious File?). When we try to import it we get an error message that complains we cannot import the OVF. A quick look indicates that strangeguest expects to be of SystemType vmx-12 or better:

admin@fileserver:/export/public/ISOs/strangeguest$ fgrep vmx- strangeguest.ovf         vmx-12
admin@fileserver:/export/public/ISOs/strangeguest$

Thing is our ESXi setup does not support vmx-12 guest in our ESXi as is a bit old and needs to be upgraded (which will be subject of another article). However, right now we need to make this work.

So we cheat.

We know the latest systemtype our ESXi support is vmx-10 by looking at the properties of the guests currently in place. So, how about if we tell strangeguest that it is vmx-10?

admin@fileserver:/export/public/ISOs/strangeguest$ sed -i -e 's/vmx-12/vmx-10/' strangeguest.ovf
admin@fileserver:/export/public/ISOs/strangeguest$ fgrep vmx- strangeguest.ovf         vmx-10
admin@fileserver:/export/public/ISOs/strangeguest$

So we try again and we get a different error (note to myself: get that error message). What did we do wrong? Well, do you remember the mysterious file? Let's see what is inside it:

admin@fileserver:/export/public/ISOs/strangeguest$ cat strangeguest.mf
SHA1(strangeguest.ovf)= 7b11b4aacead791f8aaf76e5ed3c2354349b3b20
SHA1(strangeguest-disk1.vmdk)= 9ccd4817ac2f943f7f29be970b76461850460d18
admin@fileserver:/export/public/ISOs/strangeguest$

So it has the checksum (as SHA1, which is a step about MD5 but still not to be used as it can be lied to, but I digress). Remember we edited strangeguest.ovf!

admin@fileserver:/export/public/ISOs/strangeguest$ sha1sum strangeguest.ovf
f14befc1e790b0043dd5f8e22fd8d601637997bd  strangeguest.ovf
admin@fileserver:/export/public/ISOs/strangeguest$

So, we need to update strangeguest.mf:

sed: -e expression #1, char 83: unterminated `s' command
admin@fileserver:/export/public/ISOs/strangeguest$ sed -i -e \
's/7b11b4aacead791f8aaf76e5ed3c2354349b3b20/f14befc1e790b0043dd5f8e22fd8d601637997bd/' \
strangeguest.mf
admin@fileserver:/export/public/ISOs/strangeguest$ !cat
cat strangeguest.mf
SHA1(strangeguest.ovf)= f14befc1e790b0043dd5f8e22fd8d601637997bd
SHA1(strangeguest-disk1.vmdk)= 9ccd4817ac2f943f7f29be970b76461850460d18
admin@fileserver:/export/public/ISOs/strangeguest$

And we should be rewarded with strangeguest being properly imported.

Using sed Capture Groups (Linux/Mac)

This will be a short one and belongs to the TIL bin: until 2 days ago I did not even know about capture groups and how to use it. So, I knew how to replace a matching string/pattern in sed

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v/swallow_speed/' 
As of today swallow_speed=23kph at STD
bash-3.2$ 

And how to replace from a given pattern all the way to the end of the line. Or starting from the beginning of the line to said pattern:

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*/swallow_speed=42/' 
As of today swallow_speed=42
bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/^.*swallow_v=/Can you believe that swallow_speed=/' 
Can you believe that swallow_speed=23kph at STD
bash-3.2$ 

If you are curious, the .* in the search pattern means "any character or list of characters here, be it zero or a lot of characters". The dot (.) does the any part and the asterisk (*) the how many. I think this is from regex, but don't quote me on that. Fine, what if I want to replace everything between two patterns, but leaving the second pattern alone? Tricky. You see, replacing everything between the two patterns, inclusive is not that hard

raub@desktop:~$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*k/swallow_v=25/'
As of today swallow_v=25ph at STD
raub@desktop:~$ 

But to preserve the second pattern we need to use the Capture Groups mentioned in the title of this article. And that makes sense because if it is on the title I better use it. So, we are supposed to surround the capture group pattern with parenthesis and then we can refer to them. If it does not make sense, I too was confused, so let's keep on using our test string:

raub@desktop:~$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*(k)/swallow_v=25$1/'
As of today swallow_v=23kph at STD
raub@desktop:~$ 

Er, it does not seem to have worked according to the plan. In fact, it was supposed to at least grab swallow_v=23k but as you can see it did not find the pattern. Is $1 the proper way to output the captured string? Going nowhere slowly.

After much soul searching, I found the -e requires the parenthesis to be escaped. And, the capture group pattern is output using \1 instead of $1.So we try again:

raub@desktop:~$ echo "As of today swallow_v=23kph at STD" | sed -e "s/swallow_v=.*\(k\)/swallow_v=25\1/"
As of today swallow_v=25kph at STD
raub@desktop:~$ 

much better!

What about the Mac? Same thing (the bash-3.2$ you have seen all day is it; the raub@desktop:~$ is the Linux box):

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*\(k\)/swallow_v=25\1/' 
As of today swallow_v=25kph at STD
bash-3.2$ 

I am not going to say it is perfect though:

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*\([[:blank:]]\)/swallow_v=25\1/' 
As of today swallow_v=25 STD
bash-3.2$ echo "As of today swallow_v=23kph at    STD" | sed -e 's/swallow_v=.*\([[:blank:]]\)/swallow_v=25\1/' 
As of today swallow_v=25 STD
bash-3.2$ 

References

• A site I think has lots of interesting sed examples.

Setting up zabbix using official instructions and repo: Step 1 we ain't there yet

So I am installing Zabbix. Why, well, you probably know. If not, we can talk about that in a different article. Yes I am testing my ansible playbook in a docker container, but right now that too is not important. The How I did It will be in a different article. This article is the Everything That Went Wrong and How I Got Around That one. Think of it as an insight of how I deal with me being clueless; laughing at my expense is acceptable and maybe even recommended.

I want to install latest version of Zabbix in a CentOS 7 host, as a result I will be using the official zabbix 3.2 install docs, which are the most current when I wrote this article. For now I will be lazy and use the mysql version since it is faster to setup; we can revisit that later.

Dependencies

1. Need the repo. Per the official Zabbix instructions, I am using the official Zabbix repo, which as of the time of this writing can be obtained by

   rpm -ivh http://repo.zabbix.com/zabbix/3.2/rhel/7/x86_64/zabbix-release-3.2-1.el7.noarch.rpm

   I did write a script to get the latest rpm, but it is not important right now. Now, if you are curious, here is the repo config file:

   [root@zabbix ~]# cat /etc/yum.repos.d/zabbix.repo 
   [zabbix]
   name=Zabbix Official Repository - $basearch
   baseurl=http://repo.zabbix.com/zabbix/3.2/rhel/7/$basearch/
   enabled=1
   gpgcheck=1
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX-A14FE591
   
   [zabbix-non-supported]
   name=Zabbix Official Repository non-supported - $basearch 
   baseurl=http://repo.zabbix.com/non-supported/rhel/7/$basearch/
   enabled=0
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX
   gpgcheck=1
   [root@zabbix ~]# 

   Before you ask, I am making a point to accidentally post it here for a reason, which will become clearer later.

2. The database server. Thanks to irc user leManu enlightening me, we are not supposed to install mysql (or whatever db) server on the machine that will run zabbix server. With that said, the line

   mysql> grant all privileges on zabbix.* to zabbix@localhost identified by '';

   in the official docs has a very localhost feel to it.

   We better build the db server first, and then go there and create the zabbix user, tying it to the IP for the zabbix server. I used mariadb and then grabbed the required -- FQDN, port, zabbix password -- info and came back to the zabbix server.
   

3. Packages. We have the repo setup, and database server info on standby. We might as well start installing zabbix itself, right?

   [root@zabbix ~]# yum install zabbix-server-mysql zabbix-web-mysql 
   [...]
   --> Finished Dependency Resolution
   Error: Package: zabbix-server-mysql-3.2.4-2.el7.x86_64 (zabbix)
              Requires: fping
   Error: Package: zabbix-server-mysql-3.2.4-2.el7.x86_64 (zabbix)
              Requires: libiksemel.so.3()(64bit)
    You could try using --skip-broken to work around the problem
    You could try running: rpm -Va --nofiles --nodigest
   [root@zabbix ~]# 

   Bummer. Why didn't it grab them from the normal centos repo? I guess maybe it does not have them and we will need to fetch them from another repo. But, before we add another repo, do you remember the file /etc/yum.repos.d/zabbix.repo, whose contents we pasted earlier? It has a zabbix-non-supported; how about if we take a quick look there?

   [root@zabbix ~]# yum whatprovides */fping --enablerepo=zabbix-non-supported
   Loaded plugins: fastestmirror, ovl
   Loading mirror speeds from cached hostfile
    * base: mirrors.gigenet.com
    * extras: mirror.keystealth.org
    * updates: mirror.umd.edu
   fping-3.10-1.el7.x86_64 : Scriptable, parallelized ping-like utility
   Repo        : zabbix-non-supported
   Matched from:
   Filename    : /usr/sbin/fping
   
   
   
   fping-3.10-1.el7.x86_64 : Scriptable, parallelized ping-like utility
   Repo        : @zabbix-non-supported
   Matched from:
   Filename    : /usr/sbin/fping
   
   
   
   [root@zabbix ~]# 

   Short version: grab the two packages we need from it already:

   yum install fping iksemel --enablerepo=zabbix-non-supported

4. Missing setup file (thanks Yum!). Per the docs, we are now supposed to grab a file called /usr/share/doc/zabbix-server-mysql-3.2.4/create.sql.gz and use it to initially populate the zabbix database. Thing is, I can't find that (/usr/share/doc/zabbix-server-mysql-3.2.4) directory, much less the file:

   [root@zabbix ~]# ls  /usr/share/doc/
   coreutils-8.22    pam-1.1.8             python-pycurl-7.19.0
   gnupg2-2.0.22     pygpgme-0.3           unixODBC-2.3.1
   krb5-libs-1.13.2  python-kitchen-1.1.1  zabbix-release-3.2
   [root@zabbix ~]# 

   Maybe /usr/share/doc/zabbix-release-3.2/ is the directory and the docs were off? I will have to expertly crush your hopes:

   [root@zabbix ~]# ls  /usr/share/doc/zabbix-release-3.2/
   GPL
   [root@zabbix ~]# ls -l  /usr/share/doc/zabbix-release-3.2/
   total 20
   -rw-r--r-- 1 root root 18385 Feb 15  2016 GPL
   [root@zabbix ~]# head -10  /usr/share/doc/zabbix-release-3.2/GPL 
   *****************************************************************************
   The following copyright applies to the Red Hat Linux compilation and any 
   portions of Red Hat Linux it does not conflict with. Whenever this
   policy does conflict with the copyright of any individual portion of Red Hat 
   Linux, it does not apply.
   
   *****************************************************************************
   
                       GNU GENERAL PUBLIC LICENSE
                          Version 2, June 1991
   [root@zabbix ~]# 

   Maybe it is somewhere else? Nope.

   [root@zabbix ~]# find / -name create.sql.gz -print
   [root@zabbix ~]# 

   So, where's it? Hey, don't look at me like that. I too have no idea. Let's grab the rpm and then take a look at it:

   root@zabbix:/tmp$ rpm -qlp zabbix-server-mysql-3.2.4-2.el7.x86_64.rpm | grep cre
   ate.sql.gz 
   warning: zabbix-server-mysql-3.2.4-2.el7.x86_64.rpm: Header V4 DSA/SHA1 Signatur
   e, key ID 79ea5ed4: NOKEY
   /usr/share/doc/zabbix-server-mysql-3.2.4/create.sql.gz
   root@zabbix:/tmp$ 

   That is the version we installed, right?

   [root@zabbix ~]# rpm -q zabbix-server-mysql
   zabbix-server-mysql-3.2.4-2.el7.x86_64
   [root@zabbix ~]#

   Looks like it. And yum's log, /var/log/yum.log file think so too:

   Mar 07 14:30:20 Installed: zabbix-web-mysql-3.2.4-2.el7.noarch
   Mar 07 14:30:21 Installed: zabbix-web-3.2.4-2.el7.noarch
   Mar 07 14:30:22 Installed: zabbix-server-mysql-3.2.4-2.el7.x86_64

   This really does not make sense. Let me look again at the contents of the installed package, not at the rpm:

   [root@zabbix ~]# rpm -qlv zabbix-server-mysql
   -rw-r--r--    1 root    root                      132 Mar  2 14:55 /etc/logrotate.d/zabbix-server
   -rw-r-----    1 root    zabbix                  14876 Mar  2 14:55 /etc/zabbix/zabbix_server.conf
   -rw-r--r--    1 root    root                      415 Mar  2 14:29 /usr/lib/systemd/system/zabbix-server.service
   -rw-r--r--    1 root    root                       35 Mar  2 14:29 /usr/lib/tmpfiles.d/zabbix-server.conf
   drwxr-xr-x    2 root    root                        0 Mar  2 14:55 /usr/lib/zabbix/alertscripts
   drwxr-xr-x    2 root    root                        0 Mar  2 14:55 /usr/lib/zabbix/externalscripts
   -rwxr-xr-x    1 root    root                  2220064 Mar  2 14:55 /usr/sbin/zabbix_server_mysql
   drwxr-xr-x    2 root    root                        0 Mar  2 14:55 /usr/share/doc/zabbix-server-mysql-3.2.4
   -rw-r--r--    1 root    root                       98 Feb 27 09:22 /usr/share/doc/zabbix-server-mysql-3.2.4/AUTHORS
   -rw-r--r--    1 root    root                    17990 Feb 27 09:23 /usr/share/doc/zabbix-server-mysql-3.2.4/COPYING
   -rw-r--r--    1 root    root                   742520 Feb 27 09:22 /usr/share/doc/zabbix-server-mysql-3.2.4/ChangeLog
   -rw-r--r--    1 root    root                       52 Feb 27 09:24 /usr/share/doc/zabbix-server-mysql-3.2.4/NEWS
   -rw-r--r--    1 root    root                      188 Feb 27 09:22 /usr/share/doc/zabbix-server-mysql-3.2.4/README
   -rw-r--r--    1 root    root                  1161488 Mar  2 14:49 /usr/share/doc/zabbix-server-mysql-3.2.4/create.sql.gz
   -rw-r--r--    1 root    root                      881 Mar  2 14:55 /usr/share/man/man8/zabbix_server.8.gz
   drwxr-xr-x    2 zabbix  zabbix                      0 Mar  2 14:55 /var/log/zabbix
   drwxr-xr-x    2 zabbix  zabbix                      0 Mar  2 14:55 /var/run/zabbix
   [root@zabbix ~]# ls /usr/share/doc/zabbix-server-mysql-3.2.4/create.sql.gz
   ls: cannot access /usr/share/doc/zabbix-server-mysql-3.2.4/create.sql.gz: No such file or directory
   [root@zabbix ~]# 

   It turns out (kudos to irc user TrevorH for pointing that out) that yum is configured not to install docs

   [root@zabbix ~]# grep -ir tsflags /etc/yum.*
   /etc/yum.conf:tsflags=nodocs
   [root@zabbix ~]# 

   Let's comment it out then and try again

   [root@zabbix ~]# sed -i -e 's/^tsflags=nodocs/#tsflags=nodocs/' /etc/yum.conf
   [root@zabbix ~]# yum reinstall zabbix-server-mysql zabbix-web-mysql --enablerepo=zabbix
   Loaded plugins: fastestmirror, ovl
   Loading mirror speeds from cached hostfile
    * base: dist1.800hosting.com
    * extras: mirror.eboundhost.com
    * updates: mirror.es.its.nyu.edu
   Resolving Dependencies
   --> Running transaction check
   ---> Package zabbix-server-mysql.x86_64 0:3.2.4-2.el7 will be reinstalled
   ---> Package zabbix-web-mysql.noarch 0:3.2.4-2.el7 will be reinstalled
   --> Finished Dependency Resolution
   
   Dependencies Resolved
   
   ================================================================================
    Package                   Arch         Version              Repository    Size
   ================================================================================
   Reinstalling:
    zabbix-server-mysql       x86_64       3.2.4-2.el7          zabbix       1.8 M
    zabbix-web-mysql          noarch       3.2.4-2.el7          zabbix       5.1 k
   
   Transaction Summary
   ================================================================================
   Reinstall  2 Packages
   
   Total download size: 1.8 M
   Installed size: 4.0 M
   Is this ok [y/d/N]: y
   Downloading packages:
   (1/2): zabbix-web-mysql-3.2.4-2.el7.noarch.rpm             | 5.1 kB   00:00     
   (2/2): zabbix-server-mysql-3.2.4-2.el7.x86_64.rpm          | 1.8 MB   00:01     
   --------------------------------------------------------------------------------
   Total                                              1.7 MB/s | 1.8 MB  00:01     
   Running transaction check
   Running transaction test
   Transaction test succeeded
   Running transaction
     Installing : zabbix-web-mysql-3.2.4-2.el7.noarch                          1/2 
     Installing : zabbix-server-mysql-3.2.4-2.el7.x86_64                       2/2 
     Verifying  : zabbix-server-mysql-3.2.4-2.el7.x86_64                       1/2 
     Verifying  : zabbix-web-mysql-3.2.4-2.el7.noarch                          2/2 
   
   Installed:
     zabbix-server-mysql.x86_64 0:3.2.4-2.el7                                      
     zabbix-web-mysql.noarch 0:3.2.4-2.el7                                         
   
   Complete!
   [root@zabbix ~]# ls /usr/share/doc/
   coreutils-8.22    pygpgme-0.3           zabbix-release-3.2
   gnupg2-2.0.22     python-kitchen-1.1.1  zabbix-server-mysql-3.2.4
   krb5-libs-1.13.2  python-pycurl-7.19.0
   pam-1.1.8         unixODBC-2.3.1
   [root@zabbix ~]# ls /usr/share/doc/zabbix-server-mysql-3.2.4/
   AUTHORS  ChangeLog  COPYING  create.sql.gz  NEWS  README
   [root@zabbix ~]# 

   Success at last!

I think that is enough for one article. If you expect this to have any closure or redeeming message, I have news for you sunshine. Just hope that the next zabbix article will talk about actually getting it installed and configured and running. But, I make no guarantees.

Checking if RedHat/CentOS has new updates

If you use Debian Linux derivatives, specially Ubuntu, you probably noticed when you login it will tell you (using the MOTD) that there are new updates waiting for you. And, you can use that if, say, you are writing a script to let you know about that; someone I know use that to monitor his Ubunu boxes in Nagios. But, what about in RedHat and derviatives the lazy way?

Let's do some thinking aloud and see if we can come up with something. We know that if we run yum check-update, it should reply with the list of packages needing to be upgraded if any

[root@vmhost ~][raub@duckwitch ~]$ yum check-update
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirror.supremebytes.com
 * extras: mirror.hostduplex.com
 * updates: mirror.scalabledns.com

chkconfig.x86_64                        1.3.61-5.el7_2.1                 updates
device-mapper.x86_64                    7:1.02.107-5.el7_2.5             updates
device-mapper-libs.x86_64               7:1.02.107-5.el7_2.5             updates
dracut.x86_64                           033-360.el7_2.1                  updates
glibc.x86_64                            2.17-106.el7_2.6                 updates
glibc-common.x86_64                     2.17-106.el7_2.6                 updates
iproute.x86_64                          3.10.0-54.el7_2.1                updates
kernel.x86_64                           3.10.0-327.22.2.el7              updates
kpartx.x86_64                           0.4.9-85.el7_2.5                 updates
libxml2.x86_64                          2.9.1-6.el7_2.3                  updates
ntpdate.x86_64                          4.2.6p5-22.el7.centos.2          updates
pcre.x86_64                             8.32-15.el7_2.1                  updates
selinux-policy.noarch                   3.13.1-60.el7_2.7                updates
selinux-policy-targeted.noarch          3.13.1-60.el7_2.7                updates
systemd.x86_64                          219-19.el7_2.11                  updates
systemd-libs.x86_64                     219-19.el7_2.11                  updates
systemd-python.x86_64                   219-19.el7_2.11                  updates
systemd-sysv.x86_64                     219-19.el7_2.11                  updates
tzdata.noarch                           2016f-1.el7                      updates
[raub@duckwitch ~]$

As you can see, it does not require you to run that command as root. And you can even check if a repo you use but configured to be normally disabled, like epel in the following example:

raub@duckwitch ~]$ yum check-update --enablerepo=epel
Loaded plugins: fastestmirror
epel/x86_64/metalink                                     |  11 kB     00:00
epel                                                     | 4.3 kB     00:00
(1/3): epel/x86_64/group_gz                                | 170 kB   00:00
(2/3): epel/x86_64/updateinfo                              | 584 kB   00:00
(3/3): epel/x86_64/primary_db                              | 4.2 MB   00:00
Loading mirror speeds from cached hostfile
 * base: mirror.supremebytes.com
 * epel: mirror.chpc.utah.edu
 * extras: mirror.hostduplex.com
 * updates: mirror.scalabledns.com

chkconfig.x86_64                        1.3.61-5.el7_2.1                 updates
device-mapper.x86_64                    7:1.02.107-5.el7_2.5             updates
device-mapper-libs.x86_64               7:1.02.107-5.el7_2.5             updates
dracut.x86_64                           033-360.el7_2.1                  updates
epel-release.noarch                     7-7                              epel
glibc.x86_64                            2.17-106.el7_2.6                 updates
glibc-common.x86_64                     2.17-106.el7_2.6                 updates
iproute.x86_64                          3.10.0-54.el7_2.1                updates
kernel.x86_64                           3.10.0-327.22.2.el7              updates
kpartx.x86_64                           0.4.9-85.el7_2.5                 updates
libxml2.x86_64                          2.9.1-6.el7_2.3                  updates
ntpdate.x86_64                          4.2.6p5-22.el7.centos.2          updates
pcre.x86_64                             8.32-15.el7_2.1                  updates
selinux-policy.noarch                   3.13.1-60.el7_2.7                updates
selinux-policy-targeted.noarch          3.13.1-60.el7_2.7                updates
systemd.x86_64                          219-19.el7_2.11                  updates
systemd-libs.x86_64                     219-19.el7_2.11                  updates
systemd-python.x86_64                   219-19.el7_2.11                  updates
systemd-sysv.x86_64                     219-19.el7_2.11                  updates
tzdata.noarch                           2016f-1.el7                      updates
[raub@duckwitch ~]$

What if there are no upgrades?

[root@server1 ~]# yum check-update
Loaded plugins: fastestmirror
base                                                     | 3.6 kB     00:00
extras                                                   | 3.4 kB     00:00
updates                                                  | 3.4 kB     00:00
Loading mirror speeds from cached hostfile
 * base: mirror.us.leaseweb.net
 * extras: mirror.us.leaseweb.net
 * updates: reflector.westga.edu
[root@server1 ~]#

Sounds like we need to check for the first blank line. We can do that using sed. We can find the blank line by using the /^\s*$/ search pattern in sed. So we could start with something like (the -n is there because we only care when we find said blank line)

yum check-update | `sed -n '/^\s*$/p'`

which if you run does not seem to do much. Reason is that what we really want is to know if the match was successful or not. And, sed actually has the answer: look at these entries I stole from its man pange:

q [exit-code]
              Immediately  quit  the  sed  script  without processing any more
              input, except that if auto-print is  not  disabled  the  current
              pattern  space will be printed.  The exit code argument is a GNU
              extension.

       Q [exit-code]
              Immediately quit the sed  script  without  processing  any  more
              input.  This is a GNU extension.

Let's try it then. First we will find a machine that has a package that needs to be updated. In this case it will be my KVM server, which you have met before when we talked about USB passthrough:

[raub@vmhost ~]$ yum check-update 
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
 * base: mirror.vcu.edu
 * extras: centos.mirror.constant.com
 * updates: mirror.vcu.edu

samba4-libs.x86_64                    4.2.10-7.el6_8                     updates
[raub@vmhost ~]$
So as of this writing it has one update. We then try our one-liner, which we want to return 0 if it did not find any pending upgrades and 1 if it did. And that is done by adding q1 past the search string, as in /^\s*$/q1, which means write 1 if search successful. Of course we now need to print the result, which can be done with echo $?. So, let's try it:
[raub@vmhost ~]$ yum check-update | `sed -n '/^\s*$/q1'` ; echo $?
1
[raub@vmhost ~]$ 

It thinks that it found it. I am not completely sure it works, so we need also to verify it works as it should when we find no updates:

[root@server1 ~]# yum check-update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.netdepot.com
 * extras: mirror.solarvps.com
 * updates: ftpmirror.your.org
[root@server1 ~]# yum check-update | `sed -n '/^\s*$/q1'` ; echo $?
0
[root@server1 ~]# 

Sounds like we have a winner. Now all that is left is to wrap something around it to something with that. I will leave that to you. Note that it does not differentiate between normal and security updates though.

Starting Tomcat manually in Docker

So I needed to test some settings between CentOS 6 + Apache Tomcat 6 + Java 6 and CentOS 7 + Tomcat7 + Java 8. Best way I found to do these quick tests is to build them in docker: write a quick dockerfile, spool it up, test, and blow it up.

Now, when I built and ran the tomcat 6 + CentOS 6 + Java 6 setup by connecting to the container and manually starting tomcat by typing

service tomcat6 start

It worked fine. So I then built the tomcat 7 + CentOS 7 + Java 8, and tried to start it

[root@tomcat ~]# systemctl start tomcat.service
Failed to get D-Bus connection: Operation not permitted
[root@tomcat ~]# 

Since systemd could not start it, and I could not figure out why (I do not take solace knowing I am not the only one), I tried starting it even more manually:

[root@tomcat /]# /usr/sbin/tomcat start
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory
[root@tomcat /]#

This is the time to do what everyone does at a time like this: look for answers online. All I got was someone asking the very same question. It seems if we want some answers we will need do more exploring on our own.

With that in mind, let's see those two lines we are being barked about:

[root@tomcat /]# sed -n '21p' /usr/sbin/tomcat
    . /etc/sysconfig/${NAME}
[root@tomcat /]# sed -n '39p' /usr/sbin/tomcat
  ${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
[root@tomcat /]#

Not much help here, but we will revisit that later. First, let's run the bash script again, but this time in a debugging (-x) mode:

[root@tomcat /]# bash -x /usr/sbin/tomcat start
+ '[' -r /usr/share/java-utils/java-functions ']'
+ . /usr/share/java-utils/java-functions
++ _load_java_conf
++ local IFS=:
++ local java_home_save=
++ local java_opts_save=
++ local javaconfdir
++ local conf
++ unset _javadirs
++ unset _jvmdirs
++ set -- /etc/java
++ _log 'Java config directories are:'
++ '[' -n '' ']'
++ for javaconfdir in '"$@"'
++ _log '  * /etc/java'
++ '[' -n '' ']'
++ for javaconfdir in '"$@"'
++ conf=/etc/java/java.conf
++ '[' '!' -f /etc/java/java.conf ']'
++ local IFS
++ local JAVA_LIBDIR
++ local JNI_LIBDIR
++ local JVM_ROOT
++ '[' -f /etc/java/java.conf ']'
++ _log 'Loading config file: /etc/java/java.conf'
++ '[' -n '' ']'
++ . /etc/java/java.conf
+++ JAVA_LIBDIR=/usr/share/java
+++ JNI_LIBDIR=/usr/lib/java
+++ JVM_ROOT=/usr/lib/jvm
++ _javadirs=/usr/share/java:/usr/lib/java
++ _jvmdirs=/usr/lib/jvm
++ _load_java_conf_file /root/.java/java.conf
++ local IFS
++ local JAVA_LIBDIR
++ local JNI_LIBDIR
++ local JVM_ROOT
++ '[' -f /root/.java/java.conf ']'
++ _log 'Skipping config file /root/.java/java.conf: file does not exist'
++ '[' -n '' ']'
++ _javadirs=/usr/share/java:/usr/lib/java
++ _jvmdirs=/usr/lib/jvm
++ '[' -d '' ']'
++ '[' -n '' ']'
++ '[' _ '!=' _off -a -f /usr/lib/abrt-java-connector/libabrt-java-connector.so
-a -f /var/run/abrt/abrtd.pid ']'
++ _log 'ABRT Java connector is disabled'
++ '[' -n '' ']'
+ '[' -z '' ']'
+ TOMCAT_CFG=/etc/tomcat/tomcat.conf
+ '[' -r /etc/tomcat/tomcat.conf ']'
+ . /etc/tomcat/tomcat.conf
++ TOMCAT_CFG_LOADED=1
++ TOMCATS_BASE=/var/lib/tomcats/
++ JAVA_HOME=/usr/lib/jvm/jre
++ CATALINA_HOME=/usr/share/tomcat
++ CATALINA_TMPDIR=/var/cache/tomcat/temp
++ SECURITY_MANAGER=false
+ '[' -r /etc/sysconfig/ ']'
+ . /etc/sysconfig/
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
+ set_javacmd
+ local IFS
+ local cmd
+ '[' -x '' ']'
+ set_jvm
+ local IFS=:
+ local cmd
+ local cmds
+ _set_java_home
+ local IFS=:
+ local jvmdir
+ local subdir
+ local subdirs
+ '[' -n /usr/lib/jvm/jre ']'
+ '[' -z '' ']'
++ readlink -f /usr/lib/jvm/jre/..
+ JVM_ROOT=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el7_2.x86_64
+ return
+ '[' -n /usr/lib/jvm/jre ']'
+ return
+ for cmd in jre/sh/java bin/java
+ JAVACMD=/usr/lib/jvm/jre/jre/sh/java
+ '[' -x /usr/lib/jvm/jre/jre/sh/java ']'
+ for cmd in jre/sh/java bin/java
+ JAVACMD=/usr/lib/jvm/jre/bin/java
+ '[' -x /usr/lib/jvm/jre/bin/java ']'
+ _log 'Using configured JAVACMD: /usr/lib/jvm/jre/bin/java'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ return 0
+ cd /usr/share/tomcat
+ '[' '!' -z '' ']'
+ '[' -n '' ']'
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-jul
i.jar
++ build-classpath commons-daemon
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-jul
i.jar:/usr/share/java/commons-daemon.jar
+ '[' start = start ']'
+ '[' '!' -z '' ']'
[root@tomcat /]# + /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bo
otstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon
.jar -Dcatalina.base= -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -D
java.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/conf/logg
ing.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
 org.apache.catalina.startup.Bootstrap start
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory
[root@tomcat /]#

If you never ran a bash script with the -x option, you should since it shows the steps being performed by the script, including tests, as it runs. For instance, you can see it starts by learning a lot about the current Java installation. After that, it loads some file, TOMCAT_CFG=/etc/tomcat/tomcat.conf, and then gives the first error message:

+ TOMCAT_CFG=/etc/tomcat/tomcat.conf
+ '[' -r /etc/tomcat/tomcat.conf ']'
+ . /etc/tomcat/tomcat.conf
++ TOMCAT_CFG_LOADED=1
++ TOMCATS_BASE=/var/lib/tomcats/
++ JAVA_HOME=/usr/lib/jvm/jre
++ CATALINA_HOME=/usr/share/tomcat
++ CATALINA_TMPDIR=/var/cache/tomcat/temp
++ SECURITY_MANAGER=false
+ '[' -r /etc/sysconfig/ ']'
+ . /etc/sysconfig/
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory

Now, /etc/tomcat/tomcat.conf (same thing as /usr/share/tomcat/conf/tomcat.conf) defines a few global (to tomcat) to variables. The top of the file also explains it is the file where you should define variables that are custom to your system but global to all tomcat instances being run here. For instance, when I built the tomcat6 container, I had

JAVA_HOME="/usr/lib/jdk1.6.0_41"

because that was the specific java version I wanted to run. Now, if we look not only at line 21 in /usr/sbin/tomcat but also around said line, we can see it wants to load a file in /etc/sysconfig

# Get instance specific config file
if [ -r "/etc/sysconfig/${NAME}" ]; then
    . /etc/sysconfig/${NAME}
fi

If we look at /etc/sysconfig,

[root@tomcat ~]# ls /etc/sysconfig/
network  network-scripts  rdisc  tomcat
[root@tomcat ~]#

It sure makes me think that $NAME = "tomcat" and $NAME is not defined.

For the second error message we should examine the following lines

[root@tomcat /]# + /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bo
otstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon
.jar -Dcatalina.base= -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -D
java.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/conf/logg
ing.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
 org.apache.catalina.startup.Bootstrap start
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory

That really looks like it wants to write to the log file catalina.out but can't find it. So we take a look at the lines around line 39:

if [ "$1" = "start" ]; then
  ${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
    -classpath "$CLASSPATH" \
    -Dcatalina.base="$CATALINA_BASE" \
    -Dcatalina.home="$CATALINA_HOME" \
    -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
    -Djava.io.tmpdir="$CATALINA_TMPDIR" \
    -Djava.util.logging.config.file="${CATALINA_BASE}/conf/logging.properties" \
    -Djava.util.logging.manager="org.apache.juli.ClassLoaderLogManager" \
    org.apache.catalina.startup.Bootstrap start \
    >> ${CATALINA_BASE}/logs/catalina.out 2>&1 &
    if [ ! -z "$CATALINA_PID" ]; then
      echo $! > $CATALINA_PID
    fi

where we find the line

>> ${CATALINA_BASE}/logs/catalina.out 2>&1 &

That makes me think that the $CATALINA_BASE = "/usr/share/tomcat" since

[root@tomcat ~]# ls /usr/share/tomcat/logs/
catalina.out
[root@tomcat ~]#

Now, /etc/sysconfig/tomcat knows about $CATALINA_BASE even though it really does not define it (commented out):

#CATALINA_BASE="/usr/share/tomcat"

Sounds like we need to define $NAME and $CATALINA_BASE somewhere. My vote would be for
/etc/tomcat/tomcat.conf because it claims it is where we put custom stuff.

# For tomcat.service it's /etc/sysconfig/tomcat, for
# tomcat@instance it's /etc/sysconfig/tomcat@instance.

# THE TWO LINES I MENTIONED IN THE ARTICLE
NAME="tomcat"                                   
CATALINA_BASE="/usr/share/tomcat"

# This variable is used to figure out if config is loaded or not.
TOMCAT_CFG_LOADED="1"

# In new-style instances, if CATALINA_BASE isn't specified, it will
# be constructed by joining TOMCATS_BASE and NAME.
TOMCATS_BASE="/var/lib/tomcats/"

After that, I was able to start it and verify it was indeed running

[root@tomcat tomcat]# ps -ef|grep tomcat
root       352     1  8 13:09 ?        00:00:01 /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start
root       372     1  0 13:09 ?        00:00:00 grep --color=auto tomcat
[root@tomcat tomcat]#

I wrote a shorter version of this article as a reply to the question I found online and mentioned earlier in this article. I hope it will be useful to the original poster.

Entrypoint scripts and data persistency: mounting a NFS fileshare inside a docker container

This is a quick article; you have been warned.

The traditional way to have persistent data in a docker container is to feed it with a volume from the docker host. You begin by locating a directory, be it in the local docker host drive, network mounted to the docker host, or a volume in another docker container. Then feed it to the container using the VOLUME statement defined in the Dockerfile or to command line. We all know that. But what some (very few we hope) of you might not have been aware of is the volume is only mounted/attached to the container when you tell docker to run the container, which might make running/configuring something that require files in those volumes a bit challenging.

At this point I would expect one of you to shout "Aha! But you can then use an entrypoint script to do all that monkeying that needs to happen after volume is mounted and before whatever service this container provides starts!" And you would be quite right! Here's a quick example in case we lost somebody: if our Dockerfile ended up like this:

# Put the entrypoint script somewhere we can find
COPY docker-entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

EXPOSE 22
# Start service
CMD ["/usr/sbin/sshd", "-D"]

we would have created a file called entrypoint.sh in the root path of the container, which is run just after the volume is created and before the, in this example, sshd service is started. Said entrypoint file could do, I don't know:

#!/bin/sh
set -e

# Do something in /home, which is the mounted volume
/do-something-at-home.sh

# And we  are done here
exec "$@"

point is that do-something-at-home.sh is called only after the persistent volume is mounted into the container.

What if you want to mount a network fileshare straight into the container? Before I have time to answer that, someone from the audience will jump and state "Why would you want to do that? You can mount the network fileshare in the docker host and then use VOLUME just like Wicked Witch of the West intended!" What if I don't want to do that? What if I can't for whatever reason mount a network fileshare on the docker host?

The answer is that it is completely doable; you just have to ask the entrypoint script to do the mounting for you. To show what I mean I will use the example of mounting a NFSv4 fileshare that is supposed to be used by some website. So we modify our little entrypoint script a bit

#!/bin/sh
set -e

# /export/www was created in the Dockerfile
mount.nfs4 fileserver.example.com:/export/www /www/some-website 

# And we are done here
exec "$@"

Since we are using NFSv4, chances are you might need to add something like

RUN sed -i -e '/^#Domain/a Domain = example.com' /etc/idmapd.conf

to your dockerfile in addition to telling it to get the nfs client packages, but the general idea should apply for, say, SMB or ZFS or whatever other network filesystem that you fancy: let your entrypoint script do the heavy lifting!