Showing posts with label tomcat. Show all posts
Showing posts with label tomcat. Show all posts

Wednesday, July 27, 2016

Starting Tomcat manually in Docker

So I needed to test some settings between CentOS 6 + Apache Tomcat 6 + Java 6 and CentOS 7 + Tomcat7 + Java 8. Best way I found to do these quick tests is to build them in docker: write a quick dockerfile, spool it up, test, and blow it up.

Now, when I built and ran the tomcat 6 + CentOS 6 + Java 6 setup by connecting to the container and manually starting tomcat by typing

service tomcat6 start
It worked fine. So I then built the tomcat 7 + CentOS 7 + Java 8, and tried to start it

[root@tomcat ~]# systemctl start tomcat.service
Failed to get D-Bus connection: Operation not permitted
[root@tomcat ~]# 

Since systemd could not start it, and I could not figure out why (I do not take solace knowing I am not the only one), I tried starting it even more manually:

[root@tomcat /]# /usr/sbin/tomcat start
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory
[root@tomcat /]#

This is the time to do what everyone does at a time like this: look for answers online. All I got was someone asking the very same question. It seems if we want some answers we will need do more exploring on our own.

With that in mind, let's see those two lines we are being barked about:

[root@tomcat /]# sed -n '21p' /usr/sbin/tomcat
    . /etc/sysconfig/${NAME}
[root@tomcat /]# sed -n '39p' /usr/sbin/tomcat
  ${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
[root@tomcat /]#

Not much help here, but we will revisit that later. First, let's run the bash script again, but this time in a debugging (-x) mode:

[root@tomcat /]# bash -x /usr/sbin/tomcat start
+ '[' -r /usr/share/java-utils/java-functions ']'
+ . /usr/share/java-utils/java-functions
++ _load_java_conf
++ local IFS=:
++ local java_home_save=
++ local java_opts_save=
++ local javaconfdir
++ local conf
++ unset _javadirs
++ unset _jvmdirs
++ set -- /etc/java
++ _log 'Java config directories are:'
++ '[' -n '' ']'
++ for javaconfdir in '"$@"'
++ _log '  * /etc/java'
++ '[' -n '' ']'
++ for javaconfdir in '"$@"'
++ conf=/etc/java/java.conf
++ '[' '!' -f /etc/java/java.conf ']'
++ local IFS
++ local JAVA_LIBDIR
++ local JNI_LIBDIR
++ local JVM_ROOT
++ '[' -f /etc/java/java.conf ']'
++ _log 'Loading config file: /etc/java/java.conf'
++ '[' -n '' ']'
++ . /etc/java/java.conf
+++ JAVA_LIBDIR=/usr/share/java
+++ JNI_LIBDIR=/usr/lib/java
+++ JVM_ROOT=/usr/lib/jvm
++ _javadirs=/usr/share/java:/usr/lib/java
++ _jvmdirs=/usr/lib/jvm
++ _load_java_conf_file /root/.java/java.conf
++ local IFS
++ local JAVA_LIBDIR
++ local JNI_LIBDIR
++ local JVM_ROOT
++ '[' -f /root/.java/java.conf ']'
++ _log 'Skipping config file /root/.java/java.conf: file does not exist'
++ '[' -n '' ']'
++ _javadirs=/usr/share/java:/usr/lib/java
++ _jvmdirs=/usr/lib/jvm
++ '[' -d '' ']'
++ '[' -n '' ']'
++ '[' _ '!=' _off -a -f /usr/lib/abrt-java-connector/libabrt-java-connector.so
-a -f /var/run/abrt/abrtd.pid ']'
++ _log 'ABRT Java connector is disabled'
++ '[' -n '' ']'
+ '[' -z '' ']'
+ TOMCAT_CFG=/etc/tomcat/tomcat.conf
+ '[' -r /etc/tomcat/tomcat.conf ']'
+ . /etc/tomcat/tomcat.conf
++ TOMCAT_CFG_LOADED=1
++ TOMCATS_BASE=/var/lib/tomcats/
++ JAVA_HOME=/usr/lib/jvm/jre
++ CATALINA_HOME=/usr/share/tomcat
++ CATALINA_TMPDIR=/var/cache/tomcat/temp
++ SECURITY_MANAGER=false
+ '[' -r /etc/sysconfig/ ']'
+ . /etc/sysconfig/
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
+ set_javacmd
+ local IFS
+ local cmd
+ '[' -x '' ']'
+ set_jvm
+ local IFS=:
+ local cmd
+ local cmds
+ _set_java_home
+ local IFS=:
+ local jvmdir
+ local subdir
+ local subdirs
+ '[' -n /usr/lib/jvm/jre ']'
+ '[' -z '' ']'
++ readlink -f /usr/lib/jvm/jre/..
+ JVM_ROOT=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el7_2.x86_64
+ return
+ '[' -n /usr/lib/jvm/jre ']'
+ return
+ for cmd in jre/sh/java bin/java
+ JAVACMD=/usr/lib/jvm/jre/jre/sh/java
+ '[' -x /usr/lib/jvm/jre/jre/sh/java ']'
+ for cmd in jre/sh/java bin/java
+ JAVACMD=/usr/lib/jvm/jre/bin/java
+ '[' -x /usr/lib/jvm/jre/bin/java ']'
+ _log 'Using configured JAVACMD: /usr/lib/jvm/jre/bin/java'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ return 0
+ cd /usr/share/tomcat
+ '[' '!' -z '' ']'
+ '[' -n '' ']'
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-jul
i.jar
++ build-classpath commons-daemon
+ CLASSPATH=/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-jul
i.jar:/usr/share/java/commons-daemon.jar
+ '[' start = start ']'
+ '[' '!' -z '' ']'
[root@tomcat /]# + /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bo
otstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon
.jar -Dcatalina.base= -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -D
java.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/conf/logg
ing.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
 org.apache.catalina.startup.Bootstrap start
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory
[root@tomcat /]#

If you never ran a bash script with the -x option, you should since it shows the steps being performed by the script, including tests, as it runs. For instance, you can see it starts by learning a lot about the current Java installation. After that, it loads some file, TOMCAT_CFG=/etc/tomcat/tomcat.conf, and then gives the first error message:

+ TOMCAT_CFG=/etc/tomcat/tomcat.conf
+ '[' -r /etc/tomcat/tomcat.conf ']'
+ . /etc/tomcat/tomcat.conf
++ TOMCAT_CFG_LOADED=1
++ TOMCATS_BASE=/var/lib/tomcats/
++ JAVA_HOME=/usr/lib/jvm/jre
++ CATALINA_HOME=/usr/share/tomcat
++ CATALINA_TMPDIR=/var/cache/tomcat/temp
++ SECURITY_MANAGER=false
+ '[' -r /etc/sysconfig/ ']'
+ . /etc/sysconfig/
/usr/sbin/tomcat: line 21: .: /etc/sysconfig/: is a directory
Now, /etc/tomcat/tomcat.conf (same thing as /usr/share/tomcat/conf/tomcat.conf) defines a few global (to tomcat) to variables. The top of the file also explains it is the file where you should define variables that are custom to your system but global to all tomcat instances being run here. For instance, when I built the tomcat6 container, I had

JAVA_HOME="/usr/lib/jdk1.6.0_41"

because that was the specific java version I wanted to run. Now, if we look not only at line 21 in /usr/sbin/tomcat but also around said line, we can see it wants to load a file in /etc/sysconfig

# Get instance specific config file
if [ -r "/etc/sysconfig/${NAME}" ]; then
    . /etc/sysconfig/${NAME}
fi

If we look at /etc/sysconfig,

[root@tomcat ~]# ls /etc/sysconfig/
network  network-scripts  rdisc  tomcat
[root@tomcat ~]#

It sure makes me think that $NAME = "tomcat" and $NAME is not defined.

For the second error message we should examine the following lines

[root@tomcat /]# + /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bo
otstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon
.jar -Dcatalina.base= -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -D
java.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/conf/logg
ing.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
 org.apache.catalina.startup.Bootstrap start
/usr/sbin/tomcat: line 39: /logs/catalina.out: No such file or directory

That really looks like it wants to write to the log file catalina.out but can't find it. So we take a look at the lines around line 39:

if [ "$1" = "start" ]; then
  ${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
    -classpath "$CLASSPATH" \
    -Dcatalina.base="$CATALINA_BASE" \
    -Dcatalina.home="$CATALINA_HOME" \
    -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
    -Djava.io.tmpdir="$CATALINA_TMPDIR" \
    -Djava.util.logging.config.file="${CATALINA_BASE}/conf/logging.properties" \
    -Djava.util.logging.manager="org.apache.juli.ClassLoaderLogManager" \
    org.apache.catalina.startup.Bootstrap start \
    >> ${CATALINA_BASE}/logs/catalina.out 2>&1 &
    if [ ! -z "$CATALINA_PID" ]; then
      echo $! > $CATALINA_PID
    fi

where we find the line

>> ${CATALINA_BASE}/logs/catalina.out 2>&1 &

That makes me think that the $CATALINA_BASE = "/usr/share/tomcat" since

[root@tomcat ~]# ls /usr/share/tomcat/logs/
catalina.out
[root@tomcat ~]#

Now, /etc/sysconfig/tomcat knows about $CATALINA_BASE even though it really does not define it (commented out):

#CATALINA_BASE="/usr/share/tomcat"

Sounds like we need to define $NAME and $CATALINA_BASE somewhere. My vote would be for
/etc/tomcat/tomcat.conf because it claims it is where we put custom stuff.

# For tomcat.service it's /etc/sysconfig/tomcat, for
# tomcat@instance it's /etc/sysconfig/tomcat@instance.

# THE TWO LINES I MENTIONED IN THE ARTICLE
NAME="tomcat"                                   
CATALINA_BASE="/usr/share/tomcat"

# This variable is used to figure out if config is loaded or not.
TOMCAT_CFG_LOADED="1"

# In new-style instances, if CATALINA_BASE isn't specified, it will
# be constructed by joining TOMCATS_BASE and NAME.
TOMCATS_BASE="/var/lib/tomcats/"

After that, I was able to start it and verify it was indeed running

[root@tomcat tomcat]# ps -ef|grep tomcat
root       352     1  8 13:09 ?        00:00:01 /usr/lib/jvm/jre/bin/java -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start
root       372     1  0 13:09 ?        00:00:00 grep --color=auto tomcat
[root@tomcat tomcat]#

I wrote a shorter version of this article as a reply to the question I found online and mentioned earlier in this article. I hope it will be useful to the original poster.



Friday, November 07, 2014

Running a web app from command line using powershell

Had a interesting task: we have an web-based program that is run by connecting to a given website. Arguments are passed in the url. So far so good. But we want to run the program at scheduled times, and that means it must be run automagically. And the host that will be used to call the program runs Windows. How about that for requirements?

I decided that would be a good exercise in powershell. Why doing it in powershell instead of something in visual-something-or-another? Well, the honest answer is that I suck at those visual thingies. However, instead of telling the truth I will instead say: since I came from a Linux/Unix background, I would prefer to use some sort of script if I can and save the bloatware associated with building a program for another time. If I were to do this in the Linux camp, I would probably do it in Bourne, tcsh, or bash. Or python if I felt it required some degree of sophistication. On the Windows camp, however, the closest thing to those shells is Powershell. And if you have used it, it ain't bad at all.

This might be a bit long since I will go over how this evolved; go grab some popcorn.

In Windows you can call a web browser from the command line feeding it the url for whatever site you want it to open. Something like this:

explorer "http://unixwars.blogspot.com"

would pop up this very site on your default browser. Problem with that is, well, you are loading a web browser. There are times you do want to pop a browser, but if you want to run something in the background that might not be a good idea. After all, would you want to be typing the most amazing program ever written in Ook! when suddenly a web browser comes up taking you to some website you never heard of? You know, like spammers love to do, but instead point to a page without boobs or free money offers. Even if you know it is the web-based program mentioned above being called, that would get old really quickly. So, we need another option.

Powershell allows you to call some .net libraries and classes. One of those classes is System.Net.WebRequest. Here's an example of how we could use it:

$request = [System.Net.WebRequest]::Create("http://www.example.com/nagios")
$reply = $request.GetResponse()

The first line creates the request to connect, in this case, to my nagios server. But, that does not send any traffic yet. We need to send the request, which is the job of the last line. It also captures the reply sent by the server. Now what should I get by sending a carefully improper request like that? An error message. What kind of message, you may ask. Let's see what you would get back using netcat (from my Linux laptop):

raub@black:~$ nc -v "www.example.com" 80
Connection to www.example.com 80 port [tcp/http] succeeded!
GET http://www.example.com/ HTTP/1.0
HTTP/1.1 400 Bad Request
Date: Mon, 06 Oct 2014 05:18:50 GMT
Server: Apache/2.2.15 (CentOS)
Content-Length: 311
Connection: close
Content-Type: text/html; charset=iso-8859-1



400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.


Apache/2.2.15 (CentOS) Server at www.example.com Port 80
raub@black:~$

As you can see we get a 400 Error. So, the above commands should spit back a similar message. Note at this point I do not care about getting a proper reply; I just want to connect to the web server.

Once I do that, we should take a look at the Apache access.log:

10.0.0.102 - - [)6/Oct/2014:00:18:50 -0500] "GET http://ww.example.com/ HTTP/1.0" 400 311 "-" "-"

As you can see, the logs register when netcat reached the server. Now, let's see if we can repeat the deed using powershell. I am going to run the two lines I mentioned above but as a powershell script, which I shall call gimmesite.ps1:

PS C:\Users\Administrator\dev> cat .\gimmesite.ps1
$request = [System.Net.WebRequest]::Create("http://www.example.com")
$reply = $request.GetResponse()
PS C:\Users\Administrator\dev>

Let's try to run it:

PS C:\Users\Administrator\Documents\dev> .\gimmesite.ps1
.\gimmesite.ps1 : File C:\Users\Administrator\Documents\dev\gimmesite.ps1 cannot be loaded because running scripts is
disabled on this system. For more information, see about_Execution_Policies at http://go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1
+ .\gimmesite.ps1
+ ~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess
PS C:\Users\Administrator\Documents\dev>

That does not look very happy. As the error message tries to tell us, the windows box is setup not to run any powershell script. You would need to certify it, which is something I have yet to do. There is a workaround, however which is mentioned in this thread:

PS C:\Users\Administrator\dev> powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
PS C:\Users\Administrator\dev>

As you can see, no error messages this time since we asked for an exception. Did apache see out connection attempt?

==> /var/log/httpd/access_log <==
10.0.0.105 - - [06/Oct/2014:00:23:50 -0500] "GET / HTTP/1.1" 200 - "-" "-"

It seems we are making progress. Next step is a small correction in the url we are using. You see, the site we are using to test out is http://www.example.com/nagios, not http://www.example.com/. So we edit the gimmesite.ps1 script and try again:

PS C:\Users\Administrator\dev> powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (401) Unauthorized."
At C:\Users\Administrator\dev\gimmesite.ps1:2 char:1
+ $reply = $request.GetResponse()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException

PS C:\Users\Administrator\dev>

The error message makes sense: you need login credentials to access my nagios page. If we remember the objective of the script we are trying to develop in this article -- a powershell script that runs a script in a webpage at certain intervals -- we realize we have no interest in any error message. So, we need to get rid of it. Now, powershell has some structures similar to Java and Python. The one of interest here is the try{}catch{} one. So, let's modify the script once more:

PS C:\Users\Administrator\dev> cat gimmesite.ps1
# Connect to some site we told it about
# Run it as
# powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1

$source = "http://www.example.com/nagios"
try
{
        # Let's try to reach site with our noodly tentacles:
        $request = [System.Net.WebRequest]::Create($source)
        $reply = $request.GetResponse()
}
catch
{
        # I tried really hard to care about error messages but I failed
}
PS C:\Users\Administrator\dev>

I hope the comments help a bit understand what is going on. When we run that, error messages are caught by the catch{} statement. Since we do nothing with them (we could in a fancier program), they are never printed to the screen/stdin:

PS C:\Users\Administrator\dev> powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
PS C:\Users\Administrator\dev>

So we have a working script. How about running it at predetermined intervals? Windows has no cron command, but there are equivalent commands. In fact, powershell has a set of commands to schedule tasks... which I will not use. Since this blog entry is getting long and I am getting tired of typing, I will instead use schtasks, which has been around since Windows XP. Let me add a comment to the script describing how to use it:

PS C:\Users\Administrator\dev> cat gimmesite.ps1
# Connect to some site we told it about
# Run it as
# powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
# And from the cronjobbie
# Schtasks /create /tn "Connect to site" /sc daily /st 07:00 /tr "powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1"

$source = "http://www.example.com/nagios"
try
{
        # Let's try to reach site with our noodly tentacles:
        $request = [System.Net.WebRequest]::Create($source)
        $reply = $request.GetResponse()
}
catch
{
        # I tried really hard to care about error messages but I failed
}
PS C:\Users\Administrator\dev>

The above comment shows how to setup a "schedule task" (fancy term for a cron job) that will be run every day at 7am. Since I want to test this, let's make it run every 10 minutes instead:

PS C:\Users\Administrator\Documents\dev> schtasks /create /tn "Connect to site" /sc minute /mo 10 /tr "powershell -execution policy bypass -file C:\Users\Administrator\Documents\dev\gimmesite.ps1"
WARNING: The task name "Connect to site" already exists. Do you want to replace it (Y/N)? y
SUCCESS: The scheduled task "Connect to site" has successfully been created.
PS C:\Users\Administrator\Documents\dev> schtasks /query|more

Folder: \
TaskName                                 Next Run Time          Status
======================================== ====================== ===============
Connect to site                          11/6/2014 12:46:00 PM  Ready

Folder: \Microsoft
TaskName                                 Next Run Time          Status
======================================== ====================== ===============
INFO: There are no scheduled tasks presently available at your access level.
[...]

You can see that it barked I already had created the task before. Since I want to change it, I told it to just replace it. You probably also noticed that the task is being identified by its name (the /tn option). I will not spend any time at all describing the different options available in schtasks; there are beautiful pages online describing that. Or you can do schtasks /query /?. One final point: the task name is rather important for it is how you refer to it if you have to manipulate it somehow. How about if we delete the task?

PS C:\Users\Administrator\Documents\dev> schtasks /delete /tn "Connect to site"
WARNING: Are you sure you want to remove the task "Connect to site" (Y/N)? y
SUCCESS: The scheduled task "Connect to site" was successfully deleted.
PS C:\Users\Administrator\Documents\dev>

I hope this is enough to get you started and give you evil ideas. I put a slightly more complex version of this script in github.