Running a web app from command line using powershell

Had a interesting task: we have an web-based program that is run by connecting to a given website. Arguments are passed in the url. So far so good. But we want to run the program at scheduled times, and that means it must be run automagically. And the host that will be used to call the program runs Windows. How about that for requirements?

I decided that would be a good exercise in powershell. Why doing it in powershell instead of something in visual-something-or-another? Well, the honest answer is that I suck at those visual thingies. However, instead of telling the truth I will instead say: since I came from a Linux/Unix background, I would prefer to use some sort of script if I can and save the bloatware associated with building a program for another time. If I were to do this in the Linux camp, I would probably do it in Bourne, tcsh, or bash. Or python if I felt it required some degree of sophistication. On the Windows camp, however, the closest thing to those shells is Powershell. And if you have used it, it ain't bad at all.

This might be a bit long since I will go over how this evolved; go grab some popcorn.

In Windows you can call a web browser from the command line feeding it the url for whatever site you want it to open. Something like this:

explorer "http://unixwars.blogspot.com"

would pop up this very site on your default browser. Problem with that is, well, you are loading a web browser. There are times you do want to pop a browser, but if you want to run something in the background that might not be a good idea. After all, would you want to be typing the most amazing program ever written in Ook! when suddenly a web browser comes up taking you to some website you never heard of? You know, like spammers love to do, but instead point to a page without boobs or free money offers. Even if you know it is the web-based program mentioned above being called, that would get old really quickly. So, we need another option.

Powershell allows you to call some .net libraries and classes. One of those classes is System.Net.WebRequest. Here's an example of how we could use it:

$request = [System.Net.WebRequest]::Create("http://www.example.com/nagios")
$reply = $request.GetResponse()

The first line creates the request to connect, in this case, to my nagios server. But, that does not send any traffic yet. We need to send the request, which is the job of the last line. It also captures the reply sent by the server. Now what should I get by sending a carefully improper request like that? An error message. What kind of message, you may ask. Let's see what you would get back using netcat (from my Linux laptop):

raub@black:~$ nc -v "www.example.com" 80
Connection to www.example.com 80 port [tcp/http] succeeded!
GET http://www.example.com/ HTTP/1.0
HTTP/1.1 400 Bad Request
Date: Mon, 06 Oct 2014 05:18:50 GMT
Server: Apache/2.2.15 (CentOS)
Content-Length: 311
Connection: close
Content-Type: text/html; charset=iso-8859-1





Bad Request
Your browser sent a request that this server could not understand.
Apache/2.2.15 (CentOS) Server at www.example.com Port 80
raub@black:~$ 

As you can see we get a 400 Error. So, the above commands should spit back a similar message. Note at this point I do not care about getting a proper reply; I just want to connect to the web server.

Once I do that, we should take a look at the Apache access.log:

10.0.0.102 - - [)6/Oct/2014:00:18:50 -0500] "GET http://ww.example.com/ HTTP/1.0" 400 311 "-" "-"

As you can see, the logs register when netcat reached the server. Now, let's see if we can repeat the deed using powershell. I am going to run the two lines I mentioned above but as a powershell script, which I shall call gimmesite.ps1:

PS C:\Users\Administrator\dev> cat .\gimmesite.ps1
$request = [System.Net.WebRequest]::Create("http://www.example.com")
$reply = $request.GetResponse()
PS C:\Users\Administrator\dev>

Let's try to run it:

PS C:\Users\Administrator\Documents\dev> .\gimmesite.ps1
.\gimmesite.ps1 : File C:\Users\Administrator\Documents\dev\gimmesite.ps1 cannot be loaded because running scripts is
disabled on this system. For more information, see about_Execution_Policies at http://go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1
+ .\gimmesite.ps1
+ ~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess
PS C:\Users\Administrator\Documents\dev>

That does not look very happy. As the error message tries to tell us, the windows box is setup not to run any powershell script. You would need to certify it, which is something I have yet to do. There is a workaround, however which is mentioned in this thread:

PS C:\Users\Administrator\dev> powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
PS C:\Users\Administrator\dev>

As you can see, no error messages this time since we asked for an exception. Did apache see out connection attempt?

==> /var/log/httpd/access_log <==
10.0.0.105 - - [06/Oct/2014:00:23:50 -0500] "GET / HTTP/1.1" 200 - "-" "-"

It seems we are making progress. Next step is a small correction in the url we are using. You see, the site we are using to test out is http://www.example.com/nagios, not http://www.example.com/. So we edit the gimmesite.ps1 script and try again:

PS C:\Users\Administrator\dev> powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (401) Unauthorized."
At C:\Users\Administrator\dev\gimmesite.ps1:2 char:1
+ $reply = $request.GetResponse()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException

PS C:\Users\Administrator\dev>

The error message makes sense: you need login credentials to access my nagios page. If we remember the objective of the script we are trying to develop in this article -- a powershell script that runs a script in a webpage at certain intervals -- we realize we have no interest in any error message. So, we need to get rid of it. Now, powershell has some structures similar to Java and Python. The one of interest here is the try{}catch{} one. So, let's modify the script once more:

PS C:\Users\Administrator\dev> cat gimmesite.ps1
# Connect to some site we told it about
# Run it as
# powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1

$source = "http://www.example.com/nagios"
try
{
        # Let's try to reach site with our noodly tentacles:
        $request = [System.Net.WebRequest]::Create($source)
        $reply = $request.GetResponse()
}
catch
{
        # I tried really hard to care about error messages but I failed
}
PS C:\Users\Administrator\dev>

I hope the comments help a bit understand what is going on. When we run that, error messages are caught by the catch{} statement. Since we do nothing with them (we could in a fancier program), they are never printed to the screen/stdin:

PS C:\Users\Administrator\dev> powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
PS C:\Users\Administrator\dev>

So we have a working script. How about running it at predetermined intervals? Windows has no cron command, but there are equivalent commands. In fact, powershell has a set of commands to schedule tasks... which I will not use. Since this blog entry is getting long and I am getting tired of typing, I will instead use schtasks, which has been around since Windows XP. Let me add a comment to the script describing how to use it:

PS C:\Users\Administrator\dev> cat gimmesite.ps1
# Connect to some site we told it about
# Run it as
# powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1
# And from the cronjobbie
# Schtasks /create /tn "Connect to site" /sc daily /st 07:00 /tr "powershell -ExecutionPolicy ByPass -File .\gimmesite.ps1"

$source = "http://www.example.com/nagios"
try
{
        # Let's try to reach site with our noodly tentacles:
        $request = [System.Net.WebRequest]::Create($source)
        $reply = $request.GetResponse()
}
catch
{
        # I tried really hard to care about error messages but I failed
}
PS C:\Users\Administrator\dev>

The above comment shows how to setup a "schedule task" (fancy term for a cron job) that will be run every day at 7am. Since I want to test this, let's make it run every 10 minutes instead:

PS C:\Users\Administrator\Documents\dev> schtasks /create /tn "Connect to site" /sc minute /mo 10 /tr "powershell -execution policy bypass -file C:\Users\Administrator\Documents\dev\gimmesite.ps1"
WARNING: The task name "Connect to site" already exists. Do you want to replace it (Y/N)? y
SUCCESS: The scheduled task "Connect to site" has successfully been created.
PS C:\Users\Administrator\Documents\dev> schtasks /query|more

Folder: \
TaskName                                 Next Run Time          Status
======================================== ====================== ===============
Connect to site                          11/6/2014 12:46:00 PM  Ready

Folder: \Microsoft
TaskName                                 Next Run Time          Status
======================================== ====================== ===============
INFO: There are no scheduled tasks presently available at your access level.
[...]

You can see that it barked I already had created the task before. Since I want to change it, I told it to just replace it. You probably also noticed that the task is being identified by its name (the /tn option). I will not spend any time at all describing the different options available in schtasks; there are beautiful pages online describing that. Or you can do schtasks /query /?. One final point: the task name is rather important for it is how you refer to it if you have to manipulate it somehow. How about if we delete the task?

PS C:\Users\Administrator\Documents\dev> schtasks /delete /tn "Connect to site"
WARNING: Are you sure you want to remove the task "Connect to site" (Y/N)? y
SUCCESS: The scheduled task "Connect to site" was successfully deleted.
PS C:\Users\Administrator\Documents\dev>

I hope this is enough to get you started and give you evil ideas. I put a slightly more complex version of this script in github.