Showing posts with label upgrade. Show all posts
Showing posts with label upgrade. Show all posts

Thursday, November 03, 2016

Yum Manually and multiple repos

Quick post (I hope) about something I learned today that has a bit of a Captain Obvious taste to it. But I thought some people might find that amusing... even if it is at my expense.

Like many who use Red Hat products (CentOS comes to mind) and derived distros, I use repos outside the official ones. Because those repos tend to have newer versions of some packages, I try to be careful about only upgrading the packages that are required by the package I added said repo to my list. Short answer is the Law of Unintended Consequences. Long version is that I expect that people building the official packages being quite careful about compatibility and security. So, I should only reach out to the different repos after I found out the official packages do not do what I need.

What I have been doing, and I do not claim it is the best solution, is to install and then disable the non-official repos so if I want something from them I have to specifically ask for it. So, if I want to use the remi repo, I would first disable it

sed -i -e 's/^enabled=1/enabled=0/' /etc/yum.repos.d/remi.repo

and then specifically ask for it to install, say, php

yum install php --enablerepo=remi

which would install the latest PHP version that remi has. On a side note, if you had to install php 5.6 from remi, you would use remi-php56. But, what about upgrading them? After all, yum check-update and yum update by default will not check the disabled repos even if you have packages installed from them. So, you have to use --enablerepo. Now, what I learned today is that you can just list all the repos you are using by separating them with commas.

Let me show you in action: at first we think there are no updates:

raub@pickles ~]$ sudo yum check-update
Loaded plugins: product-id, rhnplugin, search-disabled-repos, subscription-
              : manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
[raub@pickles ~]$

Now, let's list all the repos we have been using with this machine and see what it tells us:

raub@pickles ~]$ sudo yum check-update --enablerepo=remi-php56,epel,secu
rity_shibboleth
Loaded plugins: product-id, rhnplugin, search-disabled-repos, subscription-
              : manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
security_shibboleth                                      | 1.2 kB     00:00
security_shibboleth/primary                                |  15 kB   00:00
security_shibboleth                                                       96/96

libcurl-openssl.x86_64                7.51.0-2.1             security_shibboleth
opensaml-schemas.x86_64               2.6.0-1.1              security_shibboleth
shibboleth.x86_64                     2.6.0-2.1              security_shibboleth
xmltooling-schemas.x86_64             1.6.0-1.1              security_shibboleth
[raub@pickles ~]$

As you can see, we do need to update shibboleth and its dependencies! And update we shall:

[raub@pickles ~]$ sudo yum update --enablerepo=remi-php56,epel,security_shibboleth
[...]
[raub@pickles ~]$ 

Kinda neat, eh?

Wednesday, September 14, 2016

Setting up atftp in (ubuntu) Linux

I have a switch whose firmware was in dire need of being updated. Thing is, the switch will only take the upgrade firmware if it is offered by a tftp server, don't ask me why. And that is something I tend not to have running; last time I used one was to network boot Solaris boxes. Sounds like an excuse to write another article.

Just to be different I will be deploying this on a Ubuntu Linux host instead of a CentOS/RedHat one as I usually do. Why? Doing the same all the time gets boring quickly. So, let's see which versions of tftpd we can pick and choose:

raub@desktop:~$ apt-cache search tftpd
tftpd-hpa - HPA's tftp server
atftpd - advanced TFTP server
libnet-tftpd-perl - Perl extension for Trivial File Transfer Protocol Server
tftpd - Trivial file transfer protocol server
uec-provisioning-tftpd - the UEC Provisioning TFTP server
raub@desktop:~$

After careful scientific consideration, the atftpd one sounds more interesting (Multi-threaded TFTP server implementing extension and multicast), so we will pick that one. I think this is the part of the show in which we go through the steps to do the deed:

  1. A good place to start is to install it. I like command line and apt-get, so I think

    sudo apt-get install atftpd

    should do the trick. Of course you can use aptitude or the GUI. But I am lazy.

  2. Traditionally the directory used to put stuff that will be shared by tftp is /tftpboot, but current practice is to use /srv/tftp. In fact, atftpd does create /srv/tftp for you. For the same of showing how to customize things, let's say we want to be old school. And that means creating /tftpboot ourselves:

    sudo mkdir /tftpboot
    sudo chmod -R 777 /tftpboot
    sudo chown -R nobody:nogroup /tftpboot
  3. This might be a good time to put the files we want to be shared, in this example a file called image.bin in /tftpboot

  4. We need to configure it by editing /etc/default/atftpd. Here is what mine looks like

    raub@desktop:~$ cat /etc/default/atftpd
    USE_INETD=false
    OPTIONS="--tftpd-timeout 300 --retry-timeout 5 --port=69 --mcast-port 1758 
    --mcast-addr 192.168.0.0-255 --mcast-ttl 1 --maxthread 100 --verbose=7 /tftpboot"
    raub@desktop:~$

    where:

    • --port=69 we are forcing it to use the default tftp port
    • --mcast-addr 192.168.0.0-255 specifies the multicast address range we will be using. Being lazy, I am using the entire 192.168.0.0/24 range
    • /tftpboot is the directory we will be sharing as explained above. By default the config file specifies /srv/tftp which means if we put our file in /tftpboot we would get a message like

      Sep 13 13:54:00 desktop atftpd[28045]: File /srv/tftp/image.bin 
      not found

      when we try to fetch the file

    • --verbose=7 is the highest amount of verbose we can use. By default its value is set to 5.

    Once it starts properly (service atftpd start should be enough to start it), you should see something like

    raub@desktop:~$ ps -ef|grep ftp
    raub     16510 11504  0 15:26 pts/11   00:00:00 grep ftp
    nobody   28161     1  0 Sep13 ?        00:00:00 /usr/sbin/atftpd --daemon --tftpd-timeout 300 --retry-timeout 5 --port=69 --mcast-port 1758 --mcast-addr 192.168.0.0-255 --mcast-ttl 1 --maxthread 100 --verbose=7 /tftpboot
    raub@desktop:~$

How to get the file using tftp is beyond this discussion because it depends on your OS and the tftp client you are using. For instance the switch might show a webpage where you can configure the name of the tftp server -- 192.168.0.102 in my example -- and the name of the file you want to grab. What is more interesting is to see how the entire enchilada from we started the tftp server until we get the file image.bin looks like. By default (can be configured) we would see that in /var/log/syslog:

Sep 13 13:55:52 desktop systemd[1]: Starting LSB: Launch atftpd server...
Sep 13 13:55:52 desktop atftpd[28160]: Advanced Trivial FTP server started (0.7)
Sep 13 13:55:52 desktop atftpd[28153]: Starting Advanced TFTP server: atftpd.
Sep 13 13:55:52 desktop atftpd[28161]:   running in daemon mode on port 69
Sep 13 13:55:52 desktop atftpd[28161]:   logging level: 7
Sep 13 13:55:52 desktop atftpd[28161]:   directory: /tftpboot/
Sep 13 13:55:52 desktop atftpd[28161]:   user: nobody.nogroup
Sep 13 13:55:52 desktop atftpd[28161]:   log file: syslog
Sep 13 13:55:52 desktop atftpd[28161]:   not forcing to listen on local interfaces.
Sep 13 13:55:52 desktop atftpd[28161]:   server timeout: Not used
Sep 13 13:55:52 desktop atftpd[28161]:   tftp retry timeout: 5
Sep 13 13:55:52 desktop atftpd[28161]:   maximum number of thread: 100
Sep 13 13:55:52 desktop atftpd[28161]:   option timeout:   enabled
Sep 13 13:55:52 desktop atftpd[28161]:   option tzise:     enabled
Sep 13 13:55:52 desktop atftpd[28161]:   option blksize:   enabled
Sep 13 13:55:52 desktop atftpd[28161]:   option multicast: enabled
Sep 13 13:55:52 desktop atftpd[28161]:      address range: 192.168.0.0-255
Sep 13 13:55:52 desktop atftpd[28161]:      port range:    1758
Sep 13 13:55:52 desktop systemd[1]: Started LSB: Launch atftpd server.
Sep 13 13:55:59 desktop atftpd[28161]: socket may listen on any address, including broadcast
Sep 13 13:55:59 desktop atftpd[28161]: Creating new socket: 192.168.0.102:45115
Sep 13 13:55:59 desktop atftpd[28161]: Serving image.bin to 192.168.0.3:2295
Sep 13 13:56:03 desktop atftpd[28161]: End of transfer
Sep 13 13:56:03 desktop atftpd[28161]: Server thread exiting

The underline on the entry when the file image.bin is transferred was added to make it easy to see. And that is pretty much all I had to do. Once file was transfered, I stopped atftpd and then remove it

sudo apt-get remove --purge atftpd

because I do not like to have unused services running.

PS: Always backup your switch/network device's config before upgrading its firmware in case it reverts to default as part of the upgrade process. Guess who forgot to do that? Good thing I had good notes and could reconfigure it using the time-honored cut-n-paste technique

Sunday, March 23, 2014

When upgrades go bad: Installing JunOS from USB in a SRX router

So, I screwed up pretty bad. I decided to upgrade the JunOS release in this Juniper SRX210 router to the one (at the time I type this) recommended by Juniper, 11.4R10.3. When it booted up after the install, it crashed during the boot process. Well, I could have spent the time kicking myself but I am doing this upgrade off-hours and I did account for things going badly in my downtime estimate. And, this router is part of a redundant router setup using the Virtual Router Redundancy Protocol (VRRP); being down will not affect production. In other words, this is more of an annoyance than a real issue. Since I have to deal with this, how about if we learn how to restore the OS in this juniper router?

I tried a few ways and thought that the easiest one was to use a USB drive. Of course, it will not work well if you are not physically close to said router (other things will also not work well in these circumstances but that is another topic), but since I can I am doing the USB upgrade.

Procedure

  1. Get a USB drive. I know, this is a pretty obvious step but it is step 1. Ideally use a 1GB/2GB USB drive, formatted as fat16/fat32. Honestly I do not know how critical that is, but my experience with Cisco, which seems not to like the higher capacity ones, made me be leery. On the plus side, you should be able to find those rather easily as people replace their old ones with newer larger ones. If not, there are always the usual sources such as ebay or amazon.
  2. Download and copy OS image you are going to use, say junos-srxsme-11.4R10.3-domestic.tgz, into USB drive. If you are smarter than me, you would have gone to the Juniper downloads site and got all the OS images you need, placing them in your file server. I wasn't so I had to go the SRX210 download page and fetch it.
  3. Have your trusty serial cable and connect it to the router's console port. The default setup is the time-honored 9600 8N1. If you changed it, make sure you wrote than somewhere. I am lazy and I kinda like that setting.
  4. Connect USB drive to router.
  5. Reboot router after you attack the usb drive to it. It needs to know the drive exists as it boots up. Otherwise, it will bark like this:
    loader> install file:///junos-srxsme-11.4R10.3-domestic.tgz
    cannot open package (error 22)
    loader>

    When you try to install it.

  6. Now, if you boot with USB already connected to router, it will first say something like this:

    Running U-Boot CRC Test... OK.
    Flash:  4 MB
    USB:   scanning bus for devices... 4 USB Device(s) found
           scanning bus for storage devices... 2 Storage Device(s) found
    Clearing DRAM........ done
    BIST check passed.

    Some of you noticed the 2 storage devices message. It is talking about the inboard one (probably where the OS should be) and the external drive.

  7. Now, when you see

    POST Passed
    Press SPACE to abort autoboot in 1 seconds

    Please keep your fingers in your pockets. If you press space here, you will end up in the => prompt (U-boot). If you wait you will then see

    Protected 1 sectors
    Loading /boot/defaults/loader.conf
    /kernel data=0xb0f9c0+0x134788 DA(some hot action happening here)

    have your space-bar finger on standby for the next message will be

    Hit [Enter] to boot immediately, or space bar for command prompt.
  8. Then you will press space bar and get the loader> prompt. And now, it will start doing the install thingie:

    loader> install file:///junos-srxsme-11.4R10.3-domestic.tgz
    /kernel data=0xae82f0+0x12d2b8 syms=[0x4+0x88ce0+0x4+0xc6af6]
    Kernel entry at 0x801000d8 ...
    init regular console
    GDB: debug ports: uart
    GDB: current port: uart
    KDB: debugger backends: ddb gdb
    KDB: current backend: ddb
    Copyright (c) 1996-2013, Juniper Networks, Inc.
    All rights reserved.
    Copyright (c) 1992-2006 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
            The Regents of the University of California. All rights reserved.
    JUNOS 11.4R10.3 #0: 2013-11-15 06:56:20 UTC
    [...]
  9. After a while (I got bored and went to make me some tea), you will see it recreate the ssh key pairs and then finally be ready for business (apologies for the bad cut-n-pasting but my terminal console was being cute):

    |
    |                 |
    |  .o  ..         |
    |.+o .o.o.
    |X . .. .. E      |
    |oo ..            |
    |  .+             |
    |.-+
    root@uranus% omplete
    Setting initial options: .
    Starting optface configuration:
    additional daemons: eventd.
    Additional rout;/boot/modules -> /bo;
    kld netpfe drv: ifpfed_dialer default_adtwork setup:.
    Starting final network daemons:.
    setting ldconfig.
    Initial rc.mips initialization:.
    Local package initializationup access
    .
    kern.securelevel: -1 -> 1
    Creating JAIL MFS partitirade.uboot="0xBFC00000"
    boot.upgrade.loader="0xBFE00000"
    Boot mILE SYSTEM CLEAN; SKIPPING CHECKS
    clean, 78249 free (17 frags, ar 20 16:46:25 CDT 2014
    
    uranus (ttyu0)

    Note that it remembered the hostname for the router. I still went through the configs before letting it join the router cluster. But that is pretty much it! Router is back in business.

Closing Thoughts

  1. The universe is Murphian; things will go wrong. Try not to stress about that.
  2. When you schedule downtime for upgrades, account for things going badly in your time estimates.
  3. The hardest thing to do is figuring out what can go wrong. But, you could ask yourself "If this upgrade halts server or just this service, what would be my backup plan?" and then see if you can answer that question.
  4. Next time I need to upgrade the OS in this or another router, I will have the firmware/OS on standby in a USB drive. I do not know about you but I found out when I am prepared everything works out perfectly.
  5. If you can afford it, redundancy is a wonderful thing.
  6. Always save your configs somewhere, well, safe. Having to recreate them from scratch is a bit of a drag.

Tuesday, September 03, 2013

Upgrading SugarCRM command-line style

Where do we start? If you are here, chances are you know what SugarCRM is. For the lazy amongst you (like me!), the CRM in SugarCRM stands for Customer relationship management, which can be seen as a way to keep track of everything related to your customers: address, frequency and types of support questions, infomercials you sent to them, and so on.

If I can go on a detour, the concept is interesting because if you forget about the "customer" part of the acronym, you as an individual can apply it to other things. Like dealing with the IRS and other government organizations, your landlord (yes you should, trust me), job hunting, and so on. Ok, time to merge back with the main traffic.

The reason I am writing this is, as the title hints at, upgrading SugarCRM. There are two ways of upgrading this program. The first one is using the Upgrade Wizard and the other is using what they call a Silent Upgrade. Both start the same way:

  1. Find out the version you currently have installed and the one you want to upgrade to. At the time I wrote this, I had 6.5.9 installed and wanted to go to 6.5.14.
  2. Go to http://sourceforge.net/projects/sugarcrm/ and look for the upgrade file. In my case, it was in the http://sourceforge.net/projects/sugarcrm/files/1%20-%20SugarCRM%206.5.X/SugarCommunityEdition-6.5.X%20Upgrade/ directory. And the file I wanted was SugarCE-Upgrade-6.5.x-to-6.5.14.zip.
  3. Place the zip file somewhere the user running the web server can access. Since I am using Linux and am quite lazy, the location of choice was /tmp.

Now, if you were going to use the Wizard, you would follow something like http://www.siteground.com/tutorials/sugarcrm/sugarcrm-upgrade.htm... and then SugarCRM would then hang for an hour or so. I did try following the instructions in http://support.sugarcrm.com/04_Find_Answers/02KB/02Administration/100Install/Troubleshooting_Upgrade_Wizard_System_Check to no avail. I posted about this on their support site but so far no replies. So, time for a silent install.

Doing it command-line style

Don't get me wrong: using the command line to upgrade SugarCRM is not only to avoid sitting in your thumbs for an hour just to see it fail. It does not require the person doing the upgrade be a SugarCRM admin; if you can run as the web server user, you are good. You do not even need to be a user with root rights. Privilege separation in action babe!

So we go back to the steps we used to get the zip file (see above) and then add a few more:

  1. In the same directory you downloaded the upgrade file from, find a file called silentUpgrade-CE-X.Y.Z.zip where X, Y, and Z match the version you are upgrading to. In my case that was silentUpgrade-CE-6.5.14.zip.
  2. Download the silentUpgrade file you found in the previous step to the same convenient location you downloaded the upgrade file to. Once again in my case that was /tmp.
  3. Unzip the silentUpgrade file, but leave the upgrade one alone. If you are using /tmp as the staging directory, you should end up with something like this
    root@sugarcrm:/tmp# ls -lh /tmp/
    total 4.4M
    -rw-r--r-- 1 root     root      25K Jun 21 02:01 silentUpgrade_dce_step1.php
    -rw-r--r-- 1 root     root      33K Jun 21 02:01 silentUpgrade_dce_step2.php
    -rw-r--r-- 1 root     root     4.4K Jun 21 02:01 silentUpgrade.php
    -rw-r--r-- 1 root     root      42K Jun 21 02:01 silentUpgrade_step1.php
    -rw-r--r-- 1 root     root      21K Jun 21 02:01 silentUpgrade_step2.php
    -rw-r--r-- 1 root     root     4.9K Jun 21 02:01 SILENTUPGRADE.txt
    -rw-r--r-- 1 root     root     4.2M Aug 28 09:59 SugarCE-Upgrade-6.5.x-to-6.5.14.zip
    root@sugarcrm:/tmp# 
    Note that I cheated here and am running as root. You can unzip those files as yourself as long as you have permission to set their group id to the web server user (in Ubuntu that would be www-data).
  4. Run silentupgrade, telling it where to store its log file. Just to be different, I decided my log file shall be /tmp/sucre.log:
    sudo -u www-data php -f /tmp/silentUpgrade.php /tmp/SugarCE-Upgrade-6.5.x-to-6.5.14.zip /tmp/sucre.log /var/www/sugar/ admin
  5. If successful, you should see a message like this:
    ********************************************************************
    ***************This Upgrade process may take sometime***************
    ********************************************************************
    
    ********************************************************************
    *************************** SUCCESS*********************************
    ********************************************************************
    ******** If your pre-upgrade Leads data is not showing  ************
    ******** Or you see errors in detailview subpanels  ****************
    ************* In order to resolve them  ****************************
    ******** Log into application as Administrator  ********************
    ******** Go to Admin panel  ****************************************
    ******** Run Repair -> Rebuild Relationships  **********************
    ********************************************************************
    If you see an error, check the log file you created (in this case /tmp/sucre.log) for clues.

If you are curious, the actual running of silentupgrade took under a minute once all the files were properly downloaded and unzipped. I don't know about you but to me that is a big improvement from waiting an hour to find out it did not work.

References