Readers Respond: Updating VIOS on POWER7, and a Simplified invscout Script

A customer was planning to patch their VIO servers on a POWER7 machine. You may recall my mention of another customer that had success running recent code on POWER7 despite the lack of documentation.

“I mention POWER7 specifically because a client with that system was wondering about their options. If you go to FLRT and plug in 8202-E4C for the machine type model, you’ll see that 2.2.6.5 is the latest recommended version of VIOS code. For an 8202-E4D model (POWER7+), the recommendation is 3.1.2.21.

What would you do in this situation? Is there enough of a difference between POWER7 and POWER7+ that you would hesitate to deviate from the recommendations? In this case, even when told by support to stand pat, the client was brave and chose to install 3.1.2.21 on their POWER7 box, because that version of VIOS is still supported. It worked. Of course every environment is unique, so your mileage may vary. And ultimately, getting to POWER9 should be your goal if it is at all possible. But I can report that at least one client is now successfully running 3.1.2.21 on POWER7.”

In this case, the customer’s 8202-E4C was running fine on VIOS 3.1.1.25. But with FLRT showing that 3.1.2.21 was supported on an 8202-E4D, they decided to roll the dice and move to 3.1.3.14. It didn’t go well. Their NPIV mappings stopped working, and attempts to login to the SAN switch triggered errors from the HMC lsnportlogin command. Using the chnportlogin command didn’t work either. Here’s what they saw:

wwpn_status=2,logged_in=unknown,wwpn_status_reason=invalid error number

Lacking time in their change window for additional testing, they ended up restoring from a mksysb and returning to 3.1.1.25. Everything worked again. Now the plan is to stay there until the system is retired. Naturally this setup isn’t supported by IBM, but the customer feels it’s their best option. Incidentally, the customer is running another 8202-E4C box at VIOS at level 3.1.2.21 with no issues.

So there’s hope, but also a strong dose of reality. Yes, some of these older machines can be updated beyond the stated supported levels, but you’ll need to do some testing (and, obviously, capture a mksysb, run viosbr, etc., even before that. As I recently said on Twitter, nobody cares about backups, but they do care about restores). Realistically though, it appears we’ve about reached the limits of extending VIOS on POWER7 machines.

Still, I’d love to hear more about this topic, particularly if you’ve been able to get even later VIOS versions to work on POWER7 hardware.

Simplified invscout Script

I greatly appreciate reader feedback, because typically I learn something. Here’s another recent reader interaction about a script that wasn’t working.

You may have seen my TechChannel video about invscout. I had linked to a script that’s supposed to automate the download of the catalog.mic file, copy it to multiple LPARs, consolidate the output and send it to IBM. However, a reader told me the script did not work as written. Fortunately, it was a quick fix. I changed a couple URLs and did other small simplifications that the reader requested.

Obviously you’ll need a machine that can access the LPARs you select to run invscout, and of course you’ll need internet access to automate the catalog.mic file download from IBM and send the file for processing. Depending on restrictions in your environment, it may take more manual steps to get this to work, but the general idea is that you start with a device—a NIM server or something similar—that has ssh keys set up to log into LPARs.

Be sure to edit the /usr/local/etc/servers file (or change the location to something you would prefer) and verify the TEMP directory you plan to use for this. You’ll also need scp, curl and wget installed.

The original script is capable of self-cleanup, so you may want to automate that by adding those sections in as well. The original also emails the output. I added a couple of comments to highlight where the IBM URLs had changed. You may prefer running the original script with my URL changes, but I also wanted to provide another, shorter version. Given the frequency with which useful tech stuff disappears from the internet, it never hurts to double up on these things.

#!/bin/ksh
# script: generate_survey.ksh
# purpose: To generate a microcode survey html file
# where is my list of servers located?
SERVERS=/usr/local/etc/servers
# what temporary folder will I use?
TEMP=/tmp/mup
# what is the invscout folder
INV=/var/adm/invscout
# what is the catalog.mic file location for invscout?
MIC=${INV}/microcode/catalog.mic
# user check
USER=`whoami`
if [ "$USER" != "root" ];
then
    echo "Only root can run this script."
    exit 1;
fi
 
# create a temporary directory
mkdir $TEMP 2>/dev/null
cd $TEMP
 
# this url had changed from the original script
wget ftp://ftp.software.ibm.com/software/server/firmware/catalog.mic
 
# move the catalog.mic file to this servers invscout directory
mv $TEMP/catalog.mic $MIC
 
# distribute this file to all other hosts
for server in `cat $SERVERS` ; do
   echo "${server}"
   scp -p $MIC $server:$MIC
done
 
# run invscout on all these hosts
# this will create a hostname.mup file
for server in `cat $SERVERS` ; do
   echo "${server}"
   ssh $server invscout
done
 
# collect the hostname.mup files
for server in `cat $SERVERS` ; do
   echo "${server}"
   scp -p $server:$INV/*.mup $TEMP
done
 
# concatenate all hostname.mup files to one file
cat ${TEMP}/*mup > ${TEMP}/muppet.$
 
# Sometimes, the IBM website will respond with an
# "Expectation Failed" error message. Loop the curl command until
# we get valid output.
 
stop="false"
 
while [ $stop = "false" ] ; do
# The following url had changed from the original script. I also had to make sure that it was all on the #same line, it is all one command.
curl -H Expect: -F mdsData=@${TEMP}/muppet.$ -F sendfile="Upload file" http://www14.software.ibm.com/support/customercare/mds/mds> ${TEMP}/survey.html
 
#
# Test if we see Expectation Failed in the output
 
#
 
unset mytest
mytest=`grep "Expectation Failed" ${TEMP}/survey.html`
 
if [ -z "${mytest}" ] ; then
        stop="true"
fi
sleep 10
done

At this point I moved the survey.html file onto a machine with a browser, and it gave me a very useful report. Chris Gibson has some great examples of the reports and information you can expect to see. Scroll down for specifics about adapter firmware updates. The red type really pops, and once you’ve updated your firmware and rerun the report, seeing the red disappear can be very satisfying as well.

---