VIO Server Update Tips and Tricks
Last month I presented on VIOS Top Tips to the Power Virtual Users Group. During and after that presentation I picked up some tips that I would like to share here. Many thanks to those who asked questions and provided suggestions during the session. Further reading added a few more tips as well. The tips below are for VIO servers that are not running in an SSP cluster. If you’re using an SSP cluster then please pay careful attention to the instructions in the readme file specific to SSPs.
1. Taking a Snap for a VIO Server
As padmin
snap
This creates /home/padmin/snap.pax.Z. You will need to rename this file to TS????.snap.pax.X where TS???? is the problem number for the case you open with IBM. You can then upload the snap to IBM.
For dual VIO, you should take a snap on both VIO servers so IBM gets the complete picture.
2. Preparing for Updates
I always do my updates by logging into the VIO server via the HMC. I ssh to the HMC then use vtmenu to get a list of servers and LPARs and select the VIO server I want to work on. That protects me if something happens to the network on the VIO during my updates. Additionally, although all of my updates live on an NFS server I copy them into a local filesystem and do the update from there. This protects me if there is a network or NFS issue during the update process.
I also always take a clone before I start. I have a second disk in the system that I use for this.
# lspv | grep root hdisk1 00c47b30dbc9df98 altinst_rootvg hdisk2 00c47b309395f3d4 rootvg active
To take the clone I do the following:
exportvg altinst_rootvg alt_disk_copy -V -B -d hdisk1
If I have to back out later then I set the bootlist to hdisk1 and reboot the VIO on the old version. If your rootvg is mirrored, then you can unmirror it and take a clone to the disk that was freed up.
3. updateios Command
Most of us are familiar with the updateios command. This is how we put on updates to a VIO server within a version (i.e. v3.0 to 3.1). This is typically done using the updateios -dev /path -install command. I put all my VIOS updates into a directory and run updateios against that.
updateios -dev /updates/vios31314 -install -accept
I also do my SSH, SSL and Java updates the same way:
updateios -dev /updates/javasshssl -install -accept
Until recently I didn’t use updateios for putting on efixes, but it turns out that you can and should install efixes that way.
On VIO 3.1.3.14 there is a patch for lscore which can be downloaded from:
ftp.software.ibm.com/aix/ifixes.security/lscore_fix.tar
Now I put the efix into its own directory (only copy the epkg.Z file for 3.1.3.14). To determine the correct file open the Advisory.asc file and search for the level you are running. In my case it is 3.1.3.14 and there is a line in the file that shows:
3.1.3.14 IJ36810s3a.211217.epkg.Z key_w_fix
So the file I need to copy into a directory is: IJ36810s3a.211217.epkg.Z.
Now point at that directory as follows:
cd /usr/local/software/lscore_fix # ls -l -rw-r----- 1 root staff 21670 Feb 04 16:01 IJ36810s3a.211217.epkg.Z exit to padmin updateios -commit updateios -install -accept -dev /usr/local/software/flrtfixes/lscore_fix-vio
On VIO 3.1.3.14 there are two efixes that are needed (as of 2/7/2022)—the one for lscore above and openssh_fix14. You can still use emgr to check these are installed as follows:
# emgr -P PACKAGE INSTALLER LABEL ======================================================== =========== ========== bos.rte.security installp IJ36810s3a openssh.base.client installp 41617s9a openssh.base.server installp 41617s9a # emgr -l ID STATE LABEL INSTALL TIME UPDATED BY ABSTRACT === ===== ========== ================= ========== ====================================== 1 S IJ36810s3a 02/04/22 16:36:11 IJ36810 Potential security issue 2 S 41617s9a 02/04/22 16:38:26 Ifix for CVE-2021-41617
Prior to any updates (efix or other) you should always run a commit to make sure everything is committed:
updateios -commit
For updates you should also remove any efixes. That way your update won't fail due to a locked fileset. Once the update is installed and committed you can run FLRTVC (Fix level recommendation tool vulnerability checker) to determine which efixes need to be installed.
To remove an efix you can also use updateios.
First run lssw to make sure you have the label correct.
At the very end I currently see:
ID STATE LABEL INSTALL TIME UPDATED BY ABSTRACT === ===== ========== ================= ========== 1 S IJ36810s3a 02/04/22 16:36:11 IJ36810 Potential security issue 2 S 41617s9a 02/04/22 16:38:26 Ifix for CVE-2021-41617
To remove the SSH fix I would use:
updateios -remove 41617s9a
If your update fails don't forget to run "updateios -cleanup" to make sure you clean up.
The log file for efixes will be in /var/adm/ras/emgr.log even though you are using updateios to install the fixes.
4. Java, SSH and SSL
These need to be updated regularly. Java patches can be found at Fix Central and SSH/SSL are found at the AIX Web Download Pack Programs site. You will need an IBM site login id to get them. As of Feb 7, 2022 the latest versions (32 bit and 64 bit) are:
SSH 8.1.102.2104
SSL 1.1.1.1200
Java7 7.0.0.700
Java8 8.0.0.700
There is also a patch for SSH (openssh_fix14) which can be downloaded from IBM at ftp.software.ibm.com/aix/ifixes.security/openssh_fix14.tar.
5. Diagnosing Update Failures
The /home/padmin/install.log file is critical to understanding what happened. If you plan to retry the updateios command then take a copy of this file first as it may get overwritten. In particular look in the file for "Installation Summary for FAILED." There are many reasons an update can fail including losing your network connection, a bad rootvg disk, insufficient memory, efix or ifix that has locked a fileset, and missing files or directories. You should never get rid of the default directories such as /etc/perf or /home/padmin/config as these (and the files in them) are often used in updates.
6. LWI Nonstop
If you see this running or messages about waiting for it to exit then you still have systems directory installed. If systems director is not being used (which in v3 it is not) then you can remove it.
7. Fixpack Versus Servicepack and Minipack
VIOS 3.1.3.14 is available as either a fixpack or a servicepack. The fixpack allows you to upgrade from v3.1.0.0 to v3.1.3.14. The servicepack (which is really a minipack) only allows the update from v3.1.3.10 to v3.1.3.14. I always download the full fixpack plus any prereqs so am sure I have everything I may need.
Hopefully you find these updates to the tips and tricks helpful. Additional information and links can be found at the IBM Support Portal.
References
- VIOS Snap
- Uploading VIOS Snap to IBM
- updateios
- updateios and efixes
- FLRTVC
- FLRTVC hiper.csv file
- IBM Support Portal
- IBM Power Virtual User Group
- UK PowerVM Virtual User Group
- AIX Web Download Pack