Upgrading to PowerVM v4.1.0.21
IBM Champion Jaqui Lynch highlights her experience upgrading PowerVM from v4.1.0.10 to v4.1.0.21
Recently I decided to upgrade my VIO servers from 4.1.0.10 and wanted to share that experience with you. If you are on v3.1, then you would use the viosupgrade tool to upgrade as I discussed in my article on PowerVM v4 experiences. The only difference is that you would use the 4.1.0.21 flash image that you download from ESS (Entitled Systems Support). In this case I downloaded the fixpack for 4.1.0.21 from Fix Central. I also downloaded the flash image from ESS so I could build the NIM resources for VIO server restores. As a note, 4.1.0.21 requires that your NIM server be at 7.3.2.2, which mine was. 4.1.0.21 is supported only on POWER8 servers and above.
The first step after downloading the fixpack is to read the readme which can be found here. After that, I built the NIM resources for the new version.
Building the NIM Resources
On my NIM server I keep the initial images in /software/powervm41, so I upload the iso image to there.
cd /software/powervm41
The image is called:
Virtual_IO_Server_Base_Install_4.1.0.21_Flash_062024_LCD8292401.iso
I then use loopmount to mount the image so I can copy the code for NIM to use
mkdir /software/powervm41/powervm41021-base
loopmount -i /software/powervm41/Virtual_IO_Server_Base_Install_4.1.0.21_Flash_062024_LCD8292401.iso -o "-V udfs -o ro" -m /cdrom
Here I copy the mksysb image so NIM can use it to build the SPOT
cp /cdrom/usr/sys/inst.images/mksysb_image /software/powervm41/mksysb_vio41021_2024
cp -R /cdrom/* /software/powervm41/powervm41021-base
umount /cdrom
cp /software/powervm41/mksysb_vio41021_2024 /nim/images
Now I build the resources:
- Create a mksysb nim resource called mksysb_vio41021_2024 from /nim/images/mksysb_vio41021_2024.
- Now create the SPOT from the mksysb:
nim -o define -t spot -a server=master -a source=mksysb_vio41021_2024 -a location=/nim/spot spotvios41021
- Then check the resources:
nim -o check spotvios41021
lsnim -l spotvios41021
Now that the NIM resources are created you can go ahead with the update.
Performing the Fixpack Update
The first step is to download the code. Since I am at 4.1.0.10 I use Fix Central to download the updates and I save them in /software/powervm41/powervm41021-updates.
You will also need to download the following from IBM:
Latest Java8 8.0.0.826 from Fix Central
Patches from https://public.dhe.ibm.com/aix/efixes/security/
openSSH_fix17.tar
curl_fix5.tar
invscout_fix26.tar
Python3, latest openssl and openSSH from IBM Web Download Page
Python3 3.9.19.0
SSH 9.2.112.2400
SSL 3.0.13.1000
You should also download the latest perl-5.34.1.6 and rpm 4.18.1.2003 from the IBM web download site.
Make sure you have the latest flrtvc (0810) and HMCScanner (0.11.54) downloaded:
When I downloaded the VIOS 4.1.0.21 updates from Fix Central, I selected include all prerequisites.
You should also check that your current firmware and HMC levels are supported by the levels you are going to. You can find this at the flrtlite site or by running FLRT. There are some prerequisites in the README for certain adapters so make sure your I/O adapter firmware code is up to date prior to upgrading.
Make sure any media images are unloaded:
$lsvopt
If it shows something loaded, then unload it as follows, replacing ?? with the correct number.
$ unloadopt -vtd vtopt??
Always start by running errpt to check for errors. You do not want to try to update a failing system or one that has errors. Additionally, if you are mirroring rootvg you will need to unmirror it or have two spare disks for the upgrade. I also check all my client LPARs before and after each VIO update to make sure they get all their paths back before I go on to the next update.
PowerVM 4.1.0.21 Process
Before I start, I always document everything on the VIO servers and take copies of important files such as:
/etc/ssh/sshd_config
/etc/ssh/ssh_config
/etc/inetd.conf
/etc/inittab
/etc/rc.tcpip
/etc/resolv.conf
/home/padmin/config/ntp.conf
/etc/hosts
/etc/motd
I find the following commands useful for documentation and I save the output to an external file (usually a .txt file on my PC).
As padmin
ioslevel
lsnports
lspv -size
lspv -free
lspv
lsrep
lsvopt
lsmap -all -npiv | grep vfc
lsmap -all -npiv | grep fcs
lsmap -all | grep vhost
As root (oem_setup_env)
ifconfig -a
lspv
lspv | grep root
bootinfo -b
bootlist -m normal -o
ifconfig -a
Make a note of any interfaces with IP addresses and document those plus the gateway and subnet masks for them.
oslevel -s
instfix -i | grep ML
lppchk -v
lppchk -vm3
lsdev -C | grep fcs
lscfg -vpl fcs* | grep Network
lscfg -vpl fcs* | grep fcs
lsdev -C | grep ent
lsdev -C | grep Shared
Note the ent number it returns and use it in the next command instead of ent6.
lsattr -El ent6 | grep ent
lsmcode -A
In my case the VIO was already at 4.1.0.10, so the steps I followed for the update to 4.1.0.21 were:
Download the patches to /soft/powervm41.
Download the security updates to /software/flrtfixes and untar them.
Combine the ssh, java and ssl updates into one directory – /software/javasshssl-v3-vio-jun122024.
Combine the ssh, invscout and curl patches into /software/flrtfixes/vioflrt.
Take a clone:
#lspv | grep root
Note which disk is altinst_rootvg and replace ?? below:
exportvg altinst_rootvg
#alt_disk_copy -V -B -d hdisk??
Also take a mksysb type backup.
Get an HMCScanner report.
Run “#emgr -P” to look for any efixes.
Remove the efixes using emgr.
#emgr -r -L patch????
I.e., “#emgr -r -L 38408m9c”
Use updateios to do updates:
$updateios -commit
$updateios -accept -install -dev /software/javasshssl-v3-vio-jun122024
4 to go on
$updateios -accept -install -dev /software/powervm41/ powervm41021-updates
2 to go on
Then 123 to go on
$updateios -commit
Now, apply the patches. You can do this one of two ways:
- $updateios -accept -install -dev /software/ flrtfixes/vioflrt
- Use emgr is any of the above patches fail.
lslpp -L invscout.rte
invscout.rte 2.2.0.26 C F Inventory Scout Runtime
cd /software/flrtfixes/invscout_fix6
emgr -p -e is22026s1a.240514.epkg.Z
emgr -e is22026s1a.240514.epkg.Z
lslpp -L perl.rte
If not at 5.34.1.6:
cd /software/flrtfixes/perl-534
installp -apYd . perl.rte
installp -aXYd . perl.rte
lslpp -l | grep -i python
python3.9.base 3.9.18.1 COMMITTED Python 3.9 64-bit binary
cd /software/flrtfixes/python-3.9.19.0
installp -apYd . python*
installp -aXYd . python*
lslpp -l | grep -i python
python3.9.base 3.9.19.0 COMMITTED Python 3.9 64-bit binary
cd /software/flrtfixes/curl_fix5
emgr -p -e 853sa.240503.epkg.Z
emgr -e 853sa.240503.epkg.Z
cd /software/flrtvc
./flrtvc-0810-nodl.ksh
No problems should be found.
Now, run emgr -P and you should see:
PACKAGE INSTALLER LABEL
======================================================== =========== ==========
invscout.rte installp is22026s1a
openssl.base installp 301002sa
oss.lib.libcurl installp 853sa
openssh.base.client installp 92112ma
openssh.base.server installp
#updtvpkg
You should run updtvpkg any time you update the operating system, or SSL or rpm.
Run Your Checks Then Reboot
oslevel -s
7300-02-02-2420
instfix -i | grep ML
All filesets for 7.3.0.0_AIX_ML were found.
All filesets for 7300-00_AIX_ML were found.
All filesets for 7300-01_AIX_ML were found.
All filesets for 7300-02_AIX_ML were found.
oslevel -s -l 7300-02-02-2420
lppchk -v
lppchk -vm3
instfix -icqk 7300-00_AIX_ML | grep :-:
instfix -icqk 7300-01_AIX_ML | grep :-:
instfix -icqk 7300-02_AIX_ML | grep :-:
lslpp -l | grep ssh
lslpp -l | grep ssl
lslpp -l | grep Java
These should show:
SSH 9.2.112.2400
Java 8.0.0.826
SSL 3.0.13.1000
The SSH update may not update all the language sets, so I remove those:
As padmin:
updateios -remove openssh.msg.CA_ES
updateios -remove openssh.msg.CS_CZ
updateios -remove openssh.msg.DE_DE
updateios -remove openssh.msg.ES_ES
updateios -remove openssh.msg.FR_FR
updateios -remove openssh.msg.HU_HU
updateios -remove openssh.msg.IT_IT
updateios -remove openssh.msg.JA_JP
updateios -remove openssh.msg.Ja_JP
updateios -remove openssh.msg.KO_KR
updateios -remove openssh.msg.PL_PL
updateios -remove openssh.msg.PT_BR
updateios -remove openssh.msg.RU_RU
updateios -remove openssh.msg.SK_SK
updateios -remove openssh.msg.ZH_CN
updateios -remove openssh.msg.ZH_TW
updateios -remove openssh.msg.Zh_CN
updateios -remove openssh.msg.Zh_TW
updateios -remove openssh.msg.ca_ES
updateios -remove openssh.msg.cs_CZ
updateios -remove openssh.msg.de_DE
updateios -remove openssh.msg.es_ES
updateios -remove openssh.msg.fr_FR
updateios -remove openssh.msg.hu_HU
updateios -remove openssh.msg.it_IT
updateios -remove openssh.msg.ja_JP
updateios -remove openssh.msg.ko_KR
updateios -remove openssh.msg.pl_PL
updateios -remove openssh.msg.pt_BR
updateios -remove openssh.msg.ru_RU
updateios -remove openssh.msg.sk_SK
updateios -remove openssh.msg.zh_CN
updateios -remove openssh.msg.zh_TW
Before rebooting run bosboot to rewrite the boot image and use bootlist to rewrite the bootlist – assuming hdisk3 is rootvg then:
#bosboot -a -d hdisk3
#bootlist -m normal -o
#bootlist -m normal hdisk3
#bootlist -m normal -o
Once the system comes back up, I run all my checks including errpt. I also log in to the client LPARs and check that all their paths have come back. Only when I am happy with all of that do I go work on the primary VIO server and go through the same process again.
And you should always end by taking a mksysb backup and a fresh hmcscanner report. If you mirror your rootvg then wait at least a week before remirroring.
Gotchas
On these VIO servers I had gone through the issues associated with using viosupgrade to upgrade to 4.1.0.10 so I had put on the patches to avoid the padmin login issues, and I had updated /etc/security/user to ensure padmin was set as follows:
padmin:
admin = false
default_roles = PAdmin,CacheAdm
core_path = on
core_pathname = /home/ios/logs
maxage = 0
maxexpired = -1
histsize = 0
histexpire = 0
I have a backup padmin account of jlynch so I had run as padmin:
chuser -attr maxage=0 padmin
chuser -attr maxexpired=-1 padmin
chuser -attr histsize=0 padmin
chuser -attr histexpire=0 padmin
chuser -attr maxage=0 jlynch
chuser -attr maxexpired=-1 jlynch
chuser -attr histsize=0 jlynch
chuser -attr histexpire=0 jlynch
Much to my surprise, after I had put on the patches and before the reboot I could still login to jlynch but for padmin I got an error that my password had expired, and it forced a password change. So, I changed the password then changed it back and I could then login fine. The first time I found this out after the reboot, so from then on, I tested the login before rebooting. It was very strange that they expired the padmin account but not the jlynch one.
This article shows you how to do the update from VIO 4.1.0.10 to 4.1.0.21 manually. A lot of this can be done using NIM if you prefer that. I am a big fan of fully patching (efixes, etc.) so that my VIO servers are as secure as possible. Happy patching!
References
IBM Web Download Page: Download OpenSSH, openssl and the Python3 patch from here
ESS: Download the PowerVM ISO image from here
Fix Central: Download Java8 8.0.0.826 from here (or higher as needs be)
FLRTLITE and the FLRT Data Tables