Power10 Planning and Requirements
Jaqui Lynch on the new POWER10 servers and the requirements to support them
In 2021, IBM brought out the enterprise E1080, the first of the Power10 systems. On July 12, 2022, IBM announced the scale-out and midrange servers—S1014, S1022, S1022s, S1024—there are also Linux-only versions of the S1024 and S1022. There is a great deal of material published on the new servers so this article will address how to plan for the servers. This is broken down into hardware, software and power prerequisites. For Power10 IBM has also introduced Update Access Keys (UAKs) for AIX and IBM i. This is in addition to the UAKs already being used for firmware. These are discussed in the Technical Overview Redbook for each server, but they will need to be refreshed every six months and they require up-to-date support contracts.
Hardware Prerequisites
Whenever I work on an install or upgrade, I still do a technical delivery assessment (TDA) preferably prior to order. This lets me go over critical hardware issues to avoid problems. Some of these issues include:
Network
- Are connections copper or fiber
- RJ45, 10Gb, 25Gb or other
- Are there enough switch ports
- Are there enough SFPs
- Who is providing cables
- IP addresses
- LACP aggregation
SAN
- Are there enough switch ports?
- Are there enough SFPs?
- Who is providing cables?
HMC
- Is it at a supported level?
- If new, we need 2 IP addresses on admin network
- We also need a private range for the HMC to server connections
Racks
- Are there sufficient U available?
- Is there sufficient power across multiple PDUs?
- Are new PDUs and circuits needed?
- IBM requires use of rack rails that come with the server
General
- Customer install of IBM install
- Some of these servers require a lift tool or multiple people to install due to weight
- Delivery information—any issues with truck size.
- Is lift gate needed?
Software Prerequisites
OS minimum levels vary depending on the server and whether the LPAR is running with or without a VIO server. If you are planning to upgrade to Power10 then this is a good time to start upgrading your OSes to ensure a smooth transition. All of the levels listed below should be checked on the IBM website in case something has changed since announce. I normally check at the Fix Level Recommendation Tool (FLRT) site and the FLRTLITE site to make sure I am installing supported levels and combinations.
S1014, S1022, S1024
The minimum VIO server level is v3.1.3.21.
AIX 7.1 minimum level is 7100-05-06-2016 and must run under a VIO server in POWER8 compatibility mode.
AIX 7.2 runs in POWER9 compatibility mode. Under a VIO server minimum levels are 7200-04-02-2016 or 7200-05-01-2038. Without a VIO server the minimums are 7200-04-06-2220 or 7200-05-04-2220. Service Pack 7200-04-06-2220 is planned to be available from September 16th, 2022.
AIX 7.3 can run in Power10 mode—with a VIO server the minimum is 7300-00-01-2148. Without a VIO server it is 7300-00-02-2220.
The minimum levels for IBM i are:
- IBM i 7.5
- IBM i 7.4 Technology Release 6 or later
- IBM i 7.3 Technology Release 12 or later
On the S1022s or S1022 a VIO server is required and partitions must be set to “Restricted I/O” mode. An LPAR can have a maximum of four cores. IBM i is not supported on the S1022s 4-core processor model.
Both Red Hat Enterprise Linux (RHEL) and Suse Linux Enterprise Server (SLES) are supported on these models. Minimum supported levels are:
- Red Hat Enterprise Linux 8.4 for Power LE, or later
- Red Hat Enterprise Linux for SAP with Red Hat Enterprise Linux 8.4 for Power LE, or later
- Red Hat Enterprise Linux 9.0 for Power LE, or later
- SUSE Linux Enterprise Server 15 Service Pack 3, or later
- SUSE Linux Enterprise Server for SAP with SUSE Linux Enterprise Server 15 Service Pack 3, or later
E1050
The minimum VIO server level is v3.1.3.21
AIX 7.1 minimum level is 7100-05-06-2016 and must run under a VIO server in POWER8 compatibility mode.
AIX 7.2 runs in POWER9 compatibility mode. Under a VIO server minimum levels are 7200-04-02-2016 or 7200-05-01-2038. Without a VIO server the minimums are 7200-05-04-2220. Service Pack 7200-04-06-2220 is planned to be available from September 16th, 2022.
AIX 7.3 can run in Power10 mode—with a VIO server the minimum is 7300-00-01-2148. Without a VIO server it is 7300-00-02-2220.
Both Red Hat Enterprise Linux (RHEL) and Suse Linux Enterprise Server (SLES) are supported on these models. Minimum supported levels are:
- Red Hat Enterprise Linux 8.4 for Power LE, or later
- Red Hat Enterprise Linux for SAP with Red Hat Enterprise Linux 8.4 for Power LE, or later
- Red Hat Enterprise Linux 9.0 for Power LE, or later
- SUSE Linux Enterprise Server 15 Service Pack 3, or later
- SUSE Linux Enterprise Server for SAP with SUSE Linux Enterprise Server 15 Service Pack 3, or later
E1080
The minimum VIO server level is v3.1.3.10
AIX 7.1 minimum level is 7100-05-05 and must run under a VIO server in POWER8 compatibility mode.
AIX 7.2 runs in POWER9 compatibility mode. The minimum level is 7200-04-01.
At the time of announce AIX 7.3 was in beta but AIX 7.3 is supported on the E1080 starting with 7300-00-01-2148.
The minimum levels for IBM i are:
- IBM i 7.4 Technology Release 5 or later
- IBM i 7.3 Technology Release 11 or later
Both Red Hat Enterprise Linux (RHEL) and Suse Linux Enterprise Server (SLES) are supported on these models. Minimum supported levels are:
- Red Hat Enterprise Linux 8.4 for Power LE, or later
- SUSE Linux Enterprise Server 15 Service Pack 3, or later
HMC
The minimum HMC version to support the S1014, S1022, S1022s, S1024 and E1050 is v10R1.1020. The minimum HMC version to support the E1080 is V10R1. V10 requires a 7063-cr1, 7063-cr2 or virtual HMC appliance. It is not supported on the 7042 HMCs.
Power Prerequisites
Planning for electrical power is always a challenge, especially because we have to plan for what could be used rather than what is actually being used. IBM uses a redundant power supply setup with only half the power supplies drawing power at any given time. However, you need to provide sufficient power for all the power supplies in case a PDU or circuit fails. Any one of each pair of power supplies could be drawing power at any given time.
I go into this in a great deal of detail in my February 2022 article on power supplies for Power Systems. The requirements for each server are published in the various IBM Redbooks, but I have listed them below in Figure 1 for the Power10 servers. I have also included plug type, BTU for the air conditioning and the weight.
PDUs and Line Cords
All the servers require single-phase 208v power, with the exception of the tower version of the S1014 which can run on 110 but it needs four power connections to do so.
The new Power10 servers draw a lot of electrical power, and you may find that the 30-amp PDUs you have been using will not be sufficient. Due to past issues with overloading, fires, etc., PDUs in the U.S. are derated to 80% of their maximum capacity. This means that a single-phase 208V 60-amp PDU is only allowed to have up to 48 amps (9.98kVA) of equipment connected to it, and a single-phase 208V 30-amp PDU is derated to 24 amps (4.99 kVA). A three-phase 30-amp PDU can use up to 8.65 kVA (see Figure 2).
I recommend that you should order new high-function IBM PDUs when buying new Power10 servers. These high-function PDUs provide more electrical power per PDU. In addition, they are intelligent PDUs that provide insight to power usage by receptacle and remote power on and off capability for easier support by individual receptacle.
For the E1050, IBM will only integrate with their new high-function PDUs. These PDUs are highly recommended for all Power10 servers.
Usable kVA per PDU Pair for Redundant Power
E1080 Power
Each E1080 node draws 4.6 kVA using 2 pair of power supplies (4 plugs). This means a single node E1080 will require 2 x 30-amp PDUs to support it.
E1050 Power
The E1050 requires a PDU that has C19/C20 connections. It has a maximum draw of 5.36 kVA which is higher than the allowed 4.99 kVA. To support this using 30-amp PDUs you would need four PDUs that support C19 connections with one power supply being connected to each PDU. A better option would be to move to 60-amp PDUs if possible.
S1014 and S1022 Power
The S1014 rack mount draws 1.105 kVA and is also C13 so up to four of these can be supported on a 30-amp pair of PDUs.
For the S1022 and S1022s these are all C13 connections and draw 2.31 kVA. A pair of 30-amp PDUs (4.99kVA max) can support two of these servers.
S1024 Power
The S1024 draws 2.835 kVA and is also C13 and has four power supplies. Only one S1024 can be supported on a pair of 30-amp PDUs.
Designing for Electrical Power
Much of the confusion comes from the way redundant power supplies work and the requirements this puts in place for additional PDUs. For example, the E1080 has four power supplies set up as two redundant pairs. This means that two cords should go to one PDU and two should go to another, so that the server can continue to run if a circuit or PDU fails. For the most part, two of the power cords may never draw power.
However, we don’t get to choose which ones will draw power. This means that redundant power supplies can hide the problem of accidentally exceeding the 4.99 kVA loading on a 30-amp PDU in normal operation, since we’ll never see one of the PDUs in the pair being used. However, if one PDU in a redundant pair happens to fail—for example, if a PDU cord is unplugged or an input circuit fails—all devices will be forced to the remaining PDU, which will likely become overloaded and fail. At that point, one of several things will happen: The circuit breaker will trip and power everything off, the equipment on that PDU may power off or you could have a fire if the circuit breakers don’t do their job. I’ve seen overloaded PDUs several times and the result is never good.
The first step is to determine what the amperage and derated amperage is for all your circuits and PDUs. Then you build a spreadsheet that tracks what is connected where and its potential amperage or kVA so that you do not exceed the limits on any of the PDUs. When doing these calculations, don’t forget to include everything in the rack—including storage, I/O drawers, switches, etc. Remember redundancy requirements—along with the regulations—meaning that you need to plan for what you could use, not just what you normally draw. A review of the PDU layouts and usage should be done any time a server, I/O drawer or any other device that uses power is added.
Planning Ahead
The best time to avoid problems with an install is to have a preinstall planning meeting before any new equipment is ordered. This ensures that all the prerequisites for software, hardware and electrical power are in place or can be ordered in time. When I see problems during an installation, 90% of the time it is due to issues around power, network connections (copper versus fiber), or switch ports for SAN or network. All of this can be avoided with proper planning before any orders are placed.
Taking the time to plan before making changes to your environment can help you to adhere to standards, avoid outages and reduce the risk of costly complications.
References
- Power10 Information
- E1080 Technical Overview and Introduction
- E1050 Technical Overview and Introduction
- S1014, S1022 and S1024 Technical Overview and Introduction
- IBM Performance Report
- Techchannel Article on Planning for Power
- FLRT
- FLRTLITE