Making Sense of the Latest IBM i Open Source Offerings
A lot has changed in the IBM i world with open-source including a new delivery vehicle and associated tooling. This article might help you understand why and the benefits to be gained.
A lot has changed recently in the IBM i world as far as open-source software is concerned—not the least of which is a brand-new delivery vehicle and associated tooling. As a result, we decided the time was ripe for an article that might help you to understand why this change is taking place and the benefits to be gained from it. Along the way we hope to clarify a few things for those of you who, like us, didn’t grow up in the predominantly Linux-oriented open-source space. If nothing else, we’ll at least help you to understand some of the new acronyms.
The Background
Before we talk about the latest changes though, a little history might help you to understand how we got to where we are today. Although often forgotten, IBM i’s first real foray into the open-source arena came with the introduction of the Apache web server (to replace the original IBM HTTP Server) back in the early 2001 time frame (V4R5).
Things were pretty quiet after that until early 2006 when IBM got together with Zend and introduced a port of PHP to the platform. PHP proved very popular and that popularity served to increase demands on IBM to open the door to more open-source offerings. The challenge that IBM faced became, “How do we achieve this?” With PHP there had been an obvious partner (Zend) to work with IBM to port the software. With other languages such as Python, Perl and Ruby there was no such obvious partner. More importantly perhaps, the compilers and other tools typically used to build and maintain such software did not exist on IBM i. That said, ports of Ruby and Python2 were made by independent groups with some assistance from IBM and had some limited success—but it was obvious that a more comprehensive effort was needed if open source was ever to live up to its potential on IBM i.
Enter the 5733-OPS product. First introduced back in early 2015 (see “Open Source for IBM i – 5733OPS” by Dawn May), this announcement generated a lot of excitement if for no other reason than it made available the much requested node.js. It was also clearly a statement of intent from IBM that open source was important to the platform and that a major effort was underway to open the door to wider adoption of open-source technologies. However, IBM’s decision to deliver open source as a Licensed Program Product (LPP) turned out to be one of those “It seemed like a good idea at the time” type of things.
Why? Well the good part was that it delivered the software in a familiar form and existing PTF mechanisms could be used to apply fixes and updates. But using an LPP approach also created its own set of problems. For example, in order to accommodate future additions to the product set, IBM had to include a number of what were initially “empty” product options so that, as the new products became available, they could be “filled” by the PTF mechanism. This was needed because PTFs could change the name of an option and deliver its content, but not create a new option.
The result was confusing to put it mildly. For example, if you used GO LICPGM to see what options were installed/available on your system you would often only see the names of the installed components. You had to know to apply a particular PTF in order to add any newly released capabilities because it was only after the PTF itself was applied that you could see the names of the new options. Catch-22! In addition, the limited number of options meant that IBM was forced to “group” some tools together to deliver them.
Beyond the confusion issue, packaging open-source offerings in this form took a lot of effort on IBM’s part. That in turn meant that delivery of new components and fixes for existing components took more time. The bigger problem, though, was that the PTF mechanism was completely unfamiliar to open source users from other platforms. Faced with install and usage issues they could not rely on any of their normal tooling to help them resolve issues. That was troubling since one of the prime motivations for bringing OS packages to IBM i was to help introduce IBM i to young developers not yet familiar with the platform. This added level of complexity was a problem and there was no obvious way around it within the LPP/PTF infrastructure.
Yum to the Rescue!
Your first reaction may well be “What on earth is yum?” We could tell you that it stands for Yellowdog Updater Modified but that really won’t help you a lot. The only real hint there is that the word updater indicates that whatever else it does it updates things. So let’s go with a simple English description. Yum is a package manager. By “package” in this context we mean a collection of software. So a package manager is a piece of software designed to allow you to install, delete, and manage software. It provides the install/uninstall capabilities of our familiar LICPGM menu plus the ability to apply/rollback fix packs just as our traditional PTF mechanism does. But yum goes one step further because it also manages dependencies.
To see what we mean, think for a moment about installing a regular LPP. If it requires other LPs be present it will note that in the documentation, but apart from that, the best it can do is to check for the required LPPs and stop the install and inform you if they are missing. A package manager such as yum takes it to the next level and will not only identify any required packages but will, if possible, retrieve and install them for you. Similarly, when you apply a PTF it may be dependent on other PTFs, but the system will not resolve those for you. You have to read the PTF cover letter and make sure that all needed components have been updated to the correct level prior to applying the PTF. When yum updates a package it checks for any additional dependences and ensures that that they are applied for you as part of the update process. So you can think of yum as similar in function to our old familiar LICPGM and PTF mechanisms, but on steroids.
In the open source world, yum has become the de facto package manager for rpm software packages. Yes, we know we just introduced another acronym . RPM originally stood for Red Hat Package Manager. These days, it’s more formally known as RPM Package Manager, which (like PHP) ends up being a recursive acronym. RPM Package Manager set a standard for how to describe the install process, dependencies, etc. in a file format which became known, not surprisingly, as rpm. Yum goes further than the original Red Hat implementation by not only noting dependencies but by retrieving and installing them when needed. As such, it was an excellent choice on IBM’s part to be the vehicle to replace 5733-OPS as the mechanism for open source delivery. Not only is it an excellent choice, but it is widely known and used in the general open source community.
So, yum can install and update products and uses the information that is provided in rpm files to achieve this. But exactly where does it get the information and the package content? RPM packages are stored in a repository and IBM has created one containing those open source packages that have been specifically compiled for and are known to run on IBM i. Most of us will never have to know where this repository is located as the updated Access Client Services (ACS) tooling, which we will discuss in a few minutes, has the repository information built into it. However, for those who like playing around at the bits and bytes level, and don’t wish to use ACS, the repository information is available to you so that you can use tooling of your choice. This page includes the manual steps required to set up and use yum without ACS.
Using ACS to Install Open-Source Packages
OK. So now that we know what yum is all about and why it’s so useful, we need to look at how we use it. This is where the latest enhancements to ACS come in. We’re going to keep this introduction very simple because IBM has an excellent web page that takes you through the whole process step by step. You’ll find it under the snappy title of “Getting started with Open Source Package Management in IBM i ACS.” If your version of ACS is 1.1.8.0 or higher, then you have this tooling. If not, then you need to update ACS to the latest level. We should note here that officially only releases 7.2 and 7.3 of IBM i are supported, but IBM has stated publicly that in most cases the open-source packages will also work on 7.1 if you’re still stuck there.
In brief, you access the new tooling via the “Open Source Package Management” menu item on the tools menu of the main ACS window as shown here.
One you have selected that, a separate window will be opened and you will be prompted to select the system you wish to connect to and provide your user credentials. We haven’t found any formal statement to this effect but suspect that it is a really good idea to use a user profile that has SECOFR capabilities since you will be installing software.
Since the connection uses SSH, you may be prompted to confirm the authenticity of the system you are connecting to. Also, if the system you’re connecting to isn’t currently configured for SSH communication, you’re advised to correct that situation before proceeding. The previously mentioned web page provides a link to setting up for SSH if needed.
If this is the first time the new tooling has been used on your system, a dialog will appear asking if you want to set up the open-source environment. In other words, you’re being asked if you want to install yum and associated components. If you press the “Yes” button, the required components will be downloaded and installed.
Next, you’ll be presented with a screen with three tabs. The first shows the open-source packages (if any) already installed on your system. The second shows any updates available for those packages and the third lists all the available packages that are not currently installed.
To install a package, you simply select it from the “Available” list and click the Install button. You can also request information about the selected package, but don’t expect it to be that helpful in all cases. We still have a little way to go in the “user friendly” aspects of this new tooling.
The install process will query the registry and identify the location of the required files, and any dependent components, and prompt you to confirm that you want to go ahead with the install. Once you answer “Y” to the confirmation request, the files are downloaded, checked for validity and the install process then takes place.
Updating components follows the same pattern. When you select that tab, the registry is checked to see if any updated versions are available. If there are any they are listed, you can then select and install the ones you need.
That’s all there is to it. As we noted earlier, IBM’s web page provides helpful images and more to guide you through the process.
Another ACS feature that you may find useful is an SSH terminal. Think DOS window or the QP2TERM PASE command line on a 5250 screen. You’ll find the link for this in the “General” section on the left-hand side of the main ACS window. For years we’ve been using the 5250 for all of our PASE operations. We don’t know why we never got around to using an SSH terminal instead but we’ll be using it far more often in the future. It’s certainly more convenient than having to start a 5250 session just to do work in PASE.
Most open-source aficionados use an SSH terminal for their work so it is useful to have it as part of the ACS tooling. One small problem we encountered was that the SSH terminal itself doesn’t support the usual cursor key controls to retrieve and modify (a la F9) entered commands. Any attempt to use these keys resulted in weird character sequences appearing. Luckily we encountered the instructions on this page, which identified the problem and pointed out that by changing our defaults to use a bash shell we could reclaim the use of the cursor controls and retrieve and edit previous commands.
Yes, we know that we just introduced yet another acronym: “bash”. We don’t have the time to go into what shells (and in particular bash) are all about. Let’s just say that a “shell” is the command line interpreter, and that bash is one of three available in PASE. The reason that this works is that it is actually the command interpreter that is responsible for the actions of the cursor keys. If there’s enough interest we’ll go into more detail on shells in a future article or blog post. For now, just follow the instructions given and all will be well.
Wrapping Up
As you’ve seen, every time we attempt to explain one thing, another has a nasty habit of popping up. For those well-versed in the world of Linux and open source, all of this is “situation normal” and we hope we have not bored (or annoyed) you too much. For those of you whose background is in the IBM i world, this is all new and often confusing. We hope that we’ve lifted a little of the fog and cleared a few things up by explaining where we are today and how we got here.
In future articles, we’ll try to focus on specific open-source components and why you might want to use them. We should also note, for those of you already using the 5733-OPS product, it will continue for the time being, but don’t expect any new products to be added to it. In time, it will undoubtedly be withdrawn and at that time we assume that IBM will provide instructions on how to migrate to the new versions.