Manage IBM i Jobs With Service Commander
IBM’s Jesse Gorzinski outlines how his open-source tool, Service Commander for IBM i, can help manage jobs as IBM i runs more technology than ever before
IBM i can run more technology than ever before. As such, the types of jobs that can be run are quite diverse—ILE programs, Db2 tasks and PASE programs can happily coexist. This synergy is indeed powerful, but it does come with a cost. All these jobs can be difficult to manage!
Consider the simple notion of starting a job, for instance. A variety of different practices are used. For instance:
Back in 2020, I experienced this complexity firsthand. At the time, I was managing an IBM i demo system and had installed a bountiful set of open-source packages, including Apache Kafka (and a web-based visualizer), Apache Zookeeper, ActiveMQ, GitBucket, MariaDB, as well as several Python and Node.js programs. I found it tedious to manage all these jobs. I kept my own notes on where each thing was installed, how to start it and how to check on it. Eventually, though, I realized that my text file was unwieldy and inefficient, so I created Service Commander for IBM i!
The following basic example is a Node.js service that runs on port 8444. Here, I opted to give it a starting directory and a descriptive name as well.
Figure 1: A simple service definition for a Node.js app
Several other configuration options are available, too. In the following example, a Node.js application is submitted to batch under the job name "IBMIDASH" and is part of the "web_apps" group (since Service Commander can start/stop groups of services).
Figure 2: A Node.js application submitted to batch
Another common need is to run a job as a specific user. An example configuration for the PostgreSQL server shows how easy this can be. In fact, you can specify any options for the SBMJOB command that you'd like, allowing you to easily exploit subsystems, memory pools and capping groups!
Figure 3: Submitting a job as another user
However, most of the time, you don't need to create or edit YAML files. Service Commander ships with an 'scedit' tool. This utility will ask you a few basic questions and create the configuration for you.
Figure 4: Using 'scinit' to create a service definition
Figure 5: Checking which services are running
In the previous screenshot, you'll see that one of them has a short name of 'mqtt_db2.’ I can start it by running 'sc start mqtt_db2':
Figure 6: Starting a service
This particular demo program uses ActiveMQ server, and 'sc' knows to start the requisite service first. Running 'sc check' again shows both are now running:
Figure 7: Checking which services are running (again)
Also worth noting: Service Commander comes with knowledge of numerous IBM-provided services "out of the box" and they can be managed by adding the '-a' flag:
Figure 8: Checking on IBM-provided services
Figure 9: Getting job information for port 110
Want to see that job's performance information? Just run 'sc perfinfo port:110'
Figure 10: Getting performance information for the job running on port 110
Want to kill it? 'sc stop port:110' does the trick!
Figure 11: Killing the job using port 110
Even further, the tool will implicitly look for services matching the provided port number or job. Imagine a scenario where an application is failing to connect to port 9092. Someone needs to start that service. Sometimes, though, a system administrator might not even know what's supposed to be running on port 9092! Plus, how is that service even started? Good news: If Service Commander knows about this service, the job is easy! Just run 'sc start port:9092'!
Figure 12: Starting a service knowing only the port number
Figure 13: Listing all open ports
Sometimes, you might only care about your own jobs, so the tool has a '--mine' option as well:
Figure 14: Listing open ports for the current user only
As you may have noticed, Service Commander is only able to tell you the details of services that have been configured. In the above screenshot, for instance, it knows what's running on port 9233 because I have configured a service named "grafana" on that port. However, it doesn't know what's running on port 12082. I could use the ‘port:12082’ ad hoc definition to investigate further.
Consider the simple notion of starting a job, for instance. A variety of different practices are used. For instance:
- Node.js jobs are started with the 'node' command in PASE.
- Java jobs are launched with the 'java' command in PASE, or the 'java' command in QSH, or with the RUNJVA CL command.
- Apache Tomcat servers have a 'startup.sh' shell script, and WildFly servers have a 'standalone.sh'
- IBM i host servers are started with the STRHOSTSVR CL command.
- IBM i TCP servers are started with the STRTCPSVR CL command.
- IBM HTTP Server jobs are started with the STRHTTPSVR CL command.
Back in 2020, I experienced this complexity firsthand. At the time, I was managing an IBM i demo system and had installed a bountiful set of open-source packages, including Apache Kafka (and a web-based visualizer), Apache Zookeeper, ActiveMQ, GitBucket, MariaDB, as well as several Python and Node.js programs. I found it tedious to manage all these jobs. I kept my own notes on where each thing was installed, how to start it and how to check on it. Eventually, though, I realized that my text file was unwieldy and inefficient, so I created Service Commander for IBM i!
Service Commander's Features
The primary value of Service Commander is to provide a unified interface for managing all your IBM i services. The project documentation lists the following features:- Start or stop services in a unified fashion
- Check the "liveliness" of your service by either port status or job name
- Customize the runtime environment variables of your job
- Submit jobs to batch easily, even with custom batch settings (use your own job description or submit as another user, for instance)
- Specify dependencies (for instance, if one application or service depends on another); Service Commander will start any dependencies as needed
- Define custom groups for your services and perform operations on those groups (by default, a group of "all" is defined)
- Query basic performance attributes of the services
- Provide some (limited) assistance in providing/managing log files
Configuring Services
Each "service" that Service Commander is defined in a YAML-formatted file. That service's short name (used with the 'sc' commands) is simply derived from the name of the YAML file. There are only two required fields: a start command and a "check alive" criteria. The "check alive" is either a job name or a port—and is what Service Commander uses to report status of the service.The following basic example is a Node.js service that runs on port 8444. Here, I opted to give it a starting directory and a descriptive name as well.
Figure 1: A simple service definition for a Node.js appSeveral other configuration options are available, too. In the following example, a Node.js application is submitted to batch under the job name "IBMIDASH" and is part of the "web_apps" group (since Service Commander can start/stop groups of services).
Figure 2: A Node.js application submitted to batchAnother common need is to run a job as a specific user. An example configuration for the PostgreSQL server shows how easy this can be. In fact, you can specify any options for the SBMJOB command that you'd like, allowing you to easily exploit subsystems, memory pools and capping groups!
Figure 3: Submitting a job as another userHowever, most of the time, you don't need to create or edit YAML files. Service Commander ships with an 'scedit' tool. This utility will ask you a few basic questions and create the configuration for you.
Figure 4: Using 'scinit' to create a service definition
Managing Services
Once services are configured, basic functions are available from the 'sc' command line. To check which ones are running, I can use 'sc check':Figure 5: Checking which services are runningIn the previous screenshot, you'll see that one of them has a short name of 'mqtt_db2.’ I can start it by running 'sc start mqtt_db2':
Figure 6: Starting a serviceThis particular demo program uses ActiveMQ server, and 'sc' knows to start the requisite service first. Running 'sc check' again shows both are now running:
Figure 7: Checking which services are running (again)Also worth noting: Service Commander comes with knowledge of numerous IBM-provided services "out of the box" and they can be managed by adding the '-a' flag:
Figure 8: Checking on IBM-provided services
Ad Hoc Services
Even without configuring specific services, Service Commander's operations can be used on an "ad hoc service." This allows for easy interrogation and management of jobs based on port or job name. The service name for an ad hoc service looks like one of the following:- port:nnnn
- job:jobname
- job:subsystem/jobname
Figure 9: Getting job information for port 110Want to see that job's performance information? Just run 'sc perfinfo port:110'
Figure 10: Getting performance information for the job running on port 110Want to kill it? 'sc stop port:110' does the trick!
Figure 11: Killing the job using port 110Even further, the tool will implicitly look for services matching the provided port number or job. Imagine a scenario where an application is failing to connect to port 9092. Someone needs to start that service. Sometimes, though, a system administrator might not even know what's supposed to be running on port 9092! Plus, how is that service even started? Good news: If Service Commander knows about this service, the job is easy! Just run 'sc start port:9092'!
Figure 12: Starting a service knowing only the port number
Checking Open Ports
System administrators or security specialists are often interested in what ports are "open" on the server. Thankfully, Service Commander comes with a hidden gem: the 'scopenports' utility. One quick command and you can see all the open ports and what's running:Figure 13: Listing all open portsSometimes, you might only care about your own jobs, so the tool has a '--mine' option as well:
Figure 14: Listing open ports for the current user onlyAs you may have noticed, Service Commander is only able to tell you the details of services that have been configured. In the above screenshot, for instance, it knows what's running on port 9233 because I have configured a service named "grafana" on that port. However, it doesn't know what's running on port 12082. I could use the ‘port:12082’ ad hoc definition to investigate further.
What's Next?
If today's post has convinced you that Service Commander could be useful to you, please stay tuned for my next post. There, I will discuss more advanced Service Commander topics, such as:- Leveraging Service Commander to automatically start jobs at IPL
- Using Service Commander's "cluster mode" to add resiliency and scalability to your application
- How Service Commander can help with application performance analysis
- Comparing Service Commander to the popular PM2 tool
