Innovation Through IBM Middleware and Transaction Management in Hybrid Environments
For this installment of the 'IBM Z Experience,' Joe Gulla explains how CICS, IMS, Db2 and IBM MQ have grown to support APIs, cloud-native development, automation and AI
The first thing anyone learns about IBM Z servers is that they form the foundation for commerce around the world, serving as the core computing platform for 43 of the world’s top 50 banks and eight of the top 10 payment companies.
But what exactly is it that makes that foundation so strong? Running through that critical infrastructure is IBM’s middleware and transaction management (TM) software, having served as the operational backbone of many of the world’s largest enterprises for more than five decades.
This software has remained important over so many years because it continues to support applications that are key systems of record. Concurrent with this support, the IBM development teams have embraced recent industry developments like RESTful APIs and JSON data formats. As the cliché goes—it can walk and chew gum at the same time.
Growth and change within this software is ongoing and significant, so if anyone thinks that IBM middleware and TM software is stuck in the past, they have not been paying attention. The software does much more that it did 10 or 20 years ago. IBM innovates, then customers utilize the resulting work, and improved outcomes follow.
This article was written to discuss these innovations that lead to ongoing IT transformation by enterprises through use of these products.
What Middleware and TM Products?
I want to reintroduce you to CICS, IMS, Db2 and MQ. I wrote about them extensively on TechChannel last year.
IBM Customer Information Control System
IBM Customer Information Control System (CICS) is a transaction processing and application-serving platform that enables secure, high-volume online business transactions on IBM mainframe systems. The software is branded today as CICS Transaction Server. Banking is the sector most frequently associated with its use because of its requirements for real-time account updates, ATM processing, card transactions, payments and online banking. IBM CICS was first formally released on July 8, 1969.
I fell for CICS in 1978. At that time, CICS made it possible for us to take IT from the batch job era to the new age of real-time processing that most companies were anxious to embrace because of its potential impact on their business. It had the potential to enliven every function from manufacturing to sales. Its potential is still being realized today as businesses seek to use real-time data, in association with new business opportunities, leveraging CICS applications of record.
IBM Information Management System
IBM Information Management System (IMS) is a high-performance hierarchical database (IMS DB) and transaction management system (IMS TM) that supports large-scale online transaction processing on IBM mainframes. Today, IMS powers 72% of the banks on the Fortune Global 500 list. IMS is also widely used in insurance, retail, travel and communications. IBM IMS was announced in 1968, in time to support the Apollo Program, and was given its current name in 1969.
I worked with IMS starting in 1984. The systems that I worked on handled the bill-of-material for military aircraft and ships. These systems were extraordinarily complex with huge volumes of component data. We felt the need to be painstaking in our design and development because sailors and pilots might be compromised if the application failed, as we were responsible for equipment maintenance and related functions.
IBM Db2
IBM Db2 is IBM’s flagship relational database management system, providing secure, scalable data management for mission-critical applications on IBM Z, Linux, UNIX and Windows platforms. Db2 is a recognized database manager leveraging a relational model for data organization and access.
Db2 is a cornerstone of enterprise data management in banking, insurance, government, healthcare, telecommunications and other data-intensive industries. Db2 was offered starting in 1983 when DB2 Version 1, Release 1 was made available. More than a decade of IBM relational database research and proof-of-concept development, led by the pioneering System R project, preceded the first DB2 release.
IBM Message Queueing
IBM MQ is IBM’s enterprise message-oriented middleware platform that enables applications, services, APIs and systems to exchange data reliably and asynchronously across mainframe, distributed, cloud and hybrid environments. IBM MQ is used extensively in banking and financial services, where the product serves as a trusted messaging backbone for payment processing, funds transfers, trading systems and core banking integration.
IBM MQ is also widely used in insurance, retail, manufacturing, healthcare, government, telecommunications and travel industries. It provides secure, reliable, once-only delivery of business-critical messages across hybrid cloud and enterprise environments. Organizations use IBM MQ to exchange information reliably between applications and have been doing so since the first release, called MQSeries, was launched in 1993.
Then and Now
While these technologies are often associated with traditional mainframe computing, their modern role extends far beyond the data center. Today they are active participants in hybrid cloud ecosystems that connect on-premises systems, cloud-native applications, mobile clients, APIs, containers and AI platforms. The evolution of these technologies represents a remarkable story of innovation, realization and transformation. The sections that follow are a deeper dive into the four major products.
A Deeper Dive into CICS Product Function
A useful way to explain IBM CICS is to start with the long-established capabilities that built the platform and then move on to the more recent enhancements that support APIs, cloud integration, observability, DevOps and modern application development.
Legacy CICS Features
The features discussed in this section are foundational capabilities that many organizations still use today.
CICS command-level programming was introduced as a simpler alternative to macro-level programming. Command-level programming uses EXEC CICS commands embedded in COBOL, PL/I and other languages to access CICS services. It remains one of the most recognizable CICS programming models. 3270 green-screen transactions is where traditional CICS applications interact with users through IBM 3270 terminals. Users enter data on screen forms and submit transactions for processing. Personal computers are used today to emulate 3270 devices.
Basic mapping support (BMS) is important because it separates screen definitions from application logic. Developers define maps and screens independently from the transaction program, simplifying maintenance.
Temporary storage queues provide a mechanism for storing and sharing data between transactions, users and programs. They have long been used for pseudo-conversational processing and work staging.
Pseudo-conversational processing is very useful because instead of holding a terminal session open continuously, CICS ends a transaction after user input and restarts it when the next request arrives. This design convention greatly improved system scalability and became a hallmark of CICS architecture.
Transient data queues make it possible for applications to write data sequentially for later processing, reporting or routing to external systems. They remain a classic batch-integration mechanism.
Significant Modern CICS Enhancements
Recent releases of CICS Transaction Server have focused heavily on modernization, APIs, cloud integration, observability, security and developer productivity.
OpenTelemetry Distributed Tracing was introduced in CICS 6.3, allowing CICS transactions to participate in end-to-end tracing across hybrid applications without requiring application changes. Modern Java runtimes including 17, 21 and 25 are supported, enabling organizations to develop and run contemporary Java workloads directly within CICS. Developers can deploy Spring Boot applications more easily in CICS environments, bridging traditional transaction processing and modern Java frameworks.
Support for Jakarta EE allows enterprise Java applications to leverage modern standards while accessing CICS services and data. MicroProfile 6.x support helps organizations build cloud-native Java applications with standardized APIs for configuration, health checking, metrics and security.
Recent CICS releases include support for modern Node.js runtimes, enabling JavaScript applications to integrate with CICS resources. CICS continues to enhance support for packaged application deployment models and container-oriented development practices.
CICS 6.3 introduced a simplified configuration model that allows an entire region configuration to be defined in a single YAML file, making automation and DevOps practices much easier. Developers can use VS Code, IBM Z Open Editor and Zowe extensions with syntax highlighting, EXEC CICS autocomplete and resource inspection capabilities.
IBM has expanded Ansible automation capabilities for CICS resource and region management, simplifying operational tasks and DevOps workflows. Recent releases include additional security validation, stronger protection of TCP/IP communications, enhanced RACF analysis and broader support for enterprise security practices. CICS now offers stronger integration with Red Hat OpenShift environments, helping enterprises connect IBM Z workloads to hybrid cloud architectures.
A summary of these important CICS features are shown in Figure 1.
A Deeper Dive into IMS Product Function
This section explores the foundational capabilities that many organizations still run today, highlighting the significant enhancements that have modernized IMS for cloud, APIs, automation, security and large-scale operations.
Legacy IMS Features
There are many capabilities that have been part of IMS for decades and are often found in long-running enterprise applications. Message processing programs (MPP) are online transaction programs that are driven by messages from IMS queues. They process transactions, access databases and return responses to users. MPPs remain a classic IMS application model.
DL/I Programming Interface is the original IMS database access method. Programs use commands such as GU, GN, ISRT, REPL and DLET to navigate hierarchical databases. This was the primary way IMS applications accessed data before SQL-based access became common.
Batch message processing programs combine batch-style processing with online IMS database access. They can process large workloads while still interacting with IMS transaction queues.
IMS Fast Path programs were designed for very high transaction rates and rapid response times. They provide streamlined access to Fast Path databases for demanding workloads.
Hierarchical database structure is the original IMS database model that organizes data into parent-child segments rather than relational tables. This hierarchical architecture remains one of IMS’s defining characteristics. Message Format Service separates terminal-screen definitions from application logic. It was widely used to support 3270 terminal applications and simplify screen management in traditional IMS environments.
Significant Modern IMS Enhancements
The enhancements discussed in this section have expanded IMS far beyond its original transaction-processing roots.
IMS Connect provides high-performance TCP/IP connectivity to IMS applications and transactions, allowing distributed systems, web applications and mobile applications to access IMS resources. Open Database Manager (ODBM) enables standardized access to IMS databases and supports modern interfaces and tools. IBM continues to enhance ODBM as part of recent IMS releases. The IMS Catalog stores trusted metadata and definitions for databases and application program views, reducing dependence on traditional ACB libraries and improving administration.
Managed application control blocks (ACBs) automate and simplify application definition management by storing metadata directly in the IMS catalog. Modern IMS supports SQL data definition language (DDL) statements for defining databases and program views, bringing database administration closer to contemporary database practices. IMS supports Java Message Processing (JMP) and Java Batch Processing (JBP) applications, enabling organizations to modernize development while continuing to leverage IMS assets.
Through technologies such as IMS Connect and z/OS Connect, IMS applications and transactions can be exposed as RESTful APIs, making IMS services accessible to cloud and mobile applications. Recent IMS enhancements added OpenTelemetry support, allowing IMS transactions and services to participate in modern observability and distributed tracing frameworks. IMS 15.5 includes additional security controls and compliance-related auditing capabilities for IMS Connect and Operations Manager environments.
Recent releases continue to improve dynamic database definition capabilities, reducing administrative effort and increasing operational flexibility. IBM now delivers many IMS enhancements through continuous-delivery PTFs rather than waiting for major releases, allowing organizations to adopt new capabilities more rapidly. Enhanced catalog APIs allow tools and automation solutions to access metadata more efficiently, supporting modernization, governance and automation efforts.
A summary of these important IMS features are shown in Figure 2.
A Deeper Dive Into Db2 Function
IBM Db2 has evolved from one of the world’s original enterprise relational database systems into a platform that now incorporates AI, advanced analytics, automation and deep integration with modern application architectures.
Legacy Db2 Features
There are long-established capabilities that formed the foundation of Db2.
Structured Query Language (SQL) is the primary language used to define, query and manipulate Db2 data. Db2 was one of the first major enterprise databases to embrace the relational model and SQL standards.Db2 stores data in rows and columns organized into relational tables, providing a flexible alternative to hierarchical and network database models. Indexes improve access performance by allowing Db2 to locate rows without scanning entire tables. They have been a core Db2 optimization feature for decades.
Db2 provides atomicity, consistency, isolation and durability (ACID), ensuring reliable transaction processing for banking, insurance, retail and government applications. Business logic can execute within the database engine itself as stored procedures, reducing network traffic and improving performance. Distributed Data Facility (DDF) enables remote applications to connect to Db2 using distributed protocols, making Db2 data available throughout the enterprise.
Significant Modern Db2 Enhancements
Recent releases of Db2 13 for z/OS have focused on AI, automation, performance, availability and application modernization. SQL Data Insights brings AI directly into Db2, enabling semantic search, similarity discovery, relationship analysis and anomaly detection without moving data outside the database. AI_SIMILARITY Function is a built-in SQL function identifies data values that are most like a specified value, helping uncover relationships that traditional SQL might miss. AI_SEMANTIC_CLUSTER Function is a function that discovers records that belong to a semantic grouping of values, expanding Db2’s analytical capabilities.
The AI_ANALOGY Function allows Db2 to identify relationships like “A is to B as C is to ?”, enabling advanced pattern analysis directly in SQL. Compared to earlier major-version migrations, Db2 13 simplifies version upgrades through a streamlined migration process called single-phase migration. New Db2 capabilities are delivered incrementally through function levels, allowing organizations to adopt enhancements on their own schedules.
Db2 13 enables online conversion from partition-by-growth (PBG) table spaces to partition-by-range (PBR) table spaces, reducing disruption and simplifying administration. Db2 now captures utility execution history information, helping administrators analyze utility activity and improve operational efficiency. With object-level history, Db2 can maintain historical information about database objects, providing better visibility into changes and administrative activities. Application-specific lock timeout controls can specify lock timeout behavior, providing greater control over concurrency and workload management.
Db2 13 leverages modern IBM Z features including z15 and z16 processor enhancements to improve performance and scalability. Recent Db2 releases include support for continuous compliance initiatives, enhanced auditing and stronger operational controls for regulated industries.
A summary of these important Db2 features are shown in Figure 3.
A Deeper Dive into MQ Function
IBM MQ is a great example of a product that has stayed true to its original mission of reliable asynchronous messaging while continuously evolving to support APIs, cloud, containers, observability and modern security.
Legacy IBM MQ Features
This section of the article discusses the foundational capabilities that helped establish MQ as enterprise messaging middleware. MQ guarantees Reliable Message Delivery so that messages are delivered once and only once when configured for persistent messaging and transactional processing. This “assured delivery” model remains one of MQ’s defining characteristics.
MQ has store-and-forward messaging capability where data can be transmitted across multiple queue managers and networks, allowing applications to communicate even when systems are temporarily unavailable. This enables highly resilient distributed processing. Applications use queue-based messaging rather than direct program-to-program connections. This decouples applications in time, location and platform.
MQ Persistent Messaging ensures that critical business messages can be written to disk so they survive application failures, queue manager restarts and system outages. MQ can transparently route messages between queue managers located on different servers and platforms, creating enterprise-wide messaging networks. This is called distributed queueing. Although introduced years after the original queueing model, native publish/subscribe became a core MQ capability that allows publishers to send information once and have it distributed automatically to multiple subscribers. This is known as publish/subscribe Messaging.
Significant Modern IBM MQ Enhancements
Recent MQ releases have focused on cloud-native deployment, enhanced security, operational visibility and developer productivity. Administrators can now manage queue managers and MQ objects through REST interfaces instead of relying solely on traditional MQSC commands. Applications can put and get messages using standard HTTP and JSON interfaces, simplifying integration with web and cloud applications using REST Messaging APIs. IBM MQ provides containerized deployments and operator-based management for cloud-native environments through Kubernetes and OpenShift Support.
The MQ Operator automates deployment, scaling, upgrades and lifecycle management of queue managers in container platforms. Modern releases include stronger TLS security controls, certificate management improvements and updated cipher support to protect message traffic. Recent releases support modern authentication mechanisms, including token-based authentication, that simplify identity management and strengthen security.
MQ now provides built-in high availability capabilities that reduce complexity and improve resilience without requiring external clustering products. MQ 9.4 introduces enhanced visibility into queue managers, applications and MQ networks to help operators identify issues more quickly. The web-based MQ Console now provides richer dashboards, application views, queue relationships and operational overviews.
IBM-supported source and sink connectors allow MQ messages and business events to flow directly into Apache Kafka environments, helping organizations build event-driven architectures. Uniform clusters improve workload balancing and simplify large-scale client application deployment across MQ environments. Managed File Transfer (MFT) enhancements provide better diagnostics, monitoring, performance analysis and troubleshooting for enterprise file movement workloads.
A summary of these important MQ features is shown in Figure 4.
Figure 4. Legacy features and others supporting new application approaches and initiatives.
Looking Ahead
The future of enterprise computing is unlikely to involve wholesale replacement of existing transaction processing environments. Such a disruption would be ill-advised. Instead, organizations will continue pursuing strategies that integrate recognized platforms with emerging technologies. IBM development is supporting these strategies though careful and important enhancements to its transaction processing software.
Next Article in the Series
In the next article in this series, I will explore automation and orchestration tools with a focus on Ansible, Zowe and IBM Cloud Pak for Automation.