Make a Useful Pilot Project Using This Framework for z/OS Hybrid Integration
In this installment of "z/OS and Friends," Dr. Joe Gulla provides a template for gleaning insights on z/OS integration
In my previous article on messaging and integration, I explained products from IBM that are up to the task of connecting different systems and applications. These products included IBM MQ and IBM App Connect Enterprise (ACE) for z/OS, IBM App Connect, z/OS Connect Enterprise Edition (EE), IBM API Connect and IBM DataPower Gateway. I did my best to explain them and how they work together using narrative and summary tables.
In this article, I want to dig deeper into the subject. One way to get greater insights into the task of z/OS integration with other systems is to layout a project framework. If you want to explore these products in the context of a project, here are the steps that can help you organize the work and get results from a pilot project.
Overall Approach
I studied to be a project manager, an old-school project manager, and have a certificate from George Washington University to prove it. We were taught that it is helpful to approach this kind of project in structured phases, a practical way of getting things done. It also gives you checkpoints along the way for management review and approval of the upcoming steps. Not every project will require checkpoints, but I learned they are helpful. In the beginning I was weary of them, but not any longer.
For several years, I worked in an environment where we had predefined project phases for all new or emerging business opportunities. At the completion of each phase, we presented our results, answered questions and then the chair asked for a vote. They asked if the company, in this case IBM, should continue the project and fund the next phase. I remember questions like, “Do you really need $500K for the next phase?” and comments like, “I don’t want to spend any more money trying to solve this old problem.”
Sometimes the discussions got heated, but these projects taught me something about leadership, flexibility and finding imaginative ways to get things done. Let’s move on to my project framework proposal.
A Project Framework for z/OS Hybrid Integration
Here are six phases that you should use for your integration project involving z/OS systems and applications and other target integration environments. As the cliché goes, this is not rocket science, but a project like this does require thoughtful investigation, implementation and communication.
Phase 1: Define Objectives and Scope
At least three areas of focus need to be explored in this phase. These include:
- Identify business goals
- Determine integration targets
- Assess existing z/OS assets
Identify Business Goals
The most important thing to remember about business goals is that they should focus on business outcomes and not be expressed in overly technical terms. Of course, technology maps to the business needs but don’t put the cart before the horse—lead with a business focus.
Below, in Table 1, are several areas of focus. Pick the ones that resonate with you and your project. You will likely recognize these goals, as almost every IT innovation created new opportunities to improve service.
Table 1. Candidate Project Business Goals
| Goal | Description of Potential Benefits |
| Increase business agility | Agility would be improved by enabling faster response to market changes by integrating legacy systems with modern cloud platforms. Additionally, support for rapid deployment of new services, for example mobile applications and customer portals, without rewriting core mainframe logic would improve agility. |
| Improve customer experience | Delivering real-time access to mainframe data (for example, account balances or transaction history) via APIs would improve the customer experience. Also helpful would be to enable seamless omnichannel experiences across web, mobile and support systems. |
| Reduce operational costs | Avoid costly rewrites of legacy applications by exposing them via APIs. While this would reduce operational costs, infrastructure costs could also be reduced by using cloud-native services for non-core workloads. |
| Accelerate digital transformation | Modernize the enterprise architecture by integrating z/OS with cloud and SaaS platforms and enable data-driven decision-making by connecting mainframe data to analytics platforms. These two steps would accelerate digital transformation. |
| Enhance security and compliance | Centralize API access and enforce security policies using tools like IBM API Connect and DataPower Gateway. Also, ensure secure, auditable access to sensitive mainframe data across hybrid environments to better support compliance. |
| Increase developer productivity | Empower developers with modern tools like App Connect Designer and ACE flows. This will increase developer productivity through faster integrations. Also, reduce dependency on specialized mainframe skills by abstracting access through APIs. |
| Enable innovation through reuse | Reuse existing mainframe logic—for example COBOL programs and DB2 queries through new digital services. Also, create composable business capabilities that can be orchestrated in cloud-native applications. |
Additional ideas and thought triggers can be found in this IBM Think article. The focus is strategy but the company examples can trigger additional goal candidates.
Determine Integration Targets
In a z/OS hybrid integration project, theintegration targets are the external systems and platforms that themainframe will connect with. These targets should align with your business goals, such as improving customer experience, enabling digital services or reducing operational costs. Here are five specific areas to focus on.
1. Cloud Platforms
These are used to extend mainframe capabilities, host modern applications and enable scalability. Include a table that has a row for each platform and the purpose of the integration. For example:
| Platform | Purpose of Integration |
| IBM Cloud | Native support for IBM Z, MQ, App Connect and API Connect |
2. SaaS Applications
These are cloud-based business tools that benefit from real-time access to mainframe data. Include a table that has a row for each SaaS App and how it will be used. For example:
| SaaS App | How Used |
| Salesforce | CRM integration with customer data from DB2 or CICS |
3. Mobile and Web Services
These are front-end applications that interact with mainframe services via APIs. Include a table that has a row for each target and a statement of its purpose. For example:
| Target | Purpose |
| Mobile apps | Real-time access to account info, transactions or customer services |
4. Analytics and AI Platforms
These platforms consume mainframe data for insights and automation. Include a table that has a row for each platform and indicates how it will be used. For example:
| Platform | How Used |
| IBM Watson | AI-driven insights from z/OS data |
5. DevOps and Automation Tools
These help streamline development and operations across hybrid environments. Include a table that has a row for each tool and a statement of its purpose. For example:
| Tool | Purpose |
| GitHub | CI/CD pipelines for integration flows |
Assess Existing z/OS Assets
Assessing existing z/OS assets is a critical early step in a hybrid integration project. The goal is to understand what you have, what can be reused or exposed and how those assets align with your business and integration goals. Here’s a structured approach to guide your assessment in order to utilize existing z/OS assets:
1. Inventory Core Business Applications
Identify key applications running on z/OS including CICS, IMS, batch jobs and DB2 databases. Document their business functions, usage patterns and dependencies.
2. Identify Integration-Relevant Interfaces
Look for existing interfaces that can be reused or exposed including MQ queues, VSAM files, DB2 stored procedures and Web services. Remember to document any existing API wrappers or middleware.
3. Evaluate Data Assets
Catalog critical data sources including DB2 tables, IMS databases and VSAM datasets. Assess data sensitivity, volume and update frequency. Determine which data needs to be exposed to cloud or SaaS apps.
4. Analyze Workload Characteristics
Understand transaction volumes, peak usage times and performance requirements. Identify workloads suitable for real-time API access, event-driven messaging and batch offloading to cloud.
5. Review Security and Compliance Constraints
Document current security models including RACF profiles, encryption standards and audit and logging mechanisms. Also, identify compliance requirements, for example HIPAA and PCI-DSS, that affect integration.
6. Assess Existing Integration Tools
Check for existing use of IBM MQ, z/OS Connect EE and ACE for z/OS orchestration. Evaluate how well these tools are configured and maintained.
7. Identify Technical Debt and Constraints
Note any legacy code that’s hard to maintain or integrate. Identify systems with limited documentation or support. Also, flag areas where modernization may be required before integration.
8. Engage Stakeholders
Interview business and technical owners of z/OS systems. Understand their pain points, priorities and expectations. Make sure to align asset assessment with business goals, for example faster onboarding and better customer service.
This is the end of the Phase 1 discussion. Remember the importance of making this section on objectives and scope directly related to the actual challenges being experienced, like lack of business agility or operational costs. You want the reader (or approving executive) to recognize the existing challenges that make this project a necessity.
Phase 2: Architecture Planning
In this phase, create a high-level architecture using the IBM tools listed below in Table 2. The component descriptions in Table 2 should be preceded by an architectural overview. This should consist of a diagram showing the major components and their interactions as well as the deployment environments, including elements like on-premises z/OS, cloud and DMZ domains.
Table 2. Component Descriptions
| Component | Role in Architecture |
| IBM MQ | Reliable messaging between z/OS and cloud systems |
| ACE for z/OS | Orchestrates and transforms data on z/OS |
| App Connect | Connects cloud-native apps and services |
| z/OS Connect EE | Exposes mainframe programs as REST APIs |
| API Connect | Manages APIs (security, analytics, lifecycle) |
| DataPower Gateway | Secures and mediates traffic between systems |
This is the end of guidance on the high-level suggestions for the architectural planning. Next is more planning regarding a design relating to integration.
Phase 3: Integration Design
Integration design is specific to the computing environment and the specific product set. That means you should design message flows in ACE for z/OS to handle orchestration and transformation. Explain how you will use z/OS Connect EE to wrap legacy programs as REST APIs. Explain how you will use App Connect to integrate cloud apps with exposed APIs or MQ endpoints. Also, define API policies in API Connect like rate limiting, authentication and others. Additionally, explain the planned configuration for DataPower Gateway for edge security and protocol mediation.
The above section will likely be a combination of narrative, figures and tables. What about implementation steps? That is the next phase.
Phase 4: Implementation Steps
The output of this step could be an implementation plan for a pilot project expressed as a digital project plan. It would include the following group of tasks:
Table 3. Implementation Tasks in a Template Plan
| Main Task | Description | Owner | Start Date | End Date | Dependencies |
| Set up MQ Channels | Configure MQ channels between z/OS and cloud MQ servers like IBM MQ on AWS/Azure. | MQ Admin | TBD | TBD | Network setup, firewall rules |
| Develop ACE Flows | Build message flows in IBM App Connect Enterprise for z/OS to process and route messages. | ACE Dev | TBD | TBD | MQ channels operational |
| Create z/OS Connect APIs | Define and expose CICS/IMS/DB2 services as REST APIs using z/OS Connect EE. | z/OS API Dev | TBD | TBD | Backend services identified |
| Deploy App Connect Flows | Deploy App Connect flows in cloud to link cloud apps with z/OS APIs or MQ. | Cloud Dev | TBD | TBD | APIs and MQ channels ready |
| Publish APIs in API Connect | Register APIs in API Connect, apply security and rate limiting policies. | API Manager | TBD | TBD | APIs tested and documented |
| Configure DataPower Gateway | Set up DataPower for secure access, transformation and protocol bridging. | Security Engineer | TBD | TBD | API Connect and z/OS Connect configured |
There might be shortcomings, including firewall/network issues, that can be mitigated through coordination with the network team. You will need to check out API performance by conducting load testing before publishing. Also, security compliance will need to be assured with a review with your information security team. That brings us to the next phase, testing and validation.
Phase 5: Testing and Validation
You should document a set of test cases to do the following:
Table 4. Focus of Test Cases
| Main Focus | Details |
| Test end-to-end flows | Cloud app to API to z/OS to MQ giving response |
| Validate the pilot application | Performance, security and error handling are the main focuses |
| Incorporate monitoring tools | OMEGAMON and API Connect analytics should be included in the testing process |
What about deployment and optimization steps? That is the next phase.
Phase 6: Deployment and Optimization
The next step is to deploy the pilot application in a system test or production environment. It will be appropriate (and necessary) to monitor usage and optimize flows. Do they work well? Are they timely? You should gather the results of the pilot and review with stakeholders and management. This will inform future activities.
The next activity is to plan for scaling, failover and future enhancements that will likely be required by post-pilot implementations. Remember—this project was just a pilot; what comes next will require additional thought and consideration.
Maybe There Are Too Many Phases?
The breakdown of work described in this article might contain too many phases for you or your organization’s culture. How about just three—is that a better match? It is reasonable to reorganize the work described into just three, as shown in the table below:
Table 5. Consolidation of Phases
| Original | Consolidation |
| Phase 1: Define Objectives and Scope | Phase 1: Project Concept |
| Phase 2: Architecture Planning | Phase 2: Planning and Design |
| Phase 3: Integration Design | |
| Phase 4: Implementation Steps | Phase 3: Implementation and Deployment |
| Phase 5: Testing and Validation | |
| Phase 6: Deployment and Optimization |
In the above consolidation, the new Phase 2 would include the original phases 2 and 3. The new Phase 3 would include the original phases 4, 5 and 6.
Need More Information?
There are many sources of information, both business and technical, to help you put together a project plan and pilot implementation project. Here are a few useful sources that I consulted in putting together this article.
IBM Z Integration Guide for Hybrid Cloud (IBM Redbooks) 4th Edition 2020
This guide provides a comprehensive framework for integrating IBM Z systems with hybrid cloud environments. It includes strategies for:
- Inventorying z/OS applications and subsystems (CICS, IMS, DB2, MQ)
- Evaluating integration readiness
- Mapping assets to integration patterns (API, messaging, event-driven)
- Aligning technical capabilities with business goals
It is an excellent source book for a hybrid integration project.
Mainframe Application Modernization Patterns for Hybrid Cloud (IBM Redbooks) June 2023
This publication outlines modernization patterns and how to apply them to IBM Z assets. It emphasizes:
- Application discovery and analysis
- Integration architecture planning
- Data access and transformation strategies
- Event-driven and API-based modernization
Next Article
In the next article, I will focus on WebSphere Application Server and HTTP Server, highlighting what is most important to know about these products, including how they are used and for what business goals.