The Blueprint for the Agentic AI Mainframe

Patrick Stanard, Distinguished Engineer and chief mainframe architect for Kyndryl US, explains how the mainframe can be transformed to leverage the latest in agentic AI capabilities

The concept of autonomous, decision-making machines once belonged to science fiction thrillers, inspiring both fascination and concern. Today, that future has arrived with the advent of agentic AI, ushering in a new era of service delivery for mainframe environments.

Building an enterprise-grade ecosystem for agentic AI on IBM Z requires the merging of decades of mainframe expertise with contemporary AI capabilities to create a scalable, secure and extensible environment. This framework enables service providers to deliver faster, more predictive and more resilient services—improving reliability, accelerating modernization and enhancing customer outcomes.

Here’s a look at the main components that make this industry-first architecture possible. 

The Hardware

Architecturally, the agentic AI mainframe requires a centralized, high-performance environment designed for secure communication with customer systems over encrypted z/OSMF channels. Locating the mainframe within a controlled network zone enables outbound-only connectivity for access-controlled, auditable interactions with AI agents, protecting customer environments from unauthorized access.

The following hardware components help form the foundation for this next-generation service delivery:

• IBM z17 hardware – For production agent execution
• Spyre accelerator cards – For enhanced on-premises inference
• DS8K subsystems – For rapid, resilient storage

The Software

Agentic AI agents differ fundamentally from traditional automation scripts and fixed orchestration workflows. They are autonomous, context-aware software entities capable of observing their environment, making decisions and taking actions within predefined guardrails.

To support this framework, model context protocol (MCP) servers facilitate agent communication and information sharing, functioning similarly to APIs. Retrieval-augmented generation (RAG) serves as another guardrail, enabling agents to retrieve data from diverse sources, analyze and plan usage, generate responses, and validate and refine outcomes. These mechanisms ensure safe and effective agent operation.

Combined with the above components, the following core elements help operationalize agentic AI on the mainframe:

• IBM watsonx Assistant for Z as the interaction layer: This conversational interface serves as the front end for human interaction with agentic capabilities.
• An orchestrator and execution pipeline: The orchestration layer, part of IBM watsonx Assistant for Z,  identifies the required agents for each task, running them serially, in parallel or in iterative loops as complexity demands.
• Secure execution: Agents do not log directly onto client systems. Instead, they use secure HTTPS connections to z/OSMF, executing REST calls to gather data, run diagnostics, inspect workloads and validate configurations. Industry-standard security frameworks like RACF control agent permissions.
• ZRAG knowledge layer: A consolidated database, akin to collecting all IBM Redbooks into a single searchable resource, powers agentic intelligence.
• Human in the loop: For high-risk or customer-impacting actions, explicit human approval is required, for trust, accountability and intervention in extreme scenarios or nuanced cases.

Worth the Effort

With the above pieces in place, the agentic AI mainframe is equipped to revolutionize critical processes across the stack.

Rapid Issue Diagnosis and Resolution

Agents can diagnose system issues, review logs, identify causal patterns and recommend next steps in seconds, streamlining problem-solving. For example, an agent monitoring COBOL programs might detect a SOC7 abend, analyze it, suggest a code fix and generate a trouble ticket—all in under a minute.

Proactive Monitoring and Anomaly Detection

Continuous system monitoring via API calls and advanced telemetry enable agentic AI to identify anomalies earlier and more consistently than human-only monitoring, mitigating operational risks and accelerating error resolution. The AI-infused software provides robust tools for tracking agent activities.

Workflow Consistency and SLA Compliance

Agentic workflows enforce consistent task execution, maintain detailed audit trails and ensure compliance with service-level agreements (SLAs), often tightening and improving SLA adherence.

Knowledge transfer and workforce optimization

These agents also capture operational logic and expertise, guiding less experienced engineers and shortening the learning curve—an especially valuable feature given the current skills gap in mainframe operations. Shifting routine work to agents frees subject matter experts to focus on more critical infrastructure tasks.

As Modern as It Gets

The agentic AI mainframe is more than an infrastructure upgrade—it is a transformative platform for modern mainframe operations. By combining intelligent agents, secure connectivity, advanced orchestration and deep mainframe expertise, the agentic AI mainframe delivers a more predictive, efficient and resilient service model, marking a breakthrough in mainframe service delivery.

---