Rochester Drug Cooperative Uses Regulations to Springboard Modernization
Dominick Pagnotta and Seiden Group collaborated to meet new regulation requirements.
Dominick Pagnotta CIO, Rochester Drug Cooperative, Image by Matt Wittmeyer
UP CLOSESome scoff at the notion of people wearing many hats. “How can someone who does so many different things be an expert in anything?” Well, plenty of people have a large enough collection of hats to dispel that notion.
CUSTOMER: Rochester Drug Cooperative
HEADQUARTERS: Rochester, New York
BUSINESS: Distributor for independent pharmacies
CHALLENGE: Meeting the requirements for a U.S. Food and Drug Administration regulation
SOLUTION: Deploying new Android-based warehouse scanners and using an existing IBM Power Systems server for related web services
HARDWARE: An IBM Power System 750 and a Power System S814
SOFTWARE: Zend Studio, Zend Server, a largely homegrown ERP application and a third-party warehouse management system
Case in point: Dominick Pagnotta, CIO of the Rochester Drug Cooperative (RDC). He began his career as an APL programmer and then went on to learn several more languages. After moving to application design, he went into technical support and sales and then straight-up sales, including serving as the head of regional sales.
He then moved into business development for outsourcing, which naturally progressed to CIO status. During this time, he also did “a lot of web development,” he says. “This allowed me to understand the entire stack very well.”
That made him the perfect person to head up government-necessitated IT changes for RDC. To comply with a new regulation handed down by the U.S. Food and Drug Administration (FDA), the organization needed to make crucial front-end changes to its warehouse processing environment—hopefully without making too many alterations to the back-office applications running on RDC’s IBM Power Systems* server and IBM i.
Shortly after word of the FDA’s Drug Supply Chain Security Act (DSCSA) came down, Pagnotta was already mulling over ways to make the necessary IT alterations. Knowing what did and didn’t have to change, Pagnotta, the RDC team and Seiden Group began work to create an effective yet inexpensive solution.
New Efficiencies
Established in 1905, the Rochester, New York-based wholesale drug cooperative acts as drug and general-merchandise distributor for independent pharmacists, including community retail pharmacies, long-term care pharmacies and home healthcare stores. It’s currently the fifth largest such company in the U.S., comprising more than 1,300 co-op members. RDC encompasses nine states, from Maine to New Jersey to Ohio.
“It started with a group of independent pharmacies that joined together to form a co-op, so they had combined leverage with the pharmaceutical manufacturers. That was just in the Rochester area, but we’ve grown a lot since then, and we now distribute everything from toiletries to prescription drugs and everything in between,” Pagnotta says.
Examining Computing Options
Humming along in the background are two IBM Power Systems servers. The POWER7* 750, located at the organization’s Rochester warehouse, is the testing and primary production system, and the POWER8* S814, hosted at a warehouse in Fairfield, New Jersey, handles disaster recovery (DR). It also hosts an LPAR for local production use.
Two core applications support the business: an ERP system originally purchased in 1998 but that has since become essentially homegrown after RDC purchased the source code, and a third-party warehouse management system that integrates with the ERP software. “All of our invoicing, ordering processing—everything—is on the POWER7,” Pagnotta notes.
That’s why RDC had to examine its operations and computing options when the DSCSA came down the pike. It had to develop a new warehouse receiving application to comply with the regulation, which requires new traceability standards to protect consumers from exposure to drugs that might be counterfeit, stolen, contaminated or otherwise harmful. The regulation is also meant—using improved serialized-based tracking abilities—to improve the detection and removal of potentially dangerous drugs from the drug supply chain to protect U.S. pharmaceutical consumers.
As Pagnotta explains, “The regulation was phased in, and the first part had to do with how actual product has to be serialized—or labeled. So, the manufacturer has to throw a label with a unique serial number on a bottle to indicate what’s in it and the product quantity. The next requirement is that we can only receive product that’s properly labeled. And the final requirement has to do with what’s called ‘verifiable returns.’ That means that if we get a return from a pharmacy, we have to verify that there’s a legitimate manufacturer-specific serial number on the bottle.”
Evaluating Requirements
RDC began initial requirements discussions in the fourth quarter of 2017 to determine how the DSCSA would impact both its IT environment and operations, especially in its warehouses. It also looked into how it could use those requirements to improve warehouse processes and applications, rather than just meeting the FDA requirements.
Although RDC had been scanning the linear UPC codes on incoming and outgoing shipments for many years, DSCSA requirements specified that a newly designed 2-D bar code—which is similar to a QR code—be used to better track product movement. RDC’s scanners, however, weren’t capable of handling these new 2-D coding requirements.
To meet the FDA’s requirements, RDC could have written a new RPG and 5250 terminal-driven application to scan the new 2-D bar code as a new additional process at receiving and shipping, but this wouldn’t have been any more efficient than what it had been doing before.
“If we had gone down that road, there would have been no change from what we had been doing except that we would have had to put a new scanner in place that was capable of reading that new 2-D code,” Pagnotta recalls. “But we wouldn’t have received any benefits of modernization, which is something we were interested in. We wouldn’t get a new UI. We wouldn’t get the data integrity we needed. We wouldn’t gain any new efficiencies of a remote scanner. The UI was still green screen.”
RDC's modernization changes included revamping the 2-d bar codes.
Wrapping it up
And this is where Pagnotta’s background of many hats came into play. He knew several modernization roads could be considered—including inserting x86 hardware to support web services between new the 2-D scanners and RDC's Power Systems servers—but these options were clunky and expensive. Why should the organization add to the complexity of its IT environment when it could use what was on hand, especially given the dearth of Windows* skills in the organization’s IT department?
By the second quarter of 2018, RDC—with the advice and assistance of Seiden Group—decided it could meet the DSCSA requirements by refactoring the receiving and shipping applications modules of the ERP on the production Power Systems server. It had also decided to host its web services on that same system, using Zend Studio to craft the code and Zend Server to handle PHP calls in its server stack, which also included HTML, JavaScript* and CSS. RDC’s IT team was instrumental in making this happen.
Pagnotta decided to use RDC’s production system to host this because it had excess compute power, it was simple to administrator, and its existing business logic and system-of-record data resided there. RDC also had highly developed RPG, Power Systems and IBM i staff skills. He also considered the platform’s ability to be not just another web server, but a highly durable platform for the modern web stack.
The third quarter of 2018 was primarily devoted to determining application requirements and researching bar code scanning devices for the new 2-D bar code implementation. RDC eventually settled on an Android-based device running a custom scanner app developed in-house.
The graphically presented and easy-to-use app was developed in Android Studio and is delivered to the scanning devices via the dedicated mobile web server on the Power Systems server, allowing RDC to use other scanning devices in the future without the need for major device-specific code refactoring. This server-to-device app delivery system also ensures RDC can more dynamically release app upgrades, from the server without updating each individual device.
In tandem with this, RDC is also using the native XML data type of IBM Db2* for i to store and query XML documents received from manufacturers and suppliers that electronically match physical shipments. Subsequent scanning at receiving and shipping verify the product against the data, helping meet the DSCSA’s three-phase regulation.
Additionally, the organization is taking advantage of microservices. They provide access to legacy business logic through PHP Db2 calls to external stored procedures. This encapsulation preserves legacy investments and reduces the need to reverse engineer deeply buried business logic.
“If we know there’s a function written in RPG, we can just put a wrapper around it rather than rewriting that code, preserving that legacy code and turning it into a little microservice,” Pagnotta says. “This has two benefits. One, it preserves the logic; you don’t have to rewrite or reinvent it. Two, it allows reuse because if we decided that it’s an important function, somebody else is probably going to come along and say, ‘Oh, I need that same function’ and we’ll make it available to them.”
Back and Forth
Although Pagnotta is a man of many hats—and recognized the best approach RDC’s situation based on past experience—he calls out Seiden Group as a key resource. “It was good to have somebody from the outside for validation,” he says. “They helped with some of the prerequisites. They got everything configured, and then as I worked through the architecture, I would ask things like ‘Shouldn’t we be using PHP? Shouldn’t we be using the Power Systems architecture and Node.js?’ That type of external back and forth was invaluable.”
This is especially true when an organization has to meet industry-specific regulations. If the requirements aren’t met, the company could face legal hurdles or go out of business entirely. So, well-considered, architected and deployed responses to regulations are absolutely necessary. And if others follow the lead of RDC, they can wrap modernization into the effort, all while using the computing resources and skills at hand.
