Scott Klement on REST APIs, DevOps and CICD
Note: This podcast transcript is lightly edited for clarity.
Paul Tuohy: Welcome to another iTalk Business with Tuohy. I was about to say it’s great to be joined by an old friend, but when I say old friend, I think just old in that we’ve known each other for awhile. Because you’re nowhere near as old as old I am, Scott, so just get that one out of the way.
Scott Klement: Well I feel pretty old [laughs].
Paul: Programming does that to us. Programming does that to us. So Scott, it’s going to be news to a lot of people, but you are now working with Midrange Dynamics, so I think maybe a good place to start. Do you want to just give people a quick blitz about who Midrange Dynamics are in case they haven’t come across them?
Scott: Absolutely. So Midrange Dynamics, I don’t know the history real well. I’m still new but it’s a company that has focused for many years in change management software, and they’ve got the MDCMS product which I think everyone thinks of when they think of Midrange Dynamics. It’s an extremely feature-rich change management software; basically it can do whatever you want to do. I was looking for a job that was more in that area because what I see happening right now, the big change, the big shift in the world of programming is switching away from this sort of old model of programming where the developers kind of write things and test them themselves, then put it in, then there’s problems. I think we’ve kind of moved away from that to a more Agile/DevOps-based world of development. People are finding things like CICD really improves their productivity. You push your changes into the change management software and it automatically deploys it to QA to have it tested. Then when QA approves it, it automatically deploys it into production, allowing you to get your changes in faster, allowing you to make smaller changes so that you’re being more iterative in your build process. This is a huge change for the IBM i world because we’ve very used to doing things the way we’ve always done them. And so I thought this is what’s really happening today in IT, this change, and I wanted to be a part of that new sort of wave of change. So that was one of the things that brought me to Midrange Dynamics, and a happy coincidence as I became involved is that they also have a product working with REST APIs, which I’m already well known for. My open-source HTTP API project has gotten a lot of people started with calling REST APIs. I’ve given a lot of presentations on how to provide them as well, and so this gives me the opportunity to get involved at that level and really help with Midrange Dynamics products, which they already have. They’re already established products and we try to bring them forward. As the industry keeps changing we keep improving.
Paul: So was there one particular thing Scott that sort of attracted you, one project? Or was it just the whole general area of what Midrange Dynamics are working on at the moment?
Scott: What really attracted me was this change in the industry towards DevOps and CICD. I wanted to work somewhere where I would be working on that kind of a product.
Paul: Cool.
Scott: So the REST API thing has been what I’ve been focusing on in the short term, and this just happened to be something that I was already good at. But that wasn’t what attracted me at first. What attracted me at first was getting involved in DevOps and CICD, at a place that has the leading software for doing that for ILE programmers. So that was what really excited me to get started.
Paul: Yeah so let’s talk a little bit about the REST API, because I know when we were chatting about this just a little bit earlier, you—everybody, well a lot of people—are envious of you anyway, Scott. They are about to get more envious I think with this, so in your own words I think is the best way to explain what’s going on at the moment with this.
Scott: So what I am focusing on the right now is a new version of MD REST for i. That’s Midrange Dynamics REST API tool and I’m in a great position that I actually get to rewrite it and redesign it to the way that I want to do it, which is very exciting. So I released HTTP API back in the early 2000s when all of this API stuff was brand new. People didn’t really know how they were going to use it yet, and I put together what I thought was very flexible, very good at the time, but you know how we all can relate to this, I think. When you write programs and you go back and look at them years later after having had them in production and use, we all know that you look at that and you say oh, I could have done this so much better if I could do it over. Well now I can do it over, and that’s exactly what I’m doing. I’m writing from the ground up a brand new API engine for providing API and a brand new API engine for consuming APIs that is done exactly the way that I think it should be done, taking into account my 20-some years of experience with this. And it has just been really fun to be able to go in there and do things all over the way that I think that they should be done. Let me tell you about this because I’m very excited about it. The APIs today, REST APIs, basically they use what’s called the open API specification better known as Swagger to describe their APIs and how they work. So with Midrange Dynamics, we can import your Swagger document and it will bring it up on the screen, tell you what everything in it means, and then you can adjust it or you can build a new Swagger document from your database, for example—or by hand if you really want to. Then it will generate a program to use that Swagger. So this is hugely productive because it’s extremely flexible. You can make it do whatever you want to do. You can adjust it until your heart’s content, and then once it generates the code, you can go in and code your business logic however you want to. It’s just extremely flexible and extremely easy, and you really only have to focus on the business logic—although you can control everything if you want to. With my new version of this we’re actually able to really beef things up from what we had. We had this idea before, but now I’m kind of fine tuning it. The code that it generates is just going to be super elegant. What I like to say is it’s generated Scott’s way, the kind of code that I like to write and work with—so very elegant, easy to understand code, easy to maintain code, using modern designs and so forth. On top of that, we tested it. I put together my first prototype for it, and the guys that I’m working with at Midrange, they were skeptical that I knew what I was doing. So they went and they tried it out and they benchmarked it versus the old one, and mine was 16 times the speed of the old one [laughs]. So if you’re an existing customer of MD REST for i, look forward to this version because it’s really going to improve your performance. I’m very excited about it. We’re also adding support for the COBOL language, and I’ve actually taken the time to write a set of subprocedures that works like RPG’s DATA INTO and DATA GEN op codes, so that those features which COBOL didn’t have before will be available to COBOL programmers. You’ll be able to code up a data structure and generate a JSON document or an XML document from that, or the other way around: receive a JSON or an XML, and our code generator will actually generate the data structures for you. It’ll just read it into it. So the business logic is just working with those data structure. Of course RPG has had this as well, but now we’re going to generate those structures for you, which makes it a lot easier. So this is very exciting, and it will work with any ILE language. COBOL and RPG is where we expect the most customers to adopt it, but you could do it in CL if you want to. It would be weird, but you could do it.
Paul: [Laughs] I’m—I’m sorry. I’m just picturing doing REST APIs in CL.
Scott: Yeah.
Paul: I’m sorry. Why not?
Scott: Well it’s very hard to do the database part of it in CL but you can do it if you want to, and maybe there’s some good uses for that—for example to allow you to look up system information on a mobile device or something like that. You know it might be easier to write something like that in CL. So I’m very excited about what we’re doing right now, and we’re making great progress. Like I said, my prototypes are all out there and people are testing them. We’ve got more to do, but this is coming this spring and it’s going to be very exciting.
Paul: I’m just going to swing back around because I want to come back where we started, because I am really envious of you on this. I think anybody, any programmer in existence to have that opportunity to revisit something that you did years ago, and especially for you Scott, in an area where you are I think probably the recognized expert in the industry on IBM i on this….
Scott: Well it’s very exciting. The world of cloud has really made APIs come to the forefront because so many people are moving applications off site into internet machines while keeping things on premises as well and needing to combine them, integrate them. And so APIs, even though I started doing them more than 20 years ago and they were very popular already back then, they’ve really taken off and they’ve become a huge part of IT today. So here I am able to redo things exactly the way that I want to do them and improve performance and improve code readability and just make things easier for the customer. I think this is so important because I really believe when it comes to programming, you have fewer bugs if your code is easy to understand and easy to work with, because people can make changes, they understand those changes, and they aren’t going to miss some silly little thing and cause things to go all haywire. So I really believe in having the nicest, most elegant, clean code possible and I’m very excited about the ability to bring that to MD REST for i and just really ramp up what’s already a good product, but really ramp it up to the point where I think it’s going to be the best in the industry.
Paul: And just so I’m understanding this correctly then Scott, basically what you’re saying is that within the product, all of what would be the sort of the REST handling part is just simply generated from a Swagger document.
Scott: Most of the REST handling is happening in a service program that I’m writing, and we’ll still have the code for the old versions as well so that your existing programs will still work, but that’s where most of the logic is. But things like, if you’ve worked with DATA INTO or DATA GEN, you know that what you do is you code a data structure in your code and you use DATA INTO to bring your document that’s coming in from the API and put it in a data structure, and then later you take DATA GEN and you take your data structure and convert it back to a document. So what this will do is generate that code for you, it will generate those data structures for you, and it will put all the logic to make it send out to our service program that makes it into an API and receive it from our service program, and all the other odds and ends that you might need to do beyond the data structure stuff will be coded for you as examples. You can go in of course and modify it to your heart’s content to add your business logic into it.
Paul: Cool, absolutely cool. As usual, whenever I end up talking to you, I end up being jealous, Scott, of the stuff that you’re working on. So please, for me, enjoy it and keep enjoying it.
Scott: I will enjoy it.
Paul: So any idea what you might be working on after this at the moment?
Scott: After this? I have a lot of ideas of things that I would like to work on. I would like to continue to work more with the change management software. It already does all the DevOps things, CICD and so forth. I would love to get more automated testing stuff built into it but—I mean you can do that right now. There’s steps in the process where you can go in and say I want to insert my test here or what have you, but I would like to find ways to get that easier for the customer so that it kind of does more of that automatically. I don’t really know at this point if that’s what I’m going to end up doing though, so I don’t want to make promises, but that is something that I would like to do. I would like to continue to enhance the REST APIs. Of course there will be a lot of supporting that and so forth that’s going to be necessary over time and continuing to add more features to that. So there’s a lot of cool things that we can do.
Paul: Cool. So listen, before we go, Scott, I would like you to bring everybody up to date with what you’ve been doing and working on just over the last couple of days.
Scott: Oh the last couple of days I’ve actually been on vacation from work because I am a mentor for my son’s robotics team. It’s called First Robotics and the way that it works is every year they announce a new game, a completely different game from any other year, and the students have to design and build a robot to play that game. It’s like a sporting event. You compete with other schools. So I’ve been away, I’ve been down in Chicago at the Midwest regional event where we were competing against 46 teams and their robots, but I’m just so impressed because these are high school kids. They’re given a game and they all sit down and brainstorm what kind of features should our robot have. They sit down and design it. They build the blueprints for it. They have machine shops where they build all the parts for the robot. Now granted, there’s sub-assemblies like motors and so forth that they purchase, but they designed and built the whole chassis for it. They did all of the wiring. I am of course the mentor for the programming team as you might expect, and so I’ve been teaching high school kids, some of which have never written code before, how to write code and how to make the robot, but they decide what they want it to do. I just help them learn how to do that, how to make that a reality. And so we serve as advisors, we being the team mentors, serve as advisors to the kids, but the kids design and do everything and they build—you know it’s a big robot that can move very quickly across a court. It lifts up cones and cubes and places them. The cones have to be placed on a pole, so if you imagine a traffic cone and how that’s shaped, you have to set it on a pole. Think of how would you make a robotic arm that can pick that up, extend out and bring it down so that the right side of the cone is going over the pole to make it sit on that. It’s easy to do that with your hands, but think about how you would build a machine to do that, and then it also has to be able to work with cubes, you know the square things that are placed on a shelf that looks very much like the stairs in your house or something like that. So it has to be placed on that in just the right spot in order to score points. So if you think about this—these aren’t little things, these aren’t kits that you buy at the store, right? These robots are designed from the ground up by high school students, and they’re big, powerful robots. We’ve got sponsors from many different companies who work with robotics and engineering, or from factories—you need people with this expertise so that they’ll bring students into it. I’m just so proud of the kids, so proud of the kids.
Paul: Yeah, when you were telling me about this last week, I was just thinking—because initially you said to me like oh, it’s a robotics thing. Of course I immediately thought robot wars, you know those things that go and chop the heck out of each other, but I think—sorry. I had never heard of this as a program being done and I think it is just phenomenal. So you were down there. So was the competition this weekend and if so, how did you guys do in it? How did the school do?
Scott: Yeah the competition was this weekend. We ended up making it into the playoffs on the top-seeded team, and we turned up in fifth place at the end.
Paul: Hey, that’s good.
Scott: So out of 46 robotics teams.
Paul: Hey, well done.
Scott: Pretty good.
Paul: Excellent.
Scott: I mean obviously we wanted to come in first, but we did pretty well, and some of the things that they were able to do that I’m just very proud of. For example we had problems with the surface that we were playing on because we don’t have the actual surface from Chicago in our school to test with. We found that the way that the robot drove on it wasn’t the way we wanted it to be, so we ended up—there was 15 minutes between matches. We ended up taking the whole wheel assemblies off of the robot, putting different ones on, changing the program code to use completely different motors and a completely different way of doing our drive train, all in ten minutes. And then we have to have it reinspected by the judges to make sure we’re not cheating and breaking any of the rules with our new setup, because we’re making a major change to it. And we were able to do the whole thing in 15 minutes and get out for the next match. I’m so proud of them and it was just so cool to see, because we had thought about this possibility. We had designed multiple sets of code for different ways of driving the robot for this reason in case it came up. We had all the spare wheels and gears and chains and motors available so that we could swap it out. So to do all of that in 15 minutes, I’m just so proud of the kids to be able to do all this stuff.
Paul: Yeah, so if they never go into robotics, they have a backup plan as a Formula One pit team [laughs].
Scott: And that’s exactly what they call it. They call it the pits where we were working.
Paul: Okay. Well I think that is an excellent note to leave this on, Scott. Thank you for taking the time to talk to me, and I wish you all of the best both in your endeavors with Midrange Dynamics and in your future endeavors in robotics as well. So thank you, Scott.
Scott: Thank you, Paul. It was great to talk to you again.
Paul: Okay so that’s it for this iTalk, everyone. Keep tuned. There will be more in the future hopefully. See you all then.
Interested in learning more? Check out Midrange Dynamics’ North American website.