Extending AI Beyond the IT Department
Profound Logic's Brian May is seeing IBM i clients benefit from AI and has advice on getting started
This transcript is edited for clarity.
Peg Tuttle: Welcome explorers to another exciting episode of PowerTalk with Peg Tuttle. Today I am thrilled to be joined by Brian May, vice president of Product Management at Profound Logic, and an IBM Champion. Profound Logic is leading the way in AI-enhanced futurization, helping businesses seamlessly integrate advanced technologies into their operations. We’re diving into the world of AI and its transformative impact on business operations, especially for those of us in the IBM i community. Whether you’re a tech enthusiast or a business leader looking to embrace the future, you won’t want to miss this conversation. Let’s get started. I’m so happy to have you all here today. It is my pleasure to welcome back to the microphone—you all know him, you all love him—Brian May with Profound Logic. Welcome, Brian.
Brian May: Thanks for having me.
Peg: Yeah, absolutely. You guys have been so busy over at Profound Logic with your new Profound AI tool. That’s what we’re going to focus on talking about today, but before we launch into that, why don’t you give everybody just a quick update on what you’ve been up this summer. Anything fun?
Brian: Well I just got back a few weeks ago from Europe, so that was a fun trip. We had COMMON in Fort Worth, and then I was going to COMMON Europe. Of course my wife and my two daughters told me that I wasn’t going to Europe without them.
Peg: Of course [laughs].
Brian: So, a quick trip to Milan turned into basically three weeks in Europe.
Peg: I love it.
Brian: But we had a blast and managed to come down with Covid on the way home, so then we spent a week in quarantine once we got home. But overall, it was a good trip and everyone is back and well now. The girls are gearing up for classes to start back soon, and I’m just trying to get back into the rhythm of things.
Peg: Yeah for sure, and I know you guys have been working diligently to get Profound AI out the door. It released and became available a little earlier this season so we’re going to talk about that, but before we launch into talking about product, let’s talk a little bit about what we’ve been seeing in the marketspace with regards to AI, because everybody is talking about it right now. I know Charlie [Guarino] and Jesse [Gorzinski] just did a session at OCEAN. I know you’ve been talking about AI. Steve Will over at IBM is going just full bore on that whole IBM AI for RPG programmers, so it’s just been really interesting to see this huge flux of interest around AI.
Brian: Yeah it’s been interesting, and I don’t say this in a bad way necessarily, but you know the IBM i customer base has not always been the fastest to adopt new technology. But AI seems to be the exception, for lots of reasons.
Peg: Yeah. Why do you think that is? I agree with you, and I know that everybody in this marketspace would say that exact same thing. We are slow to incorporate what people are talking about or what’s hot, what’s trending, but I agree with you and I know everybody would agree as well. Why do you think that is though?
Brian: I think there are a few reasons. One is the technology aspect of it. This is a disruptive technology. This is not just one of those things that yeah, that’d be great if we had it. No, this is something that if we don’t get on board with this, we’re going to fall behind the competition, right? This is going to make or break companies in the long run, so there’s a lot of awareness with that. This is also lots of times—especially in larger companies the C level execs don’t get really caught up in the tech side of things, right? This is the exception. CEOs, COOs, CFOs, they’re all wanting to know more about AI and how it can make their businesses better. They’re hearing all the chatter. They’re reading all the articles and they want to know how they can get a piece of the pie. So I think you’ve got some momentum from the upper management side of things as well as a really interesting technology for all of us tech nerds. I think that’s just kind of culminating into something that everyone can agree that we’ve got to get started on.
Peg: Yeah absolutely, and I know Steve Will has mentioned you a couple of times when he’s been interviewed as a contributor to the IBM AI for RPG—you know they’re collecting code, they’re trying to come up with a tool that is similar to what was created for the IBM COBOL side of the house. So you can chat a little bit about what Steve Will is doing over there?
Brian: Sure. You know it’s really interesting to see what’s happening. Here at Profound Logic we’ve been using AI for code assistance for a long time. We’ve been using GitHub Copilot for a really long time for our development work in-house, and it works really great for Node.js, which we do, and Javascript, which we do a lot of development on. It works OK for RPG at this point. It can help a little bit, but it’s certainly not as well versed as it is in other languages. So it’s really exciting to hear Steve Will talking about building a model and training it specifically on RPG on IBM i so it has that in-depth knowledge in order to be able to be a really useful code assistant. There’s a lot of great uses for that, whether it’s understanding an old piece of code—and not just for the new people on the IBM i side of things. I hate looking at old code too, so having something to help me unravel some really old, monolithic spaghetti code could actually be extremely useful for me, but also just being able to make developers more efficient.
Peg: Yeah, yeah.
Brian: There is a lot of code that developers write that’s not really meaningful. It’s kind of just oh, you know, I’ve got to scaffold this out, I got to have a loop to do this and I need a function to do this. AI can take care of a lot of that for you if it’s properly trained in the language that you’re using. So developers can put their concentration on the difference-making parts of the applications, and that’s great. You know there’s talk about AI helping build test cases for RPG. I know we do that at Profound Logic for our products. We use AI to help build test cases for those. It would be great for RPG developers to be able to do the same.
Peg: I know that there will be a lot more coming out about that initiative here later this fall, so we’ll have to keep an eye on that and maybe do a quick interview with Steve about that. Well wonderful, let’s just dive right into what we’re here to talk about today, and again you know, really highlighting AI and its impact on business. So let’s just start with that really big question. How do you see AI impacting business operations beyond the IT department?
Brian: You know that’s where the value really is, honestly, and more and more of our customers are realizing that. It’s great and you see a lot of places and a lot of customers I talk to when they start, you know AI is this toy that the IT department is playing with and they’re using it for code assistance and to help their lives better. And that’s great, there’s nothing wrong with that, but where the real business value for AI comes out is when you can get it in the hands of the end users.
Peg: Sure.
Brian: When your accounting clerk can have AI help them analyze financials, when your production manager can have AI help them track bottlenecks in the manufacturing process. I mean that’s where the business gets the most value.
Peg: Right.
Brian: Now it’s really easy, being in technology and being in IT, to be a little self-serving, and that’s okay too, but what I’ve been talking to a lot of customers about is okay, we’ve got to figure out how is AI going to get outside of the IT office. How are we going to find use cases that bring real business value and make everyone else’s life easier, not just IT’s.
Peg: Right, right. I could see like other groups, you know, non-technology groups, being a little nervous or apprehensive about launching into an AI tool or that integration with AI. You know the person in accounting—well I don’t know how to use this, I don’t understand it. Or the guy on the shop floor trying to discover those bottlenecks—help me understand where this would fit in. So how are you addressing some of those initiatives where there’s no buy in from those other groups—or maybe I’m not understanding properly. Maybe there is buy in, I don’t know. What are you running into?
Brian: I’d say a year ago there was a buy in, but AI has been advancing at such a rapid pace over the last couple of years [now] that people are seeing it outside of work. They’re going to a website and interacting with a chat feature and it’s AI, and it’s clear that it’s AI. If you go into Facebook right now and you go to search for someone, it takes you directly to the Meta AI, not to the search bar anymore.
Peg: Oh, I know. I know.
Brian: It’s becoming more accepted, so I think that there’s not quite as much pushback as there was a year ago. But as far as dealing with it, a lot of it is just education.
Peg: Sure.
Brian: So implementing an AI solution is not just building a chatbot or an agent and putting it in the application. That’s part of it obviously, but a lot of it is education. It is educating the end users on the capabilities of that agent that you’ve built because obviously—or at least I hope you’ve put guard rails around that particular agent and it’s only allowed to do certain things. So you’ve got to educate your users on what it’s allowed to do, but also, there’s also education from an ethical standpoint and a legal standpoint that have to happen as well. Your users need to understand what’s acceptable AI usage and what’s not, so a lot of it just boils down to education, and most of it isn’t technical. That’s the crazy part—when you’re using natural language interacting with a large language model, you’re just talking to it. It’s like talking to a person. So the technical side of it is not that hard. It’s actually just understanding the proper usage.
Peg: Sure, sure.
Brian: A lot of it, unfortunately, is setting policies and educating people on those usage policies. Probably the most technical thing from an education standpoint that we run into as far as getting users ready is teaching them how to more effectively ask questions.
Peg: Sure, yeah.
Brian: AI is really great and sometimes you can just throw it a vague question and it will understand where you’re going with it, but sometimes you have to be a little more specific. So the concept of asking a good question comes up and you have to make sure that when you’re asking a question of the AI, you’re including all the factors, right?
Peg: Sure.
Brian: For example if I were asking about how to increase sales, I would include a phrase like based on order history. Why? So that the AI knows that I want it to take a look at this customer’s history in order to help gain insights and answer my question, right?
Peg: Right, right.
Brian: If I were talking to you directly, Peggy, I wouldn’t ask you based on order history, how do you think we can increase sales to this customer? You would know where I’m going with it. But when you’re talking to a machine, you do have to throw in those context cues. A lot of it is just getting used to how to talk to the AI. It’s not hard, it just takes a little practice.
Peg: Back to helping the business utilize AI outside of IT: How should they get started? Where should they put their focus to get the most impact? Where do you think they should start? I’m sure it’s different for everybody but what are some of those low hanging fruits?
Brian: Yeah, like in our previous conversation, you know you have to start small. You are not going to implement full process automation using AI on Day 1. That’s not going to happen. So you want to find small, attainable goals. Honestly, the key to finding successful use cases is to get out of the IT office and to get out of the boardroom. Talk to your end users and say all right, what are you struggling with? What’s something that you have to do all the time that is annoying and you wish you didn’t have to do it? What’s a piece of information that you have to go searching for every time you need it? You know, what are the things that frustrate you about how you do your day-to-day job? When you find those things out, that’s when those things start popping out. This person is spending 10 hours a month chasing down all of these different pieces of information in order to come back and be able to make a decision about something, so why can’t we get that information using AI and let AI make recommendations?
Peg: Yeah.
Brian: And so suddenly you’re saving 10 hours a month, and you do that with six, seven, eight people in an organization? Suddenly you’re talking about real money.
Peg: Yeah, I love that. Anytime you can increase productivity and save money—who doesn’t want that? Then you talked a little bit about challenges—that education piece, having the guardrails, making sure you have policies in place. What are some of the other things that might pop up as far as maybe integrating into your existing systems?
Brian: Yeah, when you’re integrating into your existing systems, there are different ways to do it. You can invoke an AI agent programmatically and have it happen automatically behind the scenes to help you do things, and those are great especially if you have things that are happening in batch behind the scenes. Of course for end users that are using an application, you have the UI side of things, the customer experience side of it, making sure that your AI features are easy to access and they are in the places where they are needed most. So yeah, those kinds of things take a little doing. That’s one of the great things about—you know, when we built Profound AI, is we wanted to make that part of it, the integration part of it, as easy as possible. That’s why we include the user interface and everything and we give you—the code you need to plug into your web based interface to just automatically integrate it onto the screens—we tried to simplify as much of that as possible. So that’s the technical side of it, but a lot of it is actually in the design side of it and understanding where it makes sense to have AI. You’re not going to throw an AI agent on every screen of your application in the beginning, so understanding where there’s the most value is a big part of it, and it’s a challenge sometimes.
Peg: Sure.
Brian: This sounds crazy when we’re talking about AI and all these infinite possibilities, but also learning to reign yourself in [laughs]. So yeah, just because you can give an AI agent access to every table in your database does not mean you should.
Peg: Yeah [laughs].
Brian: So build purpose-built AI features that are context-aware and only access the things that they should for the application that they’re a part of. Those are things you have to really take into account for a lot of reasons. But also, yes, AI can handle tons of information, but you can tell it too much and confuse it. So you want to make sure that as you’re building these things out you’re thinking those kinds of things through. That can be hard for customers sometimes because they kind of want to do select all and just let the AI figure it all out and it can, but that’s not the best way to approach things, you know, for several reasons.
Peg: Do you have example that you could share with our listeners today? I think when we talk about these concepts, it’s great, but when you can put a story to it, it really helps bring the example or bring the tool to life. So I’m just curious if you have a real-world example that you can share with our listeners.
Brian: Yeah, I can. I can’t give names or specifics, but I can talk about a use case that one of our customers is really benefiting from. We have a customer who has agents in the field that are servicing the customer’s customers, right? So in their business, they actually have people out in the field who are interacting directly with their customers. They have an application that they use, they are going out and doing all their data entry in the field. I guess I don’t know how far back, but at some point those people in the field said that they just wanted—we’ll call it a general purpose notes kind of field in the application.
Peg: Oh sure.
Brian: Basically take shorthand notes of anything that they want to do to make things quick and easy when they’re in front of a customer. And that worked great, but then they realized that now that they have all of this stuff that was really free formatted—you know big text areas with just whatever they wanted to plug into—it was really hard to get any information back out of that, because nothing was formatted the same.
Peg: Oh, sure.
Brian: Things were misspelled. People were abbreviating things, and everyone was abbreviating them differently. All of this data was being collected, but they weren’t able to get much out of it. They couldn’t even get a good search function over all of this free-formatted data, again because of different spellings or different abbreviations or different formats and all of these things.
Peg: Yeah.
Brian: So they had a task on their backlog for years to implement a search for this data, because it’s valuable data. I mean they tried a few things and they were never able to actually get anything to work. Well, they’re using Profound AI now to implement that search feature, and what’s great is that a large language model can look at multiple ways to abbreviate a word and figure out what they’re actually talking about. It can get over the fact that things are misspelled. Spelling doesn’t have to match perfectly. It can infer that that’s what you’re talking about, right?
Peg: Right.
Brian: So in a matter of, in this case, weeks. If they were doing it again now, they would probably do it a lot faster because it was their first project, but in a matter of weeks they’ve been able to actually build this thing that their user base has been begging for years, but for technical reasons and past decisions they weren’t able to deliver. They’re doing that now and rolling that out and now, all of this data that was locked away. You could go and look at an individual customer and read the information, but you couldn’t do any kind of analysis over everything because it just—everything didn’t plug together properly.
Peg: Right. Right.
Brian: Using AI, they’re now able to farm all of that information out of those free-formatted buckets of data and actually be able to bring that in and get real insights, answer real questions, find customers that have the same types of needs and pull them together based on all of this text data that has been entered in over the years. So I mean they’re getting huge amounts of value from that. Can I put a dollar figure on it? Not yet. [But] it’s something that was impossible that’s now possible, and as a solution provider that’s the highest thing that I could possibly do is make something that was impossible possible, right?
Peg: Oh my gosh.
Brian: There’s no greater praise for what I do every day then hearing someone say that. It’s been awesome to see what they’ve been able to do and what they’re planning for the future now that they realize the real value. And is that a really complicated a use case? No, it’s really not. All they had to do was point the AI at the data. It figured most of the rest of it out on its own.
Peg: Yeah. Isn’t that crazy? I just think about like everything you just said about the ability to go out, read, organize the data and then ask questions of it. Like maybe there’s a new product that they need to create. Maybe they have a glitch somewhere in their system that they need to fix that will save them lots and lots of money. There are just so many different things—from driving the business, how can we make decisions going forward that impact not just our customers but maybe our employees and the way we do business—and then again just the ability to service their customers better, faster, more economically. Yeah, that’s pretty amazing, and you said it was quite simple for them.
Brian: It was. They have one person that’s been working on this. He is not an AI expert, but he was able to take this and roll with it and come up with the solution, and this wasn’t even the first choice for the projects when we were talking about it. Whenever we start with a new customer, we talk about all that. What are some things that we want to start with? What are some easy wins? This actually wasn’t one of those things, but when he started playing with it, he said you know, I wonder if it could solve this problem, and just ran with it.
Peg: Nice, nice. I love that, and you know there’s going to be more and more success like that as customers implement Profound AI and just see what it can do around their own environments. I’m just thinking about the people in accounting and running financial analysis that take forever—you know, just to speed that up. I know you touched on this a little earlier, but that whole data security—having guardrails, making sure that you’re sharing only the information that you’re allowed to share—are really important factors in all of this.
Brian: Yeah, and there’s multiple levels to that, and that’s something we have to talk to our customers about, and they have lots of questions, obviously. Even from the product standpoint, there are things that we can control in the product and there are things that we can’t. We have to have conversations and say all right, we do everything we can to make sure that no information that you don’t intend to send to the AI is sent to the AI. That, we can control. What it does with that information once it gets there is outside of our control. I hate reading them as much as everyone else hates reading them, [but] if you’re going to use a commercial large language model, you’ve got to take the time to go through all that legalese and read their privacy policies and their usage policies on what they do with your data.
Peg: Yeah.
Brian: There’s a trust there. You have a level of trust that you have to have with whoever that vendor is. You know then some are like well, we want to do everything in-house, we want to run an open-source model. And six months ago I would have said no, that’s probably not a good idea. But honestly within the last few weeks, Meta has released Llama 3.1, which performance wise is getting very close to a GPT-4, and that is an open-source model that you can run on your own hardware.
Peg: Wow. Oh, okay.
Brian: Now, is it as good? Not yet. It’s getting there; it gets better everyday. So I mean that is becoming a reality. The main thing with the AI industry as a whole is it’s going to change.
Peg: Yeah, yeah.
Brian: It constantly changes, so that is becoming a more realistic option even for those that may not have the budget for gigantic infrastructure to run large language models. Because some of these open-source large language models, you can run in the cloud. It’s not running locally, but it is running inside of your enterprise with your cloud provider, and provided you’ve set up your security properly for your cloud, then yeah, you have control over it again. So there are all kinds of things to take into account, but a lot of customers don’t get when they start—they buy a product like Profound AI and they’re like okay, you guys are just going to secure everything. Yeah, we’re securing everything we can, but if you’re using OpenAI or Anthropic or one of the commercial large language models, you’ve got to fully understand that side of it, because that’s outside of our control.
Peg: Yeah, outside the scope.
Brian: Yeah, it’s out of scope for us. We give you the flexibility to use whoever you want. So if you decide you don’t trust them tomorrow, you can switch. We can make recommendations. We can try to help you understand it, but ultimately that’s a business decision, not a tooling decision.
Peg: Sure.
Brian: So that part has been—I won’t say it’s been difficult. Again, it’s one of those teachable things that we have to do when we start getting really involved in partnering with our customers, making sure they understand all the moving parts. Because obviously, AI is not simple. We’ve made it as simple as we can, but it’s still a computer that can act like it can think, right? So that’s complicated [laughs].
Peg: Yes, yes, it is. It’s—there’s still a person behind that computer.
Brian: Somewhere.
Peg: Somewhere. Somewhere, yeah, yeah, excellent. Excellent information. Thank you so much, Brian. As we wrap up here, is there anything else that you’d like to share that we haven’t covered yet with regards to Profound AI? I know that you know we’ve got an opportunity for folks to reach out to you, to go to the website, to download the free version, get the white papers, but is there anything that you want to share that we didn’t touch on today that you feel is necessary?
Brian: The starter tier of Profound AI—as you mentioned, as part of our 25th anniversary celebration we are giving that to customers for free as a gift. So that is, in my opinion, huge.
Peg: That is.
Brian: That is a full version of the product. The only limitation in there is that logging is turned off for all of the extensive logs that we keep up with that you would probably want in a production environment. That part is part of the business tier, which is also extremely affordable. Because these podcasts go forever, I’m not going to put a price out right here, but you can go to our website and it’s clear.
Peg: Yeah [laughs].
Brian: It’s affordable, okay? But with having that affordable production class subscription as well as the free tier for the development side of things, getting into AI has never been easier or more affordable than it is right now. For example, OpenAI just released GPT-4o mini, and I want to say—don’t hold me to it—a million tokens is 15 cents on the new mini.
Peg: Oh.
Brian: It’s ridiculously cheap. So I mean the cost barrier is all but gone, especially when you have tools like ours and the reducing costs of all of the commercial models now and the rise of the open-source models, right? All of that is coming together to just kind of a storm right now where I don’t care how small your company is—I don’t care if you’re a one-person shop or a 1,000-person shop. You can afford this.
Peg: Yeah, yeah. You have no excuse not to try it out, not to dip your toe in the water.
Brian: There really isn’t. I mean you can go to our website and download Profound AI. You can load it on your local laptop and then connect it to your databases, connect it to your IBM i and get started. You don’t even have to load on your server if you don’t want to right now. I mean there’s absolutely zero barriers to actually start experimenting—
Peg: Yeah, and it sounds like it’s kind of low risk, too, which I love.
Brian: It is. You download it, you play with it. If you don’t like it, okay.
Peg: Yeah.
Brian: What are you out other than, you know, a $20 one-month subscription to your LLM provider or something along those lines, right? Yeah, there’s really not any excuse—and the thing is that if you’re not experimenting with this now, if you’re not planning now, you’re already behind.
Peg: I don’t want people to be behind, so this is the perfect—
Brian: Exactly.
Peg: Opportunity for everybody just to go ahead and check it out and you know, become a part of the movement—or at least understand what people are talking about, what they’re experiencing. Play with the tool, be familiar with it so that you can continue to have a seat at the table and really share your knowledge, and then collect more knowledge to be a part of this big movement. So, excellent. Excellent. Thank you so much. What an excellent discussion. I just am blown away by all of the AI stuff going on in the market right now. It’s crazy. It’s crazy fun.
Brian: It is. I mean, it’s a full-time job for me now just keeping up with what’s going on and making sure that we’re staying on top of it, but it’s also a ton of fun.
Peg: Well everybody, head over to profoundlogic.com and check out Profound AI, their new tool. Again, like Brian said, you can check it out for free their 25th anniversary gift to everybody. If you need something a little more advanced, they do have the subscription model. Reach out to them with your questions, reach out to them for additional information. Like I said, you can go to the website for their download and for more information and white papers—so it’s all out there. Thank you so much, Brian, it’s always a pleasure to have you on the show.
Brian: I’m looking forward to being back soon.
Peg: Thank you. Thanks for listening in, everybody.