Steve Pitcher on Malware Threats to IBM i
New: Subscribe to TechChannel podcasts on Apple.
Paul Tuohy: Okay. Hi everyone, welcome to another iTalk with Tuohy. I’m delighted to—well, actually, I don’t know if I am delighted because we’re going to be talking about something that’s terrifying the life out of me so anyway welcome back to iTalk Steve Pitcher from iTech Solutions. Steve, how are you, my friend?
Steve Pitcher: Hi, buddy. How are you doing?
Paul: I’m doing well, thank you and I hope you’re the same. Well, you’ve got less hair than me so we’ll come to that later.
Steve: A little less hair, just a little bit.
Paul: So, Steve the topic of conversation that we’re going to be talking about—when you filled me in on the details of this, I’m sorry. I will be honest. I am gob smacked by what you told me. Okay?
Steve: Good.
Paul: So, and I’m a little bit scared. I will tell you that as well—
Steve: Okay.
Paul: So, come on. Scare everybody who is listening.
Steve: Well, gob smacked, it’s okay to be gob smacked. I’m not trying to scare people because if you’re aware that there is a threat, then at least you can do something about it so you’re now aware of the risk. That’s a valuable bit of information you know. If you’re aware of risk, then you can work to mitigate that risk or at least lessen the overall damage of that risk. So, I reached out to you there a couple of days ago, a week ago probably, and said hey, Paul, I got something to talk about regarding malware on IBM i and I’ll just give a couple of statistics on some of the things that I’ve seen and by no means is this a scientific study. This is purely antidotal, but there is some value. 2018, I think I recovered one machine from malware attack, 2019, probably four or five, 2020, was about ten, the majority of those in last half of the year and in 2021, I’ve done 13-14 system recoveries from January until April of 2021, so that’s a lot. So from a malware perspective, I think Bit Defender has estimated a 715% increase in malware from 2020-2019 to 2020 and I can’t remember who the other groups, but they estimated a 6000% increase for 2020 to 2021, so there’s a lot more stuff out there hitting customers and if on a standard day if you throw a few things at the wall and something sticks, great, but when you’re throwing millions of things at the wall, you’re going to have more things stick and I think that’s where the problems arising for the IBM i community because they’re getting hit with malware more often and because rudimentary steps are not always being followed. Those that aren’t following those rudimentary steps are getting burned and by being burned, I mean doing the restore 21 to get back in business.
Paul: So I just want to make sure that everybody is clear on this, Steve—
Steve: Sure.
Paul: So, what we are talking about here is malware on the IBM i.
Steve: Absolutely. Absolutely. Malware attacking on IBM i, not necessarily malware on the IBM i. It’s on there once it starts or the results are on there, on the system once an attack happens so yeah, historically we’ve had the viewpoint and it’s correct in many ways that IBM i is virus resistant. You know it is. If you’re operating within the QSYS environment, a program is only going to operate as a program object should a source member for RPG or SQL is going to operate as a source member should. You cannot take that and use it maliciously or use in way that the system will not allow it to be used.
Paul: You haven’t seen my code Steve. I can tell you that. Not intention but you haven’t—
Steve: If we’re talking about bad code, Paul, you haven’t seen my code. Okay. It compiled good. Let’s go. It’s not pretty but she’s going to work. So yeah, IBM i is virus resistant in that regard but when you take the system and you open up windows to that system by way of SMB protocol, file shares through Netserver, if you open up the QSYS library’s file system in a Windows SMB environment or you know any type, it doesn’t have to be Windows but if I open that up in a file share from a client be it Linux or Windows or what have you, and I have appropriate rights to those objects by way of either a lack of proper private authority on the objects or by way of all object special authority. If I have a map with those credentials that satisfy any of those two issues, then I have the ability to really take apart a system inadvertently by clinking on something I shouldn’t click on, maybe open an attachment that has a macro. The macro kicks off. I have a drive map to my IBM i’s root directory and that thing kicks off and starts systematically disabling or destroying my system. That’s usually how it happens. It’s very easy to do inadvertently. It’s not intentional on any user’s end, of course, for the most part unless it is a malicious attack but for the most part it’s an innocent move. Somebody clicked something they shouldn’t click and then you have something tearing apart your network, trying to encrypt or destroy anything it finds in its way including IBM i and it will take no prisoners. IBM i is not inherently protected against that type of attack unless you configure it to be.
Paul: So Steve, has this been a just one particular type of malware that you’re noticed or has it been malware, ransomware, just the whole gamut that’s out there?
Steve: I’d say 75% is ransomware so it’s software that will encrypt your data or your objects rather and the other 25% is literally just destructive malware. That’s a really high number for stuff that just goes out there and wants to tear things apart with no other reason than just to do it. I’m not sure why that is and I’m not sure what that the drastic increase over the last two years in malware has been. Maybe it’s two-fold. I think most people are currently vulnerable, no matter what business you’re in. You’ve had to probably alter your business for good or for bad. You know people that are not in great business to be having during a pandemic. Tourism, okay, so if you’re a company that is in the tourism industry and my wife owns a mini golf course, an indoor mini golf course and you know I see it on the bottom line. I see it when we’re in lockdown. It’s not good for business to have people unable to come and spend your money. It’s not good, so people in those industries, they have to pivot. They have to lay people off. They’re already down and this stuff is maybe taking advantage of that especially the ransomware stuff. So, it sees a company whose down and it’s a great target because it’s going to kick when they’re down, say hey, if you want to save what’s left of your business, you’re going to you know pony up some bitcoin and you’re at least back in business if they actually give you the ability to decrypt your objects, which is not always the case. The other side of the coin is that some companies are positioned very well from a pandemic perspective. I could imagine people who make sanitizers. You’re already in that type of field that you’re making—a couple of my customers are in apparel. They make clothes. What have they been doing for the last nine months, ten months? They’ve been making PPE gear, smocks and capes and blankets and this and that. They’re making hay. They’re doing really well and they’ve got deep pockets because of that or at least a good war chest because of that. They can afford to pay if their information gets encrypted so it’s kind of two-fold. Everybody is vulnerable no matter how they’re doing. If they’re doing poorly because of the pandemic and they’re doing really well because of the pandemic. I think that’s probably why these malware makers are certainly making hay right now.
Paul: Yeah. I find this a really, really scary thing, Steve, so do you think it’s a thing that historically this is just something that–I remember when the IFS first came out, back in ’94 or whatever, that on the i series and I know there was a lot of confusion that people had with it because it was a lot more complicated than shared folders were. Let’s be honest that was there before.
Steve: Yeah.
Paul: Attempting to do anything like shares and all of that, a lot of people just took the easy way. Oh, if I just share the root that will cover everything.
Steve: Sure. That would cover everything.
Paul: Yup and it covered everything, and they’re just never gone and fixed it so is that sort of the case? Is that how it’s been happening for most people?
Steve: Yeah. Well, if it’s full restore situation, Paul, guaranteed it’s a share of a critical directory either with the root or somebody shared the QIBM, QOPENSYS folders or even sharing QSYS.LIVE. I think there’s a fundamental misunderstanding of what the IFS is and I had this conversation with somebody a couple of weeks ago and the IFS is called the Integrated File System for a reason. It integrates all the file systems on the platform under one umbrella. That one umbrella is a directory, your root directory so your root file system contains QSYS.LIVE; it contains your DLO file systems; it contains QOPENSYS, all the other ones; there’s a few other smaller one their file server 400. I don’t even know what that really does, to be honest with you. I don’t think it’s used much nowadays but all these things are underneath the root so when you open up the root, you’re opening up everything on the system and if you’re opening up everything, then you are relying on your private authority on your objects. That will save you if you set them up correctly. Most of the time they’re not. Most of the time they’re shipped—there’s two things that happen when you ship an IBM i. Net new IBM i comes with the root directory read, write, execute. That’s seldom taken down to read, execute so the authority problem get inherited and propagates through the rest of the IFS for custom folders that you build. You have a payroll folder, EDI folder, whatever folder, that’s usually set to read, write, executive for *public. Not good and the other thing is QCREATE authority. That’s the default public authority setting. It’s shipped as *CHANGE, okay. It ideally should be *EXCLUDE. Next to nobody is running at *EXCLUDE so what happens? You create a library or a schema. *PUBLIC *CHANGE is what it is. You’ve got to be diligent because it’s the easiest thing to turn off, that QCREATE authority because you might have some thing that’s worked for 30 years and it builds temporary tables, temporary libraries and things like that and then that process will stop working properly so you’ve got to be diligent. I’m going to create a library. I’m going to create tables. I’m going to create programs. I’ve got to go into those things and set the authority properly so when you have a share of the IFS, that private authority may or may not in the most case help you. For those who it does help, you’ve locked everything down with proper private authorities but hey guess what? You’re running security level 20 and everybody has all object or you’re running at security level 30 and the way you got from 20 to 30 was put everybody in a group, gave the group all object, then went to QSECURITY 30 and you know it’s—and that’s the easiest way to get from 20 to 30. It is. Does it get you there? Yes. Does it really do much for you from a security standpoint? No, it does not. I’ve seen—
Paul: It keeps the auditor happy. You’re on level 30.
Steve: It keeps auditor happy, and you don’t want to get me started on financial auditors doing IT security audits because brothers, that’s an hour and a half conversation by itself.
Paul: But tell me something Steve. So, in a company, again I would say in most companies like they’re going to have a network and they’re going to have file shares on some sort of a Windows server or some sort of another server, a Linux server or whatever. So, is it a thing that when these companies get these malware attacks that those servers get hit as well?
Steve: Sure. Absolutely.
Paul: Or is it a thing that people have security set up properly on those servers, so the malware attack is minimalized on those servers just sort of that person’s personal directories?
Steve: Well, if you look at the average Linux distribution when it is installed and it’s installed pretty secure, so your risk is a lot lower in order to invoke root authority with pseudo command or something like that, the average user doesn’t get to do that. From a Windows perspective, it’s a little less in my opinion but still there are some safeguards in there. Most people aren’t on their domain as a domain administrator so their rights to a local machine on the network is usually pretty minimal which is good, but it doesn’t mean that it’s infallible because most of the time when we get a call from an IBM i customers, we’re focusing on the IBM i. Great. We’ve got these other teams that are focusing on the 132 Windows VMs that just got hosed and we have seen that most of the time so while you know it is pretty good, it doesn’t mean that you don’t have people with map drives with domain admin rights or right to those local VMs that can cause a problem so it’s not just for IBM i. It’s every operating system. If you’re in charge of your Linux distros, then you need to harden those. You’re you know in charge of your mainframe. You need to harden it. I don’t care what it is. Anything not protected properly will be destroyed.
Paul: Yeah.
Steve: That’s kind of how it works.
Paul: Okay so to sum this up then Steve, your advice to—your minimum advice to people what they should be checking on their system right now?
Steve: Minimum advice. I have two rules and this is Steve’s list of commandments. I’m going to go get a piece of steppingstone, garden stone out there and start chiseling in the rules. #1 I just had a visual of what that would look like. It was awfully foolish. No but really, there’s two rules so only share what you must share. #2. Properly protect what you share, all right? So only share what you must. There is no need, no requirement to share the root or any other critical system directory, right? The root? Yes, it’s IBM’s but technically it’s part of that user space because you can create subdirectories in the root that’s yours, but IBM owns root. Do not share root. Do not share any other critical IBM directories like QSYS, like QDLS, like QOPenSys, QIBM. Now when you do share your custom directories underneath the root, you have an EDI folder, a payroll folder, a Bob’s folder—I don’t care who it is. If you’re going to share that, ensure that hey if you don’t care about the data that’s on here. Leave it *PUBLIC read, write, execute because that’s what it’s for. You’re not protecting it. Everybody can see it and everybody can change it. It must not be that important then, so you need to check the properties or the permission rather on each on each of those directories that you do share. If it’s an EDI folder, ideally the only things that should be touching that is part of an EDI process so the profile that runs your EDI. You have people that do manual intervention with the data that comes into the EDI. We still see that all over the place—have those guys in there and if they don’t need write access, don’t give them write access. Maybe they want to just need to do an eyeball before they move it to another folder or not move it to another folder but maybe they need to eyeball something twice a day, get a number out of it, whatever it is. Don’t give them write access to it. If they need to edit or move or do something with those objects, okay fair enough. Give them the rights to do that. The goal here is to minimize risk. There is no secure system but where you can get to is an acceptable level of risk on the system for what you do share. Now those are the two rules. Underneath those, there’s a couple of caveats. You know your private authority is only going to save you if you do not have an all-object problem. You got an all-object problem then you can put all the private authority on those objects you want too.
Paul: No.
Steve: So yes, properly protect what you share also includes ensuring that you don’t have an excess amount of people on your system with all object and you know that’s just—that’s just something we’ve talked about since the beginning of AS/400. You know all object, that’s your super user. That’s your root user. I was doing an assessment for a customer literally this week and I put on some auditing on socket connections hitting their system because I’m just trying to understand where some things are coming from and I notice this local IP address hitting the system and it’s using QSEC software credentials. Bang, bang, bang, every 3 minutes or 3 seconds rather, QSEC software was hitting the system. I’m like what is the IP address because we’ve got somebody who is QSEC software credentials trying to hit the system from there. They’re like that’s a TwinQ controller attached to a printer. I’m like okay, excellent. So why does that have QSEC software credentials? Don’t do that. This is basic stuff. Don’t use QSEC software for anything at all, especially don’t hardcode the password somewhere and that’s just the basic stuff so two things you can prepare. Don’t share the critical directories and properly prepare—properly protect what you share. #3 is make sure you’ve got a good backup. I don’t care what you’re doing because if you’re not backing up your system, getting a solid user data backup every night at least get a lick at operating system save, save 22 or a save 21 every six months whether you think it needs it or not so just as long you have a recent tape that you can put your fingers on and that’ll at least get you back from a—from a bare metal recovery perspective so you know because DR and security kind of go hand in hand. Most of these events end up into a disaster recovery situation.
Paul: Okay.
Steve: And it’s not fun.
Paul: Well, that is what we want to try and avoid. So, listen before we go Steve, I touched on this at the very start and I think it’s a good one to finish on because I quite like this in the times that we’re in. Last visit to the hairdresser?
Steve: Well, it’s funny Paul because you may look at me and say he doesn’t have much of a hair cut. I’m getting a little length and I kind of like it so I’m letting it grow a little bit nicely and so I went to my stylist. I have a stylist. 42-year-old man has a stylist. You wouldn’t really say it looking at me but she’s the lady who cuts my hair. So, I went over to her house, and it was what about a week or two ago? Yeah, about two weeks ago and I’m sitting in the chair. I’m listening to the news and hey government just says that we’re in lockdown. Okay, excellent. We’re in a two-week lockdown because our Covid cases are out of control, aptly timed, I’m getting a haircut, so it was an interesting time but what’s good about it though is while I was there, she’s like you know what? I haven’t hugged anybody in about a year. I’m thinking yeah, other than my wife and kids I haven’t hugged anybody either, so you know she gave me a hug and it was like oh, this is excellent. This is first person I’ve hugged in a year and I’m thinking man, I’m a hugger. When this thing is over, you’re not going to know what a hug is people because when I meet you at a conference, Paul, look out. It’s going to be rough.
Paul: It’s the first person at the airport who’s nice to you Steve. That’s who I’m worried about.
Steve: Oh man yeah. Well, you know what? If I run into Charlie Guarino, it’s probably going to create some sort of a hug black hole that is envelope the earth or something like that. It’s going to be out of control.
Paul: Okay. I think with that vision of you and Charlie creating a black hole from a hug. So, Steve thanks for taking the time to talk to me and I do hope everybody pays attention. Please go have a look at your system. Check out what Steve has been telling us. So, Steve, thank you until the next time.
Steve: You bet buddy. Thank you.
Paul: Okay everyone. That’s it for this iTalk. Tune in again for the next one. Bye for now.