Foreign Cybersecurity Threats & Risks to ICS / OT Equipment

[00:00:00] Dino: On the plan floor, you're not getting any of that stuff. You're getting none of it, right? So you have no CIS log data that's being pumped up when somebody's making a change to A PLC or when some, a new asset comes online. All right, or, there's new protocol, or new port that was opened up or a scan that's being done.

[00:00:16] Dino: It's invisible to them. so now you're missing huge chunks of your infrastructure activity because you're not focused on it.

[00:00:57] Dino: Hi, my name is Dino Boussalaki and I'm the chief technology officer for Velta Technology.

[00:01:02] Jim: And I'm Jim Cook, the COO at Velta Technology. And thank you for joining this podcast what are we talking about today, Dino?

[00:01:10] Dino: We are going to be talking about how foreign actors have been going after critical infrastructure in our government, as far as the federal government of the United States, and the role that they've been playing in trying to hunt down and root some of these,bad actors out of environments that they've been infiltrating.

[00:01:27] Jim: Yeah. Which we've seen. Yeah, this is timely, especially with the recent, depending on when someone's listening to this, but, recent announcement, about, the federal government going in and clearing, going into businesses and homes and clearing certain, bots that are in some switches.

[00:01:43] Jim: correct. that's what we're triggered this conversation,

[00:01:46] Dino: right? Yeah, where the government's basically getting, going to the courts and getting subpoenas to giving them the, warrant, needs that they need to have to basically breach private industry and private networks in order to chase down malware and other, attacks that are being deployed by Foreign actors, primarily the Chinese is the CCP is the biggest culprit right now that he will hear, the FBI, Christopher Ray. And you can find multiple, interviews of this conversation going on in the Hill just here recently of what they're laying out and what they're finding and what they're chasing.

[00:02:20] Jim: what I find really intriguing about this is that. It's a change of operation, we've seen this once two years ago, they went after that exchange, malware, where they went in and we actually saw the decline, but it's a different tactic, this is. a shift. And it's not talked about a lot, that tactic being that they're going into private industry, right? They're going in before, they're monitoring and advising, and then they'll get involved if you bring them in, but now they're seeing things that are so bad they're getting the court order or whatever legal means that they have to go in and these are two very specific incidents that shows, that the have changed, right?

[00:02:59] Jim:

[00:02:59] Dino: showing the tactics and also the fact that these business owners aren't sophisticated enough to recognize the attack that's being perpetrated upon them. and, the work that we do, like finding the federal government doing this back in 2021 in the spring, at the, soft beverage company sitting there and watching them scan.

[00:03:22] Dino: The control system network, of a plant chasing Microsoft exchange software, right? So instead of being, surgical. In the work that they were were just taking a machete to scanning and tracking and chasing that they were doing, even if it was impacting the plant floor, right?

[00:03:41] Dino: So your business is impacted by the federal government that could be shutting down your plant because they're scanning your network. Now, we see this all the time with I. T. groups that still do that type of, Discovery by scanning physically, actively hitting these control system environments and being disruptive, right?

[00:04:01] Dino: And now you're ahead in the fact that the federal government is doing the same thing in your environment. And you're not even aware that they're doing until they send you a letter saying, Oh, by the way, we were chasing this volt typhoon, malware that the Chinese have implemented in these networking switches out there, especially for end of life stuff, right?

[00:04:20] Dino: So people who don't keep current with their technology per se, and we see this too, where, part of our work is going out and looking for end of life assets. and so this is an attack vector that is an easy target, right?

[00:04:32] Jim: that's an excellent point. One or two that I want to touch on why I find this so intriguing, right?

[00:04:37] Jim: because this tactics change, they're going out into the private industry. And now we've got another thing to worry about and we've talked to some of the guys in the CISA group and the federal security. And while there are branches that understand this industrial cybersecurity,and there are some really good guys out there, the majority of it.

[00:04:57] Jim: They don't, right? They're so focused on the I. T. that, that they're coming in and could cause the same problems that we've seen that, I. T. organizations that they don't understand. Now you got the federal government coming in we're causing disruption to save you from disruption, right?

[00:05:11] Dino: there's probably many that think that this is a geopolitical, and or military targets. It's not. they're going after critical infrastructure. They're going after businesses that supply things, whether it's part of a supply chain to your pharmaceuticals or to, petroleum or to, your rail, your water, your wastewater power.

[00:05:32] Dino: you think of some of these rural and smaller community, wastewater systems out there, that they don't have the capabilities to be able to. protect themselves from malware that may have seeped in through their building automation, building management systems through their it systems and or their control systems, right?

[00:05:49] Dino: Because we know they're not watching this stuff, when you get over 90 percent and it's, and I, that's, and I think it's an even higher number that don't have complete visibility. Of everything within their environment, I'm talking about all the way down into your control system, you're building automation, building management systems, where you have full blown, I can see the activity with these machines every day.

[00:06:12] Dino: We can walk into a plant and quickly determine, what their cyber hygiene looks like. It's let me see your current asset inventory and how current is it? Is it the last couple of minutes or the last year?

[00:06:22] Jim: for those that are listening, I think, we're critical infrastructure.

[00:06:25] Jim: We're not critical infrastructure. We're not related to them. should really rethink that because,the, it has been stated that they're going after. They're critical infrastructure, including transportation, including telecom, but you mentioned the supply chain. And okay, are you in that supply chain?

[00:06:41] Jim: And even if you don't think you are, guess what you are in? You're in collateral damage territory, right?

[00:06:47] Dino: and your IP is being stolen, right? And that's been going on for at least two decades, the intellectual properties being taken from these organizations unbeknownst to them

[00:06:55] Jim: their industries, it should know that you and I have seen it that, didn't need technology years ago that there was just an accepted practice when you were working. I want to just touch on the collateral. So that should, everything that we say should be a wide group that even though the system comes out and they insist are the feds or who's the FBI, who's ever.

[00:07:13] Jim: The one executing the message and say, okay, it's critical infrastructure. It's a supply chain. It's telco. always say, look, you could be collateral damage because what ultimately can happen is that because increase in the global environment and state actors is going on. look at it and step back and go,that's how this whole thing started.

[00:07:33] Jim: So these weapons, these cyber weapons or cyber knowledge will get handed off to proxies and you may be quasi state sponsored people. and I guess what that'll go out, roll all the way down to, just. Private bad guys. So now they've got the cyber knowledge or the cyber weapons.

[00:07:51] Jim: and they're not just worried about reconnaissance and disruption. They're going to look to see, Hey, if I can get disruption, I can get some money out of this thing. So really is a wide net. And so I always try and caution, industries. And I know you do to this, That look, just because you don't think you fit in one of those classifications.

[00:08:09] Jim: If you've got this stuff out there from the industrial side, you're going to be a target eventually, right? it's always that cybersecurity guy, even on the IT side, they said this before this came on, it's not a matter of if. It's a matter of when, and, it's been juiced up and this particular tactic change, least the way I interpret it is, Hey, this is getting bad guys so bad.

[00:08:32] Jim: We got to change our tactics and we got to go out and get this stuff. And that should be a message, right? That should be a message to industry. They're going to move

[00:08:40] Dino: past the analysis to paralysis situation that most of them are in, right? because these wholesale attacks are happening every day, Imagine, having rockets fired at you every day, all the time, you can see in the Middle East, kinetic physical. But the same things happen from a cyber perspective. Every single day, there's rockets being fired at all these businesses, And the question is it just. I can't figure it out because I don't have the people, how many projects have we worked on where clients will come back and say, we're having a bad year.

[00:09:12] Dino: So we're going to cut back on our cyber security,

[00:09:14] Jim: We had a plan, that's too much money. And even if it's too much money, so they just. Yeah. They stop and don't do anything that always gets to what we say about getting better.

[00:09:23] Jim: but what I find fascinating is what you just said, right? said it's like rockets being fired every day and again, i'm not an expert In warfare, right? But I love Reading and listening to these guys that talk about cyber warfare, And basically what's the government's? Play in this the genesis of this, elevation is really coming out of these state sponsored activities.

[00:09:43] Jim: what I've heard from them and read is that it's a bit of a mutually assured destruction in this cyber warfare, but it's different. It's not physical because it is rockets. It's constant, right? And it's basically these ongoing competitors coming at each other, And it's, I've heard terms, I wrote them down.

[00:10:01] Jim: I've heard terms that it's hybrid warfare or it's gray zone warfare, or it's just their strategic competitors. and up to now it's been, they try and figure out. The deterrence and reconnaissance. All right. They know bad guys state sponsored are in our systems. And we know that because we're in their systems, but they're not shutting everything down.

[00:10:23] Jim: because then they're worried about retaliation because this is a whole new world, right? It's, the kinetic warfare, there's rules to kinetic warfare, and there's a lot of studies about it, but we've never crossed that next step and I just see it over the past few years and these incidents being specific as well, okay.

[00:10:41] Jim: The tax change. So is that gray zone shipping a little bit, right? So what ultimately is the response again, while I find that fascinating. the core of what we do is. Everybody needs to go, this is real, right? This is real. It doesn't matter who you are, you need to ask this question because even if someone's telling you it's cyber security, you're okay.

[00:11:03] Jim: We always say, are we practicing the same due diligence, on our OT assets as we do with the IT assets, right? and that question isn't being asked in enough organizations. This, what we're seeing is coming out and go, how are you going to disrupt? those machines aren't running.

[00:11:18] Jim: That's going to be a big disruption, right?

[00:11:20] Dino: the question is, what are you trying to do to get better? the federal government can't cover everybody, right? They're going to focus on the things that really matter, as far as tier 1 rail, air, banking, telecom, oil and gas, maybe, but the rest of you out there, you're on your own.

[00:11:38] Dino: you're on your own. and so what are you going to do about it? Really? anything, because from our perspective, we recognize that it's not moving fast enough, right? And there are people even within our industry who have different tactics about. you just can't patch the control systems, right?

[00:11:56] Dino: The best you should be able to do is do asset inventory. But then what about all the other mitigating controls? Then we talk about defensible architecture, your remote access strategy, right? Your incident response. So it's a bigger, broader plan. and there's a disconnect at a lot of different levels, especially between IT and ot, I'm cautious when I see and hear and watch an IT organization driving the entire plan around cybersecurity for the OT side.

[00:12:21] Jim: you bring up an interesting point. I want to follow that now, which is getting down to, just real basic things that we see all the time, right? That's ties to this when they go,and specifically around. Remote access on the plant floor, right? So it groups have managed to really tighten it down and control who's coming in their organization from the it side.

[00:12:43] Jim: But what do we find all the time when we go out to a facility and start asking the questions that we ask and, looking at the machines and inspecting them and working with the, that the plant personnel is running. What do we always find, right? That remote access everywhere. and it could be around, it is around many times.

[00:13:02] Jim: It's around the network. Sometimes it's through their network, but they don't know where it's coming from. It's loosely

[00:13:08] Dino: defined. It's loosely put together. it's probably cobbled together and sometimes not even done by the organization that owns and uses those assets. It was put in there by the that sold them that asset.

[00:13:19] Dino: and so it's a wide range, right? And you would think that would be an easy one to go in there and start with that process to find, the remote access activity requirements into this environment, determine how it's being done, right? Which is what we do, right? And then determine here are solutions and technologies and processes to harden this remote access into this environment, control it better, right?

[00:13:41] Dino: Control it. Meaning. Read only control it so you can schedule a role based to control it so that you can only, use certain ports that you can get through. a whole wide range of things. You do no file transfer, right? There's different thing auditing recording of the sessions. All of these things that you should be using for these applications that don't have security built around them.

[00:14:03] Dino: They're unsecure by design in many cases. and so what are you doing to solve that problem? Because the OT guys aren't going to solve it by

[00:14:09] Jim: themselves. They're not going to do it. no,the problems that the OT guys are trying to solve is get that machine back up and running and producing efficiently.

[00:14:16] Jim: And that's what they're focused on. And we see this all the time where, whether it's the plant manager, whether it's the maintenance or controls engineers, whatever it might be, they know these guys are coming in and guess where all these machines are made all over the world, right?

[00:14:31] Jim: you have to start asking yourself the question,how do we know they're coming in from all over the world? We should be worried about these global,access points coming in for all over the world and by the way, maybe that's a big, huge third party supply chain risk because, let's say you got a half a dozen of different ways to get into your plant and I couldn't come up and get in your plan, go up into your I.

[00:14:53] Jim: T. Organization,that needs to be remembered out there. but you got a half a dozen different ways of coming in with a half a dozen and I'm just. Picking a number out per plant, whatever that might be. how do you know that they're exercising the proper due diligence on their own identity management?

[00:15:08] Jim: They're probably sharing IDs. Who knows? are they turning over? Do they have rogue?

[00:15:13] Dino: how do you know it's the person that's supposed to be, or the organization is, how do you know it's not the CCP, right? Because, remember the plant manager we just recently talked to, he says, I got people from all over the world that have access.

[00:15:24] Dino: System I manufacturing plant floor, at any time. All right. And he's, and of course he's referencing his OEMs and SIs again, who are all over the world. How do you know that's the valid people that you need to have that have access to your plant floor, right? And he doesn't know, and neither does IT.

[00:15:39] Dino: They have no clue, right? So now you have two groups that know nothing or very little about the interaction going on. And now you got the federal government coming in and swooping in because it's okay, it's a clown show, obviously. And we're going to go in there and hunt this stuff down because you don't seem to be sophisticated enough to figure it out on your own.

[00:15:58] Dino: that's what the federal government actually said, right? The reason why we're doing this is because the people that are being targeted aren't sophisticated enough to even know that they're being hit. And these are large global companies. We're not talking about some mom and pop shop, we're talking about very large global organizations.

[00:16:15] Jim: we would love to get that list, but I don't think they'll ever publish it. And the reason why we, the reason why I think Dino's mentioning that is because we've seen it. we've seen it on an isolated basis, but we've seen it. so we're familiar with that. we don't have insight to what that list is.

[00:16:30] Jim: I'm sure that's probably locked up in whatever, privacy laws or whatever court order they got. And,and I'm glad they have to keep that captive, but man, you're right. But again, getting to the fact that this was a global. Issue, right? It's coming from somewhere else in the world, we've had instances with our clients.

[00:16:48] Jim: that's actually something that you look for. You want to geofence this stuff and go, Hey, we don't normally have somebody from Eastern Europe or China or, Malaysia into the system. And we had one now. that's a normal practice with, IT, but there's no way if you've got six different other doors in, in, into your plant.

[00:17:08] Jim: And when I want to touch on too, is there are doors outside of your network. So they've, you may have, these machine centers that are connected to your network that have a backend internet access because that's their other door. And that's common. We see that. on top of that. So you won't see it till it's too late, right?

[00:17:27] Dino: Yeah. You don't think the CCPs that out there scouring shoan to find control systems that are sitting on the network so they can see what they can extract or what they can leave and find in there. it happens, and we find it, it's there

[00:17:41] Dino: to your point, some actor coming in the middle of the night from some foreign country. this discussion has been going on for a couple of decades, right? The whole syslogs, right? It's you get all these, all this syslog data from everything from a printer running out of ink to somebody fat finger in their login password, a badge reader on a door, somebody open it up.

[00:17:59] Dino: So you get thousands and thousands of events, billions. and so at first they were just capturing it, right? Nobody was really trying to, to mine over it. some formula algorithm that's looking for that, midnight hack, somebody's trying to log in at two o'clock in the morning from the CCP in China.

[00:18:16] Dino: So then they got Splunk. You know the SIM tools, right? And QRadar and Logarithms and these other SIM tools to start helping the IT guys have something to help find those little incidents that were going on amongst these thousands and thousands of events that are happening in a day, right?

[00:18:35] Dino: On the plan floor, you're not getting any of that stuff. You're getting none of it, right? So you have no CIS log data that's being pumped up when somebody's making a change to A PLC or when some, a new asset comes online. All right, or, there's new protocol, or new port that was opened up or a scan that's being done.

[00:18:51] Dino: It's invisible to them. so now you're missing huge chunks of your infrastructure activity because you're not focused on it. Now, the right tools in place and the right skills to be able to do this. You don't understand the O. T. what I call the infrastructure.

[00:19:06] Dino: it's foreign and it is not going to chase it down. They're just not going to do it. So then he has to start providing this information and they need to be educated. So they can say, what would you need from this environment? I. T. to help do the work that you do. Those types of dialogues and discussions need to be going on.

[00:19:25] Dino: in the federal, again, the federal government's not going to do this for you. right now they're playing whack a mole, chasing this stuff down and just trying to, squash wherever they can. they don't have the resources to do it and they're just going to protect the things that really matter.

[00:19:36] Dino: and so you're a lot of organizations And so the question then becomes is, how do you get out of your own way in order to start making some movement in the right direction to get a full understanding of your exposures, Instead of saying, I, it's just too, I can't do it.

[00:19:51] Dino: I just, it's too big. Yeah. I didn't give up. That's the point. Yeah. Unlike to your stockholders. Your shareholders. Yeah. The community that relies on the water that, they need to have, to sustain life. It's too complicated for us. We just can't figure it out.

[00:20:04] Jim: I always look at it, from back in my younger days when I first moved to the big city. And. decades ago and, I wanted to educate myself on how I should be,behaving and living in a big city. And the, the 1 thing I took away is, and I can't remember the guys, this is what late 80s and he had a big mustache and McGruff, the crime dog, it wasn't that, it was another guy anyway. But it was like, they're always going to go after the softest target and don't be that soft target on the street, take that and extrapolate it to this and go, as IT has hardened and taken steps, and it's not completely hardened, nothing's ever a hundred percent, where's the soft targets now?

[00:20:41] Jim: where's the soft targets? And then they're gonna look around and go, geez, if I can get in down here in the OT space, that's a soft target you get in there. there's no antivirus, there's, unpatched, unpatched, unprotected, the Windows devices all over the place and guess what, and maybe there, there's another door coming in.

[00:20:59] Jim: So I'll try that and push my way up through the organization. there's no way to prove any different if you haven't taken the steps, you're never going to know where it came from. It came from somewhere in this general region. So even if you have an OEM vendor out there that wasn't paying attention to their door and their third, your third party got breached and then they logged in.

[00:21:20] Jim: in the, this third party connection around your network, and it got up in there, you're never going to see it until it's too late. And then you're not going to know where it's from. And you probably don't even have a liability in there prove that it came from them. If that's what you're relying on, it's just going to be a.

[00:21:37] Jim: Appearing so I just, we probably have to,start wrapping up because it, gets to that. You can't protect what you can't see thing, but all so a wrap up, right?

[00:21:46] Dino: let me, I always had one thing, soft targets, right? So when you talk about soft targets, you're right.

[00:21:51] Dino: And they're going to go after assets that are in the hands of people that aren't cyber security professionals or the I. T. group. Makes sense. It's just like what Christopher Ray was saying in this latest round up on the hill just a week or so ago, building automation, building management systems, for example, are very weak targets.

[00:22:06] Dino: And who are they in the hands up? They're in the hands of the facilities. Organization typically right when you think about your HVAC and your environmentals and your elevators and your, water recovery systems, et cetera. and again, building automation, building management, fits into that control system or, same thing with on the plant floor where you don't have a cyber security presence.

[00:22:25] Dino: You don't have any tools. You don't have, good cyber hygiene practice in place. You are a target in that realm.

[00:22:32] Jim: That needs to be the target, right? Yeah. and that cyber insurance isn't going to save you. If you think that's, it's not a strategy. They're getting smarter, and if you start, yeah, you start reading it and go,even if you're trying to get, comes through there and your disruption, it doesn't roll up underneath your cyber insurance.

[00:22:48] Jim: as the takeaway from this is my wrap up. the genesis of this conversation was based upon what I consider a change in the tactic. And that change in the tactic to me only elevates the concern, That the global environment over the past few years is, it's a little nutty. and this is a space, in this cyber warfare gray zone, whatever we want to call it, that's continually going in it. where is that line? Nobody really can say. But we're going to have some collateral damage on this thing, right?

[00:23:19] Jim: and whether that's directly from state sponsored, proxy sponsored, or just ransomware bad guys that want your money, they're going to find those soft targets. So what do you do? One is I think you got to recognize this is realer a word? Realer than before, right? For your organization. and.

[00:23:36] Jim: Again, what are you going to do to get better in this case? And you got to start asking those questions down on your facility. The ones that we do, going to these facilities and saying, Okay. where are my weak points and what am I doing about them? and many organizations that don't have the right people or the right organization support organization to even ask those questions.

[00:23:58] Jim: That's why we get involved.

[00:23:59] Dino: even their consultants, they hire sometimes or the wrong first step, Yeah.

[00:24:03] Jim: let's find a way. who are you going to ask to come in and decide to, fix up your house and lock your doors and what are good locks and what are bad locks, what are entryways into here is the guys that work on the homes might mean,

[00:24:15] Jim: and that's what we do. That's why we, feel very confident in this space is because we're the guys that work in the plants, and know where the, windows are that can be easily pried open and, a back door, the cellar window, from a company start asking yourself that question, because this, it's only in the past.

[00:24:31] Jim: The year since Delta was launched getting worse and worse. or I should say worse and worse, the risk is getting higher and higher in that case. Anyway, I'll hand it off to you, Dino, there for your takeaway from

[00:24:41] Dino: this. my takeaway, after reading and listening to, the law agencies in this country describing this problem.

[00:24:49] Dino: at the scale that it's at and the level that it's at, what are the people thinking? These executives who are responsible for these organizations, right? When they hear this, do they just power just say, not going to happen to me, or they just go, we got people working on it.

[00:25:04] Dino: really, what kind of metrics do you have, that they would understand at the executive level to say, is my plant and my perimeter secure? okay. Data center. I get it. Networks enterprise. Okay. You guys talk a good story on that. We spend 2 million dollars a year on people and technology and stuff.

[00:25:23] Dino: what about all this other stuff that's going on that how we generate revenue, right? What are we doing on that front? what are they, questions are they asking? And what are their steps? Cause you know, you've been working with some very large organizations that have been breached have been hit billion dollars loss.

[00:25:40] Dino: And here we are a year or two later after it's been done and they still haven't moved the knee. They're

[00:25:45] Jim: still struggling. They're still struggling with it.

[00:25:48] Dino: And you have to ask yourself, why is that? if you're sitting there and you talk to the board, it's like, why can't we get off the dead center with this conversation?

[00:25:55] Dino: is it infighting? Is it politicking? the money that you would have spent to improve your cyber security posture, you lost this, these ransomware attacks that have been hit.

[00:26:04] Jim: Yeah, that'd be another interesting one just to go over we think it is, right? I think it's the disciplines themselves, honestly, that are lacking need to be tweaked all up and down the board.

[00:26:14] Jim: We've had it with, ITOT, but I consider ITOT, risk management. All these guys need to expand outside windows of what they think they're responsible for and maybe change behaviors, within an organization so they can confront it appropriately.

[00:26:29] Jim: But yeah,

[00:26:30] Dino: I think some of them are just in a CYA mode, just trying to cover their own, assets from either bad decisions that they've made or decisions that they're not making at all, in my view, Yeah, maybe it's because we're just a little bit more tactical and strategic on the area of focus that we have.

[00:26:44] Dino: so how do you get through? It's like that? Oh, what do you there's the Brueggemann Meyers or Briggs? What is it with the mindset of the person? What's that called?

[00:26:53] Jim: Dunning Kruger, that's it,

[00:26:54] Jim: Actually, we should take the Dunning Kruger and apply it to all the different disciplines in the organization. then say, here's the characteristics to, is this recognizable in your organization? That would be fun to do.

[00:27:06] Dino: Yeah, that would be a good podcast leading off with that.

[00:27:09] Dino: Put that back front stage and center because I think that's part of it.

[00:27:12] Jim: they think that they're on top of it, because I didn't get here because dummy.

[00:27:15] Jim: this is just a different discipline, different world and you got to change.

[00:27:20] Jim: what your scope is to, and everything that goes with that, ultimately, Hey, Dino, the government stepping in. So you want to be proactive as opposed to reactive. You don't want to be the guys to say, Hey. they came in and shut us down, but it was for our own safety.

[00:27:35] Jim: don't want to be that guy. All right. we're wrapping this thing up, right? Unless you got anything else, I'm sure we're over time. Appreciate everyone for joining us for this podcast until next time, thanks for joining.