The CISO & Talent Crisis: Turnover Meets OT Cybersecurity Gaps

[00:00:00] Dino: you need to pick up the phone and find out who your top OEM and sis are that you spend millions and millions and millions of dollars with every. Every year potentially that are responsible for the design and support and maintenance and warranty of your control systems and the machine centers out there and find out what they're doing from a cybersecurity perspective.

[00:00:18] Dino: what is your roadmap, ciso, CIO, that you're wanting to implement, and then you have to bring these folks in. You gotta talk to 'em, right? People know who they

[00:00:27] Craig: for sure. Hi.

[00:00:59] Craig: I'm Craig Duckworth. Welcome to the podcast. Today, what do what you got on your mind today? Dino, what are you thinking?

[00:01:06] Dino: I think we ought to talk a little bit about this recent Gartner report that came out, the article they published about the number of IT cybersecurity jobs that folks are leaving. Like 25% will be leaving the field over the next year or two. Crazy.

[00:01:20] Craig: that's a big number. I think that'd be a great topic. you know, as we go through this, how does that work into the OT space that we play? What's the repercussions or the ramifications of. Of leadership that's leaving and moving around and the turnover and the, let's call it the ramp up time for new organizations they go into in the same field and let's maybe the loss of productivity around that entire churn and burn cycle.

[00:01:48] Dino: Well, you know, and, tenure of CISOs we've known for years is the high, the turnover rate, I mean, is pretty high once their tenure. It's less than two years the job, so, Why is that? and obviously the article touches on a lot of that stuff, but just the idea that to me that means that the CISOs probably don't have the level of influence.

[00:02:10] Dino: That they need to have within the organization to really make some real hard changes within the environment. it's like, you don't want to equate it to what's going on with the federal government right. At the Doge level, but at the same point, you've got these environments that are slow to change in the realm of cybersecurity and the improper investments, and it's already hard to find talent.

[00:02:32] Craig: Very hard. Well, you got the talent piece. You've got, what's the level of support within the organization Is there, do you have the support you need from your team, from your executive leadership? And I would say that I'm starting to see a little longer. I. Tenure. As organizations are becoming more and more mature as they are beginning to embrace the role of a CISO and trying to keep them there, maybe that it plays into this turnover that we're seeing where, they take one step forward, CISO leaves, they take two steps back.

[00:03:05] Craig: You know, maybe they're trying to keep it going a little longer and give them more, let's call it rope. Maybe not to hang themselves, but a little more rope to do their job. You know, I don't know.

[00:03:14] Dino: Yeah. You know, in larger organizations you think about the role of the CSO and especially when it comes to ot, cybersecurity, and if you don't have the background or experience for it, it's in a really good understanding of what, how those plants operate and the type of control systems they have and their processes and.

[00:03:31] Dino: the skill sets that are within those facilities, especially at an engineering level, process level, facilities, production, it's almost like you think that there should be an OT cybersecurity role, which we've seen a lot of clients set up, There are OT cybersecurity positions that have been developed as roles within these companies who they report to can vary.

[00:03:52] Dino: but at the same time, it's back to that think globally, act, locally mindset that you have to have in order to be successful because the level of technology around these control systems that are using these plants vary greatly, It's not like they're highly standardized, as far as the control systems, specifically the software run the SCADA systems or the DCS or the motor control centers, et cetera, et cetera, et cetera.

[00:04:15] Dino: It's hard for it to wrap their mind around what's going on out there. And at the same time, OT people aren't thinking about cybersecurity as being their role and responsibility.

[00:04:22] Craig: well, it goes back to, and we say this every time, it's who owns the assets in the IT infrastructure. Look, I would say very consistent. they may say, we want Cisco switching and hardware throughout our infrastructure. We settle on Palo Alto firewalls throughout our infrastructure.

[00:04:43] Craig: We leverage, Juniper, miss from an ap, access point throughout our infrastructure or whatever the technology is in the OT manufacturing space, if they've got 10 plants, they've got 11 variations amongst those 10 plants. Because even if they were to have a standard and a.

[00:05:05] Craig: template that they use to make those, they're all different based on the OEMs, the manufacturers who's brought technology in, what they're using, what they as an OEM settled on that was interjected into that. Maybe one area is running Rockwell, one area is running Siemens. Maybe they've got an Emerson piece over there.

[00:05:24] Craig: There's not consistent and it will vary by plan, and that's not something that it is accustomed to very often.

[00:05:34] Dino: Well, and, in all fairness, when those machines centers were developed and designed, it didn't have a. They weren't invited to the table, I guess is what I'm trying to say. Meaning that here you're gonna put in a new $20 million packaging line. , it's gonna take a year or two to get this thing in and done.

[00:05:52] Dino: And as you're starting the front end engineering design phase and you're building out, what the client's asking for and scoping it, there's not a conversation with the IT leadership among. What's the network gonna be like within those control systems? At a control level, right?

[00:06:05] Dino: We're not talking about what the IT network. We're talking about the control network and the types of PLCs and the software that's installed in those PLCs that IE the firmware or the SCADA system. You know, picking Rockwell versus deductive automation versus going maybe with a GE line of product or whatever.

[00:06:20] Dino: Those decisions are not normally made with it's participation, so they're already on the outside looking in, and I'm not saying that that's right. It's just that that's what we witness. I mean, our company alone rarely has it leadership sitting in, large plant build in regards to the process side of the house

[00:06:41] Craig: I equate it to Thanksgiving dinner. The OT team has always sat at the kids' table and as they are growing up and maturing, they are being invited to the adults' table where they can have these conversations because it's becoming more relevant. It's, important as the leadership, the mom, the dad, the aunts and the uncles listen to the children.

[00:07:03] Craig: The manufacturing that are growing up, maturing and having relevant conversations about here's what we do and why it's important.

[00:07:13] Dino: Well, you said ot people were sitting at the table.

[00:07:16] Craig: Oh, yeah,

[00:07:17] Dino: Yeah, just who generate the.

[00:07:19] Craig: I know. Who generates the revenue.

[00:07:23] Dino: You know.

[00:07:23] Craig: no, for sure. It's cost center versus cost generation. Yeah.

[00:07:30] Dino: ' cause some of 'em carry a bigger stick than others. Right. We've seen that too, to your point. and what's best is having those two groups align and collaborative and transparent, is the best of both worlds. You know, where we're working in sync and quite frankly recognize where you are short on resources to really get to where you need to be.

[00:07:48] Craig: we hear it all the time. I got a great dose of that this week. I was at a client. hear direct from the client team that's managing their control space, their kind of their IT support and the plant manager about, look, anything that we do that Impacts our production, meaning unscheduled, unplanned downtime is a financial hit to them personally and to their budget, their revenue, their what their commitments are to the organization.

[00:08:20] Craig: And there's still a high level. Of let's call it reluctancy and uncertainty about what's being asked 'cause of the unknown of what it's gonna do and the implications it can have to them and their own personal financial self wellbeing. So it's still there.

[00:08:41] Dino: Well, yeah, I mean, especially with the OEMs who can be very difficult on allowing you to, add things into that machine center that are beneficial from a cybersecurity and or safety perspective, and getting them to sign off on that without avoiding the warranty for support. that is a balance.

[00:08:56] Dino: and the OT teams have come accustomed to relying on that OEM to fix that machine or determine, you know, what's going on wrong with it. in a very, expedited fashion, To your point, you gotta get back up and running 'cause it's costing money when that machine's not doing what it's supposed to be doing.

[00:09:10] Dino: and it is usually not tied to that, a lot of times they don't wanna be tied to that in all honesty, because of The critical nature, of what's going on inside that plant with those machine centers, right?

[00:09:20] Craig: Yeah, for sure. You know, but if you go back and you look at that Gardner report that we started with, and you look at the turnover rate and you look at how does that play into the already talent shortage that's in the space that we're operating in. it really goes back to choose a partner, choose a.

[00:09:39] Dino: Service provider right.

[00:09:41] Craig: provider that can walk with you down this road, that can provide these services that you trust, that understands your environment, that is competent, that can help you address risk. Address unscheduled, unplanned downtime. Help you build budgets for these tools, give you options for what's available that's, purpose-built unique for the environment you're looking at, and can really bridge that gap between the IT and the OT side, and help both sides understand the why behind what the organization's trying to do.

[00:10:13] Dino: All good stuff. I said, if this gap is true, what Gartner is saying, these organizations, they're gonna be, I don't wanna use the word forced, but they're gonna be pushed over into looking for managed service organizations to help 'em out. People who wanna work in this space as cybersecurity professionals, but wanna work for organizations that treat it as a priority.

[00:10:34] Dino: And that's why I think a lot of the CISOs that we see that, that the turnover rate is, if it's not a priority to the organization,

[00:10:41] Craig: Then they're not getting the support. They're not getting the, funding. They're not getting anything

[00:10:45] Dino: Dry, either. They don't wanna, they don't want the fingers pointed at them if something that bad happens, Because you wouldn't implement this policy or this procedure, or put the budget together to look for technologies and solutions or partners to really shore up these gaps that we got. I mean, the CISO that's recognize where those gaps are.

[00:11:01] Dino: But at the same time, if they're not been able to influence, you know, executive leadership in investing appropriately and properly, that I can see why they wanna move on. 'cause they don't want to be the one, hanging on the nail if something bad happens. Well, but a lot of times if, for example, if I'm a CISO and make my case, I build a. Plan and a budget, and I talk to my executive leadership, the CEO, and he says, Hey, Craig, that's a great idea. Come with me next month and let's present this to the board. The board says it's a great idea.

[00:11:31] Craig: Here's X amount of money, $5 million, pick a number, whatever it is, go implement that. Again, it gets back down to the where do they, how far can they go if it's incorporating their OT space before they kind of get, let's call it knee capped, and it gets stopped and delayed because they don't have the buy off of the plant management.

[00:11:53] Craig: They don't have the buy off of the controls engineers to go in and do whatever it is they're trying to do. In that space that the IT teams are not familiar with, and then the budget is overrun, the timeline is delayed, and then next quarter, next year, the CISO is saying, Hey, well I know you gave me this money and I'm not getting as far as I thought, so I need more money.

[00:12:16] Craig: And then where does that process stop?

[00:12:19] Dino: Well, it's the value, right? so maybe one shown the value,

[00:12:22] Craig: Yeah.

[00:12:23] Dino: especially those that have invested in tools to go down this path, specifically to OT cybersecurity. But at the same time that OT ecosystems need to, needs to demonstrate why. Having this information, even if you just looked at the five, sans critical controls, defensible architecture, incident response, vulnerability management, remote access.

[00:12:43] Dino: I'm probably missing one of of them. So,

[00:12:45] Craig: recover. Yep.

[00:12:46] Dino: yeah. so what is your plan around that from an OT perspective? so that we recognize that, if the client. Isn't really generating, a strategy around cybersecurity, then the OT teams need to bring that to the table, They need to demonstrate that we see value in it, and here's the reasons why we see value in it. You should also client, see this because we need this as part of the work that we're doing for you. Right. I'm building this packaging line for you. I've got a tight time startup, you know, I gotta get this machine up and running within a year, And we start building it out and I need to know, what can impact or is impacting my ability to do that and what risks are associated within that environment. Not only with the stuff that I'm bringing in, but the stuff that's already in your plan. Right? I wanna know if you got malware in your plan.

[00:13:32] Dino: I wanna know what the remote access looks like in there. I wanna know, you know, what it is doing in regards to, doing their due diligence, but what an impact it may have on my control systems that I'm programming and setting up for you. So, there has to be a two-way conversation.

[00:13:48] Dino: , and right now we don't see that. And, the OT organization needs to say, here's why I'm asking for this, or Here's why I am bringing these tools in, and here's the value and benefit of having this information and this technology, and within this environment.

[00:14:01] Craig: Well, and part of it, Dino, is, you brought up a great question. we see malware in this environment, you know, and I know we've gone in. We've implemented a tool or technology, and within 30 minutes we find malware in that OT environment, either sitting on a backup or sitting on a device or sitting somewhere.

[00:14:20] Craig: And when we ask the OT teams, Hey, wait, let's, let's pause a second here. We found this malware active in your environment. what do we need to do to address this right now? You know, and I know the response is, it's not our problem. we have no idea. We don't know who to ask. We don't know what to do.

[00:14:36] Craig: It's not our issue. And. Please proceed with what you're doing. We're like, well, hey. that has to be important to someone. There is someone in that organization that would say, wait a minute, stop, step back. We need to do steps one through five. We need to do these things to remove, eradicate, fix that issue before we get too far.

[00:14:57] Craig: But they just don't know. It's, the right hand doesn't understand what the left hand is doing, and neither of them are talking to each other.

[00:15:04] Dino: Like, well, they get that deer in the headlight look right. When that happens. and to Craig's point for our listening audience, if we go into 10 facilities, we're gonna find malware. And two of them, we're gonna find rogue assets in all of them. Right. that's speaking from experience and been doing this for many, many years, that's just the nature of the beast, right?

[00:15:23] Dino: And we uncover this stuff that's new information, not only to the IT who didn't really wasn't paying that close attention anyway with control system environment, but even the plant people. Are surprised by some of the things that are found, in their environment that they did just didn't know, was there, didn't believe that it was, there don't know why it, it's there, but we're finding this stuff again. Rogue assets in every plant. Remote access in every plant, warranted or unwarranted, that's being uncovered at malware. You know, sometimes a lot of misconfigurations, we uncover misconfigured, assets the way that they're communicating. So, whether it's at the switch level or at the firewall level, or even some module that they're using to talk between control systems, whether it be ProSoft or some pyramid integrator, which is going back a number of years, but there's technologies out there that span decades.

[00:16:12] Dino: These OT assets are out there for a while.

[00:16:15] Dino: Their life cycle's much different than the in the IT space. So you have to, again, think globally, but you gotta act locally for all these reasons we're describing here.

[00:16:24] Craig: Yeah. I look back and I say, okay, how can I as Systems integrator or a consultant that understands OT and understands these tools and technologies, go in and provide value, provide something to an organization to help them. And I, pull this back around to the ciso.

[00:16:46] Craig: It's leaving and this 25% are getting a new job and they're all moving around and how does that play? and part of it is, they don't understand the makeup of that OT environment. They don't recognize. and some of them probably very well do recognize, but not all of them. Yeah, high level.

[00:17:06] Craig: But they don't realize that there are multiple layers of networks underneath that top level PLC that they're not seeing because of the NIC cards that are in those devices where OT for. OT for a very long time has used those as network segmentation devices to keep it out of their hair, let's just call it what it is, because of the inherent distrust.

[00:17:30] Craig: And when a CSO or an executive walks down on that plant floor with an OT team, lead controls engineer, myself or someone equivalent to me, and they say, Hey, this. Windows seven machine that's running this, is connected your network. They're like, well, why is it on my network?

[00:17:51] Craig: Well, because it's managing this asset, this packaging line, this process sell, whatever it is, well take it off. You do know that that's gonna shut your plant down, I can turn it off, but my production is gonna stop. And you are gonna have to report to the CEO. Why We stop production So that the they?

[00:18:11] Craig: Yeah. Which is never ever gonna happen. No, but they don't understand. No, the plant's not gonna do it. They're never gonna do

[00:18:22] Dino: pivot, right? Then you gotta pivot to, well, how do I micro segment that thing? How do I put virtual patching around it? there's a lot of strategies that can be used in order to deal with that, and that's where the IT teams are at a loss on what it would take again, to secure and protect those connected physical systems.

[00:18:41] Dino: And we say connected physical systems. We're talking about data, driving physical outcomes. That is purely in the hands of the OT organization to deal with that because they're the ones that most, intimately knowledgeable of that machine operation. They may not know everything that's going on within that infrastructure and that network, per se, but what they do know is what it takes to run that machine

[00:19:03] Craig: Well, and they're all working. they've all got full-time jobs. they've got 10 tasks and they're working as much as they physically can and doing everything to the best of their ability. And the last thing they want is to try and take on something. That one is. designed to give someone outside of them an outcome that's not relevant to them, that they don't believe, and then they may or may not understand the why behind it.

[00:19:31] Craig: Back to that localization and OT adoption that we talk about. How do you help Get organizational OT sides to understand the why. What's the benefit to them as a team and what's the value? You know, tell me, InfoSec team, what data do you need? That I can give you. So you stop scanning my network and killing my PLCs every time.

[00:19:59] Craig: Just tell me what you want. I'll get it to you in a different fashion, but please stop doing what you're doing because the implications it has. But if they don't understand the why, if it's just because I was told to, as a kid, that was the worst answer in the world. Hey Craig, do this. Why? Because I said to, but that does not work for me.

[00:20:22] Dino: You gotta show it to 'em, right? You gotta demonstrate the impact, And if you don't have the right tools and technology in there, that's hard to do. that's where we get back to the ghost in the machine, I don't know why my fillers slowed down or stopped it. Just hit it.

[00:20:33] Dino: And by the time we cleaned up the mess and reset everything and got QA to give us a thumbs up to run again, and I've lost three hours. I don't know why. I haven't been able to, attach that downtime to anything specific that that's popping out at me. if I can't find a mechanical issue, a motor burning up, or filler injector, getting clogged up or whatever, if I can't point to that.

[00:20:57] Dino: I wanna do is get back up and running. I'm not really going back to trigger and try to figure out the why, especially if I don't have something there that's collecting information. Tell me the why

[00:21:04] Craig: correct.

[00:21:05] Dino: and if I go to it, it, we didn't do anything.

[00:21:09] Craig: but these tools and technologies are gonna give them that information through the OT side. They're gonna say, oh, here was a scan and here's what it did, and here's the IP address of the IT laptop that kicked this off, or whatever it was. So now they know, and it goes back to what we say all the time.

[00:21:27] Craig: We are being marriage counselors between the two teams to help understand that. Again, to have a conversation, Hey, I see that you did this. just so you know, here's what it did to my environment and my cost of downtime because you did that. Tell me what you were trying to get and let me give it to you.

[00:21:48] Dino: Well, classic example. we witness this now, as a system integrator. we don't design in EDR tool sets into the, C applications. And there's a reason for that, Somebody has to manage that EDR package and that EDR package can be very, very disruptive. based on, what kind of controls are being in place in a one or a crop start type install in my HMI environment, somebody has to manage that.

[00:22:13] Dino: And again, thinking globally versus acting locally because to your point. It will come in and say, oh, we got a compute infrastructure out there. It's got all these VMs on it. let's implement, CrowdStrike because. OT doesn't weigh in on that per se. It just did it. Then you have, a security program, any virus program sitting out there, running in these hmis that are critical to the process that when they fire off their scans or their updates that they receive.

[00:22:40] Dino: All of a sudden that HMI is becoming distracted. it's not communicating with the machines like the PLCs in a more timely manner. And those machines, if they lose communications, for a period of time, even a short, very, very short period of time, they will fault, they will go safe. because if they can't be communicated with in a proper time or didn't get their instructions in the right time, they will take a different action.

[00:23:02] Dino: And, and then of course the unforced error there is you end up with an unplanned, unscheduled downtime event, so you wanna be able to track that through. And again, that's where IT and OT aren't working well together when it just automatically pushes down an EDR package onto a compute infrastructure and a, and a manufacturer facility because they want to add AV and other cybersecurity, functionality into that environment

[00:23:24] Craig: Right.

[00:23:24] Dino: really testing it against the C applications that are in there running, because timing timing's important.

[00:23:29] Dino: You need to know when might be downtime in that area in order for you to run what you need to run from that EDR package and get your input back from that. you don't wanna do it during run times. You don't wanna do certain changeovers or critical steps process. You don't want that EER package firing off and running, doing what it's doing. You gotta tie that back to certain communication flows, which ones are warranted versus which ones are unwarranted. 'cause those EDR packages see things they don't recognize sometimes with these industrial protocols. And they will, they will prevent it from running,

[00:24:05] Craig: Great Totally get it. Hopefully, hopefully the audience understood and can kind of take some, feedback from this. Go back within their organizations, understand kind of where they are. ask some questions. Walk down on the plant floor, pick up the phone if you're a CISO or CIO and call the plant manager or call your director of VP of engineering.

[00:24:26] Craig: Take a tour and say, Hey, help me understand the business unit, what you're doing, and the implications of some of the things that we're doing. Here's what we are thinking. how would that. Impact you and your team from an operations and have that start having some collaboration it's gotta start somewhere.

[00:24:46] Dino: Yeah, and CIO, you need to pick up the phone and find out who your top OEM and sis are that you spend millions and millions and millions of dollars with every. Every year potentially that are responsible for the design and support and maintenance and warranty of your control systems and the machine centers out there and find out what they're doing from a cybersecurity perspective.

[00:25:06] Dino: what is your roadmap, ciso, CIO, that you're wanting to implement, and then you have to bring these folks in. You gotta talk to 'em, right? People know who they

[00:25:15] Craig: for sure.

[00:25:16] Dino: And find them. If you look in your, payment system, the companies that you're spending, like I said, big dollars with who are responsible for those, that equipment in your manufacturing facilities, you need to have a relationship with them, To some extent, you should be pursuing that. and if you're not interested in doing that, then there needs to be somebody that steps into take that role.

[00:25:34] Craig: Yep. Totally, totally agreed. Great conversation. Dino. Thank you very much. I appreciate your time. As always, you guys have a great, great day out there, and remember, stay safe.

[00:25:44] Dino: Happy Friday.

[00:25:45] Craig: Yep.