Eric Knapp: Why OT Cybersecurity Is A Journey

Carlos Pazos (00:02):

Welcome to Forging Connections, a podcast from Honeywell about the convergence of IT and operational technology for industrial companies. We'll talk about the future of productivity, sustainability, safety, and cybersecurity. Let's get connected. Hi everyone and welcome to Forging Connections. In this episode, we have Eric Knapp, who is our Director of Cybersecurity, R&D at Honeywell Connected Enterprise. Welcome Eric.

Eric Knapp (00:31):

Um, thank you for having me Carlos.

Carlos (00:32):

So Eric, as I mentioned, you are the Director for Cybersecurity Research at Honeywell Connected Enterprise, but just for our audiences, could you please tell us a little bit more about your role and actually what brought you to it?

Eric (00:48):

Sure. My role is actually kind of an interesting one, at least I think so. Cybersecurity products designed to help detect and protect against threats depend a lot on our understanding of the current threat landscape, obviously. And that landscape changes continuously. At the same time, detecting threats against OT requires an understanding of process control. So I have actually a pretty fun job. I get to run a team that covers that whole life cycle from identifying new threats but with a particular focus on OT all the way to inventing new ways to help protect against those threats. And then obviously working very closely with the product teams to get that technology into the hands of our customers. You know, in terms of what brought me to the role, that's a long story, I'm getting pretty old, so <laugh>, I'll try, I'll try to make it quick.

Eric (01:41):

I started my career in cybersecurity, actually in the telecommunication sector. There was talk of industrial cybersecurity back then, but the term OT didn't even exist. I found the concept though of using computers to make changes in the physical world something that was really fascinating and also a bit scary. And about 15 years ago now, maybe a bit longer, the Idaho national laboratory demonstrated a cyber-attack capable of destroying a generator, you know, so a cyber-attack destroying a physical piece of equipment. It's what was known as the Aurora project and it flipped a switch for me, no pun intended. And it, you know, kind of changed a hobby into a real career pursuit. Basically, it scared me and as I dug in and learned more about how easy it can be to cause physical damage with digital methods, I've kind of dedicated my career to helping in any way I can. In the early days, it was mostly about just getting the word out. That's improved a little bit. And so I've been able to switch my focus to trying to get ahead of that threat. And as I mentioned, you know, trying to develop new technologies to help protect against them.

Carlos (02:49):

Gotcha. Eric. So you have been driving improvements in the industrial cybersecurity, uh, area for a long time now. So how have things changed in that time?

Eric (02:58):

Oh, it's gotten so much better. You know, in the, in the early days back before we even called OT OT, before the days of Stuxnet and some other high profile cyber-attacks against critical infrastructure, I was actually laughed at on stage trying to promote the concept of OT cybersecurity to industrial process control operators. I had someone raise their hand during the Q&A and ask me if this was really what I did for a living. <laugh> It has changed an amazing amount since then. There's pretty much a blanket acceptance of the need now. However, there's still, you know, a real challenge in how we fulfill that need. OT Cybersecurity is still far behind what we think of as IT cybersecurity or traditional cyber. And, unfortunately the threat keeps advancing so we can't really afford to be behind so that's why we try to find ways to accelerate that and bring the industry back up to where it needs to be.

Carlos (04:03):

I see. So why don't we now, do a little bit of a deep dive when it comes to all this cybersecurity. Do you care, please explain, for our audience what this is and why does it need a special consideration?

Eric (04:17):

Um, sure. So, you know, IT security or really information security is about protecting information, right? Very aptly named. If somebody hacks a bank, you know, they can steal data, they can move numbers around ensure there's loss and that loss, you know, primarily of intellectual property or financial data can really impact people. So, I'm not diminishing that in any way, but in an automation system, if something gets hacked, it could cause real physical damage. It could shut off critical components of our infrastructure – electricity, water, transportation. It could kill people. It could cause, you know, massive destruction, environmental damage, you know, all sorts of potentially catastrophic outcomes. So, when it comes to consequence, OT cybersecurity has a lot more at stake. At the same time, the environments are incredibly different. At the heart of it, they often will use the same technology.

Eric (05:24):

You know, a computer used in OT and a computer used in IT are both probably windows computers running on, you know, common computing hardware, but they function very differently. How they are used is different. And in an OT system, those computing platforms are primarily used to maintain a finely balanced, real-time, industrial process control system. They operate in real time. They're closed loop operation, and they're very, very sensitive to disruption. So while if you shut off, you know, an SAP server or a, you know, an ERP or some sort of, you know, business database at an unexpected time, you might have some angry users who are trying to access those systems and can't get to them. But if you were to shut off systems in a process control environment, you could be preventing an operator from being able to monitor a complex control system.

Eric (06:28):

You might prevent them from being able to make changes to that control system which obviously puts the system at risk. And in the worst case scenario, an attacker could be clever enough to actually manipulate that system or cause disruption of it at the same time. So again, while there's similarities, the approach to it is different in that you need more consideration of the target system and the consequences of failing to protect that system. You know, the consequences of a successful cyber-attack are, or, you know, can be pretty extreme.

Carlos (07:06):

Yeah, I think you're, you're absolutely right Eric, like when it comes to industrial environments, these cyber threats truly can come from anywhere, like in any sort of environment at any time. So, can you expand a little bit more on like the biggest risks and consequences of not taking action here?

Eric (07:25):

You know, as I said, the biggest risk is that there would be some sort of a physical consequence. You know, realistically, it could be a production error that decreases yields or impacts quality. Or it could be something that could, you know, as I said, cause actual physical damage or harm to people or in the environment. If you are operating machinery, that is for example, you know, handling the flow of molten plastics or metals in some sort of, you know, injection molding operation or something, or you know, a lot of, you know, if you're rolling sheet metal or something… If there's an unplanned outage that loss could extend a lot further beyond just the fact that production is down for a while, right? If molten material hardens in places it shouldn't harden because the system's not running the way it should, you know, you could be in the position of having to repair or even replace parts of your manufacturing infrastructure.

Eric (08:18):

Obviously, it makes recovery that much harder. And, you know, during any example of where there's disruption to OT, you know, there's the potential on the business side of really altering or halting your ability to produce a good, right? And the production of that good is what manufacturing organizations use to earn revenue. So, you know, there is a real business impact as well as the potential to cause harm. You know, we've seen that in the real world. Obviously, you know, in just recent history, unfortunately there's been some examples of cyber-attack against industry where we've seen that. The colonial pipeline is a great example because the cyber-attack actually didn't manage to infiltrate the industrial control system at all, but it did take it offline because as a precaution, those systems were shut down to prevent, you know, putting them at further risk and as a result, you know, pipelines were offline. And, you know, there was a real impact to, you know, the delivery of gas on the Eastern United States. And, you know, as someone who lives in the Eastern United States, I definitely felt that.

Carlos (09:33):

Gotcha. And that's quite interesting, right? Like, starting the conversation from the perspective, like A – what is the risk and the consequences, but at the same time when for our audience, like, as they're navigating this concepts of cybersecurity, it's not something that typically they can relate to a specific terms or concepts like we hear terms like secure media exchange, things like PCN hardening truly for organizations listening these podcasts. Why are these things are important? And, how does this relate to the organization's cybersecurity efforts?

Eric (10:14):

Um, okay? I will, that's a big question. I'll, <laugh>, I'll try to break it apart and answer it. So, you know, we've talked about the consequences of OT having an incident. The good news is that in OT, there are actually very few vectors where a threat actor can try to penetrate and compromise the system. So, in terms of cybersecurity for, you know, those in the audience who aren't familiar with cyber terms, there's the concept of an attack surface, which is how sort of broad the target is. And there are concepts of attack vectors, which are the specific paths you take to try to get to that target. In OT, the attack vector can be very large and the reason, I'm sorry the attack service can be very large.

Eric (11:07):

and the reason is because sometimes, most of the times, unfortunately, the systems have been in place for a long time, patching is very difficult and because of that, the presence of, you know, hardware and software vulnerabilities is almost inevitable. But luckily the vectors are few. Industrial control systems historically have been separated physically – they've been behind air gap. So, you had to physically enter a facility, you had to physically be standing in front of a control console in order to interact with that system. You know, valves were closed and opened manually, and so physical separation was all that was necessary. As things evolved and became digitized, the air gap kind of went away. Conceptually, it's still there, but there's a need to move information between process control and the business network and the outside world and so there are allowed paths of communication, referred in the industry as conduits, that make the air gap concept a little less effective.

Eric (12:17):

But because of that, again, we have, we have a path through on the network. Hopefully, a very narrow path and a very well defined path. We have physical access, so somebody could carry a laptop or, you know, a USB thumb drive or something into the system. And really, the only other vector besides those two is the supply chain. Which is that something enters the facility, either software or hardware, that has been manipulated before it, you know, was taken into ownership and deployed in the infrastructure. In terms of cyber, that's actually good news. You know, in IT cyber, there are so many vectors that's almost impossible to keep track of them all. The most popular and the most successful vector, still in the IT world, is fishing. You know, sending an email to someone and hoping that they, you know, open that email and click a link that they shouldn't and effect a system that way. In OT, there's no access to email. So phishing can still be an effective, sort of, first step to compromise an individual or a machine that is connected to the internet, but then there's another step that has to be taken to get into process control. And that, again, there's not many choices left, right? Find a way in to pivot over the network or find a way to have your digital presence carried in physically.

Carlos (13:45):

Eric. So, so that gets me thinking like you are delivering a little bit of good news, right? In the sense that there's this limited amount of attack vectors in OT, but at the same time, like this sounds like a little bit of complexity, right? In terms of like the emerging threats and obviously industrial organizations that need to get ready for that, right? So, how do they prepare?

Eric (14:11):

That's a great question and, you know, when you think about how we prepare for something like this, the word to focus on is we because it takes a concerted effort from everyone in the industry. Certainly, industrial control operators need to take steps to help protect and, you know, design resilience systems and protect against cyber-attacks. We, as control system vendors are responsible for trying to find ways to make, you know, the devices that make up these systems more resilient and to find new ways to detect these threats. But, none of it will work on its own, right? And the operators who run the world's critical infrastructures are all doing the right things. The most important thing to remember is that security is a journey. It's a life cycle. You know, it sounds kind of cliche, but it's true.

Eric (15:06):

You need to know where you are and you need to know where you need to be, where you ultimately need to be, and also where like your next step needs to be. And it doesn't matter what stage of that journey you're on. I mean, if you are, you know, if all your control systems are directly connected to the internet and you don't have any monitoring, you don't have any staff that's trained and, you know, you're way back at the beginning – that's fine as long as you know that you have to start taking those first steps. So, the first step is really figuring out what you need to do. And there's certainly ways to get help in figuring that out. You know, assessments are a great place to start. You know, you mentioned hardening, the systems deployed in industrial control often aren't hardened.

Eric (15:54):

So, if they aren't, then that's a challenge in and of itself, right? You can't just say, okay, I'm gonna harden my systems and push a button. You have to figure out where the vulnerabilities are, what systems need to be changed and how, and when that can happen in terms of, you know, things being on process or off process. And, you know, there's a lot of complexity involved around this, but there's a lot of people available to help, you know. And at Honeywell, a lot of people don't even realize this, but, you know, we have hundreds of dedicated cybersecurity professionals around the globe that do nothing but industrial cybersecurity – highly trained, skilled people who are available to help with this journey. And as customers get a little further on that's really where, where I come in. And I'm, you know, again, my team is trying to innovate new ways to detect and protect against cyber threats.

Eric (16:53):

So, as you become more mature and your security posture improves, then that's when you need to think, okay, how can I make it even more difficult for an attacker to succeed? There's a very tried and true tenant of cybersecurity, which is defense and depth. There's never enough cybersecurity, add layers, if you think you're protected and you think everything's good, then, you know, have a pen test or do a red team, blue team exercise to see if you’re really as secure as you think you are and then find a ways to make yourself a little bit more secure because the bad guys don't rest. You know, as I said, the threat landscape is continuously evolving. We see that in, you know, in our jobs day-to-day. Certainly in the world today, there are motivations and there are high, highly skilled nation state actors that operators of critical infrastructure need to be concerned about. So, it's not necessarily a doom and gloom story. Like I said, there is a path that you can take. Figure out what that path is and start walking and Honeywell is there to help and there's a lot of resources in the industry there to help to get you along that journey.

Carlos (18:17):

Yeah, I would agree with that is the focus on the we, right? And, how everyone is responsible. Eric, could you help us describe a little bit more in terms of the roles of the market and governance and how this is affecting or driving the strategy of organizations?

Eric (18:40):

So, there are some regulations depending on what particular industry you're in. There may be compliance requirements mandated by industry organizations or government agencies on how to

improve cybersecurity of critical infrastructure in particular. There are a lot of industries that are classified as critical infrastructure. And obviously if electricity or other energy resources or clean water, you know, these are the types of things that if they were impacted could affect society, right? So, governments do have, you know, there are some regulations in place. My recommendation personally, is that you should always consider these regulations as a starting point. It not my intent to try to discredit or downplay the need for these regulations, cuz they play a very important role. But in the process of becoming a standard, anything is gonna get a little bit watered down.

Eric (19:51):

So, treat your regulations as a baseline. And again, it's not a I can check the box and I'm done – cyber never stops. You have to always keep moving. So, use that as a baseline and then from there, figure out how you can improve your cybersecurity posture even more. And it's not always just about spending money and buying shiny objects and, you know, implementing new technologies. Awareness plays a huge part. Training plays a huge part. In industrial cybersecurity, we use this analogy a lot. If you walk onto a manufacturing floor and you see somebody who isn't wearing a hard hat and they are marching boldly towards, you know, one of these, slanted, yellow caution tape lines on the floor and they're taking a step over it, somebody's gonna reach out and they're gonna grab that person by the shoulder and yank them back and probably give them a talking to and ask why they're not wearing their safety gear and why they're ignoring safety markings.

Eric (20:55):

It's inherently trained into the industry. Safety is incredibly important. But cybersecurity hasn't gotten there yet. We want the industry to behave the same ways in terms of cyber threats. If you see somebody who is about to insert a USB thumb drive into an HMI or any computing platform, really, in an industrial control environment, you want somebody to do the same thing. You want them to reach out, grab that person by the hand and say, what, what are you doing? Where did that come from? Are you sure it's clean? Are you authorized to do that? Has it been scanned? And that type of thing. We're not there yet, but we're getting there.

Carlos (21:34):

That's an amazing analogy. And I like the part that you commented of it's all about learning, right? It's about, like, moving towards that direction. When it comes to that Eric, where can people learn more about what Honeywell is doing in this area?

Eric (21:52):

You know, if you wanna learn more about Honeywell specific cybersecurity solutions, there's a lot of information available online at becybersecure.com. Again, there's a lot of information out there.  I know there's some pretty good books on the market <laugh> that people can read. The author of one said book might be familiar to listeners because it's me, but there are actually a lot of resources out there. I started this journey, you know, my own path towards OT cybersecurity a long time ago and at that time I was one of a very few people who were focusing on this. There are a lot of us out there now. There's certainly a lot more, if not, you know, if not a lot. So, I encourage everyone to learn more about cybersecurity, even if it's not your specific job responsibility or your specific role, learn about it.

Eric (22:48):

Everybody is involved in cybersecurity whether they know it or not. You know, there's sort of an adage in cyber that the weakest link in cyber defense is people. And that's very true. I, as a hacker, I don't have to breach your network and overcome your security controls and break through your firewall and the things that you see in the movies. All I really need to do is find somebody who already has access and already has the authorization and the know how to make a change and then trick that person to do what I want them to do. So, even if you don't think OT cyber is your thing, right? If you work in process control, if you work in a mill or a plant or a manufacturing facility of any kind, go to becybersecure.com. Read up on some of the challenges and some of the solutions that are out there to meet those challenges, because you will help by becoming more informed and you can help other people become more informed. And that is one way that we'll all succeed.

Carlos (23:50):

Awesome. Thanks a lot, Eric and for sharing all these insights. It was a fantastic episode and yeah, definitely, stay tuned for future episodes of Forging Connections.

Eric (24:03):

Thank you very much for inviting me. It was a pleasure.

Carlos Pazos(24:07):

This has been Forging Connections, a podcast from Honeywell. You can follow Honeywell Forge on LinkedIn and download new episodes from our website, Honeywellforge.ai. Thanks for listening.