Lawfare Daily: How Our Growing Software Dependence Threatens National Security

Transcript

[Intro]

Chad Serena: 40 years ago, would you have noticed if there were two cameras inside your house, one on your computer, three on your neighbor's house, one on a doorbell? You'd have said, "Yeah, would've stood out like a sore thumb." Now you probably walk into your office at any point in time and you don't even notice those things. You wouldn't even be aware that there's microphones or that somebody's phone's laying there that could potentially be recording you.

Jonathan Cedarbaum: It's the Lawfare Podcast. I'm Jonathan Cedarbaum, Lawfare's Book Review editor, with Colin Clark and Chad Serena, senior researchers at the Soufan Center.

Colin Clarke: You think about what the Chinese did with Volt Typhoon, pre-positioning itself within various U.S. utilities, transportation hubs. But instead of just immediate disruption, these kind of penetrations are more likely to, you know, precursors to a future pre-positioned cyberattack.

Jonathan Cedarbaum: Today we're talking about their new report, “Closing the Gap: Software Understanding in U.S. National Security.”

[Main Podcast]

Your report talks about the software understanding gap. What do you mean by that term?

Colin Clarke: Yeah, I would say just very briefly, that's our ability to, the, the way we've explained it, to understand, to verify to reason about software, which has been, you know, dramatically outpaced by its production and uptake and implementation. That, that's created a gap, and in our in our report we h- we have a graphic that kind of shows this.

And because of the increasing prevalence and importance of software to really everything we do, right? U.S. national security interests U.S. military, U.S. intelligence agencies, but also every civilian function you can think of there's tremendous risk that is built into this gap. And the gap continues to grow. It is exacerbated by a number of different factors. The newest monkey wrench is, is artificial intelligence and, and how we conceive that.

Jonathan Cedarbaum: Very good. So is the problem of the software understanding gap principally about the quality of software and how it's developed, or is it one about how users of software have limited understanding of how software works?

Chad Serena: I think it's a, I think it's an all, all of the above, Jonathan, in, in terms of the, not ju- on the understanding side, it's the what is the software going to do? How is it gonna perform under different circumstances, as Colin talked about. But it's also then how is it what are user expectations? What do users do with it? What do different types of users do with it?

And I see this spanning across a range of different types of users and organizations. So if you have chief technical officers or engineers or software engineers at one place, they're gonna understand this problem very well. If you have your individual users, like, say, any of us that are on this podcast right now, our understanding of how this works is going to be radically different.

And I see that, I see that in terms of thinking about the differences again between how security professionals would deal with this subject and how individual civilians would deal with this subject. But now the difference between what of those apply to the national security ecosystem, those civilians who don't have quite the same understanding of this and the way they think about, you know, the implications of how secure things are, this becomes more problematic.

So for instance, one of the examples that we cited in the report was smart home devices, so things like your thermostat or your refrigerator or elsewhere. Those could be tapped into technically, and then the IT systems that are governing those and the infrastructure that is then connected to those could be accessed through things in a person's home.

This was inconceivable 20, 30, 40 years ago, where you would never even stop to think about, is there something in my house that could conceivably be tied back to national security or maybe even just a security issue, a local security issue? Now that is certainly on the table and something that's possible.

Jonathan Cedarbaum: Got it. So, one of the recent seminal studies about this issue that you mention is the 2023 Software Understanding for National Security Initiative. What, what was that, and has it led to any practical initiatives?

Chad Serena: So the SUNS report, as we understand it, and we don't have... I can tell you this is one of those questions that we don't have a great answer for, Jonathan, but we can probably provide a little bit of understanding of this. I think of this as a, so you ask, you know, what was that report about? What was this? That was, that was generated, I think the “National Need for Software Understanding” was generated from a 2023 workshop that was held by Sandia Laboratories out in, out in New Mexico that then led to a number of different initiatives. So one would have been the ONCD report, the Office of the National Cyber Director, back to the building blocks, that report, and then another report through Sandia Laboratories, but with others on closing the software understanding gap, and that came out in June 2025.

And why I say we don't have the best answer to some of these questions like this is a lot of this is very new, not just to us as security practitioners, but also to people that work in this area. Colin and I spent some time asking different people we knew just anecdotally, "How well do you understand this concept of the software understanding gap, and what do you know about software understanding?" And a lot of people just didn't have an answer for us. They didn't know what the terminology meant.

And we're sort of there too. We understand most, more than, you know, the next guy or the average person. But at the same time, it's something that's only a couple years old and it's mostly been dealt with by people that are technically gifted and understand the subject very well and hasn't been explored in depth by people that understand the policy and security side of it.

Colin Clarke: And I would add to that, I would say, you know, this is one of those cases of we don't need to reinvent the wheel, right? So the, the SUNSEC initiative, you know, this group has already done substantial work on the topic. 2023, it ran a systematic research agenda, it published the technical roadmap, it helped coordinate the, the “Closing the Software Understanding Gap” report.

And there you had real interagency, you know, buy-in, right? You had CISA, NSA, OUSDR and E, DARPA. And so there's been a lot of work and I, you know, in, in some ways I look at our report as kind of building on the shoulders of that and bringing some of these issues to the forefront.

And one of the things that we really tried to do here was connect this tangibly to what's happening in the operational environment geopolitically. So how does this impact the United States vis-a-vis our adversaries, right? We're in an era of great power competition, China, Russia, Iran, North Korea a range of violent non-state actors and other adversaries with lesser capabilities, but that are improving each day. I mean, you know, my, my background is in studying transnational jihadist groups, and I've been spending a lot of time for the past year, year and a half looking at the lowering to barriers o- of entry into using a range of emerging technologies.

Now, I don't think, you know, violent non-state actors are at the low end of that spectrum and, and China would be at the high end, nation states. But as we've seen with Iran, right, and their proxies, this kind of tacit knowledge transfer to a range of terrorist and insurgent and militia groups can be quite effective. It extends the battlefield in many ways, and it keeps us, i.e., the United States, busy at places further afield.

Jonathan Cedarbaum: Understood. I want to go more deeply into those geopolitical risks with you, but before we do so, I'd like to spend just another few minutes on the Software Understanding for National Security Initiative. As you mentioned, a gathering at the Sandia National Labs in 2023, a report from the Office of the National Cyber Director in 2024, then a follow-on report in 2025 with proposals about addressing the software understanding gap involving interagency collaboration. Are there some highlights from that 2025 report that you think are most important for folks focusing on this issue to be aware of?

Colin Clarke: You know, for, for me, I think it's it's a lot about the potential that's there, but playing almost a, a convening role, right? Because there's been, you know, because it's been at the center of the interagency. So how do we elevate this to, to provide a kind of coherent, coordinated research agenda across the federal government?

Particularly for, you know, a topic that, as Chad mentioned, is technical and, you know, we- we've seen this, our backgrounds, you know, we spent years at the RAND Corporation doing some work on cyber warfare, and anytime you mention certain terms, cyber, cyber anything, right? Cyber security, cyber warfare, and now software understanding, you're gonna get a lot of people that just wholly back off, put their hands up and say, "Oh, that's too technical for me. I don't understand it."

And so, you know, one of the things we're trying to, to discuss here is elevating that, kind of bringing this out into the mainstream and, you know, really diffusing this across the federal government where you have the authorities, the resources, the ins-institutional support needed to elevate this to, I think, the priority it should be.

Jonathan Cedarbaum: Well, from what you're describing the kind of interagency coordination and elevation of the issue across the federal government, it sounds to me like the sort of issue, the sort of task that is suited for the work of the Office of National Cyber Director because, of course, the function of that office is to engage in just such coordination around the government about essential issues concerning cyber security.

So is the ONCD carrying forward the initiative you've described, whether based on that twenty twenty-five report or otherwise? Do you know if the ONCD has an active effort going on to work on these issues?

Colin Clarke: I think there are active efforts within the government. ONCD would be a, a chief player there. I do think this is one of the rare areas where we have seen bipartisan agreement. I've spent some time recently on Capitol Hill talking to lawmakers about this and so I am a little bit optimistic that this is something that we can continue to move forward.

But, you know, in, in terms of which office does it, with my researcher hat on, you know, I'm less concerned about advocating for a specific office. ONCD seems well-positioned, and I think one area that we've talked a lot about is the need for public-private partnership.

Jonathan Cedarbaum: Well, let's talk a little bit more about the nature of the problem and the nature of the risks that you identify in your report as arising from the software understanding gap. You talk about six dimensions of national security risks. I would be happy to hear you discuss any of them, but I wanted to focus on two first that struck me as less well appreciated than some of the others.

The first I wanted to highlight is what you call “inured blindness.” What do you mean by that term, and how does it come out of this problem of soft- the software understanding gap?

Chad Serena: So I tend to think of this question when we were developing this report, we were trying to think about how to categorize some of the things that have come out of the growth of the software understanding gap. Cyberspace in general, things becoming more technical over since, especially since the end of the Cold War.

And one of the things that stood out to us when we were kinda looking backwards through this were some really recent examples of blindness and what we would consider then to be inured blindness. It's something we, we've developed blindness because we just don't pay attention to our circumstances anymore.

Now, if I were to say to anyone here, 40 years ago, would you have noticed if there were two cameras inside your house, one on your computer, three on your neighbor's house, one on a doorbell? You'd have said, "Yeah, would've stood out like a sore thumb." Now you probably walk into your office at any point in time, and you don't even notice those things. You wouldn't even be aware that there's microphones or that somebody's phone's laying there that could potentially be recording you. It's not something that really occurs to you anymore, and that's the inured part.

This is no different than, say, driving down the road and discovering traffic cams anymore. I would say 20 years ago, that would've been something that would've popped out immediately. Now they're probably on every traffic light or intersection that we drive through, and we don't even notice it anymore. So that's the inured side.

The application of that or the operational side of that is to think about what happened recently in Iran. We've, we've read reports about how the Israelis had tapped into various cameras throughout Tehran to be able to do pattern of life monitoring of people coming in and out of various buildings to figure out what they were doing, who was there, who they were meeting with. We've seen it with cartels as well, being able to tap into these various systems in order to gather information and be able to track people.

Apparently, Russia was also doing this in Ukraine as well to track logistics things coming in and out of different logistics hubs like train stations. And then Hamas apparently also was able to use these different types of systems in order to gather information. Right?

What I think is particularly interesting about this is the break where if you think back to, say, any of the movies or novels we would've read in the 50s, 60s, or 70s, you start to think about how gathering this type of information would've been a really low density Jason Bourne, James Bond type of activity where you'd have to spend all this money to sneak a thirty-year trained professional into a country to gather this information.

Now it's completely different. You have established the infrastructure, put it in place, put in things that are like microphones, cameras, and everything else that I don't even have to pay for. I just have to access them and make sure you don't catch me accessing them. You've set up the surveillance and intelligence system that I then want to exploit in order to be able to engage in various nefarious activities against you, whether that's an actual kinetic strike or whether it's gathering information for information operations purposes or, or, or something else. And that, that's completely different. But the inured part is we don't even know that this is going on around us anymore 'cause we're surrounded by so much technology.

Colin Clarke: I, I would add to that. I think, you know, when you think about the concept of pre-positioning, you look at the scale and the complexity of all the software-defined systems that undergird U.S. national security. They can hide intrusions for extended periods. They allow adversaries kind of freedom to act strategically at a place and time of their choosing, before defenders even know what's occurred. S

o if you go back to kind of the SolarWinds supply chain intrusion perpetrated by the Russians, you know, there you had kind of malicious code injected through, into third-party software, updates distributed to thousands of government agencies and private companies, and it went undetected for nearly a year. So what did that allow the Russians to do? I think we still don't know really the intelligence that they gained from, from that. But certainly that long-term covert access to sensitive systems was a, a boon one would suspect, for the Kremlin. And those are the, the breaches that we know about.

I think, you know, it gets into the, I don't know. In some ways I'm reminded of the Rumsfeldian “unknown unknowns” which I realize is a kind of different rabbit hole, but, you know, and there's been other examples from a range of different adversaries as well, including some that, you know, we would maybe consider less sophisticated cyber actors, but that are getting into different targets.

And, and some benign targets, right? Civilian targets, water plants in, you know, random parts of the United States. What's the purpose of that? Is this the kind of equivalent of a weapons test? Is it just get in, hang out, see how long you can be in there before you're undetected, and then try to replicate that in a different system? You know, when you think about the, the vast realm of possibilities, it can get quite dizzying.

Jonathan Cedarbaum: How do you think this problem of inured blindness in the United States compares to the same issue in other countries? Certainly, I see what you're saying about the pervasiveness of software-enabled systems, that pervasiveness leading us to forget how we are surrounded by these systems. But certainly the United States is not the only country where ever more of these systems are controlling many physical infrastructures around us. How does the U.S. compare to either our allies or our adversaries when it comes to this issue of inured blindness? Are they also suffering from these kinds of issues?

Chad Serena: I think in part it's a, it's a matter of scale. So simply put, it's the how much technology you have of, o- of different stripes, whether it's surveillance or other types of things, but here we're, we're generally talking about surveillance, that are insecure or that we don't know how they're gonna behave and then could also be accessed for these purposes.

So in that regard, although we haven't done a, an actual count of these types of devices, I'd have to expect that the United States is probably a leader in this, but certainly Western Europe too if you think that the United Kingdom certainly has a lot of different surveillance set up throughout London and other, other cities within England. Certainly these are things that could then be tapped into.

So any of the more sophisticated, any of the more sophisticated countries that use these things for other purposes, whether it's traffic control or even detecting and evaluating criminal behavior or watching people getting on the subways where there are more of those, then it would seem to make sense that there would be more opportunity for these things to be exploited. And then if you were to go to countries where things were more sparse or people aren't that densely populated or you don't have the equipment around, it would seem that there'd be less of a risk to that.

Jonathan Cedarbaum: We've been talking about one dimension of the risks you identify in inured blindness. I wanted to switch and focus a little bit on one of the other ones next. That is what you call “the tactical becomes the strategic.” What do you mean by that?

Chad Serena: That's right. It, it, so again, thinking about how the scale and scope of these things have changed over time what we mean by the tactical becoming the strategic is that if you think of some of the things that have been taken over or manipulated in order to generate either actual or potential effects, the way that we have to think about these now would be different.

Again, and I hate to keep referring back to Cold War period or then, but it's a good reference point because a lot of this stuff has changed since the end of the Cold War, and a lot of these problems have emerged since the end of the Cold War and the increased use of software and software-enabled devices and software-defined systems.

But if you think about something like an industrial control system, which we reference in the, in the report and talking about that, if one of these were to be taken offline, say, either deliberately or accidentally in a local environment like the Iranians tried to do with the system in Aliquippa, Pennsylvania If that were to occur, it's, it's a smaller scale problem. It's a tactical problem.

However, when you have all sorts of industrial control systems scattered throughout the country that are then controlled by software or, or software-defined, now if those things could be penetrated at scale, you could have that same type of disruption occur, except you could have it occur at thousands of different water treatment facilities.

Now, what would've been a tactical problem at one point that we never would've even considered really in a, in national security sense, unless it were to happen at a military installation or somewhere else. Now, that simple tactical problem becomes a strategic issue if these things were to manipulated at scale or simultaneously.

And that applies too for other places where, whether it's transportation hubs, if you were to look at airports, you could take something that would be a very simple problem, like interfering with the computer systems or communication systems at airports, and then all of a sudden not going to be able to have airplanes taking off from a bunch of different places. That problem then compounds if people can't move. If people aren't able to move, if logistics aren't able to move, you could see where problems just cascade onwards.

Colin Clarke: Yeah. I, I would add to that, I think, you know, it, it's kind of death by a thousand paper cuts, right? You have these low-level attacks that, you know, when, when taken in aggregate or, or cumulatively, they produce strategic consequences.

So if you think about what the Chinese did with Volt Typhoon, pre-positioning itself within various U.S. utilities, transportation hubs. But instead of just immediate disruption, these kind of penetrations are more likely to, you know, precursors to a future pre-positioned cyber attack. And obviously when, when we talk about this, it's hard not to think of a, a Taiwan scenario, some kind of a future Taiwan scenario.

But, you know, play that out across a range of, of different adversaries, and not even necessarily U.S. adversaries, but as these capabilities, you know, are kind of enhanced in other theaters of conflict, right? In other interstate rivalries, you know, whether it's India-Pakistan, or Israel-Iran, or Turkey. I mean, this is really, I think something we're gonna be seeing a lot more in the foreseeable future, and, and probably not in the distant future

Chad Serena: It's important to point out too, Jonathan, that we tend to think of this, Colin and I both were certainly guilty of it because of our perspectives on the subject, we tend to think of this as a security issue, that this is something that's gonna happen during an attack or an adversary is gonna do this.

This can also happen accidentally, and sometimes it's difficult to determine whether it was an accident or whether it was an attack that caused it in the first place. So if you were to look at something like the Iranian hacks into gas stations and the digital control devices on those, normally if you're a person working at a gas station, you wouldn't think, "Well, I'm, I'm sitting here at the head of an international attack on a, a system that controls the pumps at my local gas station." You might be thinking something else, but at the same time, it could also be an accident.

Either way, if the attack or the disruption at, say, Aliquippa or some other water treatment facility was Iranian or it was accidental, it can still lead to cascading effects. So some of this is about the software. If it doesn't perform correctly, you still could have these cascading interdependencies of different systems failing or, or being unable to support each other, or you could have this done because of an attack where the software is manipulated and still have the same result.

Jonathan Cedarbaum: Understood. We've talked a little bit about how the government is beginning to address this very consequential problem. One of the elements of response that you mentioned in your report is a provision in last year's National Defense Authorization Act, the big annual statute that provides guidance to the Department of Defense.

And you mentioned that there's a provision in last year's NDAA that directs the department to develop, quote, "A comprehensive strategy for transitioning DARPA's formal methods research investments into production environments across the department." Can you translate that a little bit for folks who may not be familiar with what formal methods research investments mean? What, what is that directive telling DoD to do?

Colin Clarke: So I think in some ways, because national security functions and systems are so, you know, deeply software-defined and the risks that are posed by, as we, we kind of talked about in the intro, the software understanding gap continue to grow, this is about actually putting our money where our mouth is.

So evaluating the potential and actual effects of the gaps, but okay, we're not just observing these kind of, you know, throwing our hands up and saying, "Oh, wow, I wish we could do something about this." This is kind of trying to mobilize the cavalry, if you will, developing the policies, you know, software development requirements and, and the capabilities and practices, for example, formal methods, but also the procedures, data sharing agreements that can help at least mitigate the effects of the gap, right? We're not saying that formal methods is the, the be-all, end-all, but it certainly puts us in a better position to close this gap.

You know, and, and one of the things we've talked about, not only in the paper, but, again, Chad and I have been colleagues going back for a very long time now, I guess 20 years almost and we have these kind of long-running, sometimes philosophical, you know, conversations and debates. As you kind of close out or mitigate vulnerabilities, new ones will arise as well. But this is trying just to kind of get again, I guess I'd go back to the putting your money where your mouth is and getting this language introduced into legislation and formally, you know, added as a requirement for the government to move forward.

Jonathan Cedarbaum: Well, you mentioned how the nature of vulnerabilities changes, and that leads me to think about the issue that hovers over every issue related to digital systems today, and that is the impact of AI. How will the increasing ability of AI systems both to identify and apparently to repair software vulnerabilities affect this problem of the software understanding gap?

Chad Serena: So I think in some ways it's both in the identification of risks, whether you're doing that from the position of being an attacker or a defender is important. As we understand it, it is more difficult to defend certain things because you have to be right all the time. As the phrase goes, “you have to be correct all the time about what it is that you're trying to defend, whereas the attacker only needs to be right once in order to weaken your defenses.”

Thinking about AI, this produces a, produces quite a challenge i- i- in terms of the speed and the depth and the sorts of things that AI would be able to find for both sides, so both on the offensive and defensive side. So it opens up a lot of capabilities in terms of being able to find vulnerabilities and to test software and to really use AI to enable formal methods in order to get towards greater software understanding.

But then on the other side too, to use this as a means of exploring vulnerabilities in, in different systems and thinking about that at a macro level, you start to say, "Well, I'm going to secure X, Y, and Z." You will see, and, and we would expect to see an adaptation amongst the, the different aggressors in this, that as you start to secure targets and make targets harder, you're then going to expose softer targets that haven't necessarily been defended in the same way or haven't been evaluated in the same way. And AI will just speed that up and, a- and make that sort of exploitation that much more difficult.

We would also expect to see some sort of competition, not just amongst the different larger, like the United States, China, Russia, or nation states in terms of using AI to do these things, we would also expect for other organizations and states that are less thought of non-state actors and others to be able to use these things in order to amplify their capabilities and to do things that we wouldn't expect them normally to be able to do.

Colin Clarke: I, I would add to that, you know, AI has become a force multiplier in so many different ways, but the advantage is naturally, you know, they're not always distributed evenly. So, for attackers, they need to find a single exploitable flaw. Defenders, much more difficult task. They have to identify and remediate all of these.

So I think A- AI accelerates both tasks, but the attacker’s is fundamentally easier, gives them a structural advantage, and I think there's gonna be different, you know, this isn't a kind of static race or competition, if you will. It's highly dynamic. There's gonna be different developments on both sides, and, you know, I think when you think about the potential for AI, and I talked before about public-private partnerships.

You know, one of, one of these areas when we're specifically dealing with closing the gap is gonna be our adversaries, whether those are nation states or non-state actors, they're operating under a different set of rules, right? They're not, in many ways, governed by the same laws, authorities, policies, procedures that, you know, that, that governments are that the U.S. government is, and that tech companies are. So I think there is so much uncertainty in this area. It's really difficult to predict where, where this is gonna go and how the gap kind of, you know, diverges or, or kind of ebbs and flows over time, if you will.

Chad Serena: In some ways, you can think of this as a boon for organizations that don't have rules or have lessened rules. So if you think of Russia, Russia's, Russia's control over this and how it is that people that act on Russia's behalf or with the Russian government, whether they're non-state or semi-state actors, this allows them a greater amount of flexibility, like Colin was saying, in order to engage in these types of behaviors.

Part of the challenge here is, and why it's so important that this needs to be organized well in the United States through SUNSAC and through inter-agency cooperation, is we don't have whether, and when I say “we,” either as the United States or organizations within the United States, we don't have that sort of flexibility to operate outside of the law and do whatever it is that we want to do in order to defend ourselves or to protect ourselves.

So it's important that the, these steps be taken, these first steps in trying to at least get a coordinating body together to think about how it is that the software understanding gap can be closed because y-you're, you're gonna be perpetually behind the curve if you're trying to very slowly and piecemeal establish laws across a country with 330 million people in it. When you're thinking about these non-state actors that have nothing preventing them from doing what it is that they want to do, and in many cases, having state support in doing what it is that they're going to do, that's a, that is a tough dichotomy when you're thinking about being a defender, when you're on the side that has to follow the law and you're fighting against people that just have no interest whatsoever in following any of the rules that you would like to establish.

Colin Clarke: The, the way it was described to me by one intelligence official was, in, in many ways, violent non-state actors, terrorist insurgents are able to stay one or two steps ahead of us. There's growing concern that AI will make that three or four steps ahead and that it becomes impossible to, to close that gap over time.

Jonathan Cedarbaum: Well, on that troubling note, I think we're gonna close for today. Chad and Colin, it's been a pleasure talking with you about this very urgent problem of the software understanding gap, and I hope your report will get the wide readership that it deserves. Thanks very much.

Colin Clarke: Thanks for having us.

Chad Serena: Yes, thank you.

[Outro]

Jonathan Cedarbaum: The Lawfare Podcast is produced by the Lawfare Institute. If you wanna support the show and listen ad-free, you can become a Lawfare material supporter at lawfaremedia.org/support. Supporters also get access to special events and other bonus content we don't share anywhere else. If you enjoy the podcast, please rate and review us wherever you listen. It really does help.

And be sure to check out our other shows, including Rational Security, Allies, The Aftermath, and Escalation, our latest Lawfare Presents podcast series about the war in Ukraine.  You can also find all of our written work at lawfaremedia.org. The podcast is edited by Jen Patja with audio engineering by Noam Osband of Goat Rodeo. Our theme song is from Alibi Music. And as always, thank you for listening.