Untitled

[ nominal delivery draft, San Francisco, 24 February 2020 ] Hard Choices Dan Geer There are no introductory paragraphs here; this audience does not need them and there is no time. Nor are there predictions, per se. Instead, what we will discuss are what I will call questions, forks in the road our path through which will steer the future. Some questions are our free choice, others not so much. As Nassim Taleb observes: when the tails of a probability distribution get fatter, the predictable becomes a function of the distribution's extreme values and only those extreme values. As he puts it: [We are] undergoing a switch between [continuous low grade volatility] to ... the process moving by jumps, with less and less variations outside of jumps. The faster the rate of change, the heavier become those tails, due mostly to the growth of unrecognized interdependence between the moving parts. Such a situation requires the prudent man plan for maximal damage scenarios, not for most probable scenarios, a point to which I will return. The questions that follow are not mutually exclusive nor are they collectively exhaustive, but each calls out a fork in the road which, however it is resolved, leads to a different future. Were there more time, we could discuss the interactions between them that come from their not being mutually exclusive. The core insight given us by evolutionary biologist Stephen Jay Gould was that while change is inevitable, undirected change does not produce some steady upslope at 8% grade, rather the course of undirected change traces through periods of quietude interrupted by bursts of self-reinforcing change. He called this phenomenon a "punctuated equilibrium". When I look at our history with Gould's eye, I count three "equilibrium punctuations" to date in cybersecurity. circa 1995: a TCP/IP stack appears as a freebie in MS Windows => that exposed an unhardened operating system to the world, and => that birthed the cybersecurity industry circa 2005: opponents change over from braggarts to professionals => because defense may share but offense does not, an ever increasing fraction of exploits are closely held, and => so, in time, zero-days came to dominate risk circa 2015: automation of flaw finding (DARPA's Cyber Grand Challenge) => which is the ultimate proof that all security technology is dual use ------------------------------------------------------------------ forks in the road expressed as questions ------------------------------------------------------------------ automation: Q: Over time, does automation help defense more or does it help offense more? => If the answer over time is offense, then cybersecurity becomes more like nuclear deterrence, including that there are no precision weapons and that enforceable treaties are essential to our survival. If the answer is defense, then we turn to algorithms to do what we cannot otherwise do, protect us from other algorithms. How long do we wait to pick a winner on which we will depend? [ spw18.langsec.org/slides/Vanegue-AEGC-5-year-perspective.pdf ] --------------------------------- connectivity without management: Q: Do we allow self-modifying decision making to free-run? => Berkeley SWARMlab says that within 5yr there will be 1000 radios per person on earth while Pete Diamandis (X Prize) says within 15yr there will be 10,000 sensors per person. That scale exceeds manageability, so that technology must just be allowed to free-run, yet if that free-running is self-modifying then "trustworthy" becomes unevaluable. Does that mean that we should not pair free-running with self-modifying and, if so, by what mechanism do we make that choice stick? --------------------------------- metrics as driver: Q: Do we steer by MTBF (drive to infinity) or MTTR (drive to zero)? => Choosing Mean Time Between Failure as the core driver makes the assumption that vulnerabilities are sparse, not dense. If they are sparse, then the treasure spent finding them is indeed well spent so long as we are not deploying new vulnerabilities faster than we are eliminating old ones. If they are dense, then any treasure spent finding them is more than wasted, it is disinformative. If we cannot answer whether vulnerabilities are sparse or dense, then should we not default to instant recovery, which is to say a Mean Time To Repair of zero? Does not the paramount security engineering design goal become no silent failure (not no failure, but no silent failure)? Steering by MTTR is, in fact, consistent with planning for maximal damage scenarios. [ www.theatlantic.com/technology/archive/2014/05/should-hackers-fix-cybersecurity-holes-or-exploit-them/371197 ] --------------------------------- embedded systems: Q: Do we require devices either be updatable or have a finite lifetime? => A remote management interface is required for updatability but such an interface is itself a potential risk should it be cracked, yet doing without remote update risks having devices that are immortal *and* unfixable unless a finite lifetime is purposefully designed in. Enforcement of either mandate, that of remotely fixable or pre-limited lifetime, is an expressly government issue, and enforcement means culpability for those who fail to implement the one or the other. But that culpability means what, exactly? Note that devices which are unlocatable after deployment are a special case; they must have limited lifetimes. [ tools.ietf.org/html/rfc791 ] --------------------------------- composability: Q: Non-composability (secure A + secure B != secure A+B) is often demonstrated, but can research resolve this? Should we trust research to be able to resolve the question? => This is the inherent security-in-the-limit question for the supply chain model. With some encouragement, DARPA has become interested in looking at composability within the narrow domain of just input parsers on the argument that since hostile data input is so very often the mechanism by which exploit is triggered, composable parsing would yield much general benefit, and not just for cybersecurity. [ langsec.org ] --------------------------------- abandonment: Q: Every kind of property will be seized in the public interest if and when abandoned, be that property a car, a bank account, a baby, a house, etc. Except software is not. Should software be different and, if not, then what is the remedy? => The most straightforward remedy is to open source abandoned codebases. Another is to seize cryptographic keys of signal value, such as those of a certifying authority that goes out of business. The nascent Microsoft/GitHub arctic repository of code on which "we" depend is an example answer of serious preservation. Note that a supplier declaring that such and such a software product is "no longer supported" would be substantive abandonment and thus trigger the abandonment process. A plague of lobbyists can thus be expected. [ geer.tinho.net/ieee/ieee.sp.geer.1307.pdf ] --------------------------------- support: Q: Do we mandate that the software supply chain be checkable? => Olav Lysne has shown that, in the limit, electronic equipment from untrusted vendors cannot be verified. We thus fall back to process requirements, that is to say we make merchantability of digital goods require the seller's retained ability to recreate those digital goods as deployed. This means preservation of the build environment in working detail. The nascent, international effort on "reproducible builds" is making technical progress and should be supported enthusiastically. Or, if caveat emptor is sufficient to your world view, don't bother with any of this. [ Lysne, The Huawei and Snowden Questions: Can Electronic Equipment from Untrusted Vendors be Verified? Can an Untrusted Vendor Build Trust Into Electronic Equipment?, Springer, 2019 ] [ reproducible-builds.org ] --------------------------------- attribution: Q: Do we geocode the Internet by fiat? => A public safety argument mandated geocoding mobile phones in real time. The sanctity of all financial services starts with "know your customer". Whereas intercontinental ballistic missiles have a visible flight path and a limited number of launch-capable governments, offensive software has neither. Conspiracy junkies will yell "False flag!" as the opening volley in any argument over attribution, to which the correct rejoinder must be "So what?". The Westphalian principle is clear: if packets are exiting your territory, then you have strict liability for them regardless of any actual negligence or intent to harm on your part. Were Patrick Henry with us still, he would again say that "The preservation of liberty depends on the single chance of men being virtuous enough to make laws to punish themselves". Are we that virtuous? [ geer.tinho.net/ieee/ieee.sp.geer.1707.pdf ] --------------------------------- prioritization: Q: Does continuous machine learning require prioritizing integrity over confidentiality? => Within the classic triad of confidentiality, integrity, and availability, confidentiality has long been first among equals. As data collection grows in breadth and depth, and as data-driven algorithms take broad control of the physical world, mustn't data integrity take priority over availability, and availability take priority over confidentiality? If this is the moment of overturning that prior order, then much different design regimes must be deployed including, but not limited to, robust data chain of custody regimes uninfluenced by single points of failure. [ geer.tinho.net/ieee/ieee.sp.geer.1801.pdf ] --------------------------------- data mitigation: Q: What mitigation for immortalized data loss? => The insertion of bad things onto block chains has no reversal mechanism, no "undo". That is inherent in their design. Do those who manage blockchains without a deletion mechanism inherit strict liability for the consequences of the blockchain's misuse as publisher of illicit goods? If they do not, then who pays the piper and for how long? [ news.bitcoin.com/a-brief-history-of-hidden-messages-in-the-bitcoin-blockchain ] --------------------------------- namelessness: Q: Do we require resolvable names for some classes of services? => At present, name resolution is the double-entry bookkeeping of the Internet (scenario: I claim this name and address pair are connected; you verify my claim with third parties). In the absence of names, how do you know to whom you are connected? Is a (nameless) address baked in and, if so, baked in to what? Does a software updating engine reach out to devices that have only a network address and no name? Does an endpoint that is asked to accept an update have a name to check? Do we need a two-class liability and/or duty for checkable name-to-address pairs versus un-named addresses? What is the risk interaction of this with an IPv6 world too big to enumerate plus IPv6 protocols providing address hopping and multi-homing inherently? Will internal firewalls include a key-centric, rather than a name-centric, PKI? Does a MAC address or a UUID-in-ROM distinguish keys in a nameless world and thus imply an identity-based PKI? Either way, is the key-management job going to be harder or easier absent names? [ geer.tinho.net/fgm/fgm.geer.1812.pdf ] --------------------------------- balkanization: Q: Should the trade in network and security gear mimic the trade in arms, that is to say that you only buy or sell within your team? => A government official saying "We need you to do us a favor" is probably a seller issue more than a buyer issue. Must all sellers become holding companies so that the Apple-China team does one thing while the Apple-EU team does another (for example)? Even if there is no nation-state pressure on a supplier to install an exploitable flaw, it is flatly certain that "do us a favor" can take the form "you are forbidden to fix the following flaw". Or is nation-state the wrong granularity here? Some Google security engineers tell me that they only trust Google-manufactured phones as other manufacturers modify the Android software and may install other software as part of the base install. A few major banks no longer buy application software, they write their own. The idea will spread; does sovereignty come to require running only code you wrote? [ www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf ] --------------------------------- interoggability: Q: Do we accept dependence on black-box algorithms? => Machine learning from noisy datasets delivers equations where coefficients have no inferential meaning and are likely to have the fragility that comes from over-fitting. Article 15 (and others) in the European Union's General Data Protection Regulation creates a right to challenge any algorithmic decision with the demand of "Why?", which cannot be answered by an uninterrogable algorithm. Similarly, military users will likely be unwilling to permit algorithms to apply deadly force if those algorithms cannot explain themselves in the vernacular. At what level of criticality do we require interoggability prior to legal deployment? Does this create a new variety of "informed consent" when a medical algorithm says what must be done but cannot explain why, and so what, exactly, does "informed" in "informed consent" then mean? [ www.clarip.com/data-privacy/gdpr-article-15 ] --------------------------------- insurance: Q: What is the definition of force majeure in cybersecurity? => As Mondelez v Zurich and Merck v Allianz/AIG show, what is force majeure in cybersecurity may be judiciable. Since many instances of attack have no immediately identifiable attacker, how will this work out? Has the burden of proof for payout shifted from proving "I was harmed" to proving "I was harmed and I can prove that I was the actual target"? Besides that, how will risk pooling -- the very premise of insurance -- proceed in future? At some level of personalization there is nothing left with which to pool. [ www.bloomberg.com/news/features/2019-12-03/merck-cyberattack-s-1-3-billion-question-was-it-an-act-of-war ] --------------------------------- truth: Q: What are the trust anchors we now need? => While the usual use of the term "trust anchor" refers to the self-signed certificate at the head of a specific cryptographic hierarchy, I mean it in a policy sense, a societal sense. As Malcolm Gladwell has persuasively argued, a civil society only works if in dealing with each other its members default to trust. As digitalization of society proceeds, the presence of cyber risk makes defaulting to trust look increasingly naive. What divergence effects do you get if in meatspace you more or less have to default to trust whereas in bitspace you more or less have to default to mistrust or, as is trendily said, adopt a "zero trust" policy? [ Gladwell, Talking to Strangers, Hachette Book Group, 2019 ] --------------------------------- entity scale: Q: Are cloud computing suppliers critical infrastructure? => As Wade Baker and I have shown, the very smallest companies and the very biggest companies are safer if they do their computing on-premises whereas middling companies are safer if they do their computing in the cloud. In a target-rich world, the little don't much draw attacks while the big have the dollars to corner the talent market. The middling are therefore all but obliged to have somebody else mind the cookstove. So does it then follow that cloud computing suppliers have the duties of a critical infrastructure provider? Do we replicate the regulatory structure surrounding Systemically Important Financial Institutions (SIFI) for computing entities that are not "too big to fail" but rather "too interconnected to fail"? What is the policy analog for a cloud provider to a SIFI's capital adequacy requirements or any of the rest of the financial stress-testing regime? Since the bulk of regulation for financial services is the accumulation of past failures, shall we just wait for cloud failures of sufficiently serious kinds and design regulations then, following, as it were, Rahm Emmanuel's dictum to "Never let a serious crisis go to waste"? [ geer.tinho.net/fgm/fgm.geer.1909.pdf ] [ geer.tinho.net/fgm/fgm.geer.1412.pdf ] --------------------------------- strategy: Q: What strategy is most cost effective for cybersecurity? => Cost/benefit analysis is a waste of time; what matters is cost effectiveness. The fork in our road is to move toward proving that a body of code implements its spec and no more than its spec -OR- to embrace moving target defense. Code proving is a, if not the, gold standard, but it is also difficult enough to do that after a successful proof is in hand you must then adopt brutal change control to protect your investment. Moving target defense is easy and it may work well enough to diminish the defender/attacker strategic asymmetry that today favors attackers, but it also guarantees that how any fielded instance of code really works at any given moment on any given end-point is not obvious -- it requires an intermediary to explain. Brutal change control damps down change, which may be an acceptable price to pay for stamping out cyber exploit. Moving target defense doesn't care if there's a new set of vulnerabilities with every revision, which may be an acceptable price to pay for stamping out cyber exploit. Taking either path is a serious choice supported by serious results obtained by serious scientists; it is also a choice that is expensive to later reverse either in dollars or clock-ticks. --------------------------------- ownership: Q: Should the beneficial owner of software also own its risks? => Today, suppliers claim ownership for them and licensure for you, yet via the pervasively unreadable End User License Agreement they disclaim all responsibility for what they say they own. This contradiction now applies to countless melanges of hardware and software from the big to the small. This can either continue toward its logical conclusion, that of owning nothing and having no recourse, or it can be steered (at least in democratic countries). To do that, new constructions of strict liability and of merchantability are necessary, most particularly in light of self-modifying software where the copy you have may well be unique on all the planet. This fork in the road demonstrates more than most that not to decide is to decide. A plague of lobbyists can be expected here, too. [ The End of Ownership, Perzanowski & Schultz, MIT, 2019 ] [ geer.tinho.net/ieee/ieee.sp.geer.1907.pdf ] --------------------------------- analog fall back: Q: If a state of security is the absence of unmitigable surprise, what is the mitigation for an irrecoverable software fault? => Of all the choices enumerated here, this is the most telling. We know that optimality and efficiency work counter to robustness and resilience. We know that complexity hides interdependence, and unacknowledged interdependence is the source of black swan events. We know that the benefits of digitalization are not transitive, but the risks are. We know that because single points of failure require militarization wherever they underlie gross societal dependencies, frank minimization of the number of such single points of failure is a national security obligation. We know that because cascade failure ignited by random faults is quenched by redundancy whereas cascade failure ignited by sentient opponents is exacerbated by redundancy, we therefore also know that preservation of uncorrelated operational mechanisms is likewise a national security obligation. Those uncorrelated operational mechanisms are the analog alternative. If we are to maintain the ability to operate in the absence of the Internet and all that it delivers, then we will have to retain, in working order and fighting trim, the analog realm that came before precisely because that analog realm does not share common mode failures with the digital realm nor do analog means create pathways to cascade failure. To do that -- to retain the analog realm -- the analog realm has to be used, not left sitting on some societal shelf in hope that it still works when some emergency demands that it work. As a current example, consider the regulatory and public policy debate ongoing in Sweden on how to retain a sufficient base load of cash processing such that the mechanisms needed to use cash do not disappear. Sweden is now choosing whether to make cash processing a critical infrastructure and to do so by regulation. Mere economics will otherwise create a singleton risk of society-wide failure. In short, the intervention is to require, by force of law, that those who choose to opt out of the digital world do not do so at the price of moving to the 15th century, that accommodating those who opt out is precisely and uniquely how to maintain a base load for nearly all of the essential components of daily living. Perhaps never before and never again will national security and individual freedom jointly share a call for the same initiative at the same time. As someone who is, in fact, increasingly opting out, I can tell you that the active preservation of the analog option must come soon if it is to have the cost-effectiveness of preserving fully-amortized infrastructure and not the sky-high costs of recreating it under emergency conditions. What will we choose? [ A Rubicon, www.hoover.org/research/rubicon ] --------------------------------- At the outset, I said that the prudent man would plan for maximal damage scenarios, not for most probable scenarios. I learned from my accountant father that the accountant's convention of conservatism, also known as the doctrine of prudence, is to anticipate possible future losses but not future gains. I know from the teachings of my career that cybersecurity requires just such discipline and that such discipline is fundamentally conservative expressly because the conservative knows that ordered liberty depends on putting a speed limit to irrevocable change. It is in that spirit that I have laid out a few of the choices we face. We cannot be passive; freedom isn't free. --------------------------------- There is never enough time; thank you for yours. [ this and other material is at geer.tinho.net/pubs ]