Category Archives: Security

Überschätzte Risiken: Skynet und Killerdrohnen

Die künstliche Intelligenz hat in den letzten Jahren rasante Fortschritte gemacht. Dahinter steckt eine Mischung aus weiter wachsenden Rechenkapazitäten, verbesserten Verfahren sowie immer mehr verfügbaren Daten, mit denen sich lernende Systeme trainieren lassen. Parallel dazu entstand das Internet der Dinge beziehungsweise das Internet of Everything – vernetzt sind nicht mehr nur klassische Computer, sondern tendenziell alles, was Energie verbraucht. Kombinieren wir beide Trends, so erhalten wir intelligente autonome Roboter – Staubsauger, Autos, Fluggeräte und so weiter.

Wo sich technischer Fortschritt zeigt, sind die Mahner nicht weit. Eine KI würde den Menschen eines Tages überflügeln und sodann auslöschen, fürchten sie, und autonome Roboter könnten uns als automatische Todesschwadronen begegnen:

Nicht alle überzeugt dieses Szenario, doch die Autoren des Films halten daran fest.

Was fällt daran auf?

  1. Das Szenario spielt mit Vorstellungen aus dem Hollywood-Kino vom Angriff der Killer* (-bienen, -tomaten, usw.) bis zu Skynet. So weit ist die KI jedoch gar nicht.
  2. Eine plausible Risikobetrachtung fehlt und vielleicht muss sie das auch, weil wir es mit epistemischen Risiken zu tun haben.
  3. Die Technik wird einseitig als Waffe in der Hand eines Gegners dargestellt. Zu welchen Szenarien kämen wir, machten wir uns diese Technik zunutze? Was, wenn die Killerdrohnenschwärme für uns auf Terroristenjagd gingen oder gar Kernwaffen unschädlich machten?
  4. Strategische und taktische Diskussionen fehlen. Für wen ergäbe so einen Drohnenschwarmangriff einen Sinn? Wie würde man sich dagegen verteidigen? Ergäbe der Angriff dann immer noch einen Sinn?
  5. Die Baseline, der Bezugsrahmen bleibt unklar. Reden wir über eine neue Form von Krieg, oder reden wir über Terror im zivilen Leben? Was genau verändern die neuen Mittel?

Das Video ist gut gemacht, aber nicht so richtig ausgegoren. Es appelliert an die Emotion, nicht an den Verstand. Diesem Appell nachzugeben, ist mir zu riskant.

P.S. (2018-02-19): Im Mittelpunkt unserer echten militärstrategischen Probleme steht weiter die gute alte Atomrakete.


Securing Your PC (1980s Style)

There was a time when personal computers came with security built into their hardware. For about a decade from 1984 on, virtually every PC featured a key lock. Depending on the particular implementation, locking would prevent powering on the computer, keyboard input, hard drive access, opening the case, or a combination thereof. This video tells the story:

From today’s perspective the key lock looks like a weak if not pointless security mechanism. In the best case it makes tampering with the hardware slightly harder—attackers have to equip themselves with tools and spend some time using them—while not at all addressing all the software vulnerabilities that we care about so much today.

Nevertheless the design made a lot of sense.

First, a keylock is a usable security mechanism. Everyone is familiar with key locks and knows how to use them, no complicated setup is required, and there is little potential for mistakes.

Second, an attacker’s physical access to hardware internals defeats most software security mechanisms. Physical access control is therefore a prerequisite for security against certain threats.

Third, personal computers at that time were not really threatened by online or software attacks, perhaps with the exception of relatively harmless viruses spreading through exchanged floppy disks. Someone tampering with the computer was indeed one of the more realistic threats.

Fourth, seemingly minor security gains can be rather effective when put in context. While forcing attackers to carry tools and use them may not seem like a great complication for them, it may suffice to prevent opportunistic attacks as well as quick serial attacks against multiple consecutive targets.

Security technology has evolved to make key locks obsolete, but they made sense at the time of their introduction.

Application Layer Snake Oil

TL;DR: The author thinks Snowden’s home security app, Haven, is snake oil regardless of the algorithms it uses. Operational security is at least as hard as cryptography and no app is going to provide it for you.

Bogus cryptography is often being referred to as snake oil—a remedy designed by charlatans to the sole end of selling it to the gullible. Discussions of snake oil traditionally focused on cryptography as such and technical aspects like the choice of algorithms, the competence of their designers and implementers, or the degree of scrutiny a design and its implementation received. As a rule of thumb, a set of algorithms and protocols is widely accepted as probably secure according to current public knowledge, and any poorly motivated deviation from this mainstream raises eyebrows.

However, reasonable choices of encryption algorithms and crypto protocols alone does not guarantee security. The overall application in which they serve as building blocks needs to make sense as well in the light of the threat models this application purports to address. Snake oil is easy to mask at this level. While most low-level snake oil can be spotted by a few simple patterns, the application layer calls for a discussion of security requirements.

Enter Haven, the personal security app released by Freedom of the Press Foundation and Guardian Project and associated in public relations with Edward Snowden. Haven turns a smartphone into a remote sensor that alerts its user over confidential channels about activity in its surroundings. The intended use case is apparently to put the app on a cheap phone and leave this phone wherever one feels surveillance is need; the user’s primary phone will then receive alerts and recordings of sensed activity.

Haven is being touted as “a way to protect their [its users] personal spaces and possessions without compromising their own privacy.” The app allegedly protects its users against “the secret police making people disappear” and against evil maid attacks targeting their devices in their absence. To this end, Haven surveils its surroundings through the smartphone’s sensors for noise, movement, etc. When it detects any activity, the app records information such as photos through the built-in camera and transmits this information confidentially over channels like the Signal messenger and Tor.

Alas, these functions together create a mere securitoy that remains rather ineffective in real applications. The threat model is about the most challenging one can think of short of an alien invasion. A secret police that can make people disappear and get away with it is close to almighty. They will not go through court proceedings to decide who to attack and they will surely not be afraid of journalists reporting on them. Where a secret police makes people disappear there will be no public forum for anyone to report on their atrocities. Just imagine using Haven in North Korea—what would you hope to do, inside the country, after obtaining photos of their secret police?

Besides strongly discouraging your dissemination of any recordings, a secret police can also evade detection through Haven. They might, for example, jam wireless signals before entering your home or hotel room so that your phone has no chance of transmitting messages to you until they have dealt with it. Or they might simply construct a plausible pretense, such as a fire alarm going off and agents-dressed-as-firefighters checking the place. Even if they fail to convince you, you will not be able to react in any meaningful way to the alerts you receive. Even if you were close enough to do anything at all, you would not physically attack agents of a secret police that makes people disappear, would you?

What Haven is trying to sell is the illusion of control where the power differential is clearly in favor of the opponent. Haven sells this illusion to well pampered westerners and exploits their lack of experience with repression. To fall for Haven you have to believe the  premise that repression means a secret police in an otherwise unchanged setting. This premise is false: A secret police making people disappear exists inevitably in a context that limits your access to institutions like courts or media or the amount of support you can expect from them. Secret communication as supported by Haven does not even try to address this problem.

While almost everyone understands the problems with low-level snake oil and how to detect and avoid it, securitoys and application layer snake oil continue to fool (some) journalists and activists. Here are a few warning signs:

  1. Security is the only or primary function of a new product or service. Nothing interesting remains if you remove it.
  2. The product or service is being advertised as a tool to evade repression by states.
  3. The threat model and the security goals are not clearly defined and there is no sound argument relating the threat model, security goals, and security design.
  4. Confidentiality or privacy are being over-emphasized and encryption is the core security function. Advertising includes references to “secure” services like Tor or Signal.
  5. The product or service purports to solve problems of operational security with technology.

When somebody shows you a security tool or approach, take the time to ponder how contact with the enemy would end.

Wechselnde Prioritäten

Sicherheit ist schwer, weil das Angreiferverhalten nicht einmal statistisch konstant bleibt. Diese Erfahrung machte auch die DDR kurz vor ihrem Ableben:

Jahrzehntelang hatte man sich intensiv auf einen militärischen Konflikt mit dem Klassenfeind vorbereitet. Dem preußischen Militarismus stand die DDR in nichts nach und die männliche Bevölkerung war hervorragend  militärisch ausgebildet. Der Einsatz der Armee gegen das eigene Volk kam auch deshalb nicht in Frage – es hätte sich zu wehren gewusst und die Armee war fürwahr eine Volksarmee.

Can Penetration Testing Trigger Change in Development Teams?

Software penetration testing has become a common practice in software companies. Penetration testers apply exploratory testing techniques to find vulnerabilities, giving developers feedback on the results of their security efforts—or lack thereof. This immediate benefit is mostly uncontested, although it comes with the caveat that testers find only a fraction of the present vulnerabilities and their work is rather costly.

Security experts commonly recommend that developers should respond to the results of a penetration test not merely by fixing the specific reported vulnerabilities, but rather analyze and mitigate the root causes of these vulnerabilities. Just as commonly, security experts bemoan that this does not seem to happen in practice. Is this true and what prevents penetration testing from having an impact beyond defect fixing?

Studying Software Developers

We studied this question by observing a product group of a software company over the course of a year. The company had hired security consultants to test one of its products for security defects and subsequently train the developers of this product. We wanted to know how effective this intervention was as a trigger for change in development. To this end we conducted two questionnaires, observed the training workshop, analyzed the contents of the group’s wiki and issue tracker, and interviewed developers and their managers.

The product group in our study comprised several Scrum teams, development managers, and product management. Scrum is a management framework for agile software development. Scrum defines three roles—Development Team, Product Owner, and Scrum Master—and artifacts and ceremonies for their collaboration. The Development Team is responsible for and has authority over technical matters and development work; the Product Owner is responsible for requirements and priorities; and the Scrum Master facilitates and moderates their collaboration.

The participants of our study appreciated the penetration test results as feedback and the training workshop as an opportunity to learn. They managed to address the particular vulnerabilities reported by the consultants. They also felt that security needed more attention in their development work. Yet, after addressing the vulnerabilities uncovered by the penetration test, they returned to their familiar ways of working without lasting change.

Analyzing our observations through the lens of organizational routines, we found three key factors inhibiting change in response to the penetration test and training: successful application of existing routines, the organizational structure of roles and responsibilities, and the overall security posture and attitude of the company.

(1) Existing Routines

To address the immediate findings of the penetration test, our developers used an existing bug-fixing and stabilization routine. Defect reports arrive asynchronously and sometimes require quick response; developers therefore commonly dedicate some—variable—percentage of their working time to defect fixing in response to reports. The penetration test fed the team dozens of new defects at once, but developers hat a routine to deal with it. Moreover, management tracked the number of open defects, so that the sudden increase raised attention and created pressure on the team to get this number down.

Feature development, on the other hand—where change would have to occur—remained mostly unaffected. Feature development followed the process of Scrum and the penetration test neither had a specific impact here nor did it feed requests or ideas into this routine.

(2) Organizational Structure

Following the ideas of Scrum and agile development, a strong division of labor and responsibilities characterized the organizational structure in the setting of our study. Product management and product owners were responsible for the direction of the development work, whereas development teams enjoyed a certain autonomy in technical questions. This structure worked as a social contract: managers expected developers to take care of security as a matter of quality and developers were keen to deliver the features requested by management. However, the penetration test had little impact on the manager’s priorities beyond the pressure to reduce the number of open defects. The developers thus found themselves in a situation where security was not explicitly required and additional security work could not be justified.

(3) Business Role of Security

Finally, security had limited perceived importance for the business of the studied company, which thus far had not experienced any public security disaster and did not actively sell security. The company therefore lacked a security narrative that could have been used to justify security efforts beyond defect fixing. This, together with the inherent low visibility of security and insecurity, shaped priorities. Product managers knew that features sell their products—new features are easy to show and explain, whereas security improvements are not. Security was perceived as contributing little to the success of the product and the company, making it a low-priority requirement.

Our study points to some of the complexities of managing software development and of triggering change by interventions. While it would be tempting to assign blame to a single factors, such as agile development or negligent management, the problem is really more complicated. Organizational structures and routines exist and they are shaped by business needs. Scrum, for example, is highly beneficial for the studied company. One might even ask whether the company’s dealing with security is a problem in the first place. Are they perhaps doing the right thing for their market and customers?

Our Paper:

A. Poller, L. Kocksch, S. Türpe, F. A. Epp; K. Kinder-Kurlanda: Can Security Become a Routine? A Study of Organizational Change in an Agile Software Development Group. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW’17), February 25–March 1, 2017, Portland, OR, USA.
[DOI: 10.1145/2998181.2998191, BibTeX, slide deck]

Was heißt hier „cyber“?

Die Cyberwehr dient Neuland. Was heißt das?

Seit einigen Jahren und erst recht seit der Ankündigung des Verteidigungsministeriums, die Bundeswehr um das eben aufgestellte Kommando Cyber- und Informationsraum (KdoCIR) als neue Teilstreitkraft zu erweitern, reden alle vom Cyberkrieg. Die Online-Armee soll „Waffensysteme und Computernetze der Bundeswehr schützen, aber auch zu Angriffen in der Lage sein.“

Konkreter wird die öffentliche Diskussion selten. Von hackenden Russen und Chinesen (Spy vs. Spy?) – seltener: Amerikanern et al. – mit staatlichem Auftrag ist öfter die Rede und gelegentlich erinnert jemand an Stuxnet als erste „Cyberwaffe“. Was, wenn jemand so eine Cyberwaffe gegen kritische Infrastrukturen wie die Wasser- oder Energieversorgung unseres Landes einsetzt? Müssen wir da nicht zurückschlagen können?

Jetzt haben wir also eine Cyberwehr, die irgendwas mit Internet, Hacking und IT-Sicherheit macht, auch offensiv und im Ausland, um uns vor sowas zu beschützen. Wann braucht sie dafür ein Bundestagsmandat oder Routingrechte für die Cyber- und Informationsräume anderer Staaten? Wo verlaufen die Grenzen zwischen Angriff und Verteidigung? Ergeben solche Fragen überhaupt einen Sinn und was unterscheidet das Cybermilitär von der Security-Abteilung eines Unternehmens?

Die öffentliche Berichterstattung und Diskussion wirft munter verschiedene Dimensionen des Themas durcheinander. Ich sehe mindestens drei verschiedene Aspekte:

  1. „Cyber“ als IT-Betrieb und Sicherheitsmanagement, wie wir sie kennen: Jede Organisation muss sich um den sicheren Betrieb und das Sicherheitsmanagement ihrer vernetzten Systeme kümmern, so auch die Bundeswehr. Das ist in erster Linie eine Admin- und Abwehraufgabe und für eine Armee die Ausdehnung von Wachdienst und Spionageabwehr ins Internet sowie auf die militärischen Kommunikationssysteme und die IT der Waffensysteme. Bundestagsmandate braucht man dafür so wenig wie für das zivile Gegenstück nach BSI-Kompendium. Soldaten braucht man dafür auch nur bedingt; die Bundeswehr könnte auch einen IT-Dienstleister beauftragen.
  2. „Cyber“ als Erweiterung des Waffenarsenals und Operationsraums: Im Rahmen militärischer Einsätze werden zusätzlich oder alternativ zu herkömmlichen Waffen auch „Cyberwaffen“ eingesetzt, das heißt das Ziel der Operation wird auch durch Eingriffe in Netze und IT-Systeme verfolgt. Statt Bomben auf Kraftwerke zu werfen, könnte die Truppe beispielsweise versuchen, den Strom im Operationsgebiet mit anderen Mitteln abzuschalten. Für solche Einsätze braucht die Bundeswehr ohnehin ein Mandat.
    Abzuwarten bleibt, wie gut „Cyberwaffen“ in diesem Zusammenhang wirklich funktionieren. Kanonen, Gewehre, Geschosse und Bomben haben den Vorteil, im Kern sehr einfachen Funktionsprinzipien zu folgen. Sie lassen sich massenhaft herstellen und unter Feldbedingungen flexibel gegen verschiedene Ziele einsetzen. Ob es eine gute Idee ist, die Energieversorgung zu hacken, statt ein Kommando ins oder Bomber übers Kraftwerk zu schicken, wird sich zeigen.
  3. „Cyber“ als neuer Raum für Konflikte geringer Intensität: Die Informationstechnik ermöglicht auch neuartige Aktionsformen unterhalb der Schwelle zum offenen militärischen Konflikt. Stuxnet ist das prominenteste Beispiel: Ein Staat verfolgt Ziele im Ausland durch Eingriffe in fremde IT-Systeme. Solche Operationen haben eher einen geheimdienstlichen als einen militärischen Charakter.
    Solche Operationen sind neu und international gibt es damit wenig Erfahrung, deshalb stellen sich hier die interessanten Fragen: Was ist ein militärischer Angriff, was nur ein unfreundlicher Akt? Welche Ziele sind legitim, welche Antworten angemessen? Und wie findet überhaupt heraus, wer der Angreifer war? Aufgrund der geheimdienstlichen Natur solcher Operationen wäre die Forderung nach einem vorher zu erteilenden spezifischen Bundestagsmandat kontraproduktiv. Stuxnet illustriert all diese offenen Fragen.

Um qualifiziert und zielführend zu diskutieren, müssen wir uns auf klare gemeinsame Begriffe einigen. Wenn Euch jemand vollcybert oder von hybriden Bedrohungen schwätzt, dann fragt ihn also erst einmal, wovon er eigentlich redet. Denkt daran: „Cyber“ ist immer für eine Geschichte gut und nicht alle Geschichten stimmen auch.

(Getriggert von Holger Schauer bei G+)

PS: Thomas Wiegold erklärt nüchtern, was das KdoCIR tut. Außerdem gibt’s dort die Rede der Vercyberungsministerin zum Anhören und auszugsweise zum Nachlesen.

CfP: 3rd Workshop on Security Information Workers (WSIW 2017)

There will be another edition of WSIW this year and it will be part of SOUPS again, which is in turn co-located with the Usenix Annual Technical Conference. WSIW is concerned with  all kinds of security information work and the people doing such work, like developers, administrators, analysist, consultants, and so on. We were there last year with early results of our penetration testing in software development study. If the subject of your reseach is security work, please consider submitting to WSIW.


Workshop website:

Submissions due: 2017-05-25

Vortrag: „Security by Design?“

Triggerwarnung: work in progress

Vergangene Woche durfte ich auf dem 1. IT-Grundschutztag 2017 zum Thema Security by Design? vortragen. Das Leitthema der Veranstaltung war Application Security und ich habe passend zu unserer Forschung einen Blick auf die Softwareentwicklung geworfen. Sichere Software ist leicht gefordert, aber die Umsetzung im Entwicklungsalltag bereitet Schwierigkeiten: In frühen Phasen der Entwicklung kämpft man mit Ungewissheit und es ist schwer, Entscheidungen zu treffen; später weiß man mehr, aber die Veränderung wird schwierig – nicht nur technisch, sondern auch in den Strukturen und Routinen des Entwicklerteams.

Der Vortrag entstand aus einer früheren Fassung, die ich gemeinsam mit Andreas Poller auf dem Workshop „Partizipatives Privacy by Design“ im vergangenen Oktober in Kassel gehalten habe. Aus Andreas’ Vortrag auf der CSCW’17 habe ich mir auch Slides geborgt.

Wer die Tonspur zu den Slides hören möchte: einfach fragen.

Encryption Will Not Give You Free Speech

“Freedom of speech is the right to articulate one’s opinions and ideas without fear of government retaliation or censorship, or societal sanction.”Wikipedia

Reports of a vulnerability in WhatsApp are making the rounds today after The Guardian boosted the signal. Besides the fact that there is not really a backdoor, but rather a feature that represents a reasonable choice in a tradeoff between confidentiality and availability, the Guardian also repeats a common mistake: confounding encryption and free speech.

“Privacy campaigners criticise WhatsApp vulnerability as a ‘huge threat to freedom of speech,’” writes The Guardian. This is bullshit. As per the definition cited above, free speech means you can say things without fear. Being able to say things only in private and needing strong technical privacy guarantees is the opposite of free speech. You need encryption for that which you cannot say without fear.

Yes, encryption can be a tool against those who suppress you (though a weak one, as your adversary can easily use your use of encryption against you – or deny you due process altogether and persecute you without any trace of evidence and probable cause). But encryption will never give you free speech, it will only support your inner immigration.

Re: Offener Brief zu DNA-Analysen in der Forensik

Mahnungen vor dräuenden Gefahren verkaufen sich immer, sind doch vorhergesagte Probleme nie auszuschließen, ohne dass man ein Risiko eingeht und etwas ausprobiert. So lässt sich beliebig lange spekulieren, was alles passieren könnte, wenn man täte, was man wegen der Risiken besser bleiben ließe. Als neuester Gegenstand solcher „kritischen“ Betrachtungen bietet sich die Forderung nach einer Ausweitung der zulässigen DNA-Analysen in der Polizeiarbeit an. Folgerichtig haben Sozialwissenschaftler einen Offenen Brief zu DNA-Analysen in der Forensik verfasst der zur Vorsicht mahnt und seine Autorinnen als unverzichtbare Expertinnen anbietet. Der Tenor: Erweiterte DNA-Analysen seien viel zu kompliziert als dass man einfache Polizisten unbegleitet mit ihren Ergebnissen arbeiten lassen dürfe. Am Ende steht wenig mehr als die Schlussfolgerung, dass es zu Fehlern kommen könne. Dies jedoch ist eine banale Aussage: Fehler sind in der Polizeiarbeit Alltag und das System aus Gesetzgebung, Polizei und Justiz kann damit gut umgehen. Selbstverständlich muss man die Auswirkungen neuer Methoden betrachten, aber zur Panik gibt es keinen Anlass. Unser Rechtsstaat irrt sich recht zuverlässig zugunsten der Verdächtigen und die Forensiker wissen selbst ganz gut, wo die Grenzen der verschiedenen Analyseverfahren liegen. Unschätzbare Risiken können wir jeder Technik unterstellen, das hilft nur niemandem.


An In-Depth Study of More Than Ten Years of Java Exploitation

My colleagues Philipp Holzinger, Stefan Triller, Alexandre Bartel, and Eric Bodden had a closer look at Java and the vulnerabilities discovered in the Java runtime environment during the last decade. They started from known exploits, identified the vulnerabilities exploited, and analyzed and grouped their root causes. Philipp’s presentation of the results at CCS’16 has been recorded and published on YouTube:


The paper is also available online:

P. Holzinger, S. Triller, A. Bartel, E. Bodden: An In-Depth Study of More Than Ten Years of Java Exploitation. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS’16), Vienna, Austria, Oct. 24-28, 2016. DOI: 10.1145/2976749.2978361. Artifacts:


CAST-Workshop „Sichere Software entwickeln“ am 10. November

Auch in diesem Jahr organisieren wir einen CAST-Workshop zum Thema „Sichere Software entwickeln“. Der Workshop findet am Donnerstag, dem 10. November 2016 am Fraunhofer-SIT in Darmstadt statt. Am Vorabend laden wir zu einem Get-Together ein. Das Programm und alle weiteren Informationen zum Workshop findet Ihr hier:

P.S. Jetzt haben wir auch einen Flyer zum Ausdrucken und Verteilen.

Classifying Vehicles

Security is a classification problem: Security mechanisms, or combinations of mechanisms, need to distinguish that which they should allow to happen from that which they should deny. Two aspects complicate this task. First, security mechanisms often only solve a proxy problem. Authentication mechanisms, for example, usually distinguish some form of token – passwords, keys, sensor input, etc. – rather than the actual actors. Second, adversaries attempt to shape their appearance to pass security mechanisms. To be effective, a security mechanism needs to cover these adaptations, at least the feasible ones.

An everyday problem illustrates this: closing roads for some vehicles but not for others. As a universal but costly solution one might install retractable bollards, issue means to operate them to the drivers of permitted vehicles, and prosecute abuse. This approach is very precise, because classification rests on an artificial feature designed solely for security purposes.

Simpler mechanisms can work sufficiently well if (a) intrinsic features of vehicles are correlated with the desired classification well enough, and (b) modification of these features is subject to constraints so that evading the classifier is infeasible within the adversary model.

Bus traps and sump busters classify vehicles by size, letting lorries and buses pass while stopping common passenger cars. The real intention is to classify vehicles by purpose and operator, but physical dimensions happen to constitute a sufficiently good approximation. Vehicle size correlates with purpose. The distribution of sizes is skewed; there are many more passenger cars than buses, so keeping even just most of them out does a lot. Vehicle dimensions do not change on the fly, and are interdependent with other features and requirements. Although a straightforward way exists to defeat a bus trap – get a car that can pass – this is too expensive for most potential adversaries and their possible gain from the attack.

The Key-Under-the-Doormat Analogy Has a Flaw

The crypto wars are back, and with them the analogy of putting keys under the doormat:

… you can’t build a backdoor into our digital devices that only good guys can use. Just like you can’t put a key under a doormat that only the FBI will ever find.

(Rainey Reitman: An Open Letter to President Obama: This is About Math, Not Politics)

This is only truthy. The problem of distinguishing desirable from undesirable interactions to permit the former and deny the latter lies indeed at the heart of any security problem. I have been arguing for years that security is a classification problem; any key management challenge reminds us of it. I have no doubt that designing a crypto backdoor only law enforcement can use only for legitimate purposes, or any sufficiently close approximation, is a problem we remain far from solving for the foreseeable future.

However, the key-under-the-doormat analogy misrepresents the consequences of not putting keys under the doormat, or at least does not properly explain them. Other than (idealized) crypto, our houses and apartments are not particularly secure to begin with. Even without finding a key under the doormat, SWAT teams and burglars alike can enter with moderate effort. This allows legitimate law enforecement to take place at the cost of a burglary (etc.) risk.

Cryptography can be different. Although real-world implementations often have just as many weaknesses as the physical security of our homes, cryptography can create situations where only a backdoor would allow access to plaintext. If all we have is a properly encrypted blob, there is little hope of finding out anything about its plaintext. This does not imply we must have provisions to avoid that situation no matter what the downsides are, but it does contain a valid problem statement: How should we regulate technology that has the potential to reliably deny law enforcement access to certain data?

The answer will probably remain the same, but acknowledging the problem makes it more powerful. The idea that crypto could not be negotiated about is fundamentalist and therefore wrong. Crypto must be negotiated about and all objective evidence speaks in favor of strong crypto.

Apple, the FBI, and the Omnipotence Paradox

“Can God create a rock so heavy He could not lift it?” this is one version of the omnipotence paradox. To make a long story short, ominpotence as a concept leads to similar logical problems as the naïve set-of-sets and sets-containing-themselves constructions in Russel’s paradox. Some use this paradox to question religion; others use it to question logic; and pondering such questions generally seems to belong to the realm of philosophy. But the ongoing new round of (civil) crypto wars is bringing a tranformed version of this paradox into everyone’s pocket.

Can Apple create an encryption mechanism so strong that even Apple cannot break it? Apple claims so, at least for the particular situation, in their defense against the FBI’s request for help with unlocking a dead terrorist’s iPhone: “As a result of these stronger protections that require data encryption, we are no longer able to use the data extraction process on an iPhone running iOS 8 or later.” Although some residual risk of unknown vulnerabilities remains, this claim seems believable as far as it concerns retroactive circumvention of security defenses. Just as a locksmith can make a lock that will be as hard to break for its maker as for any other locksmith, a gadgetsmith can make gadgets without known backdoors or weaknesses that this gadgetsmith might exploit. This is challenging, but possible.

However, the security of any encryption mechanism hinges on the integrity of key components, such as the encryption algorithm, its implementation, auxiliary functions like key generation and their implementation, and the execution environment of these functions. The maker of a gadget can always weaken it for future access.

Should Apple be allowed to make and sell devices with security mechanisms so strong that neither Apple nor anyone else can break or circumvent them in the course of legitimate investigations? This is the real question here, and a democratic state based on justice and integrity has established institutions and procedures to come to a decision and enforce it. As long as Apple does not rise above states and governments, they will have to comply with laws and regulations if they are not to become the VW of Silicon Valley.

Thus far we do not understand very well how to design systems that allow legitimate law enforcement access while also keeping data secure against illiegitimate access and abuse or excessive use of legitimate means. Perhaps in the end we will have to conclude that too much security would have to be sacrificed for guaranteed law enforcement access, as security experts warn almost in unison, or that a smartphone is too personal a mind extension for anyone to access it without its user’s permission. But this debate we must have: What should the FBI be allowed to access, what would be the design implications of guaranteed access requirements, and which side effects would we need to consider?

For all we know, security experts have a point warning about weakening what does already break more often than not. To expectat that companies could stand above the law because security, however, is just silly.

PS, remember Clarke’s first law: “When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.”

PPS: Last Week Tonight with John Oliver: Encryption