Advanced Cryptography

16 min read Original article ↗

MF3d via GettyImages

On this page
  1. Introduction
  2. When to use Advanced Cryptography
  3. Identifying an appropriate solution
  4. Specific risks for Advanced Cryptography
  5. Privacy considerations
  6. Conclusion
  7. Appendix: Advanced Cryptography Techniques and Applications

Introduction

The use of cryptography to provide confidentiality, integrity and authentication is well established. Symmetric and asymmetric encryption techniques are used to provide confidentiality and integrity protection to data. Digital signatures, usually backed by supporting technologies – such as Public Key Infrastructures – are used to provide authentication. These can be combined, for example to provide authenticated encryption.

Recent developments in the scientific field of cryptography hold the promise of enabling novel ways of processing data in encrypted form, improving privacy and security, and reducing the exposure of data held or transmitted in unencrypted form.

These emerging techniques, collectively called Advanced Cryptography, can be used in scenarios where traditional cryptography may be insufficient, such as where the various parties do not fully trust each other with their unencrypted data. However, compared to traditional methods of encryption, the security of advanced techniques are not yet as well understood, and they may be significantly slower and more expensive to run.

In this white paper, we define what the NCSC means by Advanced Cryptography, and propose several real-world applications of the technology. We also discuss some of the limitations of Advanced Cryptography and present a framework for making decisions around whether an Advanced Cryptography solution is appropriate, or whether a more conventional solution would suffice. 

This white paper will be useful to cyber security professionals who make technical decisions about systems for secure processing of data at all levels of sensitivity.

What is Advanced Cryptography ?

The NCSC uses the term Advanced Cryptography to refer to a range of techniques that use cryptographic methods to provide data-processing functionality beyond that which can be achieved by applications of traditional cryptographic functions. These include techniques where multiple parties cooperate to process the data, but have only partial trust in each other so wish to keep some of their information secret. They vary in their degrees of computational complexity as practical solutions, and the extent to which their security properties are fully understood.

The NCSC defines Advanced Cryptography as Cryptographic techniques for processing encrypted data, providing enhanced functionality over and above that provided by traditional cryptography.

The following are examples of Advanced Cryptography techniques illustrating the kind of enhanced functionality that can be achieved. These examples are expanded upon in the Appendix.

  • Homomorphic encryption:

    performing calculations directly on encrypted data

  • Private information retrieval:

    conducting a query into a database, without revealing the query to the owner of the database

  • Multiparty computation:

    carrying out a calculation cooperatively, but without sharing secret inputs

  • Zero-knowledge proofs:

    proving the possession of some secret knowledge without revealing it

  • Private set intersection:

    learning which data items you have in common with someone else, without revealing your whole list

  • Attribute-based encryption:

    limiting decryption of messages to anyone possessing a specific set of attributes

Relation to Privacy Enhancing Technologies

The term Privacy Enhancing Technologies (PETs) is commonly used to refer to a variety of technologies that enable the privacy-respecting processing of data. There is some overlap between Advanced Cryptography and PETs, but the two terms are not synonymous.

Advanced Cryptography provides the enabling functionality for some PETs. However, some other PETs are applications of traditional cryptography - for example, anonymous web-browsing technologies are usually based on traditional cryptography and are not reliant on Advanced Cryptography. Other PETs are not cryptographic in nature at all, for example statistical techniques for anonymising or adding noise to datasets to enhance privacy.

This white paper focuses on the applications of Advanced Cryptography, not the other forms of PETs. 

When to use Advanced Cryptography

When deciding whether to use Advanced Cryptography, start with a clear articulation of the problem, and use that to guide the development of an appropriate solution. That is, you should not start with an Advanced Cryptography technique, and then attempt to fit the functionality it provides to the problem. 

Cryptography can be used to secure data during processing, where some of the data is sensitive and should not be shared openly. However, traditional cryptography can be a barrier if not every party engaged in processing the data has equal permission to view and/or modify all the data.

Advanced Cryptography might provide an appropriate solution to a secure data processing problem if most of the following are true:

  • multiple parties need to cooperate to achieve the desired result
  • there is no trusted third party that can perform the operation on behalf of the parties
  • a significant increase in performance overhead (such as computation cost, communications bandwidth or processing time) is acceptable
  • a small number of participating parties may be unreliable or act dishonestly.

Answering the following questions should help you to identify whether Advanced Cryptography is likely to be a good answer to your problem. If you are unsure how to proceed, you could explore these questions with your supplier base, with a consultant who has appropriate cryptographic expertise, or (for UK government and public sector use-cases) with the NCSC. 

Define the problem statement

  • What are you trying to achieve? What kinds of data are you processing, what actions need to occur, where does the data need to be, and how will data be stored and communicated?

  • What data is sensitive and requires strong cryptographic protection; what data is less sensitive?

  • What operation or computation do you need to perform? 

  • Who are the parties participating in the data processing application, and what are their roles (for example, client, server, trusted third party, untrusted proxy)?

  • Can individuals consent to having their private data processed in this way? If not, can you establish a legal basis for the processing of private data?

  • What practical constraints exist (for example, on bandwidth or response time, or on the processing power of the parties' devices)?

Understand the threat model

  • Who would benefit from compromising the sensitive data?

  • What kind of access to data do the threat actors have? For example, do they have access to the raw data at source, can they read or manipulate log files, can they forge requests or responses?

  • Can you trust each party to only send legitimate, well-formed messages (and not to try to subvert the application to access sensitive data)?

  • Can you trust each party to not attempt to discover additional sensitive data beyond what they are authorised to access, through examination of well-formed messages?

  • Is there a danger of the parties colluding with each other (or with an external threat actor) to obtain access to sensitive data?

  • Are the communication channels between the parties secured so that a threat actor cannot eavesdrop on (or interfere with) data in transit?

Explore possible solutions

  • What is the status quo; how is this problem tackled at the moment? Does this suffice?

  • Can trust be improved between the parties, for example by obtaining user consent to data processing, using a trusted intermediary or making data-sharing agreements?

  • Is it possible to design a solution to the problem built entirely with traditional cryptographic technologies (such as secure communications through TLS or SSH channels, encrypted databases, public key infrastructures)? If not, what aspects of the problem are not solved?

  • Which Advanced Cryptography technique (or combination of techniques) solves an aspect of the problem not met by traditional cryptography?

  • What additional restrictions would an Advanced Cryptography solution impose? (for example, limited formats and quantities of data, data throughput rate, processing latency)

  • Can you assess the cost of the whole solution (which may include system engineering, resource usage, new hardware, licence fees, maintenance and support contracts)?

Examine the risks

  • Does a product implementing the solution already exist, or would you need to build your own?

  • Are you confident that the proposed solution would satisfy the operational requirements? Are you able to run a pilot exercise to test its viability?

  • How will you gain confidence in the cryptographic security of the proposed solution? Note that, so far, there are few standards or certified implementations of Advanced Cryptography.

  • Have you performed a risk assessment of a failure (malicious or accidental) of the solution to cryptographically secure the data it is supposed to be protecting?

Identifying an appropriate solution

Rather than starting with Advanced Cryptography, first consider whether traditional cryptography will solve your data processing problem.

Many problems in data protection have solutions that can be formulated in terms of traditional cryptographic operations. If they suffice for the problem, these traditional cryptography solutions may be preferable to Advanced Cryptography solutions, for the following reasons:

  • Traditional cryptography exists in the form of algorithmic standards. Standardisation exposes algorithms to extensive scrutiny from academia, industry and government, so the security properties are generally well-understood.
  • Traditional cryptography is much less computationally complex than some Advanced Cryptography techniques, so will generally be highly performant and will sometimes benefit from hardware acceleration on commodity IT platforms.
  • Standards-based cryptography will offer a route to certification (for example through a cryptographic validation programme) and may be interoperable with other vendors' products, helping to avoid vendor lock-in.

Consider also whether there is an additional assumption that could be made (for example, using a centralised trusted authority, a mutually-trusted intermediary, or a different hardware processing platform such as a Trusted Execution Environment) that would enable an overall simpler solution.

Choosing an Advanced Cryptography solution

There is a small but growing number of Advanced Cryptography products available on a commercial or open-source basis. These are typically more niche than mainstream traditional cryptography products, but their vendors have identified use cases that are relevant to certain sectors – such as healthcare – where the privacy of the individual’s medical data is paramount.

From the definition of your problem statement and your threat model, you should be able to identify which of the Advanced Cryptography techniques is applicable to your use case. You will then need to research which products can implement this technique. Your research should identify the product’s

  • applicability to your problem
  • performance characteristics
  • security level
  • certifications achieved (if any)

It is a good idea to seek a demonstration of the solution in a small-scale pilot, but be aware that the performance on operational-sized problems will be slower so you should seek reassurance on the scalability of the solution. You should also ensure that the product offers the functionality that your user-base will require (for example, being able to make sufficiently general queries) and seek advice on any cyber security implications of using the product, such as what its use would mean for your cyber intrusion detection systems.

Note

In almost all cases, it is bad practice for users to design and/or implement their own cryptography; this applies to Advanced Cryptography even more than traditional cryptography because of the complexity of the algorithms. It also applies to writing your own application based on a cryptographic library that implements the Advanced Cryptography primitive operations, because subtle flaws in how they are used can lead to serious security weaknesses.

Specific risks for Advanced Cryptography

Advanced Cryptography is an emerging field, and some of the properties of its techniques may differ in unexpected ways from the more prevalent forms of cryptography.

  • Some Advanced Cryptography techniques have a very high computational burden; while optimisations continue to be developed to improve their performance, solutions may still be impractical for operational-sized problems while maintaining an appropriate cryptographic security level.

  • Advanced Cryptography solutions can be complex to deploy, perhaps requiring new infrastructure or new data handling processes, and almost certainly requiring specialist expertise to configure and support.

  • Advanced Cryptography solutions may not provide all the security properties you would normally expect from cryptography. For example, Fully Homomorphic Encryption does not provide data integrity by design, only confidentiality.

  • Some Advanced Cryptography techniques can leak sensitive information even without breaking the cryptography, for example by traffic flow analysis. These can only be mitigated through procedural means, so it is essential that users adhere to operating procedures.

  • Be cautious of introducing additional cyber security vulnerabilities through the adoption of Advanced Cryptography. Encrypted queries and encrypted responses provide a covert channel through which a system could theoretically be attacked and data leaked, and therefore present a cyber security risk. Intrusion detection systems will not be able to understand or validate the encrypted queries and responses and will therefore not be able to scan them for attacks or data leaks.

  • While there is effort towards developing standards for some techniques, there are no certification programmes for Advanced Cryptography.

  • If you do not trust another party to process your sensitive data, consider whether you can trust them to provide you with the correct data, to behave honestly in any cryptographic protocol, or not to launch a cyber attack against you. 

  • Remember that no amount of cryptographic protection can assure the quality of the data provided by another party; Advanced Cryptography cannot provide any guarantee of correctness.

Privacy considerations

Traditional cryptography, Advanced Cryptography and PETs can all help data processors comply with the data minimisation principle and the security principle enshrined in the UK GDPR. Encrypting the private data of an individual (and minimising the number of parties that have access to decrypt the private data) can be proportionate measures for minimising the exposure of the private data, helping to ensure that it is only used for the stated purpose.

Advanced cryptography will not automatically provide a ‘more compliant’ solution. Under Article 25 of the UK GDPR, data controllers must ensure that solutions are designed to provide data protection by design and by default. It is not, however, necessary to adopt the most innovative or cryptographically secure solution for protecting private data to be compliant with UK GDPR and data protection law, especially if the technology is immature or impractical.

Under the UK GDPR, data controllers have a responsibility to ensure that their data is only used by data processors for lawful purposes and in compliance with the UK GDPR. If the controller is unable to verify this directly (because, for example, the data processing is underneath a layer of encryption that they cannot read), they must have another means to ensure that the data processor is meeting its own UK GDPR compliance obligations. 

According to Article 35(1) of the UK GDPR, where the type of data processing is likely to result in a high risk to the rights and freedoms of natural persons, a data protection impact assessment is required; the use of innovative technology (such as Advanced Cryptography) and the purpose of the processing are relevant factors in determining whether the data processing is high-risk.

These are just a few examples of potential considerations, and the requirements for privacy compliance will vary from one case to another. For information on the data protection implications of various PETs, please refer to the Information Commissioner’s Office guidance on Privacy-enhancing technologies (PDF). 

If you are in doubt about the data protection implications of your use of Advanced Cryptography, you should seek specific legal advice.

Conclusion

Advanced Cryptography covers a range of techniques for protecting sensitive data at rest, in transit and in use. These techniques enable novel applications with different trust relationships between the parties, as compared to traditional cryptographic methods for encryption and authentication.

However, there are a number of factors to consider before deploying a solution based on Advanced Cryptography, including the relative immaturity of the techniques and their implementations, significant computational burdens and slow response times, and the risk of opening up additional cyber attack vectors.

There are initiatives underway to standardise some forms of Advanced Cryptography, and the efficiency of implementations is continually improving. While many data processing problems can be solved with traditional cryptography (which will usually lead to a simpler, lower-cost and more mature solution) for those that cannot, Advanced Cryptography techniques could in the future enable innovative ways of deriving benefit from large shared datasets,  without compromising individuals' privacy.

Appendix: Advanced Cryptography Techniques and Applications

This appendix describes some commonly encountered techniques in Advanced Cryptography that provide useful functionality in secure data processing.

Homomorphic Encryption

In traditional forms of encryption, the only useful operation that can be performed on encrypted data (or ‘ciphertext’) is to decrypt it again to yield the original plaintext. Homomorphic Encryption allows calculations to be performed on encrypted data directly, without first decrypting the data. For example, two ciphertexts can be added together, such that when their sum is later decrypted, the result is the sum of the corresponding plaintexts. There are different types of Homomorphic Encryption, usually called ‘Partial’, ‘Somewhat’ and ‘Fully’ Homomorphic Encryption, in increasing order of utility but also computational complexity.

Private Information Retrieval

Private information retrieval (PIR) allows a query to be conducted on a database held by another party, with the cooperation of the database owner but without revealing to them what query was submitted, which items of the database were accessed, or what the response to the query was. As an example, PIR is often implemented using Homomorphic Encryption methods.

Example applications: 

  • Searching a database when it is preferable to not disclose the query data (for example, queries for personal medical or genetic data).
  • A bank performing ‘Know Your Customer’ checks querying records held by other banks and institutions to assess financial risks without disclosing the identity of the customer.

Multiparty Computation

Multiparty Computation (MPC),  which is also known as Secure Multiparty Computation, allows a number of cooperating parties to jointly compute an output which is a function of their combined inputs, without ever revealing their inputs to each other. With traditional cryptography, this is possible if there is a trusted third party to perform the computation. Using secure multiparty computation, the trusted third party is not needed. 

Example applications:

  • In financial trading markets, exchanges can match large trade orders without exposing the intentions of their clients, or being able to exploit their privileged position to gain information themselves.
  • Private auctions, allowing the winning bid to be determined while keeping the other bids secret from participants and the auctioneer.

Zero Knowledge Proofs

Zero Knowledge Proofs (ZKP) allow one party (the ‘prover’) to demonstrate to another (the ‘verifier’) that they are in possession of some secret knowledge or have correctly run some computation, without revealing anything about that knowledge. 

Example applications:

  • Proving that a financial transaction has been recorded, without revealing details of the sender, the recipient or the amount.
  • Proving that a user is over a threshold minimum age required to access an online service without providing other proof of identity or date of birth.

Private Set Intersection

In this scenario, two parties each hold a list of data items that they are unwilling to share with the other in their entirety. However, they do agree to cooperate to determine which data items they have in common. Private Set Intersection (PSI) allows one party to compute the intersection of its list with that of another, learning only the common items (or the number of common items) and nothing more.

Example applications:

  • Detecting the appearance of your username and password in a cyber security breach dataset.
  • Fraud detection by banks cooperating to discover patterns of suspicious activity by account holders.

Attribute-Based Encryption

In an Attribute-Based Encryption (ABE) scheme, messages are encrypted so that they can be decrypted by any party possessing a specified attribute or combination of attributes, such as membership of a group, or even having an attribute within a range.

Example application:

  • Broadcasting encrypted materials, so that they can only be decrypted by paid subscribers within a certain geographic region and over a certain age.