- Introduction
- Hash Functions
- Merkle Trees
- Symmetric Encryption
- Asymmetric Encryption
- Digital Signatures
- Summary
- What did we miss?
- Readings
- Exercises
Defined as the study of secret writing, cryptography has roots as old as history in message obfuscation. Traditional methods involve applying an algorithm or a key to a message to produce a cipher text. This process is known as encryption. The message recipient then applies secret knowledge to the ciphertext, decrypting it, to recover the original message. Curious eyes and self-serving couriers may intercept the cipher text before its destination and try to crack the code. This is the risk those take for secure communication. Cryptanalysis is the science of breaking codes to reveal messages without having knowledge of the key.
A basic encryption scheme is one used by school children where letters are substituted with an alternate according to a prescribed routine. Shifting letters by a set number of characters in the alphabet is known as a Caesar cipher. Without knowing how many letters were shifted, it would take an eavesdropper at most 25 attempts to decrypt an English message by brute force.
Other notable ciphers are the Hill cipher, Vigenere cipher, and one-time pad cipher (Singh, 1999). Hill and Vigenere both have weaknesses and can be broken; a well-constructed one-time-pad cipher can never be broken. The one-time works by transposing a message using a random key the same length as the message. This prevents repetition in both within key and from common phrases of speech, such as 'the', or context such as 'Russians'. If the key is sufficiently random, only someone with knowledge of the key can decrypt the message. Each new message requires a new key, and thus the name one-time pad. This poses many practical problems such as production and finding good sources of entropy (randomness). The primary limiting factor for this type of encryption is distribution of the keys to both parties. Long message require longer keys, and many messages require many keys.
Cryptographic hash functions are a fundamental tool in the field of cryptography, serving as the basis of numerous information security applications. These deterministic algorithms take an arbitrary (variable) amount of input data and convert it into a fixed-size string of bytes, typically in the form of a hash digest. Uniquely, any minor alteration in the input data results in a drastically different output, a property known as the avalanche effect. Moreover, cryptographic hash functions exhibit essential security attributes including pre-image resistance, second pre-image resistance, and collision resistance.
A hash function takes a variable length input and produces a fixed length output with no discernible pattern. In this sense, the hash function is one-way which means that there is no decryption algorithm that can undo it. (In fact, there's no encryption here at all, as encryption implies the message can be decrypted.)
+------------+ +-------------+
| message | ---> | hash function| ---> message digest
| `x` | | H(x) |
+------------+ +-------------+
A good hash is easy to calculate (verify) but difficult to reverse. Given a hash, it should be computationally infeasible to both produce the data that created the hash (pre-image resistance) or to find a message that produces the same hash (2nd pre-image resistance). The figure below shows a simple hash function that takes characters as inputs and outputs 1 if the character is in the first half of the alphabet (a-m) and 0 otherwise.
+------------+ +-----------------+
| `code` | ---> | H(`code`) | ---> `1011`
+------------+ +-----------------+
If you begin with the hashed value of 1011, it is merely guesswork to try to determine the original string because 13 possible values could all produce a 1. After hashing more words, you realize that many other possibilities also produce 1011 as their hashed value, such as joke (or lame, lone, male, mine, etc.). If two or more different strings produce the same hash, this is called a collision.
+------------+ +-----------------+
| `joke` | ---> | H(`joke`) | ---> `1011`
+------------+ +-----------------+
Finding a hash function that doesn't output collisions
SHA-256 is a secure hashing algorithm that outputs a 256-bit message digest. It is part of a family of hash functions designed by the United States National Security Agency and published by the National Institute of Standards and Technology (see required reading). One of the key components is the bitwise XOR (exclusive-or) operation which outputs 0 if the two inputs are the same and 1 otherwise. This inclusion makes it difficult to back-track from a hash and figure out what data went into it. (Using the simple example above, if the hash is 1111
you know characters all map to the first half of the alphabet.) Just like trying to figure out the Colonel's 11 herbs and spices1 from tasting, it could take a lifetime to recreate the secret recipe.
Here you see the SHA-256 output of the message jeff
as compared to Jeff
. It is a string of 64 hexidecimal characters, which has been converted from a string of 256 bits (binary digits).
+------------+ +---------------+
| `jeff` | ---> | SHA-256 | ---> `2e0b8d61fa2a6959d254b6ff5d0fb512249329097336a35568089933b49abdde`
+------------+ +---------------+
+------------+ +---------------+
| `Jeff` | ---> | SHA-256 | ---> `aea5a5ee6792fab36f3c60e62ef64e40061a8dac7d868b359aff7a4786a32e51`
+------------+ +---------------+
Figure: SHA-256 has been used to hash the messages
jeff
andJeff
. There should be no way to link the two hashes.
Each hex character has 16 possible options, so the size of the solution space for SHA-256 is
In more manageable scientific notation this is about
Hashing is sometimes referred to as a one-way function. This is in the mathematical sense that it is easy to follow the algorithm and create a hash using some input string, but hard to start with the string and determine the data. You cannot go backwards. This property gives rise to verification. If someone sends you a hash, you can easily run your own computation to verify that the message used to produce the hash is authentic.
Verification is a key concept. If Alice sends Bob an important message such as "attack at dawn", its in Bob's best interest to verify the message is genuine and has not been altered. Alice can send the plaintext:
attack at dawn
, and the hash:d502810c71aeb17e5ea1cbf930b46b87bb645a75df45f500230d061992aeb90a
. Upon receipt, Bob needs to spend a small amount of effort using his computer to verify the hash.
The Simple Mail Transfer Protocol (SMTP) was defined by Jon Postel in 1982. As a standard, this would allow anyone to write an email client according to the SMTP guidelines and be able to send messages to anyone else that also followed the protocol. It is still used today and has undergone many updates, however SMTP has two problems: all text is sent as plaintext, and it is easy to spoof from addresses and generate spam.
Email spam is an issue that was considered by Dwork and Naor (1993) when they wrote a paper describing the computational cost incurred by a spammer before sending an email. Titled Pricing via Processing, the idea is that your computer has to solve some computational puzzle before being permitted to send an email. For the average user this would take seconds in the background and not be a nuisance, but for a spam emailer this would slow down their operation. A few years later in 1997, Back (2002) created Hashcash in a similar vein. Although Back was unaware of the previous work by Dwork and Naor, Back also implemented the idea of a cost function that has an easily verifiable solution. The cost here is the work that your computer does to run a hashing algorithm which adds up when trying to scale a spam operation. This is now known as Proof-of-Work and used by Bitcoin miners.
Recall that the links between blocks are called hash pointers. If an adversary wishes to modify data in a block, such as financial transaction data, the resulting hash pointer will have changed and need to be updated. So rather than having to store the entire dataset for verification, we can store the hash pointers and easily see if they are correct. Our adversary that altered some data would have to change every subsequent hash pointer to the tip of the blockchain. Simply storing the most recent hash in a place that can't be modified is enough to verify the whole chain. A good way to do this is by keeping multiple copies in different locations.
SHA-256 is used in various places in Bitcoin (and blockchains) such as: address generation, block pointers, Merkle trees, and proof-of-work mining.
Recall our ledger consisting of pages (transaction data) that is bundled into entire books (blocks) and linked by a numbering system (hash pointer). The pages inside the books can also be represented using hash pointers through a Merkle tree, named after computer scientist and cryptographer Ralph Merkle. The leaves of the binary tree structure are the data, and a hash pointer is created for every pair of leaves. When there are two pairs of leaves, the two hash pointers are combined and hashed into a third hash pointer. This continues until the root of the tree is a single hash representing all the data.
Figure: An individual block contains a Merkle tree of all the transactions in the block. Note it also contains the transactions (pages) themselves. The Merkle root is quick to verify that none of the transactions have changed.
Following the tree in the Figure, Page 1 is hashed as
"hash": "00000000000000000000534d3d2c7758fab39dabb98d23b954813379f053c580",
"confirmations": 1,
"strippedsize": 130543,
"size": 174555,
"weight": 566184,
"height": 620229,
"version": 536928256,
"versionHex": "2000e000",
"merkleroot": "cdfc01a6d3a9f037670be9c17dba180e6b281764cb402ead941b2a439c1d801d",
"tx": [
"d55c96ce6ccf995035338f4ded57850f307299b0756b44c61b2fe5e5c2c89a51",
"...truncated...",
"22ba973e71869d1e64cdd8572fa94754429d52d84bce61b2374f082606e02ec5",
],
"time": 1583366597,
"mediantime": 1583365586,
"nonce": 4225421315,
"bits": "17122cbc",
"difficulty": 15486913440292.87,
"chainwork": "00000000000000000000000000000000000000000d4bf930c8adffc9b1a4b136",
"nTx": 350,
"previousblockhash": "000000000000000000070be3e6873e60481b5e3c71322c8ced8315f6e44edd6e"
Figure: block header info for Bitcoin block #620229, note the
merkleroot
.
A Merkle path from leaf to root takes
Figure: Objects
A
toP
stored as leaf nodes in a Merkle tree. Tracing a path to verify a letter is present should occur in$\log n$ time. Note: *.gif doesn't play.
The cryptographic methods discussed so far, including hash functions and Merkle trees, serve two important roles. First, they obfuscate data: from a given hash, we cannot deduce the original data inputted into the hash function. This provides a level of data security. Second, these techniques enable quick and efficient data verification. For example, when we receive a software binary file along with its hash value, we can promptly verify the integrity of the software. This means checking that the hash of the received file matches the given hash, ensuring that the file has not been tampered with and accurately represents the intended source code.
These tools do not, however, allow us to communicate privately.
The Diffie-Hellman (& Merkle) key exchange algorithm was devised so that two parties could use insecure communication channels to determine a common secret key (Diffie, 1976).
This algorithm starts with two public values that are pre-computed, a large prime number
Once a primitive root
For example: taking
*Rows 3,4, and 5 have repeated roots:
Now for
These roots are all distinct up to
For example, to find the discrete logarithm of an integer such as
The left-hand side can be reduced modulo 7:
Now we look to the table to find the distinct root
The security of the Diffie-Hellman key exchange relies on the difficulty of reversing this computation above. Given
Q: So how do two people use this to exchange information publicly such that they each end up with a shared secret key that others can not deduce?
Alice and Jeff are going to agree on a large prime,
Note that both Alice (
and a cryptanalyst cannot calculate
For example, an actual 160 bit prime number:
One drawback of this system is that to communicate there has to be some back-and-forth between participants to agree on a secret key which can be particularly cumbersome if one participant is offline. If a third person wants to communicate, then a another pair of exchanges must take place. Every pair that needs to communicate needs their own secret key. A group of 40 students in this class requires 780 keys5. What if instead each person had their own key and everyone else could use that same key to communicate with them?
The search for a personal key involves finding a one-way function that some insider information can quickly reverse. This is the foundation of public-key cryptography. A user has a key that they can share and use to encrypt messages, called the public-key. They also have a trapdoor into that one-way function to decrypt messages, called the private-key. Sometimes the term backdoor is used to indicate insider access unknown to users. This is different from a trapdoor as it introduces a deliberate weakness into the cryptosystem, whereas a trapdoor is simply information that is easy to compute one-way.
In 1977, Ron Rivest came up with a scheme to deliver exactly this: a key-pair system that allows universal encryption with some additional information to allow the owner (and only that owner) to decrypt messages. Rivest and his colleagues Adi Shamir and Leonard Adleman's system is simply referred to eponymously as RSA encryption (Rivest, 1978). The details will be skipped here, but in brief, RSA uses the properties of large prime numbers to generate encryption keys that are hard to reverse engineer. It is easy to multiply two prime numbers and calculate the output (semi-prime), but given a large semi-prime it is much more difficult to determine the two prime factors.
For example, veryify that:
Now, given a roughly equivallent number,
Due to progress in calculating prime factorizations such as the quadratic sieve, RSA public keys need to be at least 1024 bits to provide adequate security. Elliptic Curve Cryptography (ECC) is a promising alternative to RSA for public-key encryption, allowing a much shorter key to be used with far less computational overhead, yet providing the same level of security as RSA against a cryptanalysis attack.
For instance, in order to provide roughly the same level of security as a 128-bit AES6 key, RSA requires a 3072-bit key, which places a computational burden on any devices using RSA. Worse still, to be equivalent to a 256-bit AES key, RSA requires a 15360-bit key, which is infeasible on devices with limited power and computation such as mobile phones. ECC however requires just double the number of bits than an AES key, making it particularly attractive for public-key encryption on limited devices. It has begun to challenge RSA and Diffie-Hellman key exchange as the preferred public-key cryptographic algorithm, since the difficulty of cryptanalysis against ECC gets harder for longer keys much faster than for either RSA or Diffie-Hellman (Stallings, 2017).
Table: Private key sizes in bits. The rows represent roughly equivallent levels of encryption in terms of computational power required to brute force the keys.
AES | RSA | ECC |
---|---|---|
symmetric | asymmetric | asymmetric |
--- | 512 | 112 |
--- | 1024 | 160 |
--- | 2048 | 224 |
128 | 3072 | 256 |
192 | 7680 | 384 |
256 | 15360 | 512 |
An elliptic curve over a field consists of the set of points
If
Further, if
Figure: Plot of a field of primes up to 17 (Antonopoulos, 2017). ECC operates on integers and when plotted appear as distinct points.*
Figure: Plot of$y^2=(x^3+7)$ over real numbers for visualization only; the same theory applies.
ECC can be utilized for key exchange by making public the chosen field (secp256k1
that is used by Bitcoin.
A digital signature arises as a consequence of public-key cryptosystems such as RSA & ECC. It allows a user to do three things:
- Verify the authenticity of the sender of a message; e.g. if the IRD sends you a tax invoice it is helpful to trust the source,
- Prevent a sender from denying they sent the message; this is called non-repudiation and prevents someone from saying they were not responsible, and
- Validate the integrity of the message to ensure it wasn't tampered with during transport; e.g. if a general sends a scout to deliver a message the recipient can be sure the scout did not change the content en route.
Any standard public-key encryption algorithm such as RSA or ECC combined with a cryptographic hash function such as the SHA-1 family can be used to create a digital signature, so long as recipients are aware of which encryption algorithm and hash function have been chosen and the correct public key is available to the recipients. Bitcoin and Ethereum both use the elliptic curve digital signature algorithm (ECDSA).
If a sender encrypts a message with their private key, anyone with the public key can decrypt the message. Since the public key is available, anyone has the ability to quickly verify that a message or transaction came from the holder of the private key. This is all done without exposing the private keys and so is as secure as having someone encrypt a message with your public key.
Today encryption is ubiquitous in digital communication. An inspection of the certificate of a secure website will often reveal one of two cryptosystems at work: RSA, or ECC. WhatsApp uses ECDH for key exchange and AES256 symmetric encryption for message sessions[^9]. Telegram uses 2048 bit RSA encryption and Diffie-Hellman key exchange with a symmetric protocol based on AES256[^10]. WeChat does not use end-to-end encryption and has been criticised for its lack of privacy features.
Contrary to popular belief there is no standardized encryption in the Bitcoin protocol. As a decentralized system of exchange, there is no need for encryption. All the transactions are stored in the blockchain, open and accessible to everyone. Access to the private keys controlling addresses is maintained solely through personal security of the user.
So can someone steal my coins? Not by hacking. Elliptical curve cryptography keeps keys safe from cryptanalysis via the difficulty in solving the discrete logarithm problem. The standard curve secp256k1
was chosen by the designer of the Bitcoin protocol. This is a choice unique to Bitcoin, as the more common secp256r1
is used in Transport Layer Security (TLS) for web browsing, email, etc. ECC was chosen because it provides the same level of relative security with smaller keys compared to RSA.
As in any cryptosystem, it is imperative that users keep their private keys secret. There are some added enhancements for encrypting passwords and wallets but these are third party additions and not built in to the protocol. This will be discussed further in a future lecture.
Digital signatures allow a blockchain user to transfer ownership of a token by verifying they have the authority to do so. Tokens are mapped to addresses and to transfer a token the owner must sign a transaction to the recipient's address. Additionally this allows the recipient to verify the message has not been altered at any time. Every transaction on a blockchain is digitally signed by the parties involved; this means they have used their private key to attest to the transaction.
- Zero-knowledge proofs are an applied branch of cryptography. We will return to these (at a high level) in our lecture on Privacy.
- Find out what encryption your browser site is using. Now check a different site, do you notice any difference?
- SHA256 - Open a terminal window.
Windows:windows key + R
, typepowershell
, press enter. Change where it saysYourString
to your message and paste in7.
$stringAsStream = [System.IO.MemoryStream]::new()
$writer = [System.IO.StreamWriter]::new($stringAsStream)
$writer.write("YourString")
$writer.Flush()
$stringAsStream.Position = 0
Get-FileHash -InputStream $stringAsStream | Select-Object Hash
OSX: Command + Space
, type Terminal
, press enter. Change where it says my name is Jeff
to your message and paste in
echo -n "my name is Jeff" | shasum -a 256
- What is the main mathematical difference between RSA and ECC cryptosystems?
- How would you go about finding the prime factorisation of
$1961$ ?
- New Directions in Cryptography by Whitfield Diffie and Martin Hellman (pdf)
- WhatsApp Encryption Overview - Technical white paper (pdf)
- NIST specification for SHA-1 (pdf) (download for proper font rendering)
- Antonopoulos, A. 2017. Mastering Bitcoin: Unlocking digital cryptocurrencies. O'Reilly Media, Inc.
- Back, A. 2002. Hashcash: A denial of service counter-measure. http://www.hashcash.org/
- Brown, D. R. L. 2010. SEC 2: Recommended elliptic curve domain parameters. Standards for Efficient Cryptography (SEC) (Certicom Research). https://www.secg.org/sec2-v2.pdf
- Diffie, W., Hellman, M. 1976. "New Directions in Cryptography." IEEE Transactions on Information Theory. 22 (6): 644–654.
- Dwork, C., Naor, M. 1993. Pricing via processing or combatting junk mail. http://www.wisdom.weizmann.ac.il/~naor/PAPERS/pvp.pdf
- Hoffman, P. 2005. "Algorithms for Internet Key Exchange (IKEv2)." RFC 4109. Internet Engineering Task Force. https://tools.ietf.org/html/rfc4109
- Postel, J. 1982. "Simple Mail Transfer Protocol." RFC 821. Internet Engineering Task Force. https://www.rfc-editor.org/rfc/rfc821
- Rivest, R., Shamir, A., & Adleman, L. 1978. "A Method for Obtaining Digital Signatures and Public-Key Cryptosystems." Communications of the ACM. 21(2), 120-126.
- Singh, S. 1999. "The Code Book." Doubleday.
- Stallings, W. 2017. Cryptography and Network Security: Principles and Practice. Pearson.
Here's this lecture recorded live July 25, 2023 on YouTube.
Footnotes
-
The true recipe, it is said, is locked away in a vault at KFC's headquarters in Louisville, Kentucky. ↩
-
A somewhat close comparison is the estimated number of atoms in the universe at about 10^80. There are 1000 times more atoms than potential hashes. This is comforting. ↩
-
mod is the modulo arithmetic operator. ↩
-
160 bits is about a 49 digit decimal number, and 1024 bits is a 309 digit decimal number. ↩
-
Key transport was a big problem in World War II. Uboats that were away for months at a time had to have an updated listing of keys to use to communicate with central command back at base. The Germans published key-books tied to a calendar so that subs could determine their key based on the day. Everything is fine as long as a code book doesn't fall into enemy hands. ↩
-
AES is Advanced Encryption Standard, published by the NIST in 2001. ↩
-
Crazy complicated to do on Windows because they won't natively handle a string, so it has to be converted to a stream ↩