What is Encryption? The Math That Keeps Your Data Private

Every time you log into a website over HTTPS, every time you send a message through Signal or iMessage, every time your bank stores your account balance — encryption is running. It's the mathematical foundation that makes private communication possible over inherently public networks.

Most developers use encryption constantly without understanding it in any depth. That's mostly fine — using AES-256-GCM through a well-audited library doesn't require a PhD in cryptography. But understanding the concepts well enough to make correct decisions about when and how to use encryption is genuinely important. Getting it wrong — using the wrong algorithm, implementing it incorrectly, or confusing encryption with related-but-distinct concepts like hashing — produces code that looks secure but isn't.

The Core Concepts

Encryption takes plaintext — readable data — and transforms it into ciphertext — scrambled, unreadable data — using an algorithm and a key. Decryption reverses the process using the corresponding key.

Plaintext:  "your bank password is hunter2"
    + Key
    + Algorithm
    = Ciphertext: "3a9f1b2c8d0e7f4a..."  (meaningless to anyone without the key)

The security of encryption rests on the key, not the secrecy of the algorithm. This is Kerckhoffs's principle, formulated in 1883: a cryptographic system should be secure even if everything about the system except the key is public knowledge. Modern encryption algorithms like AES and RSA are completely public — the source code is available, the math is published in academic papers, anyone can study them. What protects your data is the key.

This has a practical implication: the algorithm isn't what you need to keep secret. The key is. Code that hardcodes encryption keys in source files, commits them to repositories, or stores them in plaintext configuration files has destroyed the security guarantee regardless of how strong the algorithm is.

Symmetric Encryption

Symmetric encryption uses the same key to encrypt and decrypt. Both parties need the same secret key, established beforehand through some secure channel.

The most important symmetric cipher today is AES — the Advanced Encryption Standard. It was standardized by NIST in 2001 after a public competition evaluating multiple candidate algorithms. AES operates on fixed 128-bit blocks of data using keys of 128, 192, or 256 bits. AES-256 (256-bit key) provides security that no known attack can break in any practical timeframe.

We built AES from scratch in the RSA/AES/TLS premium article — constructing the S-Box, SubBytes, ShiftRows, MixColumns, and KeyExpansion from first principles. The implementation makes clear what the algorithm actually does: each round applies four transformations to a 4×4 matrix of bytes, mixing the data with round keys derived from the original key. After 10, 12, or 14 rounds (depending on key length), the output has no recoverable relationship to the input without the key.

Raw AES operates on single 128-bit blocks. Real data is almost always larger. Modes of operation define how to apply AES to arbitrary-length data:

CBC (Cipher Block Chaining) — XORs each plaintext block with the previous ciphertext block before encrypting. The initialization vector (IV) randomizes the first block. This was the standard mode for years. It's now considered legacy because it doesn't provide authentication — an attacker can flip bits in the ciphertext and produce predictable changes in the plaintext. Never use CBC without a separate MAC (Message Authentication Code).

GCM (Galois/Counter Mode) — the current standard. Provides both encryption (via Counter mode, which generates a keystream XOR'd with the plaintext) and authentication (via GHASH, a polynomial hash function). AES-GCM is an AEAD cipher — Authenticated Encryption with Associated Data. If any bit of the ciphertext is modified, decryption fails with an authentication error before any potentially tampered data is processed. This is what every TLS 1.3 connection uses.

# Production Python — always use the cryptography library, not homebrew
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
import os
 
key = os.urandom(32)    # 256-bit key — generate fresh, store securely
nonce = os.urandom(12)  # 96-bit nonce — MUST be unique per (key, message) pair
 
gcm = AESGCM(key)
ciphertext = gcm.encrypt(nonce, b"secret message", b"associated data")
plaintext = gcm.decrypt(nonce, ciphertext, b"associated data")

The nonce uniqueness requirement is critical. Reusing a nonce with the same key in AES-GCM is catastrophic — it leaks the authentication key and allows an attacker to recover the XOR of two plaintexts. The Sony PS3 firmware signing system was broken this way. Generate nonces with a cryptographically secure random source and treat them as single-use.

Symmetric encryption performance: AES on modern hardware with AES-NI instructions runs at tens of gigabytes per second. It's fast enough to encrypt everything in real time without noticeable overhead. This is why symmetric encryption handles the actual data in every real-world system.

Asymmetric Encryption

Asymmetric encryption (also called public-key cryptography) uses a mathematically related key pair: a public key that can be freely shared and a private key that must remain secret.

What makes it work: data encrypted with the public key can only be decrypted with the corresponding private key. Data signed with the private key can be verified by anyone with the public key. The two keys are mathematically linked but computationally infeasible to derive one from the other.

This solves the key distribution problem that symmetric encryption can't. Two strangers on the internet can establish a shared secret without having previously exchanged keys.

RSA is the most widely known asymmetric algorithm. Its security rests on the difficulty of factoring the product of two large prime numbers. Given n = p × q where p and q are large primes, computing p and q from n alone is computationally infeasible for sufficiently large n. We built RSA from scratch across multiple articles — starting with the core implementation, through prime generation, digital signatures, timing attack defenses, and all the way to OAEP padding and RSA-PSS in the TLS deep-dive.

The key insight: RSA doesn't encrypt bulk data. It's too slow — roughly 1000x slower than AES for the same data volume — and can only encrypt messages smaller than the key size. RSA is used to establish shared secrets or sign certificates, not to encrypt the actual data flowing through a connection.

Elliptic Curve Cryptography (ECC) achieves the same security as RSA with much smaller keys, because the underlying mathematical problem (the elliptic curve discrete logarithm problem) is harder to solve than integer factorization. A 256-bit EC key provides roughly the same security as a 3072-bit RSA key. Smaller keys mean faster operations and smaller certificates. TLS 1.3 uses ECDSA (Elliptic Curve Digital Signature Algorithm) for certificate signatures and ECDHE (Elliptic Curve Diffie-Hellman Ephemeral) for key exchange.

The quantum threat: RSA and ECC are both vulnerable to Shor's algorithm running on a sufficiently powerful quantum computer. Shor's algorithm can factor large numbers and solve the discrete logarithm problem in polynomial time — breaking both RSA and ECC. This is why the post-quantum cryptography transition is active right now. NIST standardized ML-DSA (CRYSTALS-Dilithium) for signatures and ML-KEM for key encapsulation in 2024. Chrome already uses ML-KEM for TLS key exchange. The certificate infrastructure migration to Merkle Tree Certificates is underway for 2027.

How Real Systems Work

Because asymmetric encryption is slow and symmetric encryption requires pre-shared keys, production systems combine both. The pattern is consistent across TLS, PGP, Signal, and virtually every other real protocol:

1. Key exchange using asymmetric cryptography — the two parties use public-key operations to establish a shared secret without ever transmitting it
2. Derive symmetric keys from that shared secret — using a key derivation function like HKDF
3. Encrypt all actual data with symmetric encryption (AES-GCM) — fast, efficient, authenticated

In TLS 1.3 specifically (which we built from scratch in the TLS deep-dive):

• ECDHE generates a shared secret from ephemeral key pairs
• HKDF derives handshake traffic keys and application traffic keys from that secret
• AES-128-GCM or ChaCha20-Poly1305 encrypts everything
• RSA-PSS signatures on certificates authenticate the server's identity

The RSA key in a server's certificate is never used to encrypt data. It signs the certificate to prove identity. The actual session is protected by ephemeral ECDHE keys that are generated fresh for every connection and discarded afterward — this is forward secrecy. Even if an attacker steals the server's RSA private key tomorrow, they cannot decrypt today's sessions.

Hashing is very Important

Hashing is frequently confused with encryption and they're fundamentally different things.

Encryption is reversible — you can decrypt ciphertext back to plaintext if you have the key. Hashing is one-way — a hash function takes input of arbitrary length and produces a fixed-length output (the hash or digest), and there is no algorithm to reverse it.

Input:  "password123"
SHA-256: ef92b778bafe771e89245b89ecbc08a44a4e166c06659911881f383d4473e94f

You cannot reverse that hash to recover "password123". What you can do is compute the hash of any candidate input and compare it to the stored hash.

This is how password storage works correctly:

import bcrypt
 
# Storing a password — never store the plaintext
password = "hunter2"
salt = bcrypt.gensalt()
hashed = bcrypt.hashpw(password.encode(), salt)
# Store hashed in database
 
# Verifying a login attempt
def check_password(attempt: str, stored_hash: bytes) -> bool:
    return bcrypt.checkpw(attempt.encode(), stored_hash)

The database stores the hash. When a user logs in, hash their input and compare. If the database is breached, attackers get hashes, not passwords. To recover passwords they need to hash candidates and compare — computationally expensive with algorithms like bcrypt, Argon2, or scrypt, which are specifically designed to be slow to prevent brute-force attacks.

Never use MD5 or SHA-1 for password hashing. They were not designed for passwords and are fast — attackers can hash billions of candidates per second on commodity hardware. Use bcrypt, Argon2id (recommended by NIST as of 2022), or scrypt.

Never encrypt passwords (using reversible encryption) when you could hash them. Encrypted passwords mean the encryption key is the new master key for every password in the system. Hashes are a one-way transformation with no reversal.

HMACs (Hash-based Message Authentication Codes) combine hashing with a secret key to produce a value that proves both integrity (the data hasn't changed) and authenticity (only someone with the key could have produced it). HMAC-SHA256 is used in JWT signatures, webhook verification, and other authentication contexts.

Key Management is the Part People Gets Wrong

The strength of encryption is entirely theoretical if key management is broken. In practice, key management is where systems fail.

Common failures:

• Hardcoding keys in source code (attackers find them in Git history, even after deletion)
• Committing .env files containing keys to public repositories
• Logging decryption operations in ways that expose plaintext
• Using the same key for everything (key rotation becomes impossible)
• Storing encryption keys in the same database as encrypted data (encrypting the data then giving attackers the key is theater)
• Generating keys from low-entropy sources (seeding with timestamps, predictable values)

What good key management looks like:

• Keys generated with a cryptographically secure random number generator (os.urandom() in Python, crypto.randomBytes() in Node.js, secrets.token_bytes() in Python)
• Keys stored in purpose-built secrets managers (AWS KMS, HashiCorp Vault, Azure Key Vault) separate from the data they protect
• Different keys for different purposes — one key for user data, another for internal credentials, another for signing
• Key rotation schedules with zero-downtime re-encryption for long-lived data
• Hardware Security Modules (HSMs) for the highest-sensitivity keys

For most web applications: store secret keys in environment variables, never in code. Use a secrets manager for production. Rotate keys when team members leave. Audit who has access to production secrets.

Encryption in Transit vs At Rest

These are distinct requirements that often get conflated.

Encryption in transit protects data as it moves between systems. HTTPS/TLS protects data between browsers and servers. VPNs protect data between remote workers and corporate networks. Encrypted connections between microservices protect internal traffic. The attack being defended against is interception — someone reading traffic on the network path. TLS 1.3 with perfect forward secrecy is the current standard.

Encryption at rest protects stored data. Full-disk encryption (FileVault, BitLocker, LUKS) protects data if a physical device is stolen. Database encryption protects data if the storage layer is compromised. Field-level encryption protects specific sensitive columns even if the database is breached.

The threats are different. Transit encryption is about protecting data from network attackers. Rest encryption is about protecting data from attackers who gain access to storage.

A system with strong transit encryption but no at-rest encryption is still vulnerable to an attacker who gains access to the database server. A system with at-rest encryption but weak transit encryption is vulnerable to a network attacker who intercepts credentials. You need both.

End-to-End Encryption

End-to-end encryption (E2EE) means only the communicating users can read the messages. Not the service provider, not the platform, not anyone intercepting the traffic — only the sender and recipient.

Signal's protocol is the gold standard. It uses a combination of the X3DH (Extended Triple Diffie-Hellman) key agreement and the Double Ratchet algorithm, which provides:

• Forward secrecy: past messages can't be decrypted even if keys are later compromised
• Break-in recovery: future messages recover security even if a current key is compromised
• Deniability: messages can't be cryptographically attributed to a specific sender (important for whistleblowers)

The critical distinction: services that claim to be "encrypted" but hold the keys server-side (like iCloud backups without Advanced Data Protection, or most enterprise email) provide encryption that the provider can reverse. True E2EE means the service provider genuinely cannot read the content.

Passkeys are a related application of asymmetric cryptography to authentication — the private key never leaves the device, the server only holds the public key, and authentication happens through a challenge-response that proves possession of the private key without revealing it.

The One Rule

Most encryption mistakes aren't made by people who understand the algorithms and get the math wrong. They're made by people who use well-tested libraries but misconfigure them — wrong mode of operation, reused nonces, keys in environment variables committed to git, plaintext passwords in a field named password_encrypted.

The rule that prevents most of this: use high-level, audited cryptographic libraries and follow their documentation exactly. In Python, the cryptography library. In Node.js, the built-in crypto module or libsodium.js. In Java, the standard JCA with a vetted provider. Don't build your own encryption. Don't use algorithms that aren't recommended by NIST or equivalent standards bodies. Don't modify recommended configurations because a Stack Overflow answer suggested it.

The math is solid. The implementations of the math, when done by the right people with the right review, are solid. The configuration decisions and key management practices are where things fall apart, and those are entirely within a developer's control.