How HTTPS Works: TLS Handshake, Certificates, Forward Secrecy

18 min read Original article ↗
14 min read · Guide · Network

How it works · Network · Security

HTTPS, the handshake under the lock.

The protocol that quietly protects a trillion transactions a day, and yet almost nobody who uses it can describe what happens in the milliseconds before the page loads. So let's describe it. Slowly, and in order.

Parts01 – 06 InteractiveTLS 1.2 / 1.3 PrereqTCP / DNS

Three guarantees,
non-negotiable.

HTTPS is HTTP over TLS. The encryption layer that secures every modern web request. TLS 1.3 (RFC 8446, 2018) is the current standard, replacing TLS 1.2 (2008) and the deprecated SSL series. The handshake establishes a shared secret between client and server using public-key cryptography; once established, symmetric encryption protects all subsequent data.

HTTPS isn't encryption. HTTPS is a cryptographically enforced contract that provides three distinct, non-negotiable guarantees. Without all three, the connection cannot be considered secure, and each one is broken by a different kind of attacker.

01

Confidentiality

ISPs, routers, Wi-Fi snoopers. None of them can read the payload. After the handshake the bytes are mathematically indistinguishable from random noise to anyone who doesn't hold the session key.

02

Integrity

The data cannot be modified in transit without the modification being detected. Authenticated encryption attaches a tag; flip a single bit of ciphertext and decryption fails, loudly.

03

Authentication

You are talking to who you think you are. Not an imposter on the same coffee-shop network, not a state actor with a forged cert. This relies on PKI. The chain of trust we unpack in Part 04.

Two shapes
of math.

TLS stitches together two flavours of cryptography that are almost philosophically opposed. The trick is using each for the job it's good at, and nothing else.

Asymmetric · slow

Padlock and key.

Anyone can snap the padlock shut (encrypt with a public key), but only the person holding the physical key (the private key) can open it. RSA and elliptic-curve variants work this way. The math is expensive. Too slow to stream video through. So TLS uses it only during the handshake, to safely agree on a shared secret.

Symmetric · fast

One shared secret.

A single key that both sides hold. AES-256 in GCM mode is typical. Hardware-accelerated on every modern CPU, fast enough to encrypt gigabits per second per core. Once the handshake hands over the shared secret, all bulk data is symmetric from there on.

The master formula

TLS = asymmetric math (slow) + symmetric math (fast). The slow math's only job is to safely transmit the key for the fast math. Once the fast key is in place, the slow math goes back to sleep.

The handshake,
in motion.

Before a single byte of application data is sent, the client and server perform a brief, precisely choreographed exchange called the TLS handshake. Toggle between the modern version (TLS 1.3) and its predecessor (TLS 1.2) below, and watch the round trips. TLS 1.3 finishes in one; 1.2 needs two. That difference is most of why 1.3 exists.

CLIENTBrowser198.51.100.24SERVERexample.com93.184.216.34 01ClientHello 02 ServerHello · Certificate · Finished 03 Finished 04 Application data 1-RTT · TLS 1.3

Step 01 of 04

The client announces what it can speak and pre-ships a key share — so the server can derive the shared secret on receipt, saving a round trip. SNI names the host; ALPN offers application protocols.

PKI, and the
chain of trust.

The whole handshake depends on the client verifying the server's certificate. But what stops an attacker from handing your browser a fake cert that says "I am yourbank.com"?

The answer is the chain of trust. Every operating system and every browser ships with a pre-installed root store — a digital vault containing the keys of a few dozen trusted third parties called Certificate Authorities: DigiCert, Sectigo, GlobalSign, Let's Encrypt, and so on.

  1. 01

    Domain validation

    You construct a Certificate Signing Request containing your public key and send it to a CA. They verify — usually via a file you place on your web server, or a DNS record you create — that you actually control the domain.

  2. 02

    CA signature

    Once validation passes, the CA uses its heavily-guarded private root key to cryptographically sign your certificate. The signature is math, not paperwork. It can be verified by anyone who has the CA's corresponding public root key.

  3. 03

    Client verification

    When a browser connects, your server presents the signed certificate. The browser looks at the signature, finds the matching root in its internal vault, and verifies the math. If the math holds, the browser trusts you. If not, the user sees a terrifying red warning screen.

Perfect forward
secrecy.

The classic RSA handshake had a catastrophic flaw: if a nation-state actor recorded months of your encrypted traffic and later managed to steal your server's private key, they could retroactively decrypt everything. Every session. Every byte.

The fix is called Perfect Forward Secrecy, implemented in modern TLS via Ephemeral Elliptic-Curve Diffie-Hellman (ECDHE). Instead of the server's private key being used to decrypt a shared secret, the client and server dynamically compute a brand new ephemeral key pair for every session. The math of Diffie-Hellman is magic — it lets two parties openly exchange variables and independently arrive at the same shared secret, without the secret itself ever crossing the wire.

The server's long-lived private key is relegated to just one job: authentication. It signs a message during the handshake to prove identity, but it is mathematically decoupled from the encryption itself. If the server gets popped tomorrow, yesterday's traffic stays secure. Forever.

TLS 1.3
lighter, faster, less.

Finalized in 2018 as RFC 8446, TLS 1.3 was a brutal, uncompromising rewrite. It stripped away legacy bloat, hardened defaults, and nearly doubled the speed of the handshake.

What was removed

Stripping the graveyard.

TLS 1.2 kept dozens of insecure ciphers around for backward compatibility. RC4, DES, SHA-1. Attackers exploited this with downgrade attacks: trick the negotiation into picking broken crypto. TLS 1.3 simply deleted them. Five AEAD ciphers remain. All of them are strong.

What was added

1-RTT, by default.

With only five ciphers to pick from, the client can guess what the server will accept and include its DH key share in the very first packet. The server replies with its share, its certificate, and Finished. All in one flight. Latency halved, silently.

Zero round trips, for regulars

TLS 1.3 introduced an even more aggressive optimization: 0-RTT resumption. If a user has visited recently, the client and server remember a resumption ticket. Next visit, the browser can encrypt the GET request and send it alongside the ClientHello. The connection is ready before the server has even finished replying. On reconnects, HTTPS performs identically to unencrypted HTTP. Minus the snooping.

SNI and ALPN,
the quiet extensions.

The ClientHello carries more than ciphers and randoms. Two small TLS extensions do a lot of work: one picks the certificate, the other picks the protocol that'll run on top.

SNI · Server Name Indication

Which cert, please.

One IP can host hundreds of sites. Before the server can choose which certificate to present, it needs to know which hostname the client is trying to reach — because a cert is pinned to a name. SNI puts that hostname in the ClientHello in the clear (pre-handshake, there is no encryption yet), so routers and load balancers can also dispatch to the right backend without decrypting anything.

ALPN · Application-Layer Protocol Negotiation

h2, or http/1.1?

The client advertises which application protocols it supports — h2 for HTTP/2, http/1.1 for the classic, or both in preference order. The server picks one, echoes it back in the ServerHello, and that choice is locked in for the life of the connection. No upgrade dance, no second round trip to discover HTTP/2 support. ALPN is also what HTTP/3 uses to be pointed at (via Alt-Svc and the h3 token) once QUIC is in play.

There's a subtle privacy wrinkle with SNI: because it's sent in cleartext, a passive observer knows which site you're visiting even though they can't read what you do there. Encrypted Client Hello (ECH) — the successor to ESNI. Wraps the entire ClientHello (including SNI) in a second layer of encryption using a public key the front-end publishes in DNS. Still a work in progress, but widely deployed by Cloudflare and Firefox as of 2024.

Hardening the
chain of trust.

PKI on its own has gaps. Certificates get compromised. Attackers strip HTTPS on the first visit before HSTS can kick in. CAs occasionally misissue, or get tricked. Three layers of hardening close most of the doors.

OCSP Stapling

Revocation, pre-cached.

When a cert is compromised, the CA revokes it, but how does a browser find out? The naive approach is an OCSP query to the CA at connection time, which adds latency and leaks your browsing history to the CA. Stapling inverts it: the server periodically fetches a fresh, signed OCSP response from the CA and attaches (staples) it to its own handshake. The browser verifies the short-lived signature without ever talking to the CA. Faster, and private.

HSTS

No plaintext, ever again.

An Strict-Transport-Security header tells the browser: "for the next year, refuse to talk to me over HTTP, even if the user types http:// or clicks a plaintext link." Once cached, downgrade attacks are impossible for the stickiness window. The HSTS preload list. Shipped with every browser. Closes the first-visit gap for domains that opt in.

CT Logs

Misissuance, auditable.

Certificate Transparency requires every publicly-trusted cert to be submitted to tamper-evident append-only logs. The CA receives a Signed Certificate Timestamp in return, which is embedded in the cert itself. Browsers refuse certs without SCTs. If someone gets a rogue cert for your domain, you. Or crt.sh on your behalf — can spot it in the logs within minutes.

A ClientHello,
field by field.

The handshake from Part 02 is a sequence of TLS records. Each record is at most 214 bytes (16 KiB), wraps a typed payload (handshake, application data, alert, change cipher spec), and carries a 5-byte header. The first record a client ever sends is a ClientHello. The menu of everything it can do.

# TLS RECORD HEADER (5 bytes)
16 03 01 00 a3              # type=Handshake(22), version=TLS1.0(legacy), length=0x00a3

# HANDSHAKE HEADER (4 bytes)
01 00 00 9f                  # msg_type=ClientHello(1), length=0x00009f

# CLIENT HELLO BODY
03 03                        # legacy_version = TLS1.2 (0x0303) — 1.3 negotiates via extension
54 7c…32 random bytes…       # ClientRandom (32B) — feeds the key schedule

00                           # legacy_session_id (empty in 1.3)

00 06                        # cipher_suites length = 6
13 01  13 02  13 03          # TLS_AES_128_GCM_SHA256, AES_256_GCM, CHACHA20_POLY1305

01 00                        # compression_methods = [null] — never negotiated in TLS 1.3

00 6e                        # extensions length = 110 bytes
  00 00  00 0d  …            # server_name (SNI): "api.example.com"
  00 0a  00 04  00 1d 00 17  # supported_groups: x25519, secp256r1
  00 0d  00 0a  …            # signature_algorithms: ed25519, rsa_pss_rsae_sha256, ecdsa_secp256r1_sha256
  00 33  00 26  …            # key_share: x25519 public key (32 bytes)
  00 2d  00 02  01 01        # psk_key_exchange_modes: psk_dhe_ke
  00 2b  00 03  02 03 04     # supported_versions: TLS 1.3 (0x0304)
  00 10  00 0e  …            # ALPN: h2, http/1.1

The body is a fixed prologue plus a free-form list of extensions. TLS 1.3 freezes the prologue (legacy fields are kept identical for middlebox compatibility) and moves all real negotiation into extensions: the version, the key exchange group, the chosen public key, the SNI hostname, the ALPN protocol list. The server only echoes what it accepts; any extension the server doesn't recognise is silently ignored.

SNI

The address on the envelope.

Many sites share one IP. Without Server Name Indication the server wouldn't know which certificate to present. SNI travels in the clear in the ClientHello. Which is why Encrypted Client Hello (ECH, RFC 9460) is the next step: wrapping the inner ClientHello inside an outer one whose SNI points at a generic frontend key.

Key Share

An optimistic guess.

In TLS 1.2 the client first asked which group the server supported, then sent the public key. TLS 1.3 fuses both: the client guesses (usually X25519) and includes a key share for it. If the server agrees, the handshake completes in 1-RTT. If the server wants a different group, it replies with a HelloRetryRequest and the client sends another ClientHello. Costing one extra round trip.

ALPN

Picking the protocol on top.

Application-Layer Protocol Negotiation chooses between h2, h3, and http/1.1 inside the same handshake. No second round trip after TLS finishes. ALPN is also how WebRTC (webrtc) and gRPC (h2) avoid sniffing.

In your own terminal

openssl s_client -connect example.com:443 -servername example.com -tls1_3 -trace dumps every record in this format. The -trace flag is the key. It labels each field for you. Pair it with tshark for a live wireshark filter on tls.handshake.

Five suites,
no exotics.

TLS 1.2 had 37 standard cipher suites and over three hundred legacy ones — RC4, 3DES, MD5, RSA key exchange, anonymous DH. Misconfiguration was a research field of its own. TLS 1.3 cut the menu to five, all AEAD, all forward-secret. The other axes. Key exchange, signature, certificate type. Moved to extensions, where you can change one without touching the rest.

Suite Cipher Hash (HKDF) Where it shines
TLS_AES_128_GCM_SHA256AES-128-GCMSHA-256Default. Fast on hardware with AES-NI.
TLS_AES_256_GCM_SHA384AES-256-GCMSHA-384Compliance ceilings (NSA Suite B, FIPS).
TLS_CHACHA20_POLY1305_SHA256ChaCha20-Poly1305SHA-256Mobile / no AES-NI. Constant-time, no timing attacks.
TLS_AES_128_CCM_SHA256AES-128-CCMSHA-256IoT / DTLS where GCM hardware isn't available.
TLS_AES_128_CCM_8_SHA256AES-128-CCM-8SHA-256Constrained devices. 8-byte tag instead of 16.

Key exchange

Always (EC)DHE.

Every TLS 1.3 session uses ephemeral Diffie-Hellman for the key. RSA key transport is gone. The supported_groups extension lists curves: x25519 (default; 32-byte keys, fast, formally analysed), secp256r1 / secp384r1 (NIST curves; required by FIPS), x448 (paranoia tier). Ephemeral keys mean a stolen long-term private key cannot decrypt past traffic — perfect forward secrecy is on by default.

Signature

Cert says who; signature proves it.

The server signs the transcript with the private key matching its certificate. signature_algorithms negotiates the algorithm: ed25519 (modern, fast, deterministic), ecdsa_secp256r1_sha256 (most common ECDSA), rsa_pss_rsae_sha256 (RSA, but properly padded). MD5 and SHA-1 signatures are flatly forbidden — past attacks proved a downgrade existed in TLS 1.2.

Why AEAD matters

Pre-1.3 TLS used encrypt-then-MAC as a separate composition, and a long string of attacks (Lucky13, BEAST, POODLE) lived in that gap. AEAD ciphers do encryption and authentication in one operation: every byte of ciphertext is bound to its position and to the associated plaintext header. There is no padding oracle to leak through, no MAC truncation to exploit. The 1.3 cipher list is short on purpose.

TLS, in the
command line.

Three quarters of TLS production failures are one of: wrong certificate chain, expired cert, name mismatch, protocol downgrade, or SNI not configured. Five tools cover almost every diagnosis without leaving the terminal.

  1. 01

    Inspect the chain. openssl s_client

    openssl s_client -connect example.com:443 -servername example.com -showcerts. Prints the entire certificate chain the server sent, in order. Use -tls1_3 to force the version, -status to also fetch the OCSP staple, and < /dev/null to close immediately after the handshake instead of hanging on stdin.

  2. 02

    Decode a single cert. openssl x509

    openssl x509 -in cert.pem -noout -text dumps the certificate as readable text — Subject, Issuer, validity window, Subject Alternative Names, key algorithm, fingerprint, embedded SCTs. -checkend 86400 is the trick for monitoring: it exits non-zero if the cert is within 24 hours of expiry.

  3. 03

    Walk the connection. curl -v

    curl -v https://example.com shows the resolved IP, the cipher suite, the negotiated ALPN protocol, the chain verification result, and the response headers. Add --resolve example.com:443:1.2.3.4 to test a specific origin behind a load balancer, and --cacert to point at a custom root for self-signed environments.

  4. 04

    Verify against a public scan. testssl.sh

    testssl.sh --severity HIGH https://example.com runs a full audit: protocol versions, cipher suites, vulnerability scans (Heartbleed, ROBOT, Logjam), HSTS posture, CT log presence. Open-source, no signup, runs on your laptop. Pairs with ssllabs.com/ssltest for a public report card.

  5. 05

    Watch the wire. tshark / wireshark

    tshark -i any -f "host example.com and port 443" -O tls filters the live traffic and parses every TLS record. To decrypt your own connections, set SSLKEYLOGFILE=/tmp/keys.log in the environment of the originating browser or curl; Wireshark reads the keylog and shows you the application data in plaintext.

Common error decoder

SSL_ERROR_NO_CYPHER_OVERLAP — the client's cipher list and the server's don't intersect. Usually a TLS 1.3-only server talking to an ancient client.
unable to get local issuer certificate. The server didn't send the intermediate; install the full chain, not just the leaf.
subjectAltName: no name matching. The cert's SANs don't include the requested host. CN-only certs stopped being trusted in browsers years ago.
certificate expired. Every TLS engineer's least favorite alert. Set up a x509 -checkend monitor.

Modern TLS deployment. Let's Encrypt, mTLS, post-quantum

What 2024 production TLS looks like.

Let's Encrypt has won. Free, automated, 90-day certificates via the ACME protocol have driven HTTPS to ~95% of web traffic (Chrome telemetry, 2024). The certbot, acme.sh, and Caddy's automatic HTTPS make deployment trivial. Paid certificates remain only for organisation-validated (OV) and extended-validation (EV) flavors that browsers no longer visually distinguish anyway.

mTLS for service-to-service. Mutual TLS. Both sides present certificates. Is the dominant pattern in modern service meshes. Istio, Linkerd, and AWS App Mesh issue and rotate workload certificates automatically (typically via SPIFFE-style identity). The cost of certificate management used to be prohibitive; service mesh automation made mTLS practically free.

Post-quantum readiness. Quantum computers running Shor's algorithm would break RSA and ECDH at scale. NIST standardised the first post-quantum KEM (Kyber, now ML-KEM) in 2024. Cloudflare and Google have already rolled out hybrid TLS handshakes (X25519 + Kyber) on a fraction of connections; full transition is a 5-10 year project.

HSTS preload list. 100k+ domains are on the browser-bundled HSTS preload list. Meaning the browser refuses HTTP from these domains even on first visit. Worth the (one-time) submission for any production property.

Certificate Transparency. Every cert from a public CA must appear in a public CT log within seconds of issuance. crt.sh lets you search them; production teams use this to detect rogue certificates issued for their domains.

A closing note

Ten years ago, only banks enforced HTTPS. Certificates were expensive, handshakes were slow, and most of the web stayed plaintext. Today, Let's Encrypt automated certificate issuance into a zero-cost thirty-second operation, AES-NI made encryption cost less than parsing HTML, and HTTP/2 and HTTP/3 made encryption mandatory. The protocol that once felt like a heavy security bolt-on is now the invisible, ubiquitous foundation beneath almost everything you load. Picked apart, it's mostly clever tradeoffs between old math and new.

Three quick checks before you close the tab

Three quick checks before you close the tab.

Pick an answer for each. The right one reveals a short explanation; the wrong ones do too. There is no scoring. These are here so you can confirm the mental model travels with you when you go back to the editor.

Q1. TLS 1.3 stripped one full round trip out of the handshake. How did it manage that?

Q2. A browser receives the header Strict-Transport-Security: max-age=31536000. What does that change about future requests to that origin?

Q3. A self-signed certificate triggers the big red warning page in every browser. Why?

Read
further.

Found this useful?