While Go’s netpoller solves thread efficiency, pools address the underlying connection cost problem.

That closing line sat with me for days. I understood the netpoller, the kernel queues, the handshake, pollDesc, epoll. But I had no concrete picture of what a pool actually is or how it decides when a connection is safe to reuse.

So I built one from scratch. Raw TCP, no frameworks, just net.Dial, net.Listen, and the same connection lifecycle I mapped in the previous article.

This is what I found.

Connection Pooling Starts with a Server Decision

The first thing that surprised me: connection pooling is not a client invention. It requires an explicit server choice.

A naive server closes after every response:

conn, _ := listener.Accept()
handleRequest(conn)
conn.Close()  // connection dead

This is what I built first. Clean, simple, and expensive. Every new request pays the full cost again: TCP handshake (SYN, SYN-ACK, ACK), buffer allocation, pollDesc registration, the whole setup.

A pooling-aware server does the opposite. It stays in a loop:

conn, _ := listener.Accept()
sc := bufio.NewScanner(conn)
for sc.Scan() {
    handleRequest(sc.Text())
    fmt.Fprintf(conn, "response\n")
}
// loop only exits when client disconnects or idle timeout fires

The for sc.Scan() is the deliberate decision. The server goroutine stays parked on the netpoller between requests, no OS thread consumed, connection held open. This is what gives the client something to reuse.

By not closing, the server lets the client leverage pooling. The benefit runs both ways: client avoids the handshake cost, server reuses its own goroutine instead of spawning new ones per connection.

The Framing Contract

The server staying in the loop creates a fundamental problem. How does it know where one request ends and the next begins?

The client writes:

request1\nrequest2\nrequest3\n

The server’s sc.Scan() uses \n as the boundary. It reads until it hits the delimiter, then stops. Real databases use length-prefixed frames or special terminator bytes. Same idea, different encoding.

But there’s a second problem. How does the client know when the full response has arrived and it’s safe to return the connection to the pool?

Return the connection mid-response and the next caller gets a connection with leftover bytes in the recv buffer. Corrupted data.

How different systems solve this:

• Postgres: sends CommandComplete + ReadyForQuery frames. ReadyForQuery means “full response delivered, connection is idle.”

• HTTP chunked: 0\r\n terminating chunk signals “stream over.”

• HTTP with Content-Length: client reads exactly N bytes, done.

Pooling and framing are inseparable. Without the sentinel, the pool can’t know when a connection is safe to reuse. This is why database drivers and HTTP clients always implement a framing protocol alongside the pool. They’re two halves of the same contract.

Building the Simplest Pool

I started with a buffered channel:

type ConnPool struct {
    pool chan net.Conn  // capacity = MaxConn
    Addr string
}

func (p *ConnPool) GetConn() (net.Conn, error) {
    select {
    case conn := <-p.pool:
        return conn, nil
    default:
        return net.Dial("tcp", p.Addr)
    }
}

func (p *ConnPool) ReturnConn(c net.Conn) {
    select {
    case p.pool <- c:
    default:
        c.Close()  // pool full, close to avoid fd leak
    }
}

The default case in both is key. In GetConn: pool empty, dial fresh. In ReturnConn: pool full, close the connection. Not drop, close. Dropping it leaks the fd and kernel socket struct.

A buffered channel is the synchronization primitive. No mutex needed, channels are already thread-safe.

I ran this with a pool size of 3 and created 5 connections. The first 3 went into the pool. The 4th and 5th hit the default case in ReturnConn and were closed immediately.

The proof of reuse: same ephemeral port.

iter[0]: conn.LocalAddress(127.0.0.1:55645)
iter[1]: conn.LocalAddress(127.0.0.1:55645)  ← same port

The kernel assigns one ephemeral port per TCP connection. Same port across two GetConn calls = same net.Conn returned from pool = zero new dials.

The Overflow Demo

I wanted to see what happens when more connections are created than the pool can hold.

Setup: pool size = 3, MaxConn = 3. I created 5 connections total. The first 3 should go into the pool. The last 2 should hit the default case in ReturnConn and be closed immediately.

The code:

pool := pool.New(":4040")

// grab 2 extra connections before warming pool
conn3, _ := pool.GetConn()
conn4, _ := pool.GetConn()

// warm up pool with 3 connections
conns := []net.Conn{}
for i := 0; i < 3; i++ {
    conn, _ := pool.GetConn()
    conns = append(conns, conn)
}
for _, conn := range conns {
    pool.ReturnConn(conn)  // pool now full: 3 connections
}

// first loop: grab from pool, use, return
for i := 0; i < 3; i++ {
    conn, _ := pool.GetConn()
    handleConn(i, conn, pool)  // i = 0, 1, 2
}

// second loop: same 3 connections reused
for i := 0; i < 3; i++ {
    conn, _ := pool.GetConn()
    handleConn(i, conn, pool)  // i = 0, 1, 2 again
}

// overflow: these two hit pool full → closed
handleConn(3, conn3, pool)
handleConn(4, conn4, pool)

Each handleConn writes one line with the iteration number i, reads the echo response, calls ReturnConn:

func handleConn(i int, conn net.Conn, pool *pool.ConnPool) {
    fmt.Printf("iter[%d]: conn.LocalAddress(%v)\n", i, conn.LocalAddr())
    fmt.Fprintf(conn, "[%d]: hi\n", i)  // writes "[0]: hi\n" or "[1]: hi\n"
    
    sc := bufio.NewScanner(conn)
    sc.Scan()
    fmt.Println(sc.Text())
    pool.ReturnConn(conn)
}

The [0], [1], [2] in the logs are the iteration numbers from the loops. They’re not goroutine IDs (the server is single-threaded in the client, looping sequentially). The server echoes back whatever it receives, so [0]: hi in the request becomes [0]: hi in the response.

Server-side code:

listener, _ := net.Listen("tcp", ":4040")
i := 0
for {
    conn, _ := listener.Accept()
    go func(conn net.Conn, gri int) {
        defer conn.Close()
        sc := bufio.NewScanner(conn)
        who := conn.RemoteAddr()
        
        log.Printf("[go-routine-%d][%s]: connected\n", gri, who)
        
        for sc.Scan() {
            t := sc.Text()
            log.Printf("[go-routine-%d][%s]: %s\n", gri, who, t)
            fmt.Fprintf(conn, "[go-routine-%d]echo: %s\n", gri, t)  // echo back
        }
        
        log.Printf("[go-routine-%d][%s]: disconnected\n", gri, who)
    }(conn, i)
    i++
}

The gri (goroutine ID) is just an incrementing counter captured at Accept() time. Each new connection gets the next number. The server stays in the for sc.Scan() loop, handling multiple requests on the same connection until the client disconnects or the connection is closed.

Client logs showed this:

iter[0]: conn.LocalAddress(127.0.0.1:55645)
[go-routine-2]echo: [0]: hi
iter[1]: conn.LocalAddress(127.0.0.1:55646)
[go-routine-3]echo: [1]: hi
iter[2]: conn.LocalAddress(127.0.0.1:55647)
[go-routine-4]echo: [2]: hi
iter[0]: conn.LocalAddress(127.0.0.1:55645)   ← same port, reused
[go-routine-2]echo: [0]: hi
iter[1]: conn.LocalAddress(127.0.0.1:55646)   ← same port, reused
[go-routine-3]echo: [1]: hi
iter[2]: conn.LocalAddress(127.0.0.1:55647)   ← same port, reused
[go-routine-4]echo: [2]: hi
iter[3]: conn.LocalAddress(127.0.0.1:55643)   ← overflow connection
[go-routine-0]echo: [3]: hi
iter[4]: conn.LocalAddress(127.0.0.1:55644)   ← overflow connection
[go-routine-1]echo: [4]: hi

Server logs showed this:

[go-routine-0][127.0.0.1:55643]: connected
[go-routine-1][127.0.0.1:55644]: connected
[go-routine-2][127.0.0.1:55645]: connected
[go-routine-3][127.0.0.1:55646]: connected
[go-routine-4][127.0.0.1:55647]: connected
[go-routine-2][127.0.0.1:55645]: [0]: hi
[go-routine-3][127.0.0.1:55646]: [1]: hi
[go-routine-4][127.0.0.1:55647]: [2]: hi
[go-routine-2][127.0.0.1:55645]: [0]: hi   ← same goroutine, second request
[go-routine-3][127.0.0.1:55646]: [1]: hi   ← same goroutine, second request
[go-routine-4][127.0.0.1:55647]: [2]: hi   ← same goroutine, second request
[go-routine-0][127.0.0.1:55643]: [3]: hi
[go-routine-0][127.0.0.1:55643]: disconnected                  ← immediately, pool full
[go-routine-1][127.0.0.1:55644]: [4]: hi
[go-routine-1][127.0.0.1:55644]: disconnected                  ← immediately, pool full
[go-routine-4][127.0.0.1:55647]: disconnected                  ← on process exit
[go-routine-2][127.0.0.1:55645]: disconnected                  ← on process exit
[go-routine-3][127.0.0.1:55646]: disconnected                  ← on process exit

Reading the logs step by step:

Initial dial (5 connections created):

• Ports 55643, 55644 — the two overflow connections (conn3, conn4)

• Ports 55645, 55646, 55647 — the three warm-up connections, returned to pool

First loop (iter 0, 1, 2):

• Client grabbed ports 55645, 55646, 55647 from pool

• Same ports = same net.Conn = zero new dials

• Server: go-routine-2, 3, 4 handle these requests

• Client returns all 3 to pool after use

Second loop (iter 0, 1, 2 again):

• Client grabbed the same three ports again from pool

• Server: go-routine-2, 3, 4 handle their second request on the same connection

• This is connection reuse in action: same goroutine, same connection, two separate request-response cycles

• Client returns all 3 to pool again

Overflow (iter 3, 4):

• Client calls handleConn(3, conn3, pool) and handleConn(4, conn4, pool)

• Both write their request, get a response, call ReturnConn

• Pool is already full (3/3)

• ReturnConn hits the default case: c.Close()

• Server receives FIN on ports 55643 and 55644

• go-routine-0 and go-routine-1 wake from sc.Scan(), see EOF, loop exits

• “disconnected” logs fire immediately after handling the request

Process exit:

• Client process exits

• OS closes all remaining fds (55645, 55646, 55647)

• FINs go out

• go-routine-2, 3, 4 wake from sc.Scan(), see EOF, disconnect

The key proof: go-routine-2 handled two requests on port 55645 (iter[0] in both loops). Same ephemeral port = same TCP connection = same kernel tcp_sock structure = zero new handshakes. The connection was returned to the pool after the first loop and grabbed again in the second loop.

go-routine-0 and go-routine-1 disconnected immediately because ReturnConn closed them, not because the server decided to. The pool was full, the default case fired, c.Close() sent a FIN.

Adding MaxConnsPerHost

A buffered channel pool has no burst protection. 1000 goroutines hitting GetConn simultaneously dial 1000 connections to the server at once.

Fix: a semaphore.

// pre-filled with MaxConnsPerHost tokens
maxConnsPerHost := make(chan struct{}, MaxConnsPerHost)
for i := 0; i < MaxConnsPerHost; i++ {
    maxConnsPerHost <- struct{}{}
}

// GetConn: acquire token before dialing
<-p.MaxConnsPerHost  // blocks when at limit, parks goroutine

// ReturnConn: release token when connection is closed
p.MaxConnsPerHost <- struct{}{}

chan struct{} not chan int. struct{} is zero-size, no allocation per token.

I ran 10 goroutines with MaxConnsPerHost=6:

02:26:36 [go-routine-6] blocking for MaxConnsPerHost   ← 4 parked immediately
02:26:36 [go-routine-7] blocking for MaxConnsPerHost
02:26:36 [go-routine-8] blocking for MaxConnsPerHost
02:26:36 [go-routine-4] blocking for MaxConnsPerHost
02:26:36 [go-routine-1] using conn.LocalAddress(127.0.0.1:58963)   ← 6 dial
02:26:36 [go-routine-3] using conn.LocalAddress(127.0.0.1:58964)
02:26:36 [go-routine-2] using conn.LocalAddress(127.0.0.1:58966)
02:26:36 [go-routine-5] using conn.LocalAddress(127.0.0.1:58967)
02:26:36 [go-routine-0] using conn.LocalAddress(127.0.0.1:58965)
02:26:36 [go-routine-9] using conn.LocalAddress(127.0.0.1:58968)
                                                         ← 2 seconds later
02:26:38 [go-routine-6] using conn.LocalAddress(127.0.0.1:58971)   ← 3 unblock
02:26:38 [go-routine-8] using conn.LocalAddress(127.0.0.1:58970)
02:26:38 [go-routine-7] using conn.LocalAddress(127.0.0.1:58969)

Wave pattern: burst hits cap, excess goroutines park on the semaphore. As in-flight connections complete and return, parked goroutines unblock in waves.

IdleConnTimeout

Idle connections sitting forever create a problem: servers close them after their own timeout, sending a FIN. Client grabs the stale connection, writes a request, gets an error.

Fix: timestamp each connection on return. A background goroutine wakes periodically and evicts connections that have been idle too long.

type ConnPool struct {
    mu   sync.Mutex
    pool []struct {
        idleSince time.Time
        conn      net.Conn
    }
    MaxIdleConnsPerHost int
    MaxConnsPerHost     chan struct{}
    IdleConnTimeout     time.Duration
}

Why switch from channel to slice: a buffered channel can’t be peeked without consuming. To check expiry, you need to inspect elements without removing them. A slice as a FIFO queue works: append at the back (newest last), pop from back for GetConn (warmest first), check the front for eviction (oldest first).

Eviction interval: IdleConnTimeout / 2. Worst case: connection becomes idle just after a wakeup. One interval later it’s checked, age is IdleConnTimeout / 2. One more interval, evicted. Maximum overshoot: one interval. Go’s net/http uses exactly this internally.

I set IdleConnTimeout = 5 seconds for testing. After the client exited, the eviction goroutine woke 5 seconds later:

2026/05/10 00:32:04 [pool-idle-conn-eviction] closing conn: 127.0.0.1:57050
2026/05/10 00:32:04 [pool-idle-conn-eviction] closing conn: 127.0.0.1:57051
2026/05/10 00:32:04 [pool-idle-conn-eviction] closing conn: 127.0.0.1:57052

Server logs confirmed all 3 goroutines woke from sc.Scan() and disconnected at the same timestamp. The eviction goroutine freed them, not process shutdown.

Always set IdleConnTimeout below the server’s timeout. If the server times out first and sends a FIN, the client’s next attempt on that stale connection will error.

Dedicated-Host vs Shared-Host

The pool I built so far has Addr string hardcoded at construction. One pool per target server. This is a dedicated-host pool.

All knobs (MaxIdleConnsPerHost, MaxConnsPerHost, IdleConnTimeout) scope naturally to that one host. Simple, predictable.

A shared-host pool, one pool instance for all hosts, needs different structure. The flat slice becomes a map:

pool map[string][]struct {
    idleSince time.Time
    conn      net.Conn
}

MaxConnsPerHost becomes map[string] → chan struct{}, one independent semaphore per host.

Now two scopes appear:

• MaxIdleConnsPerHost (per-host), caps stale idle accumulation on any one host

• MaxIdleConns (global), caps total idle connections across all hosts

Without the global cap, a client talking to 50 hosts with MaxIdleConnsPerHost=10 could hold 500 idle fds, most unused.

The Semaphore Invariant

Three operations touch both semaphores (IdleConns global, MaxConns[addr] per-host). The core invariant:

tokens_in_IdleConns + connections_in_pool = MaxIdleConns  (always)

Operation MaxConns[addr] IdleConns global Dial new conn consume 1 no change ReturnConn → pool no change consume 1 (entering idle) ReturnConn → per-host cap release 1 (closed) no change ReturnConn → global cap release 1 (closed) no change GetConn → pool hit no change release 1 (leaving idle) Eviction release 1 (closed) release 1 (leaving idle)

Eviction must release both. An evicted connection held a per-host slot (consumed at dial) and a global idle slot (consumed at ReturnConn). Releasing only one causes permanent token starvation.

The Knobs

http.Transport exposes the same knobs I built:

• MaxIdleConnsPerHost, warm idle connections per host. Default: 2 (very conservative). First thing to tune under high load.

• MaxIdleConns, global ceiling across all hosts. Only relevant when one http.Client talks to multiple hosts.

• MaxConnsPerHost, hard ceiling on total connections (idle + in-use). When hit, new requests block waiting. Default: 0 (no limit).

• IdleConnTimeout, evict idle connections after this duration. Default: 0 (connections sit forever, vulnerable to server-side RST).

MaxConnsPerHost protects the server from being overwhelmed. Set it higher than MaxIdleConnsPerHost to absorb burst: extra connections created under load get closed on ReturnConn (pool already full), naturally draining back to the idle cap.

There’s no global MaxConns. A global connection limit across all hosts has no coherent meaning. Throttling connections to api.stripe.com because you have too many open to api.github.com protects neither server.

SQLAlchemy Mapping

I recently worked on Airflow where many parallel DAG tasks were connecting to Postgres simultaneously. The default config was throwing connection errors.

The fix was tuning two SQLAlchemy knobs:

AIRFLOW__DATABASE__SQL_ALCHEMY_POOL_SIZE: "20"
AIRFLOW__DATABASE__SQL_ALCHEMY_MAX_OVERFLOW: "20"

These map directly:

• POOL_SIZE = MaxIdleConnsPerHost, idle connections kept ready

• MAX_OVERFLOW = extra connections allowed beyond pool size

• MaxConnsPerHost = POOL_SIZE + MAX_OVERFLOW = 40 total

Airflow defaults: POOL_SIZE = 5, MAX_OVERFLOW = 5, 10 total. Under parallel DAG execution that exhausts fast.

Same semaphore pattern, same wave behavior, different language.

HTTP Framing

HTTP is the same pool, with a self-describing framing protocol.

Raw TCP framing:

• Request end: \n

• Response end: implicit, one line echoed back

• Connection fate: server loop stays open

HTTP framing:

• Request end: \r\n\r\n after headers + Content-Length bytes

• Response end: Content-Length bytes read OR 0\r\n\r\n in chunked encoding

• Connection fate: Connection header decides

http.Transport is a shared-host pool. The same knobs map directly: MaxIdleConnsPerHost, MaxIdleConns, MaxConnsPerHost, IdleConnTimeout.

HTTP/1.1 default is keep-alive. Connection: close is the opt-out. Connection: keep-alive is redundant. I verified this by removing it from the server, nothing observable changed in pool behavior.

resp.Body.Close() is not optional. Without draining and closing the body, http.Transport doesn’t know the response is complete. The connection can’t be returned to the pool. It leaks.

HOL Blocking

HTTP/1.1 with keep-alive: one connection, one request in-flight at a time.

I built a server that sleeps 5 seconds when the request body is "slow". Sent 10 sequential requests: fast, fast, fast, fast, slow, fast, fast, fast, fast, fast.

22:54:46  ConnectStart → :53594          ← one dial, one connection
22:54:46  GotConn :53594 → fast          ← instant
22:54:46  GotConn :53594 → fast
22:54:46  GotConn :53594 → fast
22:54:46  GotConn :53594 → fast
22:54:46  GotConn :53594 → [waiting]     ← slow request holds the connection
22:54:51  response: slow                 ← 5 seconds later
22:54:51  GotConn :53594 → fast          ← all unblock instantly

One slow request blocked five fast requests. Not because the server was busy but because they all shared one connection with no way to label which response belongs to which request.

The root cause: ordering without identity. The TCP byte stream has no per-request label. Responses must come back in the same order as requests. Fix: HTTP/2 adds a 31-bit stream ID to every frame. The receiver reassembles each response independently. One connection, N concurrent streams, no HOL at the application layer.

The pool design determines HOL exposure: one idle connection = full HOL, more connections = isolated per-connection, HTTP/2 = eliminated at app layer (TCP HOL remains).

What’s Still Open

I now understand what a connection pool is and how it decides when a connection is safe to reuse. The framing contract is inseparable from pooling. The knobs (MaxIdleConnsPerHost, MaxConnsPerHost, IdleConnTimeout) map directly between raw TCP pools and http.Transport.

What I still don’t understand well enough: how HTTP/2 multiplexing actually works at the frame level. I know it uses stream IDs, but I want to see the bytes on the wire. How QUIC eliminates TCP HOL blocking by implementing per-stream reliability over UDP. What the trade-offs are between dedicated-host pools (one per service) and shared-host pools in a microservice environment.

That’s where this goes next.

func main() {
    lis, err := net.Listen("tcp", ":8080")
    // ...

    for {
        conn, err := lis.Accept()
        // ...

        go func(conn net.Conn) {
            sc := bufio.NewScanner(conn)
            who := conn.RemoteAddr()

            for sc.Scan() {
                fmt.Printf("[%s]: %s\n", who, sc.Text())
            }
            // ...
        }(conn)
    }
}

Concurrent. Each client got its own goroutine. The server stopped being unavailable.

But I had a question I couldn’t shake.

What actually happens inside that goroutine? What is conn? What did net.Listen do before Accept() was even called? What does the kernel know about this connection that my code doesn’t?

I knew the textbook answer. In college I drew this:

SYN, SYN-ACK, ACK. Three-way handshake. Four-way teardown. The state machine labels on the left for the client, on the right for the server. I could reproduce those arrows from memory.

What I never understood was what each arrow cost. What got allocated when the SYN arrived. Who actually sent the SYN-ACK. What Go does when you call conn.Read() and there’s nothing there yet. What TIME_WAIT actually is and why it exists.

I wanted to see this at the kernel level, not in theory, but through code I wrote and understood line by line.

So I built a raw TCP echo server and client in Go, no net/http, no frameworks, just net.Dial, net.Listen, net.Conn, and drew out every layer until I understood it.

This is what I found.

First, What Is a File Descriptor

Before any of this makes sense, one concept is worth grounding first.

A file descriptor is just an integer. fd=5, fd=10, fd=3. That’s all it is on the surface.

Every process has its own per-process file descriptor table, a kernel-maintained mapping from these integers to actual resources in kernel memory. Those resources can be files on disk, pipes, or in our case, sockets. fd=5 in our Go server process points to one socket. fd=5 in a completely different process points to something else entirely. The integer means nothing across process boundaries. Within a process, it is unique.

When you call socket(), the kernel allocates a tcp_sock structure in RAM and returns the next available integer from our process’s fd table. That number is your handle. Every syscall you make, bind(), listen(), accept(), read(), write(), close(), takes that integer and the kernel looks up the actual resource behind it.

This is why “frees fd=10 from the process fd table” during teardown is significant. The integer 10 is released back to the table. The kernel can hand it to the next accept() call for a completely different client. The underlying socket structure may still exist in kernel memory, in TIME_WAIT, but our process has no handle to it anymore. The fd and the socket are two different things that happen to be linked for the duration of a connection.

A Connection Is Not What I Thought It Was

Before I wrote any code, I assumed a “connection” was the thing listener.Accept() returned.

It’s not. Or rather, that’s the end of a much longer process.

When you call net.Listen("tcp", ":4040"), three syscalls happen in sequence.

socket() asks the kernel to create a TCP socket, a data structure in kernel memory, and hand back a file descriptor. That fd, say fd=5, is just a number. A handle. The socket itself lives in the kernel. At this point it has no address, no port. It just exists.

bind(fd, "0.0.0.0", 4040) attaches the socket to a port. The kernel checks if port 4040 is already in use, checks process permissions, then updates its internal port-to-socket mapping. Now fd=5 owns :4040.

listen(fd) is where something interesting happens. The socket transitions from a plain socket to a listening socket, and the kernel creates two queues:

tcp_sock
state: LISTEN
local: :4040
syn_queue:    []
accept_queue: []

These two queues are the kernel’s waiting room for incoming clients. net.Listen returns. Our Go code has a listener. Nothing has connected yet.

And then our goroutine calls listener.Accept(). If the accept queue is empty, it parks. How exactly — that's what the next section is about. But the short version: Go's netpoller watches the fd on our behalf, and wakes the goroutine when something arrives. More on the netpoller in detail further below.

The Handshake Happens Without You

When a client sends a SYN packet, our goroutine is not involved.

Not partially involved. Not notified. Completely absent.

The SYN packet arrives at the NIC. The NIC fires a hardware interrupt. The kernel’s interrupt handler picks it up. The packet walks up the network stack, Ethernet → IP → TCP. The kernel looks up which socket owns the destination port. It finds fd=5.

Then the kernel creates a new socket for this specific client, cli:tcp_sock, with the client’s address already filled in, and places it in the SYN queue. Sends SYN-ACK. When the client’s ACK arrives, the kernel moves cli:tcp_sock to the accept queue and marks it ESTABLISHED.

Our goroutine is parked the entire time. The three-way handshake completes before Accept() returns. The goroutine only wakes up to collect the result.

That was the first thing that surprised me. I had always imagined my server code as participating in the handshake. It doesn’t. The kernel does all of it.

Look at the diagram above again. The g_server column is completely empty throughout the handshake. That emptiness is the point.

There’s one more detail worth noting. The kernel creates cli:tcp_sock when the SYN arrives, not when the ACK arrives. It pre-allocates the socket structure mid-handshake. The connection only becomes real, both sides agreeing on sequence numbers, when the final ACK lands. But the kernel commits resources earlier.

What “Blocking” Actually Means in Go

This is where Go’s netpoller comes in.

The netpoller is a component of the Go runtime that sits between our goroutines and the Linux kernel’s epoll mechanism. Its job is simple: watch file descriptors for I/O readiness events, and when one becomes ready, wake up whichever goroutine was waiting on it. It runs on its own OS thread, calling epoll_wait() in a loop. Our goroutines never call epoll_wait() directly. They park, and the netpoller does the watching.

This is what makes Go capable of handling millions of concurrent connections without millions of OS threads. The netpoller converts what looks like blocking network I/O in your code into event-driven I/O underneath — our goroutine writes conn.Read() as if it blocks, but the runtime never actually blocks the OS thread. Instead it parks the goroutine and frees the thread to run other goroutines. The OS thread stays available for CPU work or other connections while the kernel watches the fd. When data arrives, the netpoller wakes the right goroutine and the thread picks it up again.

Park goroutines. Free threads. Handle more. That’s the model.

The netpoller tracks which goroutine is waiting on which fd using a pollDesc — a small struct it maintains for every registered fd:

type pollDesc struct {
    fd int
    rg *g  // goroutine waiting to read
    wg *g  // goroutine waiting to write
}

This is the bridge between a file descriptor and a goroutine. When g_server calls Accept() and the accept queue is empty, the full sequence is:

• The runtime stores g_server in pollDesc[fd=5].rg

• Registers fd=5 with epoll via epoll_ctl(ADD, fd=5, EPOLLIN | EPOLLET)

• Calls gopark() — g_server is removed from the run queue, the scheduler’s list of goroutines ready to execute

• The OS thread that was running g_server is freed and picks up other goroutines

When the handshake completes and something lands in the accept queue, the netpoller’s epoll_wait() returns fd=5 as ready. The netpoller looks up pollDesc[fd=5].rg, finds g_server, clears the field, and calls runqueue.add(g_server) — putting it back on the scheduler’s run queue. An OS thread picks it up and resumes it exactly where it parked, inside Accept(), as if nothing happened in between.

Accept() makes the syscall. The kernel dequeues cli:tcp_sock from the accept queue and returns conn (fd=10).

g_server immediately calls go handleConn(conn), spawning g_conn_1 with ownership of the connection, and loops back to listener.Accept(). The infinite for loop. The server is available again before g_conn_1 has done anything at all.

This is the whole model:

func main() {
    l, err := net.Listen("tcp", ":4040")
    // ...
    for {
        conn, err := l.Accept()
        // ...
        go handleConn(conn)
    }
}

g_server parks at Accept(). Kernel does the handshake. Netpoller wakes g_server. g_server spawns g_conn_1 and parks again. No OS thread sleeps waiting for a client. No thread is wasted.

Inside the Connection Handler

g_conn_1 takes over conn (fd=10) and calls conn.Read(buf). There’s no data yet, the client hasn’t sent anything. Same pattern repeats.

The Go runtime registers fd=10 with epoll, epoll_ctl(ADD, fd=10, EPOLLIN | EPOLLET), stores g_conn_1 in pollDesc[fd=10].rg, and parks it.

Now look at the pollDesc table:

{fd:5,  rg: g_server,  wg: nil}
{fd:10, rg: g_conn_1,  wg: nil}

Two goroutines. Two fds. One epoll instance. Zero OS threads consumed by either.

This is why Go can handle tens of thousands of concurrent connections with a handful of OS threads. Every goroutine waiting for I/O is just a data structure in memory. The OS threads are free.

When the client sends data, the NIC fires an interrupt. The kernel places the bytes in rcv_buff of fd=10, doing TCP reassembly along the way, ordering out-of-order segments, discarding duplicates. The kernel marks fd=10 as ready in epoll. The netpoller’s epoll_wait returns fd=10. The netpoller finds g_conn_1 in pollDesc[fd=10].rg and puts it back on the run queue. g_conn_1 resumes, conn.Read(buf) drains from rcv_buff into buf, and the handler processes the request.

The Client Side — EINPROGRESS

Building the client taught me the mirror image of all of this.

net.Dial("tcp", ":4040") calls socket() then connect(). No bind() call, the kernel assigns an ephemeral port automatically from the range 32768–60999. So the client’s socket gets local: :54321, remote: :4040 without our code ever choosing the port number.

But here’s where it diverges from the server side. The netpoller registers fd=3 with epoll for EPOLLOUT, not EPOLLIN. And the pollDesc entry is:

{fd:3, wg: g_client, rg: nil}

wg not rg. Write goroutine, not read goroutine.

connect() on a non-blocking socket returns immediately with EINPROGRESS, handshake started, not done yet. The goroutine is parked.

When the SYN-ACK arrives, the kernel completes the handshake and marks fd=3 as writable. For a connecting socket, writable means connected. The send buffer is ready, which only becomes true once the connection is established. The kernel reuses “writable” as the signal for “you are now connected.”

The netpoller sees fd=3 is ready via EPOLLOUT, finds g_client in pollDesc[fd=3].wg, puts it back on the run queue. g_client resumes at connect(), Go checks getsockopt(SO_ERROR) to confirm the connection actually succeeded, and net.Dial returns conn (fd=3).

The client is now ESTABLISHED. The first conn.Write([]byte) puts bytes into send_buff. The kernel drains send_buff onto the wire.

Building a Raw TCP Echo Server and Client

To understand teardown properly, I needed both sides of the connection. So I built a raw TCP echo server and client, no net/http, no frameworks, just net.Listen, net.Dial, and net.Conn directly.

The server spawns a goroutine per connection, reads lines via bufio.Scanner, and echoes each one back:

go func(conn net.Conn) {
    defer conn.Close()
    sc := bufio.NewScanner(conn)
    who := conn.RemoteAddr()

    for sc.Scan() {
        t := sc.Text()
        fmt.Fprintf(conn, "echo: %s\n", t)
    }
    // ...
}(conn)

defer conn.Close() is important. Without it, the goroutine exits but the kernel-side socket stays open. The client’s reader blocks forever waiting for data that never comes. The fd leaks. Over time fds accumulate, the process hits its fd limit, and Accept() starts failing. defer covers all exit paths cleanly.

The client reads from stdin and writes to the connection, and simultaneously reads echoes back from the server and prints them to stdout.

That “simultaneously” is the problem. Two directions, one connection.

Run them sequentially and the first blocks forever.

done := make(chan any, 1)

// writer: stdin → conn
go func() {
    io.Copy(conn, WrappedReader{r: os.Stdin})
    conn.(*net.TCPConn).CloseWrite()
}()

// reader: conn → stdout
go func() {
    io.Copy(os.Stdout, conn)
    done <- 1
}()

<-done
conn.Close()

The reader signals done when it finishes. Main waits on done.

Why the reader and not the writer? The reader finishing means the server closed its side, nothing more is coming. The writer might finish, user typed the sentinel, while the server is still sending the last echo back. Waiting on the reader catches the true end of the conversation.

Here’s what it looks like running:

Client:

go run client/main.go
connected to server
hi
echo: hi
by
echo: by
bye
echo: bye
EOF

Server:

go run server/main.go
[127.0.0.1:50472]: connected
[127.0.0.1:50472]: hi
[127.0.0.1:50472]: by
[127.0.0.1:50472]: bye
[127.0.0.1:50472]: disconnected

Each line the client types comes back prefixed with echo:. When I type EOF, the WrappedReader returns io.EOF, the writer goroutine calls CloseWrite(), the server sees the connection close, logs “disconnected”, and shuts down its side. The full teardown plays out exactly as the diagrams show.

CloseWrite Is Not Close

When the writer goroutine finishes, I don’t want to close the whole connection. The reader is still waiting for the server’s final echoes.

conn.Close() would close both directions immediately. The reader goroutine would break mid-flight.

conn.(*net.TCPConn).CloseWrite() closes only the write side. Underneath it calls shutdown(fd, SHUT_WR). The kernel sends a FIN to the server signalling “I’m done sending”, but fd=3 stays open for reading. The pollDesc entry stays registered with epoll. The g_client reader goroutine keeps running.

I had to look this up to understand the type assertion. net.Conn is an interface. It doesn’t expose CloseWrite(). We have to assert to *net.TCPConn first:

conn.(*net.TCPConn).CloseWrite()

A half-close. Each direction closed independently, when each side was ready.

The Sentinel Protocol and Its Limits

To signal end-of-input from the terminal without Ctrl+D, I built a WrappedReader around os.Stdin:

type WrappedReader struct {
    r io.Reader
}

func (wr WrappedReader) Read(p []byte) (int, error) {
    n, err := wr.r.Read(p)
    s := string(p[:n])
    if s == "EOF\n" {
        return 0, io.EOF
    }
    return n, err
}

If it reads "EOF\n", it returns io.EOF to the caller instead of passing the bytes through.

Three things I got wrong building this before getting it right.

string(p) not string(p[:n]). p is a 32KB buffer. Only n bytes are valid. The rest is garbage. Reading the full p gives you junk after the actual input.

Terminal stdin is line-buffered. You get "EOF\n" as one chunk, not byte by byte. So the comparison works cleanly.

io.Copy calls Read in a loop. Returning io.EOF is the signal to stop. Not an error, just done.

It worked. And it immediately showed me its own problem.

What if the user actually wants to send the literal string "EOF"? The protocol misinterprets it as a control signal. What if the payload is binary, a JPEG, a serialised struct? Any byte sequence could accidentally match a text sentinel.

This is the problem real protocols solve with framing.

HTTP uses Content-Length or Transfer-Encoding: chunked. Redis’s RESP uses type-tagged length-prefixed messages, $6\r\nfoobar\r\n. gRPC uses binary length-prefixed frames over HTTP/2.

The sentinel approach breaks the moment the payload itself contains the sentinel string. Framing works on everything.

What Happens to the Resources

The teardown is where I saw the full picture of what a connection actually costs.

When the writer goroutine finishes and CloseWrite() fires, shutdown(fd=3, SHUT_WR) is called. The kernel flushes and closes the send buffer. The FIN packet goes out. The socket transitions to FIN_WAIT_1. But fd=3 is still open. The rcv_buff is still alive. The reader goroutine is still parked, waiting.

fd=3 is still registered with epoll. Still in the interest list. The netpoller is still watching it.

On the server side, the FIN arrives at the NIC. The kernel places an EOF marker into rcv_buff of fd=10. The kernel marks fd=10 as ready in epoll, EOF counts as readable.

The netpoller unparks g_conn_1. conn.Read returns the EOF. sc.Scan() returns false. The for loop exits.

defer conn.Close() fires on the server. Three things happen in order.

• epoll_ctl(DEL, fd=10) removes fd=10 from the epoll interest list.

• The pollDesc entry for g_conn_1 is cleared.

• The netpoller is no longer watching this fd.

close(fd=10) syscall. The kernel decrements the reference count on the socket. If it hits zero, fd=10 is freed from the server’s process fd table, and the kernel sends a FIN to the client.

The server socket transitions to LAST_ACK, waiting for the client’s final ACK.

The client’s reader goroutine sees the server’s FIN — the kernel placed an EOF marker in rcv_buff of fd=3, epoll fired, the netpoller unparked g_client. io.Copy(os.Stdout, conn) returns. done <- 1. Main unblocks. conn.Close() is called on the client side.

On the client, conn.Close() triggers its own three-step sequence. epoll_ctl(DEL, fd=3) removes fd=3 from the epoll interest list. The pollDesc entry for g_client is cleared. close(fd=3) frees fd=3 from the client’s process fd table — the Go process can reuse that integer for a new connection immediately. As part of close(fd=3), the kernel sends the final ACK to the server and transitions the client socket to TIME_WAIT.

The server receives the final ACK. cli:tcp_sock transitions to CLOSED. The kernel frees everything on the server side — the tcp_sock structure, rcv_buff, send_buff, all TCP state — sequence numbers, timers, everything.

On the client side, the socket lingers in TIME_WAIT for up to 120 seconds (2 * MSL, Maximum Segment Lifetime). The rcv_buff stays alive through this period — the kernel needs it in case the server retransmits its FIN. Only after 2*MSL expires does the kernel free the client’s tcp_sock, rcv_buff, and all remaining state.

Two reasons TIME_WAIT exists. If the final ACK got lost, the server will retransmit its FIN — the client needs to still be alive to respond. And if the same 4-tuple gets reused for a new connection too quickly, a stale packet from the old connection could arrive and corrupt it. TIME_WAIT prevents both.

The netpoller’s involvement ends at conn.Close(). Everything after — LAST_ACK, TIME_WAIT, final memory free — is the kernel’s TCP state machine running on its own.

The Resource Lifecycle, in Full

The fd dies at conn.Close(). The pollDesc dies with it. On the server side, conn.Close() sends the FIN, the client’s final ACK arrives, and the kernel frees everything — tcp_sock, both buffers, all TCP state — when it reaches CLOSED.

On the client side it’s different. conn.Close() fires the final ACK as part of close(fd=3), and the socket transitions to TIME_WAIT. But the rcv_buff stays alive through TIME_WAIT — the kernel needs it in case the server retransmits its FIN. The client has to be able to receive and respond. Only after 2*MSL expires does the kernel free the tcp_sock, rcv_buff, and all remaining state.

The fd and pollDesc die immediately at conn.Close(). The kernel socket outlives them both.

Every row in that table is real kernel work. Every connection teardown goes through it.

This Is Why Connection Pools Exist

Look at that table again. Every connection teardown fires epoll_ctl(DEL), clears the pollDesc, closes the fd, sends a FIN, waits for ACK, frees the socket structure, frees both buffers, waits out TIME_WAIT. And that’s just the teardown. The setup side is just as expensive — socket(), bind(), connect(), the three-way handshake, buffer allocation, pollDesc registration.

Every new connection pays that cost twice. Once to set up, once to tear down.

In the teardown diagrams above, I marked every resource cleanup event in green — epoll_ctl(DEL, fd=10), the pollDesc entry being cleared, the fd freed from the process table, the final kernel free of tcp_sock, rcv_buff, send_buff, and all TCP state. Each green box is real kernel work that happens on every single connection close.

Connection pooling is the realisation that most of that work is unnecessary if the connection can be reused. Instead of tearing a connection down after each request, a pool keeps it alive and hands it back for the next one. None of those green boxes fire. The tcp_sock stays in kernel memory. The buffers stay allocated. The pollDesc entry stays registered with epoll. The next request skips the handshake entirely and goes straight to writing bytes into an already-established connection.

The netpoller makes Go capable of holding thousands of those alive connections open simultaneously without wasting OS threads. But the connections themselves still cost kernel memory, fd table entries, and buffer allocations. That cost doesn’t disappear just because goroutines are cheap.

Go’s netpoller solves the thread problem. Connection pools solve the connection cost problem. They operate at different layers and both matter.

Pool them. Reuse them. The teardown is expensive.

What’s Still Open

I now understand what happens when you call net.Listen and net.Dial. The kernel owns more of the process than I realised. The handshake, the queues, the buffers, the teardown, most of it runs without our code ever being called.

Go’s contribution is making the interface to all of that feel sequential. Accept() looks blocking. Read() looks blocking. Neither holds a thread while waiting. The netpoller and epoll handle the gap between “nothing ready” and “resume here.”

What I still don’t understand well enough: what happens when thousands of connections are open simultaneously and I don’t want to pay the handshake cost every time. How connection pools work. How HTTP’s request-response framing sits on top of exactly this primitive. How the send and receive buffers interact with TCP’s flow control when one side is slow.

That’s where this goes next.

The diagrams in this article are from my Excalidraw canvas where I mapped the full TCP connection lifecycle while building this. The full canvas is in the repo here.

I used to think logging was this:

log.Info("request ended")

One line. Done.

Turns out logging under high concurrency is where I finally understood what serialization actually costs. I’d seen memory spikes in CloudWatch before when traffic spiked 1,000+ requests/min on a single EC2 instance, memory usage climbing. Understanding how logging works under concurrency made allocation pressure visible.

What a log write actually is

When you call:

log.Info(ctx, "Request ended", "duration", "8.2s", "status", 200)

Three things happen.

Formatting. Individual fields get assembled into a byte sequence:

"duration" + "8.2s" + "status" + 200
        ↓
{"time":"...","level":"INFO","msg":"Request ended","duration":"8.2s","status":200}

This assembled byte sequence lives in a scratch buffer in memory while being built.

Write. The formatted bytes get written to fd=1 (stdout):

w.Write(buf)  // syscall → kernel → fd=1 → wherever stdout points

Discard. The scratch buffer is done, goes out of scope.

Simple enough for one goroutine. Now add 1000 concurrent request handlers all logging simultaneously.

The concurrent write problem

Two goroutines writing to fd=1 simultaneously without coordination:

goroutine A: {"level":"INFO","msg":"Request en
goroutine B: {"level":"ERROR","msg":"upload failed"}
goroutine A: ded","duration":"8.2s"}

What lands on disk:

{"level":"INFO","msg":"Request en{"level":"ERROR","msg":"upload failed"}ded","duration":"8.2s"}

Not valid JSON. Log parsers break. Loki can’t index fields. Grafana shows garbage. A real production incident becomes unreadable at the worst possible moment.

The fix is a mutex around the write.

slog‘s JSONHandler does exactly this:

type JSONHandler struct {
    mu *sync.Mutex
    w  io.Writer
}

func (h *JSONHandler) Handle(ctx context.Context, r Record) error {
    h.mu.Lock()
    defer h.mu.Unlock()
    // format entire record into internal buffer
    // write complete JSON line to w
}

One goroutine holds the mutex, formats and writes, releases. Others wait. Each log line lands as a complete JSON object, no interleaving.

The mutex bottleneck

The mutex solves correctness. But it covers the entire format+write operation:

[goroutine A holds mutex: building JSON string... writing... done]
[goroutine B waits                                               ]
[goroutine C waits                                               ]

Under 1000 concurrent goroutines, 999 are waiting while 1 formats. The expensive part, assembling the JSON string, is serialized even though it could be done in parallel. Each goroutine is working with its own data, there’s no reason they can’t format simultaneously.

The insight: formatting doesn’t need shared state. Only the write to fd=1 needs coordination.

slog:    [mutex: format + write]          ← mutex too wide
zerolog: format (no mutex) → [mutex: write only]  ← mutex as narrow as possible

zerolog’s approach, format outside the lock

zerolog moves formatting outside the mutex by giving each goroutine its own scratch buffer to format into:

// goroutine A                    // goroutine B
buf_A := pool.Get()               buf_B := pool.Get()
buf_A.Write(`{"level":"INFO"...`) buf_B.Write(`{"level":"ERROR"...`)
// both formatting in parallel, no contention

mutex.Lock()                      // B waits here
w.Write(buf_A)
mutex.Unlock()
                                  mutex.Lock()
                                  w.Write(buf_B)
                                  mutex.Unlock()

Formatting happens in parallel. The mutex window shrinks to just the write syscall, microseconds instead of milliseconds.

But where do the scratch buffers come from?

The naive approach: allocate a fresh []byte for every log call. Under 1000 requests/sec, that’s 1000 allocations/sec just for log buffers, continuous GC pressure. This is where sync.Pool comes in.

sync.Pool: object pooling for scratch buffers

sync.Pool is a pool of reusable objects. Instead of allocating and discarding:

allocate → use → discard → GC collects → allocate → use → discard → GC collects ...

You reuse:

allocate once → use → reset → return to pool → use → reset → return to pool ...

Basic usage:

var pool = &sync.Pool{
    New: func() any {
        return bytes.NewBuffer(make([]byte, 0, 1024))
    },
}

buf := pool.Get().(*bytes.Buffer)   // checkout: reuse existing or allocate new
buf.WriteString(`{"level":"INFO"}`) // format into it
// ...write to fd=1...
buf.Reset()                          // clear contents, keep underlying memory
pool.Put(buf)                        // checkin: available for next caller

The key: buf.Reset() clears the contents but keeps the allocated []byte underneath. The memory is reused, not garbage collected and reallocated.

A catch: buffer sizes vary. One log message might need 500 bytes, another 5KB. But bytes.Buffer grows dynamically, and keeps that capacity after Reset():

buf := pool.Get().(*bytes.Buffer)  // capacity: 1024 (initial)
buf.WriteString(veryLongJSON)      // content: 8KB → buffer grows to 8KB capacity
buf.Reset()                        // content cleared, capacity: still 8KB
pool.Put(buf)                      // 8KB-capacity buffer goes back to pool

Reset() zeroes the length but doesn’t touch the underlying []byte array. The slice capacity stays. So the pool naturally holds buffers at the high-watermark of what’s been written through them.

After a warm-up period, pool buffers converge to the size of typical log lines, not the initial 1024 you allocated in New. Each buffer has organically grown to fit real usage. Occasional large entries cause a one-time growth, then that larger buffer circulates.

The trade-off: an unusually large log line (say, a 500KB debug dump) causes one buffer to grow to 500KB. That buffer now lives in the pool permanently, until GC collects it via the victim cycle. You’re holding 500KB of RAM for a buffer that 99% of requests will use 2KB of.

But you’re still orders of magnitude better than allocating fresh on every call. Pooling doesn’t assume uniform buffer sizes. It assumes that buffers, once grown to fit real usage, will keep fitting real usage. The growth cost is paid once. The reuse benefit is paid on every subsequent call.

This is what allocation pressure looks like. Concurrent request handlers each allocating scratch buffers. Without pooling, every goroutine allocates fresh. Under high traffic, those allocations pile up faster than GC can collect them. Memory climbs. The spike isn’t a leak. It’s just allocation pressure under concurrency.

This is what sync.Pool prevents.

Why sync.Pool is lock-free in the common case

A naive pool would use a mutex-protected slice. Under high concurrency, goroutines would contend on that mutex, no better than what we started with.

sync.Pool is smarter. It exploits Go’s scheduler structure.

Go’s scheduler has P processors (one per CPU core by default). Every goroutine runs on a P. sync.Pool gives each P its own local slot:

Pool
  local[0] → buf   ← P0's slot
  local[1] → buf   ← P1's slot
  local[2] → nil   ← P2's slot, empty
  local[3] → buf   ← P3's slot

When a goroutine calls pool.Get():

1. Which P am I on? Say P0.

2. Is local[0] non-nil? Yes → take it. Done. No lock needed.

3. If nil → check other P’s slots (needs lock) → if all empty → call New

The common case (P has an object in its slot) is lock-free. Only the uncommon case (P’s slot is empty, need to steal from another P) requires a lock.

The goroutine stays on P0 the entire time. When P0’s slot is empty, the goroutine (still on P0) reaches across and takes an object from another P’s slot. The object moves, the goroutine doesn’t.

This is called work stealing, the same pattern the Go scheduler uses, but here we’re stealing objects, not goroutines.

The GC interaction: victim cache

Here’s where it gets interesting.

sync.Pool is not a cache. Objects in the pool can disappear at any GC cycle. But there’s a grace period.

When GC runs, sync.Pool doesn’t immediately drop everything. It moves the current pool contents into a victim cache, a secondary holding area. The victim cache survives for exactly one more GC cycle.

GC #1: current pool → victim cache
       pool is now empty

During the next GC interval:
  - Get() checks current pool first
  - If empty, checks victim cache
  - If both empty, calls New()

GC #2: victim cache is cleared
       (objects from GC #1 are finally eligible for collection)

Why does this matter?

After a traffic spike, allocation rate drops. GC runs. Without the victim cache, the pool would be empty immediately, and the next request would allocate fresh buffers even though the spike just ended.

The victim cache gives the system two GC cycles to smooth the transition. If traffic stays low, objects eventually get collected. If traffic spikes again before the second GC, the victim cache prevents re-allocation.

The pattern appears everywhere

Once I understood why sync.Pool exists for logging, I started seeing it everywhere.

HTTP request/response body buffers

When Go’s net/http receives a request, it reads raw TCP bytes into a buffer before parsing headers, body, etc. Thousands of concurrent connections = thousands of concurrent buffer allocations without pooling. Go’s HTTP server uses sync.Pool internally for exactly this.

TCP socket → pool buffer → HTTP parser → request struct
                ↑ pooled

JSON encode/decode scratch buffers

json.Marshal builds a []byte while encoding your struct to JSON. json.Unmarshal needs scratch space for tokenizing the input. Both operations are short-lived and happen on every API request.

Go struct → scratch buffer (JSON bytes being built) → write to TCP socket
                ↑ pooled

The buffer contains the in-progress JSON string, exists only for the duration of the marshal call, then done. Libraries like jsoniter and easyjson use pools aggressively for this reason.

Database query result buffers

The database driver reads raw bytes from the TCP connection (the database protocol wire format) into a scratch buffer, then deserializes into your Go structs. The buffer is the temporary home for raw wire data between “arrived from network” and “parsed into application types.”

TCP bytes (DB wire protocol) → pool buffer → deserializer → your Go struct
                                    ↑ pooled

Under concurrent queries, each goroutine needs its own buffer, pool provides them without per-query allocation cost.

Protobuf marshalling buffers

Same as JSON, protobuf encoding builds a byte sequence before writing to the network. protoc-generated Go code uses pools internally for the scratch buffer.

The general pattern

Any time you see:

• A scratch buffer allocated at the start of an operation

• Used temporarily to hold intermediate bytes

• Discarded when the operation completes

• Under concurrent load

That’s a candidate for sync.Pool.

The smell in code: make([]byte, ...) or new(SomeStruct) inside a hot path that runs thousands of times per second.

Object pooling vs connection pooling

Connection pooling reuses TCP connections instead of creating new ones per request. Creating a TCP connection is expensive: 3-way handshake, TLS negotiation, kernel buffer allocation.

Buffer/object pooling reuses memory allocations instead of allocating per operation. Creating a buffer is cheap individually but expensive at scale: GC pressure, allocation spikes, increased latency variance.

Same underlying idea: expensive-to-create resource, reuse instead of recreate. Different resources, same pattern.

                    Connection Pool              Buffer Pool
─────────────────────────────────────────────────────────────────────
Resource            TCP connection               Memory allocation
Expense avoided     Handshake + kernel setup     GC pressure + allocation
Go primitive        http.Transport               sync.Pool
Lifecycle           Long-lived                   Ephemeral (reset per use)
Pool size           Bounded (MaxIdleConns)       Unbounded, GC-managed

The hidden cost of serialization

Understanding buffer pooling leads to a bigger realization: serialization is not free, and we introduce it constantly without thinking about it.

Every log write serializes fields into JSON bytes. We studied this carefully, scratch buffers, GC pressure, mutex contention. But the exact same cost exists at every API boundary in a system:

Service A
    ↓
marshal to JSON/protobuf
(scratch buffer, GC pressure, CPU cycles building bytes)
    ↓
network
    ↓
unmarshal
(scratch buffer, GC pressure, CPU cycles parsing bytes)
    ↓
Service B

Every microservice call pays this twice. Every database query pays it. Every cache read pays it. Every message queue publish/consume pays it.

What serialization actually costs

At the code level it looks like one line:

json.Marshal(myStruct)

What’s actually happening:

• Allocate a scratch buffer

• Reflect over every field of the struct

• Convert each field to its string/byte representation

• Write delimiters, quotes, brackets

• Return the completed byte sequence

• GC eventually collects the scratch buffer

And then on the receiving end, unmarshal does the reverse, tokenize the bytes, allocate new struct fields, parse each value back into its Go type.

Under high concurrency, this is happening thousands of times per second across every service boundary.

The invisible tax of microservices

In a monolith, a function call has zero serialization cost:

result := orderService.GetOrder(id)  // in-process, just a pointer

Split that into two services:

Client
    ↓
marshal request
    ↓
HTTP
    ↓
unmarshal request
    ↓
[Service processes]
    ↓
marshal response
    ↓
HTTP
    ↓
unmarshal response
    ↓
Client receives result

Four serialization operations for what was zero. Every service boundary you add multiplies this cost across every request that crosses it.

Teams have measured 30-40% of total request latency in some microservice architectures being pure serialization overhead, not business logic, not database queries, not network RTT. Just converting data between bytes and structs.

Why we don’t notice

Frameworks hide it. gin.ShouldBindJSON(&req) looks like one line. gRPC stubs look like function calls. The cost is real but invisible until you profile under load and see serialization dominating your flame graph.

The design implication

Every time you say “let’s add an API” or “let’s split this service,” you’re adding a serialization boundary. That’s sometimes the right call, isolation, independent deployments, team ownership. But it’s a cost, not just an architectural choice.

Questions worth asking before adding an API boundary:

• Can this be an in-process call instead?

• If it must be remote, can we use a binary format (protobuf) instead of JSON?

• Can we batch calls to amortize the serialization cost?

• Is the data structure designed to serialize efficiently (flat vs deeply nested)?

Protobuf vs JSON: not just syntax

This is why protobuf exists. JSON serialization uses reflection and string manipulation. Protobuf uses generated code with direct field access no reflection, smaller wire format, faster parse. The trade-off: schema must be defined upfront, less human-readable.

At high volume, this matters:

JSON:     ~500ns to marshal a typical request struct
Protobuf: ~100ns for the same struct

5x difference. Across millions of requests per day, that’s real CPU time.

The logging connection

This is what made logging non-trivial: it’s serialization on every request, under concurrency, in the hot path. The solutions, zerolog, sync.Pool, narrow mutexes, are the same solutions high-performance serialization libraries use everywhere.

Logging just made the problem visible because it happens on every single request, not just at service boundaries.

What I learned

A log write is format + write. Format is parallelizable, write must be serialized. Putting both under the same mutex is the conservative but suboptimal choice.

sync.Pool is for ephemeral scratch space. Short-lived, frequently allocated, cheap to reset. Not for objects with meaningful state.

Pool size is dynamic. Grows under spikes, shrinks after two GC cycles. No manual sizing.

Per-P slots make Get/Put lock-free in the common case. The GC interaction (victim cache) gives a two-cycle grace period to smooth post-GC allocation spikes.

Buffer pooling and connection pooling are the same idea. Reuse expensive-to-create resources instead of creating them fresh per operation.

The pattern appears everywhere in high-performance Go. net/http, encoding/json, database drivers, protobuf all use pooling internally.

Serialization is not free, and we introduce it constantly. Every API boundary pays serialization cost twice (marshal + unmarshal). Every microservice split multiplies this across every request that crosses the boundary. In-process function calls have zero serialization cost.

“Let’s add an API” is a serialization decision. Before adding a service boundary, ask: can this be in-process? If remote is necessary, use binary formats (protobuf) over text (JSON), batch calls, and design flat data structures. 30-40% of request latency in some microservice architectures is pure serialization overhead, not business logic, not network RTT.

Logging made serialization visible because it’s on every single request. The solutions (zerolog, sync.Pool, narrow mutexes) are the same patterns high-performance serialization libraries use everywhere. Logging is just the place where the cost became impossible to ignore.

The upload finished too fast to observe. An 838-byte test file completed in under a millisecond. To actually see progress events stream back to the client, I needed to slow it down.

That led to building a token bucket rate limiter from scratch. Not because the service needed throttling, but because understanding how to control data flow at the io.Reader layer turned out to be the clearest way to understand streaming itself.

This is Part 1 of HTTP Protocol Deep Dives. Each article covers one prototype built from scratch to understand a specific HTTP concept at the wire level.

What Gets Built

Three endpoints:

POST /upload/initiate         # → registers upload, returns {upload_id, upload_url}
PUT  /upload/{uploadid}       # → receives file, streams to disk
GET  /upload/progress/{uploadid}  # → SSE stream of upload progress

No S3 pre-signed URLs. No message queues. The goal is to understand the HTTP machinery directly, not abstract it away.

The upload endpoint accepts multipart/form-data and streams the file to disk using io.Copy. The progress endpoint opens a Server-Sent Events stream and polls upload state every 200ms. Two separate HTTP connections for one upload.

Where Data Lives

Before walking through the implementation, here’s where the data actually sits at each stage.

For an 8GB upload:

• Socket buffer: ~128KB (kernel-managed)

• io.Copy buffer: 32KB (user-space, reused)

• Total RAM: ~160KB

The rest is either in transit on the network, already on disk, or waiting in the OS page cache to flush.

Streaming means never accumulating.

This is what makes io.Copy work. It allocates a 32KB buffer once and reuses it in a loop. The buffer size doesn’t grow with file size.

The io.Reader contract

type Reader interface {
    Read(p []byte) (n int, err error)
}

The caller allocates the buffer p. The reader fills it. Returns n, the number of bytes written. The caller only looks at p[:n], never p[:].

io.Copy does this:

buf := make([]byte, 32*1024)  // allocated once
for {
    nr, er := src.Read(buf)
    if nr > 0 {
        nw, ew := dst.Write(buf[0:nr])
    }
    if er == io.EOF { break }
}

Every src.Read asks “give me up to 32KB.” The reader returns however many bytes are available, could be 32KB, could be 1 byte, could be 0 with an error. The caller handles all three.

The Upload Flow

Here’s what happens when a file moves through the system.

Step 1: Initiate

POST /upload/initiate

Curl command:

curl -i -X POST http://localhost:8080/upload/initiate

Server generates an upload ID, creates an entry in the progress store:

{
  "upload_id": "0f033e91-5317-4476-bc89-4a64bc3354d0",
  "upload_url": "/upload/0f033e91-5317-4476-bc89-4a64bc3354d0"
}

The code:

func (u *UploadAPI) InitiateUpload(ctx context.Context, w http.ResponseWriter, r *http.Request) error {
    id := uuid.NewString()
    u.ups.SetProgress(id, uploadprogress.Progress{})
    apis.WriteJson(w, http.StatusOK, apis.ApiResponse{
        Data: struct {
            UploadId  string `json:"upload_id"`
            UploadUrl string `json:"upload_url"`
        }{
            UploadId:  id,
            UploadUrl: fmt.Sprintf("/upload/%v", id),
        },
    })
    return nil
}

The progress store entry exists before either connection starts. This solves the race, the SSE handler can connect and start polling immediately without waiting for the upload to begin.

Step 2: Two connections open

PUT /upload/0f033e91-5317-4476-bc89-4a64bc3354d0
  ↳ sends file as multipart/form-data

GET /upload/progress/0f033e91-5317-4476-bc89-4a64bc3354d0
  ↳ keeps connection open, receives progress events

Curl commands:

# Upload file
curl -i -X PUT http://localhost:8080/upload/0f033e91-5317-4476-bc89-4a64bc3354d0 \
-F file=@./move-crew-details.csv

# Watch progress (in a separate terminal)
curl -i -N http://localhost:8080/upload/progress/0f033e91-5317-4476-bc89-4a64bc3354d0

Two separate HTTP connections. HTTP/1.1 is half-duplex, a single connection can’t receive a large PUT body and send SSE events simultaneously. One connection uploads, the other streams progress.

Step 3: The complete HTTP PUT request

Here’s what actually arrives at the server:

PUT /upload/0f033e91-5317-4476-bc89-4a64bc3354d0 HTTP/1.1
Host: localhost:8080
Content-Length: 838
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="move-crew-details.csv"
Content-Type: text/csv

(file data bytes)
------WebKitFormBoundary7MA4YWxkTrZu0gW--

The Content-Type header declares multipart/form-data and includes a boundary parameter. That boundary string marks where each part begins and ends in the body. The spec doesn’t require ------ prefixes, that’s just curl’s convention.

Step 4: Validate Content-Type

ctype := r.Header.Get("content-type")
mtype, params, err := mime.ParseMediaType(ctype)
if err != nil {
    return apis.NewErrorf(http.StatusBadRequest, "invalid content type. expected multipart/form-data", "content-type sent in request: %s", ctype)
}

if mtype != "multipart/form-data" {
    return apis.NewError(http.StatusBadRequest, "invalid content type. expected multipart/form-data")
}

_, ok := params["boundary"]
if !ok {
    return apis.NewErrorf(http.StatusBadRequest, "boundary not found in multipart/form-data content type", "content-type sent in request: %s", ctype)
}

Validation happens in three steps:

1. Parse the header with mime.ParseMediaType

2. Verify media type is multipart/form-data

3. Verify boundary parameter exists

The server doesn’t validate the boundary value itself. It just passes it to the multipart reader.

Step 5: Stream to disk

reader, err := r.MultipartReader()
if err != nil {
    u.log.Error(ctx, "failed to get multipart reader", "action", "r.MultipartReader", "err", err)
    return apis.NewError(http.StatusInternalServerError, "something went wrong. try again.")
}

for {
    part, err := reader.NextPart()
    if err == io.EOF {
        break
    }
    if err != nil {
        u.log.Error(ctx, "error reading next part", "action", "reader.NextPart", "err", err)
        return apis.NewError(http.StatusInternalServerError, "failed to read part")
    }
    
    // extract filename from Content-Disposition header
    cdtype := part.Header.Get("content-disposition")
    cdmtype, cdparams, err := mime.ParseMediaType(cdtype)
    
    filename, ok := cdparams["filename"]
    if ok {
        basepath := filepath.Base(filename)
        tmp, err := os.Create(fmt.Sprintf("./%s", basepath))
        if err != nil {
            return apis.NewError(http.StatusBadRequest, "something went wrong")
        }
        defer tmp.Close()
        
        wr := uploadwriter.New(tmp, uploadid, u.ups)
        n, err := io.Copy(wr, part)
        
        u.ups.SetProgress(uploadid, uploadprogress.Progress{
            Err:        err,
            Total:      uint64(n),
            SoFar:      uint64(n),
            IsComplete: true,
        })
    }
}

r.MultipartReader() returns a streaming reader. reader.NextPart() gets each part one at a time. io.Copy allocates a 32KB buffer once, reuses it in a loop.

Step 6: Track progress via wrapped writer

The uploadwriter wraps the file handle, intercepts every Write:

type Writer struct {
    w     io.Writer
    id    string
    sofar uint64
    ups   *uploadprogress.Store
}

func (uw *Writer) Write(p []byte) (int, error) {
    n, err := uw.w.Write(p)
    uw.sofar += uint64(n)
    uw.ups.SetProgress(uw.id, uploadprogress.Progress{
        Err:   err,
        SoFar: uw.sofar,
    })
    return n, err
}

Every chunk written to disk updates the progress store. The SSE handler polls this store every 200ms, streams events to the client.

Step 7: SSE streams back

func (u *UploadAPI) UploadProgress(ctx context.Context, w http.ResponseWriter, r *http.Request) error {
    w.Header().Set("content-type", "text/event-stream")
    w.Header().Set("cache-control", "no-cache")
    w.Header().Set("connection", "keep-alive")
    
    id := r.PathValue("uploadid")
    
    for {
        select {
        case <-r.Context().Done():
            return nil
        default:
            progress, err := u.ups.GetProgressById(id)
            if err != nil {
                return apis.NewError(http.StatusNotFound, "upload-id not found")
            }
            
            if progress.IsComplete {
                fmt.Fprintf(w, "event: upload-progress\ndata: {\"progress\": \"%v\", \"complete\": %v}\n\n", progress.SoFar, progress.IsComplete)
                w.(http.Flusher).Flush()
                u.ups.DeleteProgressById(id)
                return nil
            }
            
            if progress.SoFar == 0 && !progress.IsComplete {
                time.Sleep(2000 * time.Millisecond)
                continue
            }
            
            if progress.Err != nil && !progress.IsComplete {
                fmt.Fprintf(w, "event: upload-progress\ndata: {\"error\": \"%v\"}\n\n", "failed to upload. Try again")
                w.(http.Flusher).Flush()
                return nil
            }
            
            fmt.Fprintf(w, "event: upload-progress\ndata: {\"progress\": \"%v\", \"complete\": %v}\n\n", progress.SoFar, progress.IsComplete)
            w.(http.Flusher).Flush()
            
            time.Sleep(200 * time.Millisecond)
        }
    }
}

Flush() is critical. Forces each event to be sent as its own HTTP chunk rather than accumulating in the response buffer. Without it, the client sees nothing until the upload completes.

SSE wire format:

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
Transfer-Encoding: chunked

event: upload-progress
data: {"progress": "180", "complete": false}

event: upload-progress
data: {"progress": "380", "complete": false}

event: upload-progress
data: {"progress": "838", "complete": true}

The blank line between events is the SSE delimiter. Transfer-Encoding: chunked gets applied automatically by net/http when writing without a known Content-Length.

Here’s what the client actually sees:

curl -i -N http://localhost:8080/upload/progress/0f033e91-5317-4476-bc89-4a64bc3354d0
HTTP/1.1 200 OK
Cache-Control: no-cache
Connection: keep-alive
Content-Type: text/event-stream
Date: Sat, 18 Apr 2026 08:10:00 GMT
Transfer-Encoding: chunked

event: upload-progress
data: {"progress": "60", "complete": false}

event: upload-progress
data: {"progress": "80", "complete": false}

.....

event: upload-progress
data: {"progress": "820", "complete": false}

event: upload-progress
data: {"progress": "838", "complete": true}

Controlling the Flow: Token Bucket Rate Limiting

Now, what if we want to control how fast data flows through this pipeline?

For a small test file, io.Copy is fast. The entire upload completes in under a millisecond, no SSE events, instant completion. To observe the SSE progress stream meaningfully, the upload needs to be slow enough to emit multiple events.

That’s where the token bucket comes in.

Where the Token Bucket Sits

The token bucket sits between the TCP socket buffer and the io.Copy buffer. It’s an io.Reader wrapper, it wraps the multipart part and controls how fast bytes get pulled from the source.

Here’s how it gets wired in:

wr := uploadwriter.New(tmp, uploadid, u.ups)
n, err := io.Copy(wr, slowreader.New(part))

Without slowreader.New(part), the code would be:

n, err := io.Copy(wr, part)

That would be full speed. By wrapping part in slowreader.New(part), every read from part now goes through the token bucket first.

Let’s zoom in on how the bucket actually works.

The Token Bucket Model

Think of it like this:

Tokens = currency
Read(p) = purchase
refillRate = how fast you earn currency (100 bytes/sec)
maxBurst = maximum currency you can hold at once (20 bytes)

The bucket starts with some tokens. Every time io.Copy calls Read(p), the bucket checks: do I have enough tokens to fulfill this read? If yes, read and deduct tokens. If no, wait until enough tokens accumulate.

Tokens refill passively over time at refillRate bytes per second, up to a maximum of maxBurst.

Here’s the actual implementation:

type slowReader struct {
    r              io.Reader
    tokenBal       int64
    refillRate     int64
    lastConsumedAt time.Time
    maxBurst       int64
}

func New(r io.Reader) *slowReader {
    return &slowReader{
        r:              r,
        tokenBal:       1 * 20,  // 20 bytes
        refillRate:     100,     // 100 bytes/sec
        maxBurst:       1 * 20,  // 20 bytes max
        lastConsumedAt: time.Now(),
    }
}

Initial state:

• tokenBal: 20 bytes (start with a full bucket)

• refillRate: 100 bytes/sec

• maxBurst: 20 bytes (can’t hold more than this)

• lastConsumedAt: timestamp of last read

The Read Implementation

Every time io.Copy calls Read(p), this is what happens:

func (s *slowReader) Read(p []byte) (int, error) {
    // Step 1: Calculate how many tokens we've accumulated since last read
    elapsed := time.Since(s.lastConsumedAt).Seconds()
    newTokens := elapsed * float64(s.refillRate)
    bal := min(s.tokenBal+int64(newTokens), s.maxBurst)
    
    // Step 2: Decide what to do based on current balance
    // (three cases below)
}

Step 1 calculates token refill. If 0.5 seconds passed since the last read, and refillRate is 100 bytes/sec, then newTokens = 0.5 × 100 = 50 bytes. But bal is capped at maxBurst = 20, so bal = min(20 + 50, 20) = 20.

Step 2 splits into three cases based on how many tokens are available.

Case 1: Enough tokens for the full read

if bal >= int64(len(p)) {
    n, err := s.r.Read(p)
    if err != nil {
        return n, err
    }
    s.tokenBal -= int64(n)
    s.lastConsumedAt = time.Now()
    return n, err
}

io.Copy passes a 32KB buffer as p. If the bucket has 20 tokens and len(p) = 32768, this condition is false. But if len(p) were smaller (say, 10 bytes) and bal = 20, this would trigger.

Read the full buffer, deduct n tokens (actual bytes read), update timestamp, return.

Case 2: Some tokens, but less than len(p)

if bal > 0 && bal < int64(len(p)) {
    temp := make([]byte, bal)
    n, err := s.r.Read(temp)
    if err != nil {
        return n, err
    }
    copy(p, temp[:n])
    s.tokenBal -= int64(n)
    s.lastConsumedAt = time.Now()
    return n, err
}

This is the common case. We have 20 tokens, but io.Copy is asking for 32KB. We can’t give 32KB, so we give what we have.

Steps:

1. Create a temp buffer sized to bal (20 bytes)

2. Read into that temp buffer (read at most 20 bytes from the source)

3. Copy those bytes into the caller’s buffer p

4. Deduct n tokens

5. Return n (could be 20, could be less if the source gave us less)

The caller (io.Copy) asked for 32KB but got 20 bytes. That’s fine. The io.Reader contract says: return however many bytes are available, up to len(p). The caller handles partial reads.

Case 3: No tokens, wait

// here bal == 0
remaining := min(int64(len(p)), s.maxBurst)

waitTime := (float64(remaining)) / float64(s.refillRate)
time.Sleep(time.Duration(waitTime * float64(time.Second)))

temp := make([]byte, remaining)
n, err := s.r.Read(temp)
copy(p, temp[:n])

s.tokenBal = remaining - int64(n)
s.lastConsumedAt = time.Now()

return n, err

No tokens available. We need to wait for tokens to accumulate.

Critical line:

remaining := min(int64(len(p)), s.maxBurst)

The caller asked for len(p) = 32768 bytes. But the bucket can only ever hold maxBurst = 20 bytes. Waiting for 32KB worth of tokens would take 32768 / 100 = 327 seconds.

That’s the bug that happened. The upload hung for 327 seconds per read.

The fix: wait only for min(len(p), maxBurst) = min(32768, 20) = 20 tokens.

Wait time calculation:

waitTime := (float64(remaining)) / float64(s.refillRate)

remaining = 20, refillRate = 100, so waitTime = 20 / 100 = 0.2 seconds.

Sleep for 0.2 seconds, then tokens have accumulated. Read up to remaining bytes (20 bytes), copy into caller’s buffer, update state.

After the sleep, the bucket doesn’t necessarily have exactly remaining tokens (some might have accumulated before the sleep, some during). The line:

s.tokenBal = remaining - int64(n)

assumes we used n tokens out of the remaining we waited for. If n < remaining, we have leftover tokens for the next read.

Why Three Cases?

The three-case structure handles all possible states:

1. Bucket is full or has enough → read immediately

2. Bucket has some tokens → read what we can, don’t wait

3. Bucket is empty → wait for refill, then read

Without case 2, every partial-token situation would fall through to case 3 and sleep unnecessarily. Case 2 says: “I have 5 tokens right now, you asked for 32KB, here’s 5 bytes, come back for more.”

Why Float Division Matters

Original broken code:

waitTime := remaining / s.refillRate  // both int64

20 / 100 = 0 in integer division. Zero wait, zero throttling.

Fixed code:

waitTime := float64(remaining) / float64(s.refillRate)

20.0 / 100.0 = 0.2. Sleep for 0.2 seconds.

A single type mismatch made throttling disappear silently. No error, no panic, just wrong behavior.

Watching It Work

With maxBurst = 20, refillRate = 100, uploading an 838-byte file, here’s what happens:

Read 1:

• io.Copy asks for 32KB

• Bucket has 20 tokens

• Case 2 triggers: read 20 bytes, return 20 bytes

• Tokens left: 0

Wait ~0.2 seconds (next io.Copy iteration)

Read 2:

• io.Copy asks for 32KB

• Bucket has ~20 tokens (refilled during the wait)

• Case 2 triggers: read 20 bytes, return 20 bytes

• Tokens left: 0

This repeats. Each read yields ~20 bytes. Total reads: 838 / 20 ≈ 42. Each read waits ~0.2 seconds.

Expected time: 42 × 0.2 = 8.4 seconds.

SSE output:

event: upload-progress
data: {"progress": "60", "complete": false}

event: upload-progress
data: {"progress": "180", "complete": false}

event: upload-progress
data: {"progress": "380", "complete": false}

...

event: upload-progress
data: {"progress": "838", "complete": true}

Server logs:

go run cmd/prototypes/main.go
{
  "time": "2026-04-18T13:39:36.837574+05:30",
  "level": "INFO",
  "file": "internal/server/server.go:26",
  "msg": "starting server",
  "port": "0.0.0.0:8080"
}
{
  "time": "2026-04-18T13:39:44.493091+05:30",
  "level": "INFO",
  "file": "internal/middlewares/tracing/tracing.go:26",
  "msg": "Request started",
  "trace": {
    "trace_id": "6be9b4e5-30e6-4cc5-aa30-ee0693aba156",
    "request_method": "POST",
    "request_endpoint": "/upload/initiate"
  }
}
{
  "time": "2026-04-18T13:39:44.493278+05:30",
  "level": "INFO",
  "file": "internal/middlewares/tracing/tracing.go:32",
  "msg": "Request ended. request took",
  "duration": "54.159µs",
  "status": 200,
  "trace": {
    "trace_id": "6be9b4e5-30e6-4cc5-aa30-ee0693aba156",
    "request_method": "POST",
    "request_endpoint": "/upload/initiate"
  }
}
{
  "time": "2026-04-18T13:39:58.593116+05:30",
  "level": "INFO",
  "file": "internal/middlewares/tracing/tracing.go:26",
  "msg": "Request started",
  "trace": {
    "trace_id": "50580d5f-469e-4f40-aa81-f11471abdb3c",
    "request_method": "GET",
    "request_endpoint": "/upload/progress/0f033e91-5317-4476-bc89-4a64bc3354d0"
  }
}
{
  "time": "2026-04-18T13:40:00.146181+05:30",
  "level": "INFO",
  "file": "internal/middlewares/tracing/tracing.go:26",
  "msg": "Request started",
  "trace": {
    "trace_id": "2891a706-c295-408c-bd7c-142745735f04",
    "request_method": "PUT",
    "request_endpoint": "/upload/0f033e91-5317-4476-bc89-4a64bc3354d0"
  }
}
{
  "time": "2026-04-18T13:40:08.385989+05:30",
  "level": "INFO",
  "file": "internal/middlewares/tracing/tracing.go:32",
  "msg": "Request ended. request took",
  "duration": "8.239561818s",
  "status": 200,
  "trace": {
    "trace_id": "2891a706-c295-408c-bd7c-142745735f04",
    "request_method": "PUT",
    "request_endpoint": "/upload/0f033e91-5317-4476-bc89-4a64bc3354d0"
  }
}
{
  "time": "2026-04-18T13:40:08.426012+05:30",
  "level": "INFO",
  "file": "internal/middlewares/tracing/tracing.go:32",
  "msg": "Request ended. request took",
  "duration": "9.832609934s",
  "status": 200,
  "trace": {
    "trace_id": "50580d5f-469e-4f40-aa81-f11471abdb3c",
    "request_method": "GET",
    "request_endpoint": "/upload/progress/0f033e91-5317-4476-bc89-4a64bc3354d0"
  }
}

The PUT took 8.239 seconds. Expected: ~8.4 seconds.

The SSE stream stayed open for 9.832 seconds, slightly longer than the upload because it waited for the final completion event before closing.

The SSE handler polls every 200ms. Each poll sees progress incrementing by ~20–40 bytes (depending on timing). The progress climbs steadily until completion.

What This Shows

HTTP headers are declarative. The client declares content type in the Content-Type header. Don’t sniff the body.

Streaming means never accumulating. r.MultipartReader() + io.Copy keeps RAM at ~160KB for any file size.

Go’s io.Reader contract is precise. Caller allocates, reader fills, only p[:n] is valid.

SSE is HTTP with flushing. Content-Type: text/event-stream + Transfer-Encoding: chunked + Flush(). No magic.

Rate limiting is three cases, not one. Token bucket Read splits into: enough tokens, some tokens, no tokens.

Precision matters. Integer division 20/100 = 0. Float division 20.0/100.0 = 0.2. One type mismatch kills throttling silently.

What’s Next

Prototype 02: HTTP Client Connection Pool Analyzer.

The goal is to visualize connection reuse, test pool configurations, understand how http.Client actually manages connections under the hood.

Full code: https://github.com/vishal2098govind/http-deep-dive

ssh -i geek-ec2-kp.pem ec2-user@54.226.167.59

It works. I get a shell. I move on.

But with this “Building Trust on the Internet” series sitting in the back of my mind, I started paying attention to something I’d been ignoring.

The first time I connect to a new instance, the terminal pauses:

The authenticity of host '54.226.167.59 (54.226.167.59)' can't be established.
ED25519 key fingerprint is SHA256:WPHp1jPiPTmWCriZegalziBOTc3W0464HIYAdj0XxUU.
Are you sure you want to continue connecting (yes/no)?

I’d always typed yes without thinking. It’s part of the ritual. But this time I stopped.

I already had a key. That’s what the -i geek-ec2-kp.pem was for. So what was this fingerprint? What was SSH asking me to trust, and why did it matter?

Part 1 was about TLS — how browsers trust servers through certificate chains and PKI. Part 2 was about IAM — how AWS services authenticate without passwords. Both built trust through infrastructure.

SSH does something different.

1. Two Separate Keypairs

It took me a while to untangle this, but the answer is: SSH uses two separate keypairs in every connection.

I drew this out to make sure I had it straight:

The server host keypair (red circle, left side):

Generated by the remote machine itself when sshd is first installed or the instance boots for the first time. The private key stays in /etc/ssh/ssh_host_ed25519_key and never leaves the server. The public key gets sent to clients during the handshake, and clients store it in their ~/.ssh/known_hosts file.

This keypair proves “I am this specific EC2 instance, not a man-in-the-middle.”

The user authentication keypair (blue circle, right side):

➜  Downloads cat geek-ec2-kp.pem
-----BEGIN RSA PRIVATE KEY-----
<base64-encoded private key>
-----END RSA PRIVATE KEY-----%

This is the one I’m more familiar with. When I created the EC2 key-pair in AWS, AWS generated both keys. I downloaded the private key (the .pem file) and AWS injected the public key into the instance at /home/ec2-user/.ssh/authorized_keys during launch.

The diagram shows this flow: I keep the private key on my laptop, the instance gets the public key way before the SSH handshake even starts — either manually via authorized_keys or automatically via EC2’s cloud-init during instance launch.

This keypair proves “I am authorized to log in as ec2-user.”

The critical insight from the diagram:

These two flows are completely independent. The arrow on the left (server host keypair’s public key) goes from the server to my machine during the handshake. The arrow on the right (user auth keypair’s public key) goes from my machine to the server before any SSH connection even happens.

The .pem file I downloaded from AWS? That’s the user authentication private key. The fingerprint SSH was showing me during first connection? That’s the hash of the server’s host public key.

Two different keys, two different purposes, two different moments in time.

2. The Handshake Flow

The next question was: when does each keypair actually get used?

I traced through the byte-level handshake and split it into three phases:

Phase 1 - Setup, Banner Exchange, Algorithm Negotiations

The diagram shows the AWS setup at the top — when I created the EC2 key-pair, AWS sent the public key to the instance during launch. That public key sits in /home/ec2-user/.ssh/authorized_keys. I downloaded the private key (the .pem file) and saved it locally.

When the instance launched, it installed sshd and generated its own server-host-keypair. That private key stays in /etc/ssh/ssh_host_* on the instance.

Now I run: ssh -i geek-ec2-kp.pem ec2-user@54.226.167.59

Handshake Begins

Banner exchange (SSH protocol version exchange)

Server sends "SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5\r\n"
Client sends "SSH-2.0-OpenSSH_9.3\r\n"

Both sides announce their SSH version. Plain text, unencrypted. This is just protocol negotiation — “I speak SSH-2.0, here’s my software version.”

Encryption Algorithm negotiations

Both server and client decide on encryption algorithms to be used going further. This includes:

• Key exchange method (like curve25519-sha256 or ecdh-sha2-nistp256)

• Encryption cipher (like aes128-gcm)

• MAC algorithm

• Compression

Still unencrypted at this point. They’re just agreeing on what tools they’ll use once the secure channel is established.

Phase 2 - Key Exchange, Server Authentication, Client Requests User Auth

Key exchange

This is where the server’s host keypair comes in.

Client sends its ephemeral public key

Just for this session, used for deriving the shared secret and symmetric key. The client generated an ephemeral keypair (only lives for this one connection).

Server sends:

This is where I got confused initially — the server sends two public keys:

1. Server-host-keypair’s public key along with a signed hash to authenticate itself (for client to verify server)

2. Server’s ephemeral public key for key exchange

3. Signature of a hash (on strings which client also can derive on its own end) signed using the server-host-keypair’s private key

I had to sit with this to understand why two keys. One is for server authentication, the other is for the actual key exchange so both sides can derive a shared secret. The ephemeral one is short-lived, only exists during this handshake. The host key is long-lived — it’s the same key that gets sent every time from this server during every handshake, which is why the client adds it to known_hosts.

Both sides can now compute the same shared secret via ECDH. But the signature is the critical part. It proves “I own the private key matching this host public key, and I’m binding my identity to this specific key exchange.”

The hash being signed includes the banners, the algorithm lists, both ephemeral keys, and the shared secret from ECDH. The server signs it, the client verifies it.

Server authentication by client

The client takes the server’s host public key and checks ~/.ssh/known_hosts.

The client verifies the server by preparing the same hash and comparing it with the server’s hash (obtained by decrypting the signature using the server-host-keypair’s public key which the server sent during key exchange).

It also adds this public key (base64-encoded) to known_hosts against this particular host-name that was used during ssh -i /path/to/pem ec2-user@<host-name>.

The <host-name> can be a public IP address of EC2 instance or domain name if given.

If it’s a first connection, the client shows me the fingerprint and asks if I trust it. If it’s a repeat connection, the client verifies the key matches what’s stored. If there’s a mismatch, connection refused — that’s the MITM warning.

From here on, the channel is encrypted

As the diagram shows, the shared session keys (symmetric keys) are established. All further communication uses these symmetric keys for encryption.

Client initiates/requests user authentication

Client sends SSH_MSG_SERVICE_REQUEST and server sends SSH_MSG_SERVICE_ACCEPT.

This is because SSH can be used for other purposes as well like port-forwarding, and not only remote-login. Since here we are using SSH to authenticate to an AWS EC2 Linux instance, it means that I want to authenticate myself by sending SSH_MSG_SERVICE_REQUEST and server says “ok go ahead and authenticate yourself” by sending SSH_MSG_SERVICE_ACCEPT.

Phase 3 - User Authentication (Probe + Signature), Handshake Done

User authentication by server

Now, inside the encrypted tunnel, I finally prove I’m authorized to log in.

This happens in two steps:

Step 1: Probe request (User Authentication - Probe Phase)

Signing is an expensive process, so first the client will send a probe request (SSH_MSG_USERAUTH_REQUEST (Probe)) without signature, just for the server to verify basic details like if the requested user (say ec2-user or ubuntu) is present or not in the system.

After the server validates basic details, it sends the client SSH_MSG_USERAUTH_PK_OK saying “you can now send your signature and I will verify using my public key which I was shared when the ec2-key-pair was attached during my launch (or user shared with me when he did ssh-keygen).”

After basic validation check, server responds with SSH_MSG_USERAUTH_PK_OK.

Step 2: Actual signature (User Authentication by server)

Now the client prepares a signature_blob which the server can also prepare during its verification, and signs it using the ec2-key-pair’s private key which was downloaded while creating the key-pair (or created while doing ssh-keygen).

Client sends SSH_MSG_USERAUTH_REQUEST (with signature).

The server then verifies the signature using the ec2-keypair’s public key which was given to it while launching the ec2 instance, and if verified, authenticates client.

Server responds with SSH_MSG_USERAUTH_SUCCESS.

If valid, shell granted.

Handshake done

The diagram shows the Amazon Linux 2023 prompt — I’m now logged into the instance.

The key insight from these diagrams

Encryption happens before user authentication. The server proves its identity (via host key signature) before I ever prove mine. SSH secures the pipe first, then checks authorization.

The user keypair is only used once, during login. After that, all communication is encrypted with the session keys — symmetric keys derived from the ephemeral key exchange.

This mirrors what I covered in Part 1: asymmetric encryption (the host keypair, the user keypair) is used to establish trust and identity, but the actual data transmission uses symmetric encryption (AES-GCM) because it’s much faster for large amounts of data. The session keys are what handle all the terminal input/output, file transfers, everything after login. Asymmetric crypto did its job during the handshake. From here on, it’s just fast symmetric encryption.

3. known_hosts Isn’t “Trusted Hosts”

When I typed yes at that fingerprint prompt, SSH saved the server’s host public key to ~/.ssh/known_hosts:

54.226.167.59 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDxD6gi+FNijA1n40q0xZy5t8YyPL/CeqX2HTzmLhKnR

This isn’t a list of “servers I trust.” It’s a list of “this server has this public key.”

The difference matters.

On the next connection to 54.226.167.59, SSH doesn’t ask me again. It checks: does the public key the server just sent match what’s stored in known_hosts? If yes, connect silently. If no, scream.

SSH tracks servers by cryptographic identity, not by hostname.

The fingerprint (SHA256:WPHp1jPiPTmWCriZegalziBOTc3W0464HIYAdj0XxUU) is just a hash of that public key, displayed in a human-friendly format. Comparing a 43-character hash is easier than comparing a 256-byte key.

4. The Experiments

I wanted to see what actually triggers the warnings. So I spun up EC2 instances and ran through different scenarios.

Experiment 1: First connection

➜  Downloads ssh -i geek-ec2-kp.pem ec2-user@54.226.167.59
The authenticity of host '54.226.167.59 (54.226.167.59)' can't be established.
ED25519 key fingerprint is SHA256:WPHp1jPiPTmWCriZegalziBOTc3W0464HIYAdj0XxUU.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '54.226.167.59' (ED25519) to the list of known hosts.

That fingerprint is the SHA-256 hash of the server’s host public key. SSH is asking me to verify it. I typed yes. It got saved to ~/.ssh/known_hosts.

Experiment 2: Reboot

➜  Downloads echo "I am restarting my ec2 instance now"
I am restarting my ec2 instance now
➜  Downloads ssh -i geek-ec2-kp.pem ec2-user@54.226.167.59
   ,     #_
   ~\_  ####_        Amazon Linux 2023
  ~~  \_#####\
  ~~     \###|
  ~~       \#/ ___   https://aws.amazon.com/linux/amazon-linux-2023
   ~~       V~' '->
    ~~~         /
      ~~._.   _/
         _/ _/
       _/m/
[ec2-user@ip-172-31-20-136 ~]$

No prompt. SSH logged me in silently. Same instance, same host keys. The files in /etc/ssh/ssh_host_* survived the reboot.

Experiment 3: Stop/Start (new IP, same instance)

➜  Downloads echo "now i have stopped and restarted"
now i have stopped and restarted
➜  Downloads ssh -i geek-ec2-kp.pem ec2-user@54.196.250.140
The authenticity of host '54.196.250.140 (54.196.250.140)' can't be established.
ED25519 key fingerprint is SHA256:WPHp1jPiPTmWCriZegalziBOTc3W0464HIYAdj0XxUU.
This host key is known by the following other names/addresses:
    ~/.ssh/known_hosts:42: 54.226.167.59
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

Same fingerprint as before. Different IP. SSH noticed: “I’ve seen this key before, on 54.226.167.59.”

Stop/start preserves the EBS volume. The host keypair files stayed intact. New IP, same cryptographic identity.

I checked known_hosts:

➜  Downloads cat ~/.ssh/known_hosts | grep 54.196.250.140
54.196.250.140 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDxD6gi+FNijA1n40q0xZy5t8YyPL/CeqX2HTzmLhKnR

That base64 blob is the server’s host public key. The fingerprint is just SHA256(that_blob) for human readability.

Experiment 4: Terminate/Launch (new instance, same user keypair)

➜  Downloads echo "now lets try to terminate the instance and launch new instance with same geek-ec2-kp keypair"
now lets try to terminate the instance and launch new instance with same geek-ec2-kp keypair
➜  Downloads ssh -i geek-ec2-kp.pem ec2-user@54.242.80.236
The authenticity of host '54.242.80.236 (54.242.80.236)' can't be established.
ED25519 key fingerprint is SHA256:jrRK6H1dssHviNzrP2zm7ucoZOzG2zPjVprL7Bz26Pw.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

Different fingerprint. New instance, new host keys generated on first boot. I’m still using the same user keypair (geek-ec2-kp.pem), but the server’s identity changed.

Experiment 5: The MITM trap (same IP, different instance)

This is the scenario SSH is built to catch.

I am trying to reproduce MITM warning. so I am using Elastic IP to do that.

➜  Downloads echo "i am trying to reproduce MITM warning. so I am using Elastic IP to do that."
i am trying to reproduce MITM warning. so I am using Elastic IP to do that.
➜  Downloads ssh -i geek-ec2-kp.pem ec2-user@98.90.74.60
The authenticity of host '98.90.74.60 (98.90.74.60)' can't be established.
ED25519 key fingerprint is SHA256:p8l/VDBRYXPbiHLijdOjdSf8aiRWAblFJH29CWaTDLM.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

Trusted it. Then I terminated that instance and launched a new one with the same Elastic IP.

➜  Downloads echo "now I will terminate this instance and use the same Elastic IP to attach it to another instance"
now I will terminate this instance and use the same Elastic IP to attach it to another instance
➜  Downloads ssh -i geek-ec2-kp.pem ec2-user@98.90.74.60
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:ZeRBDXzEqoTaF4lI2cC1ihBoUST27cXg/AdjQ2cqrUA.
Please contact your system administrator.
Add correct host key in /Users/govind/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /Users/govind/.ssh/known_hosts:48
Host key for 98.90.74.60 has changed and you have requested strict checking.
Host key verification failed.

Same IP. Different fingerprint. SSH refused to connect.

An attacker could steal my Elastic IP, launch their own server, associate the IP. But they can’t generate the same host keypair. Different server, different public key. Always.

To fix it, I removed the old entry:

➜  Downloads ssh-keygen -R 98.90.74.60
# Host 98.90.74.60 found: line 47
# Host 98.90.74.60 found: line 48
/Users/govind/.ssh/known_hosts updated.
Original contents retained as /Users/govind/.ssh/known_hosts.old

Then connected again:

➜  Downloads ssh -i geek-ec2-kp.pem ec2-user@98.90.74.60
The authenticity of host '98.90.74.60 (98.90.74.60)' can't be established.
ED25519 key fingerprint is SHA256:ZeRBDXzEqoTaF4lI2cC1ihBoUST27cXg/AdjQ2cqrUA.
Are you sure you want to connect (yes/no/[fingerprint])? yes

Fresh trust established with the new instance.

5. TOFU: Trust On First Use

On that first connection, SSH asked me to verify the fingerprint. It didn’t automatically trust the server. There’s no certificate authority. No pre-installed list of trusted servers. No DigiCert for SSH.

I am the trust anchor.

If I verify the fingerprint out-of-band — check the AWS console system log, call the admin, whatever — I can type yes safely. The fingerprint in the console matches what SSH showed me, so I know I’m talking to the real server.

AWS provides the fingerprint in the system log:

EC2 Console → Instance → Actions → Monitor and troubleshoot → Get system log

-----BEGIN SSH HOST KEY FINGERPRINTS-----
256 SHA256:ZeRBDXzEqoTaF4lI2cC1ihBoUST27cXg/AdjQ2cqrUA root@ip-172-31-22-221 (ED25519)
-----END SSH HOST KEY FINGERPRINTS-----

Match this with what SSH shows you. If they match, type yes. You’ve verified it out-of-band. No MITM possible.

If I blindly type yes without checking, I’m vulnerable to a man-in-the-middle on that first connection. An attacker sitting between me and the real server could show me their host key instead. I’d save it to known_hosts, and from that point on, all my SSH traffic would go through the attacker.

But after the first connection, SSH enforces consistency. The host key is locked in. Any change triggers the scary warning.

This is the opposite of TLS.

TLS uses a hierarchy of certificate authorities. My browser ships with ~100 pre-trusted root CAs. DigiCert, Let’s Encrypt, Sectigo. Any server with a valid certificate from any of these CAs is automatically trusted. I never see a fingerprint prompt when I visit a new website.

SSH has no global trust infrastructure. You can run your own SSH CA if you want — generate a CA keypair, sign your servers’ host keys, configure clients to trust your CA’s public key. But it’s opt-in. And it’s rare. Most people use TOFU and manually verify fingerprints for critical servers.

The trust models reflect the use cases:

• TLS: millions of websites, unknown in advance, need automatic trust

• SSH: dozens of servers, infrastructure you control, can verify manually

6. Why AWS Doesn’t Use SSH Certificates

When you launch an EC2 instance, AWS generates the host keypair during first boot. AWS doesn’t sign it with a CA. The instance just presents a raw public key.

AWS could run an SSH CA. They could sign every EC2 instance’s host key with an AWS root CA, distribute that CA’s public key to every AWS customer, and eliminate TOFU entirely.

They don’t.

Why? Because the trust boundary is different. AWS doesn’t want to be the global SSH authority. You launched the instance. You control it. You verify it.

In practice, most people skip the fingerprint verification and rely on context. “I just launched this instance 30 seconds ago, the timing suggests it’s real, I’ll trust it.” That’s TOFU. It works most of the time. It’s not bulletproof.

7. Closing the Loop

Three models, three answers to the same question: how do you trust someone you’ve never met?

TLS (Part 1): Trust a hierarchy. DigiCert vouches for the server, my browser trusts DigiCert. Scales to millions of websites.

AWS IAM (Part 2): Trust temporary credentials. Roles define boundaries, STS issues tokens, they expire every hour. Scales to ephemeral infrastructure.

SSH (Part 3): Trust what you verify yourself. On first connection, you check the fingerprint. After that, SSH watches for swaps. Scales to dozens of known servers.

None of these is “better.” They’re optimized for different threat models.

The fingerprint prompt I used to ignore? It’s SSH asking: “Have you verified this server’s identity out-of-band, or are you trusting on first use?” That’s the entire security model in one question.

I know the answer now.

The next part will probably be JWT and OAuth — how applications represent identity and delegate access outside of a single platform. Or it might be something else entirely, depending on what breaks next.

In Part 1, I traced how HTTPS works from the ground up. Asymmetric cryptography, certificate chains, the TLS handshake. By the end, a browser and a server could talk securely over an untrusted network.

But that only solves one part of the problem.

TLS answers: “Can I trust this server?” AWS IAM answers: “Can I trust this caller, and what are they allowed to do?”

These are different questions. And inside AWS, the second one comes up constantly — every time a Lambda reads from S3, every time a service talks to another service, every time a developer runs a CLI command.

This article is about that second question. Not the theory. The actual mechanics.

(JWT and OAuth are coming in a separate part. This article focuses on how AWS itself solves auth at the infrastructure level.)

1. The Problem AWS IAM Solves

When I first started building on AWS, the instinct was to use access keys. Generate a key, drop it into an environment variable, done. Lambda reads from S3. EC2 writes to DynamoDB. Glue pulls from RDS. Each service authenticates with a static credential.

That falls apart quickly.

• Keys leak through logs, environment dumps, or accident

• Rotating them across dozens of services is painful

• There’s no natural expiry

• A leaked key stays valid until someone manually revokes it

AWS IAM solves this with a different model entirely.

No passwords. No static keys in services. Every identity is temporary, scoped, and auto-rotating.

The mechanism that makes this possible is STS — the Security Token Service.

2. Who Are You in AWS?

Before getting into STS, there’s a foundational concept worth getting right.

In AWS, every entity that makes a request is called a Principal — the thing that is authenticated and making the request.

Principal Type      Example
---------------------------------------------------------------------------
IAM User            arn:aws:iam::123456789:user/vishal
IAM Role            arn:aws:iam::123456789:role/my-lambda-role
AWS Service         lambda.amazonaws.com
AWS Account         arn:aws:iam::123456789:root
Federated identity  via SAML / OIDC / Cognito
Everyone            *

I used to think of IAM Users as the primary identity in AWS. In practice, IAM Roles are the more important primitive. Roles aren’t tied to a person. They’re tied to a purpose. Any entity that AWS trusts can assume a role and act with its permissions.

That word — assume — is load-bearing. Everything else flows from it.

3. The Two Policies Every Role Has

An IAM Role is defined by two policies. Both matter.

Trust Policy — who is allowed to assume this role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

This says: only the Lambda service is allowed to assume this role.

Permission Policy — what this role is allowed to do:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}

This says: whoever assumes this role can read and write objects in my-bucket.

Two separate concerns. Who can become this identity. What this identity can do.

Trust policy = the door. Permission policy = what’s behind it.

4. How a Request Actually Gets to AWS

Before following a request end-to-end, there’s one more piece to understand: how do humans and applications even reach AWS?

• AWS Console — password + MFA. Log in, get a session.

• AWS CLI — access key + secret, configured once. The CLI signs every request on behalf of the caller.

• AWS SDK — same access key + secret, OR an IAM Role if running on AWS compute. The SDK resolves credentials automatically via a credential provider chain — it checks env vars, then credentials file, then the instance metadata service, and so on. The same code works locally and on EC2 without modification.

• Direct API — everything above is ultimately a wrapper. Under the hood, every AWS API call is an HTTPS request signed with Signature Version 4 (SigV4).

SigV4 is the equivalent of TLS’s handshake signature — it proves the caller holds a specific key, and it binds the request to a specific time, region, and service so it can’t be replayed or redirected.

I’ll show exactly what SigV4 looks like later in this article.

5. Following a Request End-to-End

The clearest way to understand this is to follow one request through the entire system. A Lambda function reading a file from S3.

Step 1: A role is attached to the Lambda

The Lambda is created with an IAM Role attached to it. The trust policy on that role says lambda.amazonaws.com is allowed to assume it. The permission policy says the role can call s3:GetObject on my-bucket.

aws lambda create-function \
  --function-name my-function \
  --role arn:aws:iam::123456789012:role/my-lambda-role \
  ...

Nothing has happened yet. The role is just a definition.

Step 2: Lambda is invoked, AWS runtime calls STS

The function is invoked. Before the handler code even starts, the AWS Lambda runtime calls STS automatically:

POST https://sts.amazonaws.com/

Action=AssumeRole
&RoleArn=arn:aws:iam::123456789012:role/my-lambda-role
&RoleSessionName=my-function-execution-abc123
&DurationSeconds=3600

This happens invisibly. No code written, no configuration needed. The platform handles it before handing control to the handler.

Step 3: STS returns temporary credentials

STS checks the trust policy — does it allow lambda.amazonaws.com to assume this role? Yes. It returns:

{
  "Credentials": {
    "AccessKeyId":     "ASIAXXXXXXXXXXX12345",
    "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "SessionToken":    "FwoGZXIvYXdzEJr...long-token...Tuw==",
    "Expiration":      "2026-04-08T11:00:00Z"
  }
}

The AccessKeyId starts with ASIA. That prefix always means temporary STS credentials. Permanent IAM user keys start with AKIA. One letter carries the whole story.

These credentials are injected into the Lambda execution environment before the handler starts.

Step 4: SDK picks up credentials — without any explicit configuration

The handler code is simply:

cfg, err := config.LoadDefaultConfig(context.TODO())
s3Client := s3.NewFromConfig(cfg)

No credentials passed explicitly. The SDK walks the credential provider chain:

1. Hard-coded in code?               → no
2. AWS_ACCESS_KEY_ID env var?        → no
3. ~/.aws/credentials file?          → no
4. ECS container endpoint?           → no
5. Lambda runtime env vars?          → YES ✓
   AWS_ACCESS_KEY_ID     = ASIAXXXXXXXXXXX12345
   AWS_SECRET_ACCESS_KEY = wJalrXUtnFEMI/...
   AWS_SESSION_TOKEN     = FwoGZXIvYXdzEJr...

The STS call already happened before your code started. The SDK’s job here is purely credential discovery, not credential generation. It finds the credentials sitting in env vars and picks them up.

This is why the same code works locally — locally it picks up from ~/.aws/credentials. On Lambda it picks up from env vars. Not a single line of code changes.

Step 5: SDK builds the canonical request

Your code calls s3Client.GetObject(...). The SDK translates this into an HTTP request, but before sending it, it has to sign it. To sign it, it first needs to normalize it.

The request could be expressed in many equivalent ways — different header casing, different query string order, trailing slashes. HMAC-SHA256 is sensitive to every character. If the signer and verifier normalize differently, the signatures won’t match.

The solution is a canonical request — one strict, agreed-upon representation of the request:

GET
/my-file.txt

host:my-bucket.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afb...
x-amz-date:20260408T100000Z
x-amz-security-token:FwoGZXIvYXdzEJr...

host;x-amz-content-sha256;x-amz-date;x-amz-security-token

e3b0c44298fc1c149afb...

The word canonical means exactly that — one authoritative form. Both the SDK (signing) and AWS (verifying) apply the same normalization rules and arrive at the identical string. Then they both sign it. The signatures match, or the request is rejected.

Normalization rules:

• HTTP method → always uppercase

• URI path → always normalized

• Query string → sorted alphabetically

• Header names → always lowercase, sorted alphabetically

• Header values → whitespace trimmed

• Body → always SHA256 hashed, never included raw

Step 6: SigV4 signing

# Hash the canonical request
SHA256(canonical_request) = "3b4c5d6e..."

# Build string to sign
AWS4-HMAC-SHA256
20260408T100000Z
20260408/us-east-1/s3/aws4_request
3b4c5d6e...

# Derive signing key — scoped per date, region, service
kDate    = HMAC-SHA256("AWS4" + SecretAccessKey, "20260408")
kRegion  = HMAC-SHA256(kDate,    "us-east-1")
kService = HMAC-SHA256(kRegion,  "s3")
kSigning = HMAC-SHA256(kService, "aws4_request")

# Final signature
Signature = HMAC-SHA256(kSigning, string_to_sign)

The signing key is derived fresh for every date, region, and service combination. A key valid for S3 today cannot sign a DynamoDB request. A key for us-east-1 cannot be used in eu-west-1. The scope is baked into the key itself.

The signature goes into the Authorization header:

Authorization: AWS4-HMAC-SHA256
  Credential=ASIAXXXXXXXXXXX12345/20260408/us-east-1/s3/aws4_request,
  SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token,
  Signature=fe5f80f77d5fa3beca...

Wait — where are the private and public keys?

Coming from TLS, this was the question I had too. SigV4 doesn’t use asymmetric cryptography at all. There are no private or public keys here.

TLS uses asymmetric signing because the verifier (the browser) doesn’t know the signer (the server) in advance. The public key travels via a certificate, and anyone with the public key can verify the signature without knowing the private key.

SigV4 uses HMAC — a symmetric scheme. Both sides use the same secret:

TLS   → Sign(private_key, data)   | Verify(public_key, signature, data)
SigV4 → HMAC(secret_key, data)    | HMAC(secret_key, data) → compare

The shared secret is the SecretAccessKey from the STS credentials. The SDK has it because STS returned it. AWS has it because STS generated it in the first place. When S3 receives the request, it asks STS for the SecretAccessKey associated with this AccessKeyId and SessionToken, then recomputes the signature independently.

HMAC works here because the trust model is different — AWS issued the secret itself, so both sides already share it. No certificate infrastructure needed. And since the secret is short-lived (expires in an hour), the risk profile is much lower than a long-lived private key.

Step 7: S3 validates everything

S3 receives the request and runs through a checklist:

1. Extracts AccessKeyId from the Authorization header. ASIA prefix means it also expects a SessionToken.

2. Calls STS internally to validate the SessionToken — is it valid, non-expired, which role does it belong to?

3. Independently recomputes the canonical request and signature using the same rules. Compares with what arrived in the header.

4. Checks IAM — does my-lambda-role have s3:GetObject on this specific bucket and key?

5. Any explicit Deny anywhere? Any bucket policy that blocks this?

6. All checks pass. Response returned.

The request signature is verified by recomputing it independently. AWS never trusts the signature — it re-derives it. This is the same principle as TLS certificate verification — trust is established by recomputing, not by accepting.

Step 8: Credentials expire, cycle repeats

At Expiration, the credentials become useless — even if an attacker intercepted them. The Lambda runtime calls STS again automatically before expiry. Fresh credentials are injected. The handler code sees none of this cycle.

6. Who Makes the STS Call?

The Lambda example showed the platform calling STS before your code runs. But that’s not universal.

Service                              Who calls STS
------------------------------------------------------------------------------
Lambda, ECS, Fargate, App Runner     AWS platform, before code starts.
                                     Credentials pre-injected into env vars
                                     or container credentials endpoint.

EC2                                  IMDS at 169.254.169.254 calls STS and
                                     caches the result. The SDK calls IMDS,
                                     never STS directly.

EKS (via IRSA)                       A different mechanism — worth its own
                                     article.

Cross-account, Federation, CLI       Explicit AssumeRole call in application
                                     code.

The mechanism is always the same. STS, temporary credentials, signed requests. What changes is who initiates the STS call.

EC2 and IMDS — a closer look

Lambda is short-lived, so pre-injecting credentials before the handler starts works fine. EC2 is different — an instance can run for months. Pre-injected credentials would expire with no runtime to refresh them.

AWS solves this with the Instance Metadata Service (IMDS) — a special HTTP server running on every EC2 instance at a fixed link-local IP:

http://169.254.169.254

This IP is not routable. Nothing outside the instance can reach it. Only processes running on that specific EC2 instance can hit it.

IMDS exposes the instance’s IAM credentials at a well-known path:

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/my-ec2-role

{
  "AccessKeyId":     "ASIAXXXXXXXXXXX12345",
  "SecretAccessKey": "wJalrXUtnFEMI/...",
  "Token":           "FwoGZXIvYXdzEJr...",
  "Expiration":      "2026-04-08T12:00:00Z"
}

IMDS called STS internally when the role was attached, cached the result, and serves it via this endpoint. It also handles rotation — before Expiration, it calls STS again and updates the cache. The SDK just reads from IMDS. It never touches STS directly.

SDK credential provider chain
        ↓
GET http://169.254.169.254/latest/meta-data/iam/security-credentials/<role>
        ↓
IMDS returns (AccessKeyId, SecretAccessKey, Token)
        ↓
SDK uses these to sign the request

IMDSv2 — closing an SSRF gap

SSRF (Server-Side Request Forgery) is an attack where the attacker tricks the server itself into making a request to an internal resource the attacker can’t reach directly. A classic example — a web app has a “fetch this URL for me” feature. Intended for external URLs. But if an attacker passes http://169.254.169.254/latest/meta-data/iam/security-credentials/my-ec2-role, the server dutifully fetches it and hands back valid AWS credentials. The attacker never touched IMDS directly — they couldn’t, it’s only reachable from inside the instance. But the server could, and the server was fooled into doing it.

The original IMDS had exactly this problem. Any code running on the instance — including code exploiting an SSRF vulnerability in the application — could hit 169.254.169.254 and walk away with valid credentials.

IMDSv2 added a session-token requirement. Before fetching credentials, the caller must first do a PUT request to get a short-lived session token, then use that token in the GET:

# Step 1: get a session token
TOKEN=$(curl -X PUT \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600" \
  http://169.254.169.254/latest/api/token)

# Step 2: use the token to fetch credentials
curl -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/iam/security-credentials/my-ec2-role

SSRF exploits typically follow HTTP redirects but can’t make a PUT with a custom header. That one constraint closes the attack vector. AWS SDKs handle IMDSv2 automatically — this is just what’s happening underneath.

This connects back to the same thesis from section 8 — the architecture is designed so there’s nothing static to steal. Even the path to credentials has a time-limited gate in front of it.

7. Cross-Account Access — The Same Story, One Level Up

At work, I ran into this exact situation. A Step Function in one AWS account needed to trigger a Lambda in a completely different account. At first it felt like a special case — some obscure AWS feature I hadn’t seen before. Turns out it’s the same STS story, just applied across account boundaries.

Everything so far has been single-account. But real systems span multiple accounts. Production in one account, data in another, logging in a third.

By default, AWS accounts are completely isolated. A Lambda in Account A has zero access to anything in Account B.

The solution is the same STS story — just applied across account boundaries.

Account A (123456789012) — Lambda lives here
Account B (999988887777) — S3 bucket lives here

Setup: Two sides must agree

In Account B, a role is created with a trust policy explicitly allowing Account A’s Lambda role to assume it:

{
  "Principal": {
    "AWS": "arn:aws:iam::123456789012:role/my-lambda-role"
  },
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringEquals": {
      "sts:ExternalId": "shared-secret-xyz"
    }
  }
}

In Account A, my-lambda-role is given permission to call AssumeRole on Account B’s role:

{
  "Action": "sts:AssumeRole",
  "Resource": "arn:aws:iam::999988887777:role/cross-account-role"
}

Both sides must agree. Account B’s role trusts Account A. Account A’s role is allowed to call AssumeRole on Account B’s role. Missing either side means access denied.

The flow

The Lambda code explicitly calls STS:

result, err := stsClient.AssumeRole(context.TODO(), &sts.AssumeRoleInput{
    RoleArn:         aws.String("arn:aws:iam::999988887777:role/cross-account-role"),
    RoleSessionName: aws.String("lambda-cross-account-session"),
    ExternalId:      aws.String("shared-secret-xyz"),
})

STS checks both sides, and if both agree, returns a second set of temporary credentials — scoped entirely to Account B.

The Lambda ends up holding two layers of credentials simultaneously:

Layer 1 — Account A credentials (from runtime, automatic)
  → used for anything in Account A
  → used to call STS AssumeRole for Account B

Layer 2 — Account B credentials (from explicit AssumeRole call)
  → used only for resources in Account B
  → separate session, separate expiry, separate rotation

The mechanism is identical. The STS story just runs twice.

Why ExternalId matters even more here

In single-account, a compromised role damages only that account. In cross-account, if a third-party vendor’s AWS account is compromised, an attacker could try to assume the role from their account.

ExternalId is the defence. Even with full access to the vendor’s account, an attacker doesn’t know the ExternalId. The AssumeRole call fails. This is the confused deputy problem — a service being tricked into acting on behalf of someone it shouldn’t trust.

8. The Thesis: No Passwords Anywhere

Step back and look at the pattern.

A Lambda reads from S3. No password was configured. No key was hardcoded. No secret was shared between the two services. Yet the request was authenticated, authorized, and completed.

How?

• A role defined who can act and what they can do

• STS issued short-lived credentials when needed

• The credentials signed every request cryptographically

• The signature was verified by recomputing it independently

• When credentials expired, STS issued fresh ones automatically

This is the same insight as TLS — except TLS applied it to network connections, and AWS IAM applies it to service-to-service calls inside a cloud platform.

In TLS: asymmetric cryptography establishes trust, then symmetric keys handle communication. Neither side stores a shared password.

In AWS IAM: roles define trust, STS issues tokens, SigV4 signs every request. No service stores another service’s password.

The architecture is designed so that there is nothing static to steal.

Leaked credentials expire. Intercepted requests can’t be replayed — the timestamp and scope are signed. A key valid for one service can’t be used against another.

The security property isn’t that secrets are well-hidden. It’s that secrets are designed to be short-lived and narrowly scoped from the start.

9. What’s Still Open

This article followed a single request through IAM and STS. But there’s more to the full picture.

Policy evaluation logic — when a request arrives at AWS, how exactly is the allow/deny decision made? Identity-based policies, resource-based policies, and Service Control Policies all interact. The logic is non-trivial and worth its own article. The mental model is similar to what OPA/Rego does — a policy engine evaluating structured rules against a request context — but AWS’s version is closed and AWS-specific.

Federation — real humans in large orgs don’t get IAM users. They authenticate via SAML or OIDC against a corporate identity provider, and AWS maps that identity to a role via AssumeRoleWithSAML or AssumeRoleWithWebIdentity. The STS story is the same, the entry point is different.

JWT and OAuth — how applications represent and delegate identity outside of AWS. Coming in the next part.

The chain of trust keeps extending. TLS secured the channel. IAM secured the caller. What comes next is how identity is represented and carried across the boundaries of individual systems.

If something landed differently than expected, or if there’s a gap I didn’t cover, I’m curious to hear it.

Why Does AWS Show GiB Instead of GB?

I was browsing EC2 instance types on AWS when I noticed something odd.

Memory is listed in GiB. Storage is listed in GB. Network is in Gigabit.

Three different units. Same console. All measuring data.

My first instinct was that AWS was being inconsistent. But the more I looked into it, the more I realised each one is deliberate, and understanding why took me through CPU architecture, a 1980s marketing decision, and an OS labeling bug the industry decided was too painful to fix.

Two Ways to Count to a Billion

Everything starts here.

There are two competing definitions of “Gigabyte”, and they’ve been quietly causing confusion for decades.

Base 10: the human way

Powers of 10, like we learned in school.

1 KB  = 1,000 bytes
1 MB  = 1,000,000 bytes
1 GB  = 1,000,000,000 bytes
1 TB  = 1,000,000,000,000 bytes

Base 2: the computer way

Powers of 2, because computers think in binary.

1 KiB = 1,024 bytes
1 MiB = 1,048,576 bytes
1 GiB = 1,073,741,824 bytes
1 TiB = 1,099,511,627,776 bytes

1 GiB is about 7.4% larger than 1 GB. Small at small scales, but it compounds fast.

And yes, that lowercase “i” sandwiched between G and B in GiB is genuinely easy to miss. Sit a foot further from the screen and it reads exactly like GB. I missed that ‘i’ for longer than I’d like to admit.

The “bi” in KiB, MiB, GiB stands for binary. The IEC standardised these units in 1998 specifically to end the ambiguity. Before that, “KB” was used loosely for both 1,000 and 1,024 bytes depending on who was writing.

So why does AWS use GiB for memory but GB for storage?

It’s not inconsistency. It turns out each unit is accurately describing what’s actually there.

RAM Is Binary By Nature

RAM isn’t just stored data, it’s addressed data.

Every byte in RAM has a unique memory address. The CPU locates it using binary addressing. A 32-bit address bus gives exactly 2³² addressable locations, 4,294,967,296 bytes, or exactly 4 GiB. Not 4 GB. Not 4,000,000,000 bytes. The math doesn’t leave room for clean decimal boundaries.

This flows all the way down to how memory chips are manufactured. RAM comes in 256 MiB, 512 MiB, 1 GiB, 2 GiB, 4 GiB slots. A “1,000,000,000 byte” RAM stick doesn’t exist, it would break the addressing model.

So when AWS lists an EC2 instance as “8 GiB RAM,” it’s being precise. That’s 8 × 2³⁰ bytes, because that’s what RAM physically is.

Engineers always knew this. When they said “1 GB of RAM” years ago, they meant 2³⁰ bytes, they were just using imprecise language before the right words existed. GiB makes the implicit explicit.

Storage Is Decimal By Choice

Hard drives don’t have the same constraint.

A disk is a linear sequence of sectors. There’s no binary addressing requirement forcing capacity to be a power of 2, a drive can hold any number of bytes.

So when manufacturers were scaling up in the 1980s, they had a choice: count in base 2 or base 10?

They picked base 10. And the reason wasn’t engineering.

It was marketing.

Base 10 numbers are larger. A drive marketed as “10 MB” sounds better than the same drive marketed as “9.5 MiB.” As capacities scaled into gigabytes and terabytes, that gap grew from a minor rounding difference into a meaningfully larger-sounding number on the box.

The entire storage industry followed suit. It’s been base 10 ever since. When a manufacturer says “1 TB SSD,” they mean exactly 1,000,000,000,000 bytes, no tricks. That’s just their definition of TB.

AWS follows the same convention for EBS and S3.

So Where Does the Missing Storage Go?

This is where it clicked for me.

The OS receives a drive with exactly 1,000,000,000,000 bytes. Internally, it counts in base 2, consistent with how memory addressing works across the whole system. So it calculates:

1,000,000,000,000 ÷ 2⁴⁰ = 0.9094 TiB
                          ≈ 931 GiB

The number (931) is correct.

The label (”GB”) is wrong, it should say “GiB.”

But that label has been wrong for so long that fixing it would cause a different kind of chaos. Imagine waking up one morning and a “500 GB” drive suddenly reads “465 GiB.” The number dropped by 35. Most people would, reasonably but incorrectly, conclude the OS update deleted their data. Support forums melt down. Headlines scream “New update shrinks your hard drive.”

So the industry made a quiet collective decision: leave the wrong label in place.

The confusion of an incorrect unit is less damaging than the panic of visibly shrinking numbers.

The One OS That Actually Fixed It

In 2009, Apple shipped macOS Snow Leopard with a quiet but significant change, Finder switched to true base 10 reporting.

A 1 TB drive now showed as approximately 1 TB in Finder. The label and the math finally agreed.

The result? Mac users reported their drives had suddenly gained storage after the update.

Same drives. Same bytes. Just a different counting system.

Linux has been cleaning this up gradually. Modern tools like df -h and lsblk now correctly display GiB when calculating in base 2. It varies by distro, but the direction is toward precision.

Windows still shows base 2 numbers with “GB” labels.

And the “Gigabit” for Network?

Network speeds add one more wrinkle, they’re measured in bits, not bytes.

When AWS says “Up to 5 Gigabit” network performance, that means 5,000,000,000 bits per second. To get usable throughput in bytes, divide by 8:

5 Gbps ÷ 8 = 625 MB/s

Networking protocols historically transmitted data serially, one bit at a time over a wire. Measuring in bits was natural for the hardware, and it stuck. The same logic applies to ISP speeds, “100 Mbps broadband” gives about 12.5 MB/s of actual file transfer speed.

The “up to” also matters. It means burstable, the ceiling under ideal conditions, not a guaranteed baseline.

Putting It Together

Once I understood this, the AWS console stopped looking inconsistent and started looking accurate:

What I see in AWS Unit Why EC2 memory: “8 GiB” GiB (base 2) RAM is binary by architecture EBS volume: “100 GB” GB (base 10) Storage follows manufacturer convention Network: “5 Gigabit” Gbps (base 10, bits) Networking measures in bits historically

And the mystery of a 1TB drive showing 931 GB? The OS counted correctly, in GiB, and then slapped the wrong label on it.

The storage was never missing.

The units were just speaking different languages.

The Engineer’s Notebook. Things I learn, written down so I don’t forget them. If it helped, there’s more coming.

This article builds that system from first principles starting with basic cryptography and ending with modern TLS handshake.

1. The Building Blocks

1.1 Symmetric Encryption - Fast but problematic

Symmetric encryption uses the same key for encryption and decryption.

• Example algorithm: AES

• Strength: extremely fast and is used for large data

• Problem: the same key needs to be there with receiver. How to send the key securely?

  • If we already had a secure way to share the key, why not just use that to send the data?

1.2 Asymmetric Encryption - Solving Trust, not speed

Asymmetric encryption introduces key pairs:

• Public key (shared openly)

• Private key (kept secret)

Two different keys, different uses

• Encrypt using public key → Decrypt using private key

  • Purpose: Confidentiality

• Sign using private key → Verify using public key (Digital Signatures)

  • Purpose: Authenticity + Integrity

Asymmetric encryption is not used for bulk data since it’s too slow. It’s used to establish trust and identity

1.3 Hashing ensures integrity

A hash function (like SHA-256)

• Takes in input of any size

• produces fixed length output

• is deterministic

• is irreversible

Why Hashing?

To verify data integrity

If HASH(data1) == HASH(data2), then data1 == data2

2. Digital Signatures

Digital signature combines Hashing and asymmetric encryption

Let’s define:

• HASH(k) → hash of data k

• Sign(pvt, k) → sign using private key

• Verify(pub, sig, k) → verify using public key

Signing

h = HASH(k) , sig = Sign(pvt, h)

Verification

h1 = HASH(k), h2 = Verify(pub, sig)

if: h1 == h2 → trusted source

What does this give us?

• Integrity → data wasn’t changed while it reached destination

• Authenticity → signed by owner of private key

3. The Real Problem: Trust

At this point, we can verify signatures, but the bigger problem is:

How do we trust a public key?

Anyone can generate a (pub, pvt) key pair

So how do we know if the key actually belongs to example.com?

4. Public Key Infrastructure (PKI)

PKI solves the trust problem using hierarchy:

• Root Certificate Authority (RCA)

• Intermediate Certificate Authority (ICA)

• Web Servers (Websites)

Key Concepts

• Certificate → binds identity + public key

• CA → entity that signs certificates

  • ICA → Intermediate Certificate Authority

  • RCA → Root Certificate Authority

• Chain of Trust → Layered verification

How are certificates Issued

The diagram above shows how websites prepare for building trust among browsers (or User Agents) to be able to securely connect to them. To understand the flow, read the diagram from right to left.

Lets dig in deeper:

• pub → public key

• pvt → private key

• HASH(k) → hash function

• Sign(pvt, k) → signature

Step 1 : ICA gets signed certificate from Root CA

ICA generates:

ICA.pub, ICA.pvt

ICA sends a CSR (Certificate Signing Request) to Root CA.

Root CA:

h = HASH(ICA.pub)
sig = Sign(RCA.pvt, h)

Returns certificate:

(ICA.pub, sig)

Step 2: Server gets signed by ICA

Web Server (WS) generates:

WS.pub, WS.pvt

Sends CSR to ICA.

ICA:

h = HASH(WS.pub)
sig = Sign(ICA.pvt, h)

Returns:

WS Certificate:
  WS.pub
  Sign(ICA.pvt, HASH(WS.pub))

+ ICA Certificate:
  ICA.pub
  Sign(RCA.pvt, HASH(ICA.pub))

Result: Chain of Trust

Root CA → ICA → Web Server

6. How the Browser Verifies Trust

The browser (or OS) already contains trusted Root CA public keys.

Verification Flow

Step 1: Verify ICA

Verify(RCA.pub, ICA.signature)

If valid → browser now trusts ICA

Step 2: Verify Server

Verify(ICA.pub, WS.signature)

If valid → browser now trusts the web server (Bob)

7. TLS Handshake — Establishing a Secure Session

Now that trust is established, we need a shared symmetric key.

Old Approach (RSA Key Exchange)

• Client generates symmetric key

• Encrypts with server public key and sends to server

• Server decrypts using private key

Problem

If attacker gets server private key later:
→ past sessions can be decrypted

Modern Approach: Diffie-Hellman (ECDHE)

Instead of sending the key, both sides derive it independently.

Intuition

• Both sides agree on g and p

• Client picks secret a

• Server picks secret b

Exchange:

Client → g^a mod p
Server → g^b mod p

Then:

Client computes: (g^b)^a mod p
Server computes: (g^a)^b mod p

Both get:

Key Insight

The shared secret is never transmitted.

Why this is powerful

• Provides Forward Secrecy

• Even if private key is compromised later:
  → past sessions remain secure

8. Final TLS Flow (Putting Everything Together)

1. Browser connects to server

2. Server sends certificate chain

3. Browser verifies:

   • Root → ICA

   • ICA → Server

4. Trust is established

5. Client & server perform ECDHE key exchange

6. Shared symmetric key is derived

7. All further communication uses symmetric encryption (AES)

9. Why This Design Works

This system combines:

• Asymmetric crypto → identity & trust

• Hashing → integrity

• Signatures → authenticity

• PKI → global trust system

• Diffie-Hellman → secure key exchange

• Symmetric crypto → performance

10. Final Mental Model

Think of HTTPS as:

• Certificates → prove identity

• Signatures → prove authenticity

• PKI → establish trust

• TLS handshake → agree on secret

• AES → secure communication

Closing Thoughts

What looks like a simple 🔒 in our browser is actually:

• a distributed trust system

• built on cryptography

• standardized through PKI

• and optimized over decades

In this article, we built the foundation:

• Asymmetric cryptography → identity and signatures

• Hashing → integrity

• PKI → global trust

• TLS → secure communication channel

But this only solves one part of the problem.

TLS ensures we are talking securely to a server.
It does not answer who the user is or what they are allowed to do.

What Comes Next

Once a secure channel is established, modern systems need to answer two critical questions:

1. Who is the user? (Authentication)

2. What is the user allowed to do? (Authorization)

In the upcoming articles, we’ll build on this foundation and explore how real-world systems solve these problems:

• JWT (JSON Web Tokens)
  → How identity is represented and verified across services

• JWKS (JSON Web Key Sets)
  → How public keys are distributed and rotated at scale

• OAuth
  → How applications securely delegate access

• Authorization Models

  • Role-Based Access Control (RBAC)

  • Attribute-Based Access Control (ABAC)

  • Policy-Based Access Control

• Real-World Systems like AWS IAM
  → How these concepts come together in production systems

The Bigger Picture

If TLS answers:

“Can we trust this server and communicate securely?”

Then the rest of the system answers:

“Can we trust this user, and what are they allowed to do?”

By the end of this series, we won’t just understand individual concepts like JWT or OAuth in isolation—we’ll understand how they fit together into a complete, real-world authentication and authorization system.

Can this problem be solved serially in a simple, obvious way?

Take the counting problem.

Count from 1 to 10,000

A single loop does this perfectly fine.

• Easy to read

• Easy to reason about

• Zero coordination cost

• No overhead

If your program only needs to count 10,000 numbers, introducing concurrency would be like hiring 8 people to count 10 pages — you’d spend more time coordinating than counting.

This way of thinking applies not just to abstract numbers, but to how humans naturally organize work at scale.

When Concurrency Becomes a Question

Now change the problem slightly:

Count from 1 to 1,000,000,000

Suddenly, the serial solution has a cost:

• The CPU is busy for a long time

• Other work might be blocked

• The program becomes unavailable for extended periods

This is the key transition point.

We don’t introduce concurrency because the task is big.
We introduce concurrency because the serial execution makes something unavailable.

I realized that the problem of counting currency notes maps extremely well to how concurrency in Go — and even the Go scheduler — is designed.

If we think about the currency note counting problem, when there are only a few bundles, we don’t need to think much — a single person and a single counting table are more than sufficient.

But if we are supplied with a huge number of currency note bundles — say, suitcases full of them — counting alone no longer makes sense, even if the person is perfectly capable.

So the key takeaway here is this: unless the scale of the problem is large enough, we don’t need to jump into thinking in terms of concurrency.

Keeping that in mind, let’s understand concurrency design by co-relating it with the currency note counting problem.

Counting Hall Story

Deciding on the Cast

• The hall has a limited number of counting tables (say 8).

• The hall receives many currency note bundles of different denominations.

• There are many people who know how to count notes — workers.

• Condition: a worker can count only when they acquire both a table and a bundle.

• Sometimes, we deliberately assign more workers than tables.

  • This ensures that tables never sit idle — if someone finishes early or pauses, another worker can immediately take the chair.

• Even if a worker has been waiting near one table, they may notice another table becoming free and rush to occupy it.

• In addition, we introduce two special roles:

  • Producers, who distribute bundles to workers

  • Accumulators, who collect and aggregate results

Deciding on the Implementation

• Conceptually, the implementation consists of two pipes:

  • a job pipe

  • a result pipe

• This separation ensures unidirectional flow and clear ownership of responsibilities.

• Every worker is connected to:

  • the receiving end of the job pipe

    • when a worker acquires a table, they receive a currency bundle from this job pipe

  • the sending end of the result pipe

    • after counting the bundle, the worker prepares a result chit and sends it through the result pipe to the accumulator

• The producer is connected to the sending end of the job pipe.

• The accumulator is connected to the receiving end of the result pipe.

Co-relating with Go

The Corresponding Cast

If we map this to a Go implementation, the cast looks like this:

• The machine has a limited number of CPU cores (say 8).

  • The machine maps to the hall and each CPU maps to table with chair

• We have a set of note bundles of different denominations.

• Each worker go-routine, when in the Running state, is capable of counting notes from a given bundle when assigned CPU time.

  • Each worker go-routine maps to a worker (human knowing counting)

• Condition: a go-routine can count only when it is given both a CPU and a bundle.

• The extra worker go-routines remain in the Runnable / Ready state — not counting yet, but ready to run and waiting near a CPU.

• Conceptually, the behavior of acquiring the first available chair maps closely to Go’s work-stealing scheduler, where idle processors pull runnable go-routines from other queues to keep CPUs busy.

  • Can read more about work-stealing in Go’s scheduler here.

• The producer and accumulator go-routines are special go-routines that do not count notes themselves, but instead facilitate work (bundle) distribution and result aggregation.

The Corresponding Implementation

• The pipes are implemented using Go channels.

Now, if we look at the following Go implementation, the analogy becomes much clearer.

Output

Co-relation

• Each worker go-routine counts a bundle of notes received through the jobs channel (pipe) from the producer go-routine and sends the result through the results channel (pipe).

  This behavior is enforced by the signature of the worker function:

  • jobs parameter has the type <-chan Bundle, a read-only channel, which explicitly restricts the worker to only receiving from the jobs channel and not sending into it.

  • results parameter has the type chan<- CountResult, a send-only channel, which explicitly restricts the worker to only sending results into the results channel and not receiving from it.

  This makes the worker’s role unambiguous: it consumes work from one pipe and produces results into another.

• The main go-routine acts as the accumulator, holding the receiving end of the results channel, into which all worker go-routines send their results.

• Each CPU core acts as a counting table.

• The []Bundle{} slice represents the collection of bundles to be counted.

• CountResult represents the result chit prepared by each worker and sent through the results channel (pipe) to the accumulator.

Conclusion

In the end, I don’t want us to lose sight of the availability over speed perspective on concurrency.

Throughout the counting hall story, the goal was never to make a single person count faster.

The goal was to make sure that counting never stops.

We added more workers not to increase individual speed, but to ensure that:

• when one worker pauses, another can immediately take a table,

• when one table becomes free, some worker is always ready to occupy it,

• and no table ever sits idle while work is waiting.

That is exactly what Go’s concurrency model optimizes for.

Go-routines are cheap to create not so that they all run at once, but so that there is always work ready when CPU time becomes available.

Runnable go-routines exist so that the system can immediately make progress when capacity frees up.

Work stealing exists so that idle CPUs don’t wait while work is stuck elsewhere.

All of this is about keeping the system responsive and productive, even as the amount of work scales.

That brings us back to the core idea:

Concurrency is not about making one person faster.

It’s about keeping the hall productive even when work scales.

And that, more than raw performance, is what good concurrency design is about.

At its core, good concurrency prioritizes:

• availability — the system is always ready to do useful work,

• clarity — each component has a well-defined role,

• structure — work flows in one direction with clear coordination,

• and human-like organization — because that’s how large systems naturally scale.

Then I wrote a tiny TCP server.

What broke wasn’t performance — it was availability.

A single blocking Accept() call meant:

• one client connected

• everyone else waited

Adding a goroutine didn’t make the server faster.
It made the server usable.

That’s when concurrency stopped feeling like a language feature and started feeling like a design decision.

I wrote about this realization while evolving a raw TCP server in Go 👇

I used to think Go concurrency was about goroutines and channels.

You know the advice:

“Just put it in a goroutine.”

But none of that really clicked for me until I wrote a very small, very boring TCP server — and watched it behave in ways I didn’t expect.

This post is about how I slowly evolved a raw TCP server in Go and discovered what concurrency actually means — not as a concept, but as behavior.

Stage 1: The simplest possible TCP server

I started with the smallest thing that could work:

• listen on a TCP port

• accept one connection

• print who connected

Output

What this taught me

Once a client connected, the server stopped making progress.

No new connections. No errors. Just… stuck.

That’s when it hit me:

Accept() is a blocking call.
The server wasn’t idle — it was blocked.

Concurrency hadn’t entered the picture yet, but the problem already existed.

Stage 2: Reading input — still single-client

Next, I tried reading from the connection.

Output

What this taught me

The server now did something useful.

But it still handled only one client.

If a second client tried to connect while the first was active, it had to wait.

The server was alive — but unavailable.

That distinction mattered more than I expected.

Stage 3: Accepting multiple connections… serially

My next instinct was obvious:

for {
    conn, _ := lis.Accept()
    handle(conn)
}

Surely this meant multiple clients, right?

Output

What this taught me

Even with a loop, clients were handled one after another.

The server could accept multiple connections — just not at the same time.

This is when I wrote this line in my notes:

Accepting connections is easy.
Handling them concurrently is the real problem.

That sentence stayed with me.

Stage 4: Goroutine per connection — the first breakthrough

Then came the smallest change that changed everything:

for {
    conn, _ := lis.Accept()
    go handleConn(conn)
}

Output

Messages from different clients started interleaving naturally.

What this taught me

This wasn’t about speed.

It was about availability.

Each connection now had its own execution path.
No client blocked another.

For the first time, goroutines stopped feeling like magic and started feeling necessary.

What this taught me about concurrency

Before writing this TCP server, I thought concurrency in Go was mainly about:

• doing more work in parallel

• improving throughput

• “making things faster”

This exercise completely changed that.

The biggest improvement didn’t come from optimization.
It came from availability.

Before goroutines:

• one blocking Accept()

• one blocking read

• one client could make the entire server unavailable

The server wasn’t broken.
It was just unreachable.

Adding a goroutine per connection didn’t make the server faster.
It made the server usable.

Multiple clients could connect.
One slow or silent client no longer blocked everyone else.

That’s when concurrency stopped feeling like a language feature and started feeling like a design decision.

The mental shift that finally clicked for me

I used to ask:

“How do I make this code concurrent?”

Now I ask:

“What is blocking, and who does it make unavailable?”

Once I started thinking in those terms:

• goroutines felt obvious

• concurrency felt justified

• the design choices made sense

Go didn’t “add concurrency” to my server.
It removed unnecessary unavailability.

That distinction changed how I look at backend systems.

Where this goes next

This server is concurrent — but still incomplete.

It doesn’t coordinate clients.
It doesn’t handle slow consumers.
It doesn’t shut down gracefully.

In the next part, I want to explore what happens after availability:

• how concurrent goroutines coordinate

• why channels change the design

• and how ownership keeps systems sane

But this first realization deserved its own space.

If this post saved you even one hour of confusion, it did its job.