Async Timeouts with CompletableFuture: Fixing Blocking Java Code the Right Way

Let me tell you about the day I inherited a Java application with a 1,000-thread pool where every single thread was blocked on the exact same line of code.

Not a deadlock. Not a race condition. Just future.get(). A thousand threads, each holding a stack frame open, each burning memory, each doing absolutely nothing but waiting for a response that might never come — or might come in 50 milliseconds, there was no way to tell. The application was supposed to be low-latency. It was not low-latency. Under any meaningful load, the thread pool exhausted itself and new requests started queuing up behind it. The system couldn't keep up with the incoming request rate because it was too busy waiting.

This article is the full story of fixing that: why Future.get() is a trap, how CompletableFuture doesn't automatically save you from it, and the specific pattern — a within() utility built on a single-threaded ScheduledExecutorService — that finally made the whole thing non-blocking with proper timeouts and correct error propagation.

The Problem: Why Future.get() Kills You at Scale

Here's the original code, stripped down to its core:

public void serve() throws InterruptedException, ExecutionException, TimeoutException {
    final Future<Response> responseFuture = asyncCode();
    final Response response = responseFuture.get(1, SECONDS);
    send(response);
}

private void send(Response response) {
    // send the response downstream
}

Looks fine, right? You kick off an async operation, wait for it with a 1-second timeout, send the result. The timeout is even there. What's wrong?

The problem is the thread model. Future.get() is a blocking call. When you call it, the current thread suspends and sits there waiting. It is not doing anything useful. It's not processing other requests. It's not running other tasks. It's occupying a thread, burning a stack frame worth of memory (typically 256KB–1MB depending on JVM settings), and holding that thread hostage until the future completes or the timeout expires.

If you understand how the JVM heap and thread stacks work, you already know where this goes. With a pool of 1,000 threads and each one potentially blocking for up to 1 second:

• At ~512KB per thread stack, that's 500MB of stack memory doing nothing but waiting
• Any new incoming request that needs a thread gets queued behind the blocked ones
• If request rate > completion rate, the queue grows without bound
• Under load spikes, latency explodes — not because your code is slow, but because requests can't even start executing

The 1-second timeout in get(1, SECONDS) doesn't help as much as you'd think. Yes, each thread eventually unblocks. But 1,000 threads × 1 second = you can handle at most ~1,000 requests per second in the absolute best case, with every single thread pegged. Any burst beyond that and you're queuing. In practice it's much worse because not all futures complete in exactly 1 second and some threads are doing other work.

This is what thread-per-request models look like under load. This is exactly why virtual threads exist, and why libraries like vtstream are valuable — but more on that in a moment. For now, let's fix this properly.

Step 1: Switch to CompletableFuture — But Don't Make This Mistake

The first instinct is to swap Future<Response> for CompletableFuture<Response>:

public void serve() throws InterruptedException, ExecutionException, TimeoutException {
    final CompletableFuture<Response> responseFuture = asyncCode();
    final Response response = responseFuture.get(1, SECONDS);
    send(response);
}

And here's the brutal truth: this changes nothing. You still have .get(). You still have a blocking call. You've swapped the type on the left-hand side and accomplished nothing meaningful. The threads are still blocked, the pool is still exhausted, the latency is still terrible.

CompletableFuture is not magic. It's not a promise that your code is non-blocking. It's a tool that enables non-blocking code — only if you actually use its callback-based API instead of reaching for .get().

The right move: use thenAccept().

public void serve() {
    final CompletableFuture<Response> responseFuture = asyncCode();
    responseFuture.thenAccept(this::send);
}

Now serve() returns immediately. No blocking. No thread sitting around waiting. When responseFuture completes, send() will be called on whatever thread completed the future — typically the thread from whatever ExecutorService or async operation was backing asyncCode().

There are two ways you end up with a CompletableFuture in the first place:

// Option 1: You control the async task submission
// Use supplyAsync to get a CompletableFuture directly instead of a raw Future
CompletableFuture<Response> future = CompletableFuture.supplyAsync(
    () -> callRemoteService(),
    executorService
);

// Option 2: You have a callback-based API (a "promise" pattern)
// Create an incomplete CompletableFuture and complete it from the callback
CompletableFuture<Response> promise = new CompletableFuture<>();
asyncClient.call(
    request,
    (response) -> promise.complete(response),           // success
    (error)    -> promise.completeExceptionally(error)  // failure
);

The promise pattern is particularly important for integrating with old callback-based libraries. You create an empty CompletableFuture, hand completion responsibility to the callback, and your calling code gets a future it can chain on.

Thread pool for callbacks: By default, thenAccept(this::send) runs send() on whichever thread completed the future. If that's a thread from your HTTP client's connection pool, you probably don't want expensive work running there. Use thenAcceptAsync to dispatch to a separate pool:

responseFuture.thenAcceptAsync(this::send, sendPool);

Step 2: Exception Handling — The Silent Killer

When you use the blocking .get() pattern, exceptions are obvious: get() throws ExecutionException (wrapping the real cause) and you deal with it in the same method, right there.

When you go callback-based, exceptions disappear into the void if you're not careful. Watch:

// This is dangerous. If asyncCode() fails, the exception is silently dropped.
public void serve() {
    final CompletableFuture<Response> responseFuture = asyncCode();
    responseFuture.thenAccept(this::send);
    // serve() returns. Future completes exceptionally. Nobody knows.
}

The correct pattern is to always attach an exceptionally() handler:

public void serve() {
    final CompletableFuture<Response> responseFuture = asyncCode();

    responseFuture
        .thenAccept(this::send)
        .exceptionally(throwable -> {
            log.error("Failed to get response", throwable);
            return null;
        });
}

But there is a subtle trap here that trips up almost everyone the first time. The ordering of exceptionally() and thenAccept() matters in a non-obvious way. Look at this:

// WRONG: This has a bug that isn't obvious
responseFuture
    .exceptionally(throwable -> {
        log.error("Unrecoverable error", throwable);
        return null;    // ← recovery value
    })
    .thenAccept(this::send);  // ← called even on failure!

Here's what happens: exceptionally() is a recovery operator, not just an error handler. When responseFuture fails, exceptionally() catches the exception, logs it, and returns null as a recovered value. The CompletableFuture returned by exceptionally() is now successfully completed with null. Then thenAccept(this::send) sees a successfully completed future and calls send(null).

Your send method just got called with a null response. Depending on what send() does, that's either a NullPointerException or silent corruption. Neither is good.

The fix is to chain exceptionally() after thenAccept(), so it only fires if the original future or the send operation itself failed — not as a recovery that lets thenAccept execute with junk data:

// CORRECT: exceptionally only fires if the future failed
responseFuture
    .thenAccept(this::send)
    .exceptionally(throwable -> {
        log.error("Failed to send response", throwable);
        return null;
    });

If you need to handle exceptions and still perform cleanup regardless, use whenComplete:

responseFuture
    .whenComplete((response, throwable) -> {
        if (throwable != null) {
            log.error("Request failed", throwable);
            metrics.incrementFailureCount();
        } else {
            metrics.incrementSuccessCount();
        }
    })
    .thenAccept(this::send)
    .exceptionally(throwable -> {
        log.error("Send failed", throwable);
        return null;
    });

whenComplete receives both the result and the throwable (one of them will be null). It runs whether the future succeeded or failed, making it perfect for metrics, logging, and cleanup code that needs to run unconditionally.

Step 3: Adding Timeouts Without Blocking — The Real Problem

Here's where it gets interesting. When we switched from future.get(1, SECONDS) to thenAccept(), we silently dropped our timeout. Now if asyncCode() never completes — network partition, slow service, hung connection — our callback never fires. The future just... sits there forever.

We need to get the timeout back without going back to blocking.

The approach: create a synthetic future that always fails after a given duration. Then race it against our real future — whichever completes first wins.

import java.time.Duration;
import java.util.concurrent.*;
import com.google.common.util.concurrent.ThreadFactoryBuilder; // Guava

private static final ScheduledExecutorService scheduler =
    Executors.newScheduledThreadPool(
        1,
        new ThreadFactoryBuilder()
            .setDaemon(true)            // Don't prevent JVM shutdown
            .setNameFormat("timeout-scheduler-%d")
            .build()
    );

public static <T> CompletableFuture<T> failAfter(Duration duration) {
    final CompletableFuture<T> promise = new CompletableFuture<>();
    scheduler.schedule(
        () -> {
            final TimeoutException ex =
                new TimeoutException("Operation timed out after " + duration);
            promise.completeExceptionally(ex);
        },
        duration.toMillis(),
        TimeUnit.MILLISECONDS
    );
    return promise;
}

Let's break this down carefully, because there's a deliberate design choice at every line.

new CompletableFuture<>() — This creates a promise: a CompletableFuture with no backing computation. It won't complete on its own. We're going to complete it manually from the scheduler. This is distinct from CompletableFuture.supplyAsync() which creates a future backed by actual work.

scheduler.schedule(...) — We're scheduling a single task to fire after the given duration. When it fires, it calls promise.completeExceptionally(ex), which marks the future as failed with a TimeoutException. The call to completeExceptionally is thread-safe and idempotent — if the promise was already completed (by the real work finishing first), this call does nothing.

Single-threaded scheduler — One thread is enough for scheduling arbitrary numbers of timeouts. The scheduled task itself runs in microseconds (just calls completeExceptionally). This is not an accidentally-downsized thread pool — a ScheduledExecutorService with 1 thread genuinely handles thousands of scheduled timeouts simultaneously because the overhead per timeout is negligible. This is a real design choice, not a demo simplification.

setDaemon(true) — The scheduler thread is marked as a daemon thread. Daemon threads don't prevent JVM shutdown — if your application decides to exit, you don't want a lingering timeout thread holding the JVM alive.

TimeoutException vs ExecutionException: One thing worth knowing: if you ever do call .get() on a future that failed with TimeoutException via completeExceptionally, you'll get an ExecutionException wrapping the TimeoutException. This is unavoidable — get() always wraps causes in ExecutionException. In the callback-based pattern we're building, the throwable passed to exceptionally() or whenComplete() is the TimeoutException directly, unwrapped. This is actually nicer to handle.

Step 4: Racing the Futures — acceptEither

Now we have two futures: responseFuture (our real work) and oneSecondTimeout (our failure timer). We want to proceed with whichever completes first.

public void serve() {
    final CompletableFuture<Response> responseFuture = asyncCode();
    final CompletableFuture<Response> oneSecondTimeout = failAfter(Duration.ofSeconds(1));

    responseFuture
        .acceptEither(oneSecondTimeout, this::send)
        .exceptionally(throwable -> {
            log.error("Request failed or timed out", throwable);
            return null;
        });
}

acceptEither runs the given Consumer with the result of whichever future completes first. It ignores the slower one entirely.

The race plays out in two scenarios:

Scenario A: asyncCode() completes within 1 second. responseFuture completes with a Response. acceptEither calls this::send with that response. oneSecondTimeout eventually fires and tries to complete exceptionally, but the overall future is already done — the call is a no-op.

Scenario B: asyncCode() takes longer than 1 second. oneSecondTimeout fires first and completes exceptionally with TimeoutException. acceptEither sees an exceptionally-completed future and propagates the exception. exceptionally catches it and logs it. this::send is never called.

Important guarantee: this::send and exceptionally are mutually exclusive. You cannot get both. If acceptEither fires send, the exceptionally on the resulting chain only fires if send() itself throws. If acceptEither propagates an exception, send never runs.

What if both complete simultaneously? acceptEither uses the first one to complete. If they complete at exactly the same nanosecond (hardware-level simultaneous completion), it picks one — whichever the JVM's internal compare-and-set wins. send() is called with a response and the timeout exception is discarded. This is the correct behavior.

Step 5: The Clean Solution — within()

acceptEither works but it's not the cleanest API. Every call site has to create the timeout future manually and remember to chain acceptEither. There's a better pattern: wrap the original future in a new future that enforces the timeout transparently.

This is inspired directly by com.twitter.util.Future.within() from Twitter's util library — one of the cleaner APIs for this problem in any JVM language.

public static <T> CompletableFuture<T> within(
        CompletableFuture<T> future,
        Duration duration) {
    final CompletableFuture<T> timeout = failAfter(duration);
    return future.applyToEither(timeout, Function.identity());
}

applyToEither is the Function-returning sibling of acceptEither (which takes a Consumer). We pass Function.identity() so the result is forwarded unchanged — we're just racing the two futures and taking the winner's value directly.

The result is a new CompletableFuture<T> that:

• Completes with the value from future if it finishes before the timeout
• Completes exceptionally with TimeoutException if the timeout fires first
• Is completely transparent to callers — they just get a future

Here's the final, clean calling code:

public void serve() {
    final CompletableFuture<Response> responseFuture =
        within(asyncCode(), Duration.ofSeconds(1));

    responseFuture
        .thenAccept(this::send)
        .exceptionally(throwable -> {
            log.error("Request failed or timed out: {}", throwable.getMessage(), throwable);
            return null;
        });
}

This is clean. serve() returns immediately. Timeouts work. Errors propagate. No threads are blocked. The entire thing is event-driven.

Production-Grade Implementation

Here's the complete, copy-paste-ready implementation with all the pieces assembled, plus a few production hardening details the incremental walkthrough didn't cover:

import java.time.Duration;
import java.util.concurrent.*;
import java.util.function.Function;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class AsyncService {
    private static final Logger log = LoggerFactory.getLogger(AsyncService.class);

    // Single-threaded scheduler for all timeout futures.
    // Daemon thread so it doesn't block JVM shutdown.
    private static final ScheduledExecutorService TIMEOUT_SCHEDULER =
        Executors.newSingleThreadScheduledExecutor(r -> {
            Thread t = new Thread(r, "async-timeout-scheduler");
            t.setDaemon(true);
            return t;
        });

    // Dedicated pool for send() callbacks.
    // Prevents send() from running on the async client's thread pool.
    private final ExecutorService sendPool;

    public AsyncService(ExecutorService sendPool) {
        this.sendPool = sendPool;
    }

    /**
     * Returns a CompletableFuture that wraps the given future with a hard timeout.
     * If the underlying future does not complete within the given duration,
     * the returned future fails with TimeoutException.
     */
    public static <T> CompletableFuture<T> within(
            CompletableFuture<T> future,
            Duration duration) {
        final CompletableFuture<T> timeout = failAfter(duration);
        return future.applyToEither(timeout, Function.identity());
    }

    /**
     * Returns a CompletableFuture that never completes successfully — it always
     * fails with TimeoutException after the given duration.
     */
    public static <T> CompletableFuture<T> failAfter(Duration duration) {
        final CompletableFuture<T> promise = new CompletableFuture<>();
        TIMEOUT_SCHEDULER.schedule(
            () -> promise.completeExceptionally(
                new TimeoutException("Timed out after " + duration)
            ),
            duration.toMillis(),
            TimeUnit.MILLISECONDS
        );
        return promise;
    }

    /**
     * Handle a single request. Non-blocking. Returns immediately.
     */
    public void serve() {
        within(asyncCode(), Duration.ofSeconds(1))
            .thenAcceptAsync(this::send, sendPool)   // run send() on dedicated pool
            .whenComplete((ignored, throwable) -> {
                if (throwable != null) {
                    // Distinguish timeout from other failures
                    Throwable cause = throwable instanceof CompletionException
                        ? throwable.getCause()
                        : throwable;

                    if (cause instanceof TimeoutException) {
                        log.warn("Request timed out: {}", cause.getMessage());
                    } else {
                        log.error("Request failed unexpectedly", cause);
                    }
                }
            });
    }

    /**
     * Simulates an async operation — replace with real implementation.
     */
    private CompletableFuture<Response> asyncCode() {
        return CompletableFuture.supplyAsync(() -> {
            // Simulate variable latency
            try {
                Thread.sleep((long) (Math.random() * 1500)); // 0-1500ms
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
                throw new RuntimeException(e);
            }
            return new Response("OK");
        });
    }

    private void send(Response response) {
        log.info("Sending response: {}", response);
    }

    // --- Demo ---
    public static void main(String[] args) throws InterruptedException {
        ExecutorService sendPool = Executors.newFixedThreadPool(4);
        AsyncService service = new AsyncService(sendPool);

        // Fire 10 requests concurrently — none block the calling thread
        for (int i = 0; i < 10; i++) {
            service.serve();
        }

        // Wait for everything to resolve
        Thread.sleep(3000);
        sendPool.shutdown();
        TIMEOUT_SCHEDULER.shutdown();
    }

    record Response(String body) {}
}

A few things added beyond the walkthrough:

CompletionException unwrapping — When a stage in a CompletableFuture chain fails, the exception gets wrapped in a CompletionException at certain chain boundaries. The cause of the CompletionException is the actual exception. The unwrapping logic in whenComplete handles this so you're checking for TimeoutException directly rather than CompletionException wrapping TimeoutException.

thenAcceptAsync with sendPool — send() runs on a dedicated pool rather than the thread that completed the original future. This is important when send() does any meaningful work (serialization, network I/O, downstream calls) — you don't want that work competing with the thread pool driving your async operations.

Thread.currentThread().interrupt() in the simulation — Whenever you catch InterruptedException, you must re-interrupt the thread before rethrowing. If you swallow the interrupt, the thread's interrupted status is lost and any upstream code that checks for interruption (like your thread pool's shutdown logic) may behave incorrectly.

Java 9+ Alternative: orTimeout() and completeOnTimeout()

If you're on Java 9 or later (and you should be — Java 8 is long past end of life), CompletableFuture gained two built-in timeout methods that make this even cleaner:

import java.util.concurrent.TimeUnit;

// orTimeout: fails with TimeoutException if not completed within the given time
CompletableFuture<Response> future = asyncCode()
    .orTimeout(1, TimeUnit.SECONDS);

future
    .thenAccept(this::send)
    .exceptionally(throwable -> {
        // throwable may be a TimeoutException (wrapped in CompletionException)
        Throwable cause = throwable.getCause() != null ? throwable.getCause() : throwable;
        log.error("Failed: {}", cause.getMessage());
        return null;
    });

// completeOnTimeout: provides a default value instead of throwing
CompletableFuture<Response> futureWithDefault = asyncCode()
    .completeOnTimeout(new Response("default"), 1, TimeUnit.SECONDS);

futureWithDefault.thenAccept(this::send);

orTimeout() is equivalent to our within() utility — same semantics, built into the JDK. completeOnTimeout() goes further by letting you provide a fallback value instead of failing, which is useful for optional enrichment calls where a timeout means "use the default" rather than "fail the request."

For Java 8 specifically (as in the original codebase this article came from), the within() utility is the right approach. If you're greenfielding on Java 21+, reach for orTimeout() directly and save the infrastructure code.

Thread Models and Why This Matters

The reason this problem was so painful in the original application is fundamentally about thread model. The thread-per-request model (one thread blocked for the duration of each operation) works fine at low concurrency but degrades badly under load. At 1,000 blocked threads, you're not just wasting CPU — you're allocating hundreds of megabytes of stack memory that's sitting idle. The JVM heap and thread stack model means every blocked thread is a real cost.

The callback model we've built here is a manual version of what reactive frameworks automate. Libraries like Project Reactor (Mono/Flux) and RxJava (Observable/Single) provide richer operator sets — retry, throttle, flatMap, merge, zip — on top of the same fundamental idea: don't block threads, use callbacks.

For the specific case of CPU-bound parallelism over collections, virtual threads in Java streams (vtstream) are worth understanding too — they give each element its own lightweight virtual thread rather than blocking a platform thread, which is the logical end state of the thread model evolution.

But if you're on Java 8 in a legacy codebase, CompletableFuture with within() is the pragmatic answer. It's built into the JDK, it composes cleanly with existing ExecutorService infrastructure, and it gets you from 1,000 blocked threads to exactly zero blocked threads.

The Conclusion

The transformation looks simple in retrospect: swap future.get() for thenAccept(), wrap with within() for timeout enforcement, chain exceptionally() properly. But the path to getting here required understanding why each step was wrong before the final one:

• Future.get() blocks. All blocking on get() is bad at scale, even with a timeout.
• Swapping Future for CompletableFuture doesn't help if you still call .get().
• thenAccept() makes it non-blocking but drops your timeout.
• failAfter() gives you a timeout future.
• acceptEither() races two futures — but within() wraps this into a clean API.
• exceptionally() after thenAccept(), not before it, or you'll call send(null).

The final pattern handles all of these correctly:

within(asyncCode(), Duration.ofSeconds(1))
    .thenAccept(this::send)
    .exceptionally(throwable -> {
        log.error("Failed", throwable);
        return null;
    });

Non-blocking. Timeout-enforced. Errors handled. One thread for all your timeouts instead of 1,000 threads blocked on get().