Parallelizing Java Streams with Virtual Threads: Building vtstream

You know what's funny about parallelStream() in Java? Everyone acts like it's this magic performance switch. Just slap .parallel() on your stream and suddenly your code is running on all your CPU cores, right?

Wrong. Or at least, not in the way you'd want.

parallelStream() uses the common ForkJoinPool. That's a shared, JVM-wide thread pool. Every parallel stream operation in your entire application competes for the same pool. And the pool is sized to your CPU core count — which is perfect for pure CPU-bound number crunching, but absolutely awful for anything that involves waiting. Database queries. HTTP calls. File I/O. You block one of those threads on a network call and suddenly everything using that pool slows to a crawl.

I've seen this exact thing happen in a production application. Parallel stream processing a batch of records that each needed a database lookup. Everything looked great in testing. Then load hit it in prod, the ForkJoinPool saturated, and latency spiked across the entire application because some background report was hogging all the parallel threads. Not a good day.

If you want to read more about fighting async issues in Java, we covered the deeper pain of blocking futures and how to handle them properly in Async Timeouts with CompletableFuture — go check that out.

So how do you get real parallelism for I/O-heavy stream workloads? That's exactly the problem the vtstream library was built to solve. It's an open-source library by Peter Verhas that takes a completely different approach: instead of batching elements onto a shared pool, it gives every single stream element its own virtual thread. And because virtual threads are absurdly cheap compared to platform threads, this actually works.

Let's break down exactly how it's built and why that design makes sense.

Why parallelStream() Falls Short for I/O Work

Before we get into the library, let's be precise about the limitation we're solving.

Java's standard parallelStream() uses the ForkJoinPool.commonPool() under the hood. By default, this pool has Runtime.getRuntime().availableProcessors() - 1 threads. On a quad-core machine, that's 3 parallel workers.

For CPU-intensive tasks — sorting, computing, transforming data in memory — this is great. You don't want more threads than cores because you'd just be paying context-switch overhead for nothing.

But for I/O-intensive tasks, this is terrible. When one of those 3 threads blocks waiting for a database response, it's stuck. It can't do anything else. Your parallelism has just degraded to 2 workers.

The traditional fix was to create a dedicated ExecutorService with more threads and somehow inject it into the stream pipeline. There are tricks to run a parallel stream inside a custom ForkJoinPool, but as the Java documentation itself warns, this is an "implementation trick" and is not guaranteed to work reliably across JVM versions.

Enter Project Loom and virtual threads, finalized in Java 21 (JEP 444). Virtual threads are JVM-managed lightweight threads that park themselves (without blocking the underlying platform thread) when waiting on I/O. A machine that can handle maybe 10,000 platform threads can comfortably run millions of virtual threads. The overhead per virtual thread is tiny compared to a platform thread.

vtstream exploits this directly. Each element gets its own virtual thread. All of them run concurrently. When one parks waiting on a database, the others keep running. No shared pool saturation, no artificial parallelism limit.

Adding vtstream to Your Project

The library is on Maven Central and GitHub. Add the dependency:

<dependency>
    <groupId>com.github.verhas</groupId>
    <artifactId>vtstream</artifactId>
    <version>1.0.1</version>
</dependency>

Always verify the current version on Maven Central or the GitHub releases page before using it. This article covers version 1.0.1.

Requirements: Java 21+ (virtual threads require JEP 444, which was finalized in Java 21).

The Core Idea: A Linked List of Stream Operations

The library has two primary classes: ThreadedStream<T> and Command. Understanding how they relate to each other is the key to understanding the whole thing.

ThreadedStream<T> implements Java's Stream<T> interface directly:

public class ThreadedStream<T> implements Stream<T> {
    // ...
}

This is smart API design. It means you can use ThreadedStream anywhere you'd use a normal Stream, and it composes naturally with the rest of the Java stream ecosystem. No new API to learn.

Command is a family of nested classes representing intermediate stream operations:

public static class Filter<T> extends Command<T, T> { ... }
public static class Map<T, R> extends Command<T, R> { ... }
public static class AnyMatch<T> extends Command<T, T> { ... }
// ...and so on

Each Command subclass encapsulates a specific operation. Filter holds a Predicate. Map holds a Function. They're just value objects that describe what work needs to be done.

Here's the key structural insight: when you chain operations on a ThreadedStream, you're building a linked list. Each new ThreadedStream holds a reference to the previous one:

ThreadedStream<T> {
    Command    command;     // the operation to perform at this stage
    ThreadedStream downstream; // the previous stage in the chain
    Stream     source;     // the original data source (only set at the head)
    long       limit;      // optional element limit
    boolean    chained;    // whether this is part of a chain
}

Walk through what happens when you write this:

ThreadedStream<String> stream = ThreadedStream.threaded(Stream.of("apple", "banana", "cherry"));
ThreadedStream<String> filtered = stream.filter(s -> s.startsWith("a"));
ThreadedStream<String> mapped = filtered.map(String::toUpperCase);

After these three lines, you have a linked list: mapped → filtered → stream. The stream node holds the source data. The filtered node holds the Filter command and a pointer to stream. The mapped node holds the Map command and a pointer to filtered.

The map method that builds this:

public <R> ThreadedStream<R> map(Function<? super T, ? extends R> mapper) {
    return new ThreadedStream<>(new Command.Map<>(mapper), this);
}

Nothing executes yet. No threads. No work. This is the standard lazy evaluation model Java streams use — all intermediate operations are deferred until a terminal operation triggers execution.

Stream Execution: Where the Magic Happens

Terminal operations trigger execution. When you call something like .collect(), the library first converts the ThreadedStream back into a conventional Stream via toStream(), then calls the terminal operation on that:

@Override
public <R> R collect(Supplier<R> supplier, 
                     BiConsumer<R, ? super T> accumulator,
                     BiConsumer<R, R> combiner) {
    return toStream().collect(supplier, accumulator, combiner);
}

The real work happens inside toStream(). It inspects whether the source stream is parallel (via source.isParallel()) and dispatches to either ordered or unordered execution.

Unordered Execution: Results As They Arrive

If you don't need results in the same order as the source, unordered execution gives you maximum throughput. Results are collected as each virtual thread finishes, regardless of which element started first.

private Stream<T> toUnorderedStream() {
    final var result = Collections.synchronizedList(new LinkedList<Command.Result<T>>());
    final AtomicInteger n = new AtomicInteger(0);
    final Stream<?> limitedSource = limit >= 0 ? source.limit(limit) : source;

    limitedSource.forEach(t -> {
        Thread.startVirtualThread(() -> result.add(calculate(t)));
        n.incrementAndGet();
    });

    return IntStream.range(0, n.get())
        .mapToObj(i -> {
            while (result.isEmpty()) {
                Thread.yield();
            }
            return result.removeFirst();
        })
        .filter(f -> !f.isDeleted())
        .peek(r -> {
            if (r.exception() != null) {
                throw new ThreadExecutionException(r.exception());
            }
        })
        .map(Command.Result::result);
}

Let's walk through each piece carefully, because there's a lot happening here.

Thread.startVirtualThread(() -> result.add(calculate(t))) — This is the core. For every element t in the source stream, we spin up a brand new virtual thread and hand it the work. calculate(t) traverses the linked list of Command objects — running the filter, the map, whatever operations were chained — and returns a Command.Result<T> holding either the final value, a "deleted" marker (if a filter eliminated it), or an exception. Because these are virtual threads, creating thousands of them is cheap. Each one parks itself waiting on I/O rather than blocking a platform thread.

Collections.synchronizedList(new LinkedList<>()) — Multiple virtual threads are writing results concurrently. We need thread-safe writes. synchronizedList wraps the LinkedList with a mutex, making add and removeFirst atomic. This is the collection point where finished results land.

AtomicInteger n — We can't know ahead of time how many elements the source stream has (streams can be lazy or infinite). We count them as we dispatch with an AtomicInteger, then use that count to know how many results to wait for.

IntStream.range(0, n.get()).mapToObj(...) — This builds the output stream. We iterate n times — once for each element we dispatched. For each iteration, we spin-wait on result.isEmpty(), calling Thread.yield() to not burn CPU, then grab the first available result. Because results can arrive out of order, we get them as they finish.

.filter(f -> !f.isDeleted()) — Filter operations mark results as "deleted" rather than removing them from the result list. This keeps the counting logic consistent. We strip deleted results out here.

.peek(r -> { if (r.exception() != null) throw... }) — Exceptions thrown inside virtual threads need to propagate back to the caller. The calculate method catches any exception and stores it in the Command.Result. This peek rethrows it wrapped in ThreadExecutionException so callers can handle it normally.

Ordered Execution: Preserving Sequence

Sometimes order matters. If your downstream code depends on elements coming back in the same sequence as the source, you need ordered execution. The library handles this with a local Task class:

private Stream<T> toOrderedStream() {
    class Task {
        Thread workerThread;
        volatile Command.Result<T> result;

        static void waitForResult(Task task) {
            try {
                task.workerThread.join();
            } catch (InterruptedException e) {
                task.result = deleted();
            }
        }
    }

    final var tasks = Collections.synchronizedList(new LinkedList<Task>());
    final Stream<?> limitedSource = limit >= 0 ? source.limit(limit) : source;

    limitedSource.forEach(sourceItem -> {
        Task task = new Task();
        tasks.add(task);
        task.workerThread = Thread.startVirtualThread(() -> task.result = calculate(sourceItem));
    });

    return tasks.stream()
        .peek(Task::waitForResult)
        .map(f -> f.result)
        .peek(r -> {
            if (r.exception() != null) {
                throw new ThreadExecutionException(r.exception());
            }
        })
        .filter(r -> !r.isDeleted())
        .map(Command.Result::result);
}

The key insight here is that we preserve order by keeping the tasks in a list in source order, then waiting for them sequentially.

volatile Command.Result<T> result — The volatile keyword matters. The virtual thread writes to task.result from a different thread than the one reading it. Without volatile, the JVM is allowed to cache the read in a register and never see the write. volatile forces reads and writes to go to main memory, ensuring visibility.

task.workerThread.join() — The join() call blocks the current thread until workerThread finishes. Because we're iterating tasks in order, we wait for element 0 to finish before moving to element 1. All elements still run concurrently — we're just consuming results in order. If element 5 finishes before element 3, it just sits waiting in its Task object until we get to it.

Compare ordered vs. unordered execution timing:

Source elements: [A, B, C, D, E]

Unordered:
- All 5 virtual threads start simultaneously
- Results collected as they finish: [D, A, C, E, B] (whatever order)
- Total time: max(A, B, C, D, E) processing time

Ordered:
- All 5 virtual threads start simultaneously
- Results consumed in order: [A, B, C, D, E]
- Total time: still max(A, B, C, D, E) — all running in parallel
             (unless A is the slowest, in which case everything waits for A)

Both approaches launch all virtual threads at the same time. Unordered gives you elements as they complete. Ordered gives you elements in source sequence, but still benefits from parallel execution during the processing phase.

Real-World Usage Examples

Let's look at how this actually plays out in code you'd write against the library.

Parallel HTTP Calls

The canonical use case — fetching data from multiple endpoints in parallel:

import javax0.vtstream.ThreadedStream;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.URI;
import java.util.List;
import java.util.stream.Collectors;

public class ParallelFetcher {

    private final HttpClient client = HttpClient.newHttpClient();

    public List<String> fetchAll(List<String> urls) {
        return ThreadedStream.threaded(urls.stream())
            .map(url -> {
                try {
                    HttpRequest request = HttpRequest.newBuilder()
                        .uri(URI.create(url))
                        .GET()
                        .build();

                    HttpResponse<String> response = client.send(
                        request,
                        HttpResponse.BodyHandlers.ofString()
                    );

                    return response.body();
                } catch (Exception e) {
                    throw new RuntimeException("Failed to fetch: " + url, e);
                }
            })
            .collect(Collectors.toList());
    }
}

With parallelStream() on a quad-core machine, you'd have at most 3 concurrent HTTP requests. With vtstream, all requests launch simultaneously as virtual threads. When each one blocks waiting on network I/O, it parks — not burning a platform thread. The throughput difference on large batches is significant.

Parallel Database Lookups

This is where I've found vtstream most useful in practice. Enriching a list of records with database lookups:

import javax0.vtstream.ThreadedStream;
import java.util.List;
import java.util.stream.Collectors;

public class UserEnricher {

    private final UserRepository userRepository;
    private final ProfileRepository profileRepository;

    public UserEnricher(UserRepository userRepository, ProfileRepository profileRepository) {
        this.userRepository = userRepository;
        this.profileRepository = profileRepository;
    }

    public record EnrichedUser(User user, Profile profile) {}

    public List<EnrichedUser> enrichUsers(List<Long> userIds) {
        return ThreadedStream.threaded(userIds.stream())
            .map(userId -> {
                User user = userRepository.findById(userId)
                    .orElseThrow(() -> new RuntimeException("User not found: " + userId));

                Profile profile = profileRepository.findByUserId(userId)
                    .orElse(Profile.empty());

                return new EnrichedUser(user, profile);
            })
            .filter(eu -> eu.user().isActive())
            .collect(Collectors.toList());
        }
}

Each user lookup happens in its own virtual thread. The database queries run concurrently. With 100 user IDs and each query taking ~10ms, the sequential version takes ~1 second. vtstream brings that down to roughly 10ms (plus overhead), because all queries fire simultaneously and park waiting on the DB.

Mixing with Standard Stream Operations

Because ThreadedStream implements Stream<T>, you can seamlessly compose it with standard collectors and downstream stream operations:

// Works fine - ThreadedStream produces a standard stream via terminal ops
Map<String, Long> countsByCategory = ThreadedStream.threaded(productIds.stream())
    .map(id -> productService.fetchProduct(id))  // parallel HTTP/DB calls
    .filter(product -> product.isAvailable())
    .collect(Collectors.groupingBy(
        Product::category,
        Collectors.counting()
    ));

The parallel work happens during the map phase. By the time collect is called, vtstream has already converted back to a conventional stream, so standard collectors work exactly as you'd expect.

Understanding the Command/Result System

The Command.Result<T> type that flows through the execution is worth understanding explicitly, because it's how the library handles three different outcomes cleanly:

// Simplified representation of what Command.Result communicates:
// 1. Success - a value was produced
Command.Result<String> success = Command.Result.of("APPLE");

// 2. Deleted - a filter eliminated this element
Command.Result<String> deleted = Command.Result.deleted();

// 3. Exception - something went wrong during processing
Command.Result<String> failed = Command.Result.exception(someException);

This tri-state result model means that filter operations don't actually remove elements from the parallel pipeline prematurely — they mark them as deleted. The counting logic in toUnorderedStream always knows how many results to expect, because every element produces exactly one Command.Result. Clean.

The exception model is also worth noting. In standard parallel streams, exceptions thrown inside stream operations get wrapped and rethrown in sometimes confusing ways. vtstream stores exceptions in the result object and re-throws them deterministically during the output iteration with ThreadExecutionException, which wraps the original cause. This makes error handling predictable.

The Virtual Thread Advantage: Some Numbers

Virtual threads were introduced as a preview in JEP 425 (Java 19) and finalized in JEP 444 (Java 21) as part of Project Loom. The overhead numbers compared to platform threads are striking:

For a streaming workload with 10,000 I/O-bound elements, spawning 10,000 platform threads would likely crash your JVM or cause severe contention. With virtual threads, it's routine.

There's an important caveat though: virtual threads are for I/O-bound work, not CPU-bound work. If your stream operations are doing heavy computation (sorting large arrays, complex math, image processing), you still want parallelStream() with the ForkJoinPool, because it matches the thread count to core count deliberately. vtstream's one-thread-per-element approach would over-subscribe the CPU in that scenario.

The mental model: virtual threads excel at wait-heavy work. Platform threads in a sized pool excel at compute-heavy work.

If you want to understand how Java manages the underlying memory for all of this — where these thread stacks actually live — take a look at our deep dive into Java Heap Memory. And if you want to understand how the JVM cleans up after itself when threads finish, the custom garbage collector article walks through the GC mechanics that make this possible.

When to Use vtstream vs. Standard Alternatives

You have several options for parallel stream processing in Java. Here's when each makes sense:

parallelStream() — Use it when your work is CPU-bound (pure computation, in-memory transformations). Avoid it for blocking I/O.

CompletableFuture + ExecutorService — Powerful and flexible but verbose. Good for complex async pipelines with dependencies. vtstream is better for simple parallel map/filter patterns over collections.

vtstream — Use it when you have a collection of elements that each need blocking I/O (DB queries, HTTP calls, file reads) and you want to parallelize that I/O with minimal boilerplate. The Stream API stays familiar; you just swap stream() for ThreadedStream.threaded(stream).

Stream Gatherers (Java 22+) — For custom intermediate stream operations and complex transformations. Complements vtstream rather than competing with it.

Caveats and Production Readiness

The library author is honest about this, and so should we be: vtstream has been reviewed by a concurrent programming expert (Istvan Kovacs) but hasn't been battle-tested at large production scale. It's an open-source library that you should evaluate and test thoroughly before deploying to high-stakes systems.

Specific things to verify in your use case:

Thread pinning — Virtual threads can get "pinned" to their carrier (platform) thread in certain situations: when holding a synchronized lock, or when calling certain native code. If your stream operations call libraries with synchronized blocks (old JDBC drivers, for example), you may not get the full virtual thread benefits. Check with JVM flags like -Djdk.tracePinnedThreads=full.

Memory pressure — Creating millions of virtual threads doesn't consume 1MB each, but there is still some memory footprint. For extremely large streams, profile memory usage under load.

Error handling — Test your exception handling paths explicitly. The ThreadExecutionException wrapping is clean, but make sure your callers handle it appropriately.

The synchronized result list — The unordered execution path uses Collections.synchronizedList(). Under extremely high contention (thousands of elements completing simultaneously), this could become a bottleneck. For serious high-throughput use, you'd want to profile this with your actual workload.

The Conclusion

Java's parallel stream support has always had a hole in it: parallelStream() is great for CPU work and bad for I/O work. For years, the workaround was complicated executor injection tricks that the documentation itself warned against relying on.

Virtual threads, finalized in Java 21, change the fundamental math. When threads are cheap enough to create one per element, you don't need to batch or pool anything. You just fire and wait. vtstream takes exactly this approach — a clean Stream API wrapper that puts each element in its own virtual thread, handles ordering guarantees, propagates exceptions properly, and integrates seamlessly with the existing stream ecosystem.

The design is elegant: a linked list of Command objects built lazily as you chain operations, executed in parallel via virtual threads only when a terminal operation triggers it. The separation between ordered (toOrderedStream) and unordered (toUnorderedStream) execution gives you control over the tradeoff between throughput and sequence preservation.

Is it production-ready for every workload? The honest answer is: test it with your specific patterns and data sizes. But as a demonstration of what becomes possible when virtual threads are cheap, and as a practical tool for I/O-heavy stream pipelines, it nails the design.

If you're not on Java 21 yet and want to understand what you're missing on the concurrency front — or if you want to understand exactly what virtual threads replaced in the old threading model — the Java ecosystem coverage here at CoderOasis has you covered. Start with Heap Memory in Java to understand the runtime underpinnings, then Async Timeouts with CompletableFuture to see what the old world looked like before virtual threads simplified the story.