Virtual Threads in Java 21 comparing with Cached Thread and Fixed Thread

This example demonstrates the advantage of virtual threads introduced in Java 21 over other methods of executing tasks in threads.

Introduction

Starting from Java 8, concurrent programming is a lot of improved. Introduced two important parts of the Concurrent API: Atomic Variable and Concurrent Maps. Also, in Java 8 concurrent programming, are introduced lambda expression and functional programming. In short line, improvements in Java 8 are:
• Threads and Executors
• Synchronization and Locks
• Atomic Variables and ConcurrentMap

Java 21 brings several new features. Some of them are listed in the picture bellow:

Probably, the most important feature in Java 21 are Virtual Threads.

In Java 21 the basic concurrence model of Java is unchanged and the Stream API is still the preferred way to process large data sets in parallel.

Using Virtual Threads, introduced in Java 21, concurrent API has better performance. Today, we have microservices architecture and server application scales, and that will cause the number of threads must grow. A main goal of virtual threads is to enable scalability of server applications written in the simple thread-per-request style.

Virtual Threads

Before Java 21, the JDK’s current implementation of threads implements threads as thin wrappers around operating system (OS) threads. However, as OS threads are expensive:

if each request consumes an OS thread for its duration, then the number of threads often becomes the limiting factor when attempting to scale.
thus, an application's throughput is capped even when thread pooling is employed (given that pooling does not increase the actual number of threads). The goal is to break this 1:1 relationship between Java threads and OS threads. Virtual thread using the idea applied in Virtual Ram memory. Illusion of plentiful memory is applied mapping a large virtual address space to limited amount of the physical RAM. Similarly, the Java runtime can give the illusion of plentiful threads by mapping a large number of virtual threads to a small number of OS threads. Platform threads, implemented in the traditional way, are thin-wrapper around an OS thread. Virtual threads are not tied to any particular OS thread. A virtual thread can run any code that a platform thread can run. It is good, because existing Java code that processes requests will easily run in a virtual thread. Virtual threads exist on platform threads. These platform threads (“carriers”) are then scheduled by the OS as usual.

For example, executor with a Virtual thread you can create like this:

ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

Example with comparison

Virtual threads only consume OS threads while performing calculations on the CPU. A virtual thread can be mounted/unmounted on different carriers over the course of its lifetime. Typically, a virtual thread will unmount when it blocks (e.g. an I/O or database operation). When the blocking operation is completed, the virtual thread is mounted on any available carrier. The mounting and unmounting of virtual threads happens frequently and transparently without blocking any OS threads.

Example – Source code

Example01CachedThreadPool.java Executor is created using Cached Thread Pool:

var executor = Executors.newCachedThreadPool()

package threads;

import java.time.Duration;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;

/**
 *
 * @author Milan Karajovic <milan.karajovic.rs@gmail.com>
 *
 */

public class Example01CachedThreadPool {

    public void executeTasks(final int NUMBER_OF_TASKS) {

        final int BLOCKING_CALL = 1;
        System.out.println("Number of tasks which executed using 'newCachedThreadPool()' " + NUMBER_OF_TASKS + " tasks each.");

        long startTime = System.currentTimeMillis();

        try (var executor = Executors.newCachedThreadPool()) {

            IntStream.range(0, NUMBER_OF_TASKS).forEach(i -> {
                executor.submit(() -> {
                    // simulate a blicking call (e.g. I/O or db operation)
                    Thread.sleep(Duration.ofSeconds(BLOCKING_CALL));
                    return i;
                });
            });

        } catch (Exception e) {
            throw new RuntimeException(e);
        }

        long endTime = System.currentTimeMillis();
        System.out.println("For executing " + NUMBER_OF_TASKS + " tasks duration is: " + (endTime - startTime) + " ms");
    }

}

package threads;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

/**
 *
 * @author Milan Karajovic <milan.karajovic.rs@gmail.com>
 *
 */

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class Example01CachedThreadPoolTest {

    @Test
    @Order(1)
    public void test_1000_tasks() {
        Example01CachedThreadPool example01CachedThreadPool = new Example01CachedThreadPool();
        example01CachedThreadPool.executeTasks(1000);
    }

    @Test
    @Order(2)
    public void test_10_000_tasks() {
        Example01CachedThreadPool example01CachedThreadPool = new Example01CachedThreadPool();
        example01CachedThreadPool.executeTasks(10_000);
    }

    @Test
    @Order(3)
    public void test_100_000_tasks() {
        Example01CachedThreadPool example01CachedThreadPool = new Example01CachedThreadPool();
        example01CachedThreadPool.executeTasks(100_000);
    }

    @Test
    @Order(4)
    public void test_1_000_000_tasks() {
        Example01CachedThreadPool example01CachedThreadPool = new Example01CachedThreadPool();
        example01CachedThreadPool.executeTasks(1_000_000);
    }

}

Test results on my PC:

Example02FixedThreadPool.java Executor is created using Fixed Thread Pool:

var executor = Executors.newFixedThreadPool(500)

package threads;

import java.time.Duration;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;

/**
 *
 * @author Milan Karajovic <milan.karajovic.rs@gmail.com>
 *
 */

public class Example02FixedThreadPool {

    public void executeTasks(final int NUMBER_OF_TASKS) {

        final int BLOCKING_CALL = 1;
        System.out.println("Number of tasks which executed using 'newFixedThreadPool(500)' " + NUMBER_OF_TASKS + " tasks each.");

        long startTime = System.currentTimeMillis();

        try (var executor = Executors.newFixedThreadPool(500)) {

            IntStream.range(0, NUMBER_OF_TASKS).forEach(i -> {
               executor.submit(() -> {
                   // simulate a blicking call (e.g. I/O or db operation)
                  Thread.sleep(Duration.ofSeconds(BLOCKING_CALL));
                  return i;
               });
            });

        }   catch (Exception e) {
            throw new RuntimeException(e);
        }

        long endTime = System.currentTimeMillis();
        System.out.println("For executing " + NUMBER_OF_TASKS + " tasks duration is: " + (endTime - startTime) + " ms");
    }

}

package threads;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

/**
 *
 * @author Milan Karajovic <milan.karajovic.rs@gmail.com>
 *
 */

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class Example02FixedThreadPoolTest {

    @Test
    @Order(1)
    public void test_1000_tasks() {
        Example02FixedThreadPool example02FixedThreadPool = new Example02FixedThreadPool();
        example02FixedThreadPool.executeTasks(1000);
    }

    @Test
    @Order(2)
    public void test_10_000_tasks() {
        Example02FixedThreadPool example02FixedThreadPool = new Example02FixedThreadPool();
        example02FixedThreadPool.executeTasks(10_000);
    }

    @Test
    @Order(3)
    public void test_100_000_tasks() {
        Example02FixedThreadPool example02FixedThreadPool = new Example02FixedThreadPool();
        example02FixedThreadPool.executeTasks(100_000);
    }

    @Test
    @Order(4)
    public void test_1_000_000_tasks() {
        Example02FixedThreadPool example02FixedThreadPool = new Example02FixedThreadPool();
        example02FixedThreadPool.executeTasks(1_000_000);
    }

}

Test results on my PC:

Example03VirtualThread.java Executor is created using Virtual Thread Per Task Executor:

var executor = Executors.newVirtualThreadPerTaskExecutor()

package threads;

import java.time.Duration;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;

/**
 *
 * @author Milan Karajovic <milan.karajovic.rs@gmail.com>
 *
 */

public class Example03VirtualThread {

    public void executeTasks(final int NUMBER_OF_TASKS) {

        final int BLOCKING_CALL = 1;
        System.out.println("Number of tasks which executed using 'newVirtualThreadPerTaskExecutor()' " + NUMBER_OF_TASKS + " tasks each.");

        long startTime = System.currentTimeMillis();

        try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {

            IntStream.range(0, NUMBER_OF_TASKS).forEach(i -> {
               executor.submit(() -> {
                   // simulate a blicking call (e.g. I/O or db operation)
                  Thread.sleep(Duration.ofSeconds(BLOCKING_CALL));
                  return i;
               });
            });

        }   catch (Exception e) {
            throw new RuntimeException(e);
        }

        long endTime = System.currentTimeMillis();
        System.out.println("For executing " + NUMBER_OF_TASKS + " tasks duration is: " + (endTime - startTime) + " ms");
    }

}

package threads;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

/**
 *
 * @author Milan Karajovic <milan.karajovic.rs@gmail.com>
 *
 */

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class Example03VirtualThreadTest {

    @Test
    @Order(1)
    public void test_1000_tasks() {
        Example03VirtualThread example03VirtualThread = new Example03VirtualThread();
        example03VirtualThread.executeTasks(1000);
    }

    @Test
    @Order(2)
    public void test_10_000_tasks() {
        Example03VirtualThread example03VirtualThread = new Example03VirtualThread();
        example03VirtualThread.executeTasks(10_000);
    }

    @Test
    @Order(3)
    public void test_100_000_tasks() {
        Example03VirtualThread example03VirtualThread = new Example03VirtualThread();
        example03VirtualThread.executeTasks(100_000);
    }

    @Test
    @Order(4)
    public void test_1_000_000_tasks() {
        Example03VirtualThread example03VirtualThread = new Example03VirtualThread();
        example03VirtualThread.executeTasks(1_000_000);
    }

    @Test
    @Order(5)
    public void test_2_000_000_tasks() {
        Example03VirtualThread example03VirtualThread = new Example03VirtualThread();
        example03VirtualThread.executeTasks(2_000_000);
    }

}

Test results on my PC:

You see clear difference between the duration time in ms which different implementation of the executor spent to process all NUMBER_OF_TASKS. Also, you can experiment with different values for NUMBER_OF_TASKS. _We can see clear difference when we have big value for the NUMBER_OF_TASKS, how faster virtual threads process big value of the number of the tasks. _
The difference is very obvious when we have 1_000_000 tasks. It is shown in the table below:

Conclusion

I am sure, after this clarification, if your application process a lot of tasks using concurrent API, you will definitely move on Java 21 and start to use virtual threads, which will in many ways improve performance of your application.

Source code: https://github.com/Milan-Karajovic/comparing-threads-Java21-with-Virtual-Threads

Contact and support

author: Milan Karajovic
Portfolio: milan.karajovic.rs

GitHub: https://github.com/Milan-Karajovic/comparing-threads-Java21-with-Virtual-Threads

Follow me on LinkedIn: https://lnkd.in/dYGUKR3C

Milan Karajovic @milan_karajovic_45443e6d6