Virtual thread

Last updated

In computer programming, a virtual thread is a thread that is managed by a runtime library or virtual machine (VM) and made to resemble "real" operating system thread to code executing on it, while requiring substantially fewer resources than the latter.

Contents

Virtual threads allows for tens of millions of preemptive tasks and events on a 2021 consumer-grade computer, [1] compared to low thousands of operating system threads. [2] Preemptive execution [3] is important to performance gains through parallelism and fast preemptive response times for tens of millions of events.

Earlier constructs that are not or not always preemptive, such as coroutines, green threads or the largely single-threaded Node.js, introduce delays in responding to asynchronous events such as every incoming request in a server application. [4]

Definition

Virtual threads are preemptive

Virtual threads can hop over the execution units of all processors and cores

Virtual threads require no yield or similar interventions by the programmer

Virtual threads can number in the tens of millions by featuring small often managed stacks

Virtual threads can be allocated quickly

Virtual threads share memory like operating system threads

Virtual threads offer parallelism like operating system threads

Motivation

Java servers have featured extensive and memory consuming software constructs allowing dozens of pooled operating system threads to preemptively execute thousands of requests per second without the use of virtual threads. Key to performance here is to reduce the initial latency in thread processing and minimize the time operating system threads are blocked. [6]

Virtual threads increase possible concurrency by many orders of magnitudes while the actual parallelism achieved is limited by available execution units and pipelining offered by present processors and processor cores. In 2021, a consumer grade computers typically offer a parallelism of tens of concurrent execution units. [7] For increased performance through parallelism, the language runtime need to use all present hardware, [8] not be single-threaded or feature global synchronization such as global interpreter lock.

The many magnitudes of increase in possible preemptive items offered by virtual threads is achieved by the language runtime managing resizable thread stacks. [9] Those stacks are smaller in size than those of operating system threads. The maximum number of threads possible without swapping is proportional to the amount of main memory. [10]

In order to support virtual threads efficiently, the language runtime has to be largely rewritten to prevent blocking calls from holding up an operating system thread assigned to execute a virtual thread [11] and to manage thread stacks. [12] An example of a retrofit of an existing runtime with virtual threads is Java's Project Loom. [13] An example of a new language designed for virtual threads is Go. [14]

Complexity

Because virtual threads offer parallelism, the programmer needs to be skilled in multi-threaded programming and synchronization.

Because a blocked virtual thread would block the OS thread it occupies at the moment, much effort must be taken in the runtime to handle blocking system calls. Typically, a thread from a pool of spare OS threads is used to execute the blocking call for the virtual thread so that the initially executing OS thread is not blocked.

Management of the virtual thread stack requires care in the linker and short predictions of additional stack space requirements.

Implementations

Google Chrome Browser

Virtual threads are used to serialize singleton input/output activities and available to developers extending the browser. When a virtual thread is executing, it can hop on a different OS thread. [15]

Go

Go's goroutines became preemptive with Go 1.4 in 2014 and are a prominent application of virtual threads.

Java

Java introduced virtual threads in 2023 with Java 21, with the limitation that any code running on a virtual thread which uses synchronised blocks or native calls will become pinned to its carrier OS thread. [16] The former limitation was fixed in Java 24. [17]

Other uses of the term

Intel [18] in 2007 referred to an Intel compiler specific optimization technique as virtual threads.

See also

References

  1. Rudell, Harald (2022-03-19). "massivevirtualparallelism".
  2. baeldung (2022-01-02). "Maximum Number of Threads Per Process in Linux | Baeldung on Linux". www.baeldung.com. Retrieved 2022-03-30.
  3. "Go 1.14 Release Notes - The Go Programming Language". go.dev. Retrieved 2022-03-30.
  4. Node.js. "Don't Block the Event Loop (or the Worker Pool)". Node.js. Retrieved 2022-03-30.
  5. Lu, Genchi (2021-07-22). "Java's Thread Model and Golang Goroutine". Medium. Retrieved 2022-04-05.
  6. "Principles to Handle Thousands of Connections in Java Using Netty - DZone Performance". dzone.com. Retrieved 2022-03-30.
  7. "MacBook Pro 14-inch and MacBook Pro 16-inch". Apple. Retrieved 2022-03-30.
  8. "Frequently Asked Questions (FAQ) - The Go Programming Language". go.dev. Retrieved 2022-03-30.
  9. "JEP draft: Virtual Threads (Preview)". openjdk.java.net. Retrieved 2022-03-30.
  10. Rudell, Harald (2022-03-22). "Maximum number of virtual threads in Go".
  11. Szczukocki, Denis (2020-03-18). "Difference Between Thread and Virtual Thread in Java | Baeldung". www.baeldung.com. Retrieved 2022-03-30.
  12. "Why you can have millions of Goroutines but only thousands of Java Threads". rcoh.me. 2018-04-12. Retrieved 2022-03-30.
  13. "Main - Main - OpenJDK Wiki". wiki.openjdk.java.net. Retrieved 2022-03-30.
  14. "The Go Programming Language". go.dev. 2022-03-22. Retrieved 2022-03-30.
  15. "Threading and Tasks in Chrome". chromium.googlesource.com. Retrieved 2022-04-05.
  16. "Virtual Threads". Oracle Help Center. Retrieved 2024-09-10.
  17. "JEP 491: Synchronize Virtual Threads without Pinning". OpenJDK. Retrieved 2025-03-30.
  18. "Intel Technology Journal" (PDF).