In addition, this implementation detail might help in gaining some insight: "The synchronous networking Java APIs, when run in a virtual thread, switch the underlying native socket into non-blocking mode. When an I/O operation invoked from Java code does not complete immediately (the native socket returns EAGAIN - “not ready” / “would block”), the underlying native socket is registered with a JVM-wide event notification mechanism (a Poller), and the virtual thread is parked. When the underlying I/O operation is ready (an event arrives at the Poller), the virtual thread is unparked and the underlying socket operation is retried."

https://inside.java/2021/05/10/networking-io-with-virtual-th...

Yeah but if for some reason, no other (or few other) request is coming in concurrently, then the CPU will just sit by idly. I don't understand why you wouldn't issue the "callRemote" calls concurrently, that seems like it's missing the point of writing non-blocking code.

Exactly. That's the point. It is beneficial only when you have a huge number of requests relative to the number of platform threads.

The idea is that you can map a million of these virtual threads on to a small number of platform threads, and the JVM will schedule the work for you, to achieve maximum throughput without needing to write any complicated code. Virtual threads are for high throughput on concurrent blocking requests.

EDIT: the sequential calls to "callRemote" still process in sequence, blocking on each call. Don't get confused there. But the overall HTTP request itself is running on a virtual thread and does not block other HTTP requests while waiting for the callRemote invocations to complete.

> EDIT: the sequential calls to "callRemote" still process in sequence, blocking on each call. Don't get confused there. But the overall HTTP request itself is running on a virtual thread and does not block other HTTP requests while waiting for the callRemote invocations to complete.

Yeah I got that. But then the use case seems to be much narrower, because not every scenario is one where many concurrent requests come in at the same time. If you write non-blocking code you'll get high throughput no matter the number of requests. Ok, maybe non-blocking code is harder to write (and I don't think it's actually that hard if you use, say, Kotlin coroutines), but honestly this seems to me like something that developers should learn eventually anyway.

Or maybe it's me being weird. But I remember 10 years ago when node.js was being hyped for being "fast" because it was "async" and now suddenly using threads and blocking operations is again all the rage now.

The difference now is that it's implemented in the JVM instead of a library / framework. It is easier, simpler, and probably more efficient. You can get higher throughput from existing code with minor refactoring.

  Thread thread = Thread.ofVirtual().start(() -> System.out.println("Hello"));
  thread.join();

Also there are some traceability issues involved in using asynchronous APIs: "In the asynchronous style, each stage of a request might execute on a different thread, and every thread runs stages belonging to different requests in an interleaved fashion. This has deep implications for understanding program behavior: Stack traces provide no usable context, debuggers cannot step through request-handling logic, and profilers cannot associate an operation's cost with its caller. "

https://openjdk.org/jeps/444

In theory an async-aware runtime can stitch together a coherent and useful backtrace, but in practice most legacy tooling won’t :(

Collecting those potentially very expensive stack traces, specially in those highly concurrent environments with high throughput requirements would be great for debugging but probably is gonna kill the performance of the system. But it would be nice (even if as a very heavy hammer) as an option.

It removes the need to write async code in many cases. It is a more straightforward and efficient way to accomplish what we have already been doing for years on the JVM.

You don't need to switch frameworks

https://spring.io/blog/2022/10/11/embracing-virtual-threads

You certainly could make the callRemote calls in parallel, and there are easy and safe ways to do that (good old CompletableFuture or the new structured concurrency stuff). But doing that or not is completely independent of using Nima, so I think here they're just showing some very simple code, because this is an example of using Nima, not an example of writing concurrent code in general.

Because concurrent calls are confusing and create bugs.

The point this train wreck of an example is trying to make (and failing) is that it's fine to write sequential blocking code in your request handlers. They can take as long as they want ... seconds, minutes, days .... because the handler threads are now an infinite resource as far as the JVM is concerned.