Asynchronous Result vs Blocking Futures

This is a single client ping call from a Baratine service to another Baratine service in the same JVM. In the first case, the client uses an asynchronous callback:

Result<String> resultCallback = ...;


In the second case, the client uses a blocking future:

String result = hello.hello();

For a single client thread, the results are:

  • 120k rps - blocking call or future (1 client thread)
  • 4900k rps - async result (1 client service)

For multiple clients, the results are a bit closer:

  • 380k rps - blocking call (4 client threads)
  • 1100k rps - async result (4 client services)

A client service using async calls can have multiple requests in flight simultaneously. Baratine automatically batches both the calls and the responses.

  • 5300k rps - async result (4 batched calls, 1 client service)

While the latter case shows some of the advantages of batching, the larger advantage comes from remote calls, where the TCP overhead of a single call is high, but batched calls are efficient.

Remote Client Batching

The performance improvement of batching of multiple on a single server is more dramatic for remote calls, because batching remote messages improves the slow network overhead. The following comparison is the same hello ping call as for the single server, but the service is on a remote server from the client.

  • 21k/s (1 connection)
  • 52k/s (50 connections)
  • 170k/s (50 threads, 1 connection)
  • 380k/s (50 batched calls, 1 connection)

The batched calls on a single connection are significantly more efficient using multiple independent connections.

As explained below, the batching can also be useful for services. For example, a logging service can postpone flushing to disk (or socket) until the batch is complete.