Baratine on GitHub

Baratine Service

See also the following subsections:

What is a Baratine service?

A Baratine @Service is a loosely-coupled POJO microservice. It forms the basis of all built-in Baratine services. A Baratine service has the following properties:

  • Strict encapsulation boundary
  • An application service implementation
  • An optional application API
  • Thread safe concurrent proxy
  • Service thread for synchronization, cpu affinity, and isolation
  • Internal inbox queue to sequence and synchronize calls
  • Internal proxy and stub to marshal methods to messages

Typically, only the proxy API and service implementation are visible to the application. The inbox queue, service thread, proxies and stubs are provided by Baratine invisibly.

../../_images/service-proxy-stub.png

From an application perspective, the service looks like a single-threaded implementation called by a proxy using continuation-style methods. A hello service looks like the following:

1
2
3
4
5
6
7
 public class Hello
 {
   public void hello(Result<String> result)
   {
     result.ok("hello");
   }
 }

If you wanted to create the hello service as a standalone service, you could create a new service manager, register the hello service, and retrieve its proxy as follows:

1
2
3
4
5
6
7
8
9
public void test()
{
  Services services = Services.newManager().get();

  Hello hello = services.newService(Hello.class)
                        .as(Hello.class);

  hello.hello((value,exn)-> System.out.println("Hello: " + value));
}
  • newService() encapsulates the service with its inbox, stub, and thread.
  • as() creates a proxy.

At the minimum, a service consists of an inbox, a POJO to process requests, and an outbox. The inbox is a ring-buffer queue that allows the service to process requests serially and in batches. A single thread retrieves requests from the inbox and invokes it upon the POJO’s methods. The result is serialized to the outbox to be sent out in batches to recipients.

Service POJO

Any plain-old-java-object (POJO) annotated with @Service is a service where the object’s public methods become callable service methods.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
@Service
public class CounterService
{
  private long _count;

  public void addAndGet(long value, Result<Long> result)
  {
    _count += value;

    result.ok(_count);
  }
}

The service can be registered in the main method.

1
2
3
4
5
6
7
8
public static void main(String []args)
{
  Web.service(CounterService.class);

  Web.include(HelloRest.class);

  Web.go(args);
}

And its proxy can be injected in a web service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
public class HelloRest
{
  @Inject @Service
  CounterService _counter;

  @Get
  public void addAndGet(@Param("v") long value, Result<Long> result)
  {
    _counter.addAndGet(value, result.of());
  }
}

The service can also be looked up programmatically by obtaining a Services manager and creating a proxy.

1
2
3
// if within Baratine
MyCounter counter = Services.current()
                            .service(ServiceCounter.class);

Then a call to counter.addAndGet() would look like:

System.out.println("start");

counter.addAndGet(123, count -> {
    System.out.println("count is: " + count);
});

System.out.println("end");

The code would print out:

start
end
count is: 123

Behind the scenes:

  1. client: user calls counterService.addAndGet(123)
  2. client: client proxy serializes the call as a message and sends it to the service
  3. service: service receives message and queues it into its inbox
  4. client: caller receives OK response and returns from the call
  5. service: service’s POJO is loaded and initialized (if it hasn’t been already)
  6. service: POJO’s addAndGet(123) is invoked and response is serialized and sent back to the caller
  7. client: caller receives response and executes its Result continuation (i.e. callback)

Result is a continuation and it can be passed around to other methods/services, chained together, or even saved. It allows you to do asynchronous processing and to respond only when you are ready.

Service Proxy

The client proxy is decoupled from the service implementation. This means that the client may use any API class to call the service. Here we are casting the proxy to our MyCounter interface:

MyCounter counter = Services.current()
                            .service("/CounterService")
                            .as(MyCounter.class);

where MyCounter is the client’s API class:

public interface MyCounter
{
  public void addAndGet(long value, Result<Long> result);
}

The client’s addAndGet() does not have to match the implementation’s addAndGet(). All of the following interface methods map to the same service method:

// asynchronous call with continuation (caller does not wait for a response)
public void addAndGet(long value, Result<Long> result);

// fire-and-forget
public void addAndGet(long value);

// blocking and synchronous (call service and wait for a result to come back)
public long addAndGet(long value);

We recommend using the asynchronous call inside Baratine because Baratine services should never block for for high thoroughput.

For testing, however the blocking call is useful for clients outside of the Baratine system. It’s a bad idea to use the blocking call inside a service because it would block the single-threaded service. If you need a multi-threaded service, then you should use @Workers.

Result

Result is a continuation (i.e. callback) that is executed when a response comes back to the caller. It is executed in the caller’s context.

Results have two primary methods: ok() and fail(). ok() is for a normal return. fail() is for exceptions.

It also has a convenience method for lambdas, handle(), which combines the two

1
2
3
4
5
6
7
8
9
public interface Result<X>
{
  default void ok(X value) { ... }
  default void fail(Throwable exn) { ... }

  void handle(X value, Throwable exn);

  ...
}

You can create a Result by implementing a class that extends it or with a JDK8 lambda. The lambda will use the handle method:

// JDK8 lambda
addAndGet(123, (value,exn) -> {
  System.out.println("count is: " + value);
});

The caller does not maintain state between a call and its Result because the Result is actually sent along with the call. The response includes the caller’s Result with which the caller would execute after it examines the response. This design allows for high-performance RPC and is core to Baratine’s model.

A Result may be completed right away or it may be passed to another service for it to be completed there. It can be in any position in a method’s argument list but the only restriction is that there may only be one Result per method:

// valid
public void addAndGet(long value, Result<Long> result);
public void addAndGet(Result<Long> result), long value);
public void addAndGet(long value);

// invalid
public void addAndGet(long value, Result<Long> resultA, Result<Long> resultB);

ok()

Result.ok() completes a request and sends a reply to the caller. The request remains active until it calls ok() or fail() even if the method completes.

fail()

Result.fail() completes a request with an exception and sends the exception to the caller.

Result.of()

Result.of() chains a sub-call, such as a web-service calling an authentication service or gateway service. When the sub-call completes, the of() handler processes the sub-result and returns the final result.

Chaining Results

If the Results are compatible, you may pass in an argument Result to another service for it to be completed there:

@Get
public void addAndGet(long value, Result<Long> result)
{
  _counter.addAndGet(value, result.of());
}

If the Result type is not compatible or you want to transform the result, then you would need to chain them:

@OnSave
public void onSave(Result<Boolean> result)
{
  _store.put(_storeKey, result.of(x->true));
}

The above lambda expression is a function that returns true.

Result has a of() method that helps with chaining:

@Get
public void addAndGet(long value, Result<Long> result)
{
  CounterService counter = Services.current()
                                      .service(CounterService.class);

  // 1. create a lambda that takes the result of decrementAndGet() and multiplies it by 2
  // 2. then stuff the multiplication result into addAndGet()'s Result
  me.add(result.of(count -> count * 2));
}

ResultFuture

A ResultFuture is a type of Result that is also a blocking future:

ResultFuture<Long> future = new ResultFuture<Long>();

counter.addAndGet(123, future);

future.get();

// or get with a timeout exception
future.get(5, TimeUnit.SECONDS);

In general ResultFuture is not appropriate inside of Baratine because futures are blocking. It can be useful for testing or when using Baratine services outside of Baratine itself.

Even when considering using ResultFuture, it’s better to create an interface with a synchronous return instead of using ResultFuture, because of the compile-time checking and code readability.

Service Encapsulation

To preserve the strict service encapsulation, Baratine has callback pinning and shim generation.

Callbacks must execute in the owner’s context, not the service it’s registered with. @Pin and the pin method support pinning callbacks.

Shims simplify transfering data while avoiding boilerplate transfer-object copy code. Because a service must never expose its live data outside its thread context, any getters must return copies of data or immutable objects. Shims transfer data from the service to the caller.

@Pin - callback encapsulation

In a proxy API @Pin pins a callback to a service.

@Pin preserves service isolation when using callbacks. A callback must execute in its own service’s context, with its own service’s thread; it must not use the service context that calls the callback. In other words, it works like a dynamic proxy to the calling service.

The following code creates two pinned callbacks: one for the calling registration, and one for the returned Cancel object, which is used to unregister.

1
2
3
4
public interface MyApi
{
  void register(@Pin MyCallback callback, @Pin Result<Cancel> result);
}

With the MyApi client calls register, the proxy creates a callback proxy pinned to the client. The service will call to callback proxy and the callback code will be executed in the client’s context.

The Cancel is similar, but for a callback from the service to the client. When the service returns a Cancel for the registration, the service stub will convert the Cancel into a proxy for the client. When the client cancels the registration, the cancel code will execute in the service’s context.

Programmatic Pinning

Callbacks can be pinned programmatically using the ServiceRef.

1
2
3
4
ServiceRef serviceRef = ServiceRef.current();

myCallback = serviceRef.pin(new MyCallback())
                       .as(MyCallback.class);

Shimming

Service data is encapsulated; only the service may access its own data. This encapsulation is a core requirement of Baratine’s data model. When services need to return their data, like a Book asset returning its contents, the service must return a transfer object instead of returning the data directly.

Result.okShim() helps avoid writing unnecessary transfer code by copying fields from service data directly. The service stub will copy the service data.

A Book asset might return its value using itself as a transfer object. In the following code, the get method returns a new copy of MyBook with its data copied from the service, which preserves encapsulation of the book: only the book itself sees the active book data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
@Service
public class MyBook
{
  private IdAsset id;
  private String title;

  public void get(Result<MyBook> result)
  {
    result.okShim(this);
  }
  ...
}

Services

Services is used to

  • create services programmatically
  • lookup bound services by their URL address or type
public interface Services
{
  static Services current();

  static Builder newManager();

  ServiceRef lookup(String address);

  ServiceRef.Builder newService();

  ServiceNode getNode();

  void setSystemExecutorFactory(Supplier<Executor> factory);
}

Services.lookup

A typical programmatic lookup inside a Baratine service:

Services manager = Services.current();

Hello hello = manager.lookup("pod://hello/hello")
                     .as(Hello.class);

Services.newService

An embedded service creation:

Services manager = Services.current();

Hello hello = manager.newService()
                     .address("/hello")
                     .service(new HelloImpl())
                     .build()
                     .as(Hello.class);

Services.newManager

An embedded manager creation:

Services manager = Services.newManager().build();

Services.setSystemExecutorFactory

setSystemExecutorFactory is for integration with external frameworks and for calls outside of Baratine, because Result callbacks need a thread and a context. When an external framework has its own thread and context management and the Result is executed in that framework context, the setSystemExecutorFactory can launch the Result in the proper context.

Services.getNode

ServiceNode and getNode is useful in multi-pod environments when a service needs to know which node it’s running in.

session: scheme

Session services use the “session:” scheme. A session service will instantiate a new implementation class for each client. So the session service can use local fields to manage the session.

@Service("session:///my-session")
public class MySessionImpl implements MySession
{
  private boolean _isLogin;

  public void login(Result<String> result)
  {
    _isLogin = true;

    result.ok("ok");
  }

  public void hello(Result<String> result)
  {
    if (_isLogin) {
      result.ok("hello, world");
    }
    else {
      result.ok("not logged in");
    }
  }
}

Batching

Baratine does implicit batching for high performance and efficiency that every service benefits from. For fine-grained batching control (and possibly higher efficiency), you may use @BeforeBatch and @AfterBatch:

@BeforeBatch
public void beforeBatch(Result<Boolean> result)
{
  // do nothing
}

public void log(String msg, Result<Boolean> result)
{
  _file.write(msg.getBytes());
  _file.write("\n".getBytes());

  result.ok(true);
}

@AfterBatch
public void afterBatch(Result<Boolean> result)
{
  // flush the file
  _file.flush();
}

@Workers - multi-threaded services

You may make your service multi-threaded with a @Workers annotation on the class. @Workers accepts an argument designating the maximum number of threads for the service. A single inbox serves multiple consuming worker threads.

import io.baratine.service.Service;
import io.baratine.service.Workers;

import java.sql.ResultSet;
import java.sql.Connection;
import javax.sql.DataSource;

@Workers(10)
@Service("public:///db-service")
public MyDatabaseServiceImpl
{
  private DataSource _ds;
  private Connection _conn;

  @BeforeBatch
  public void beforeBatch()
  {
    _conn = _ds.openConnection();
  }

  public int getCount(String tableName)
  {
    // javax.sql queries are blocking, so need @Workers or else you can only
    // do one query at a time
    ResultSet rs = _conn.query("SELECT count(*) from " + tableName);

    return rs.getInt(0);
  }

  @AfterBatch
  public void afterBatch()
  {
    Connection conn = _conn;
    _conn = null;

    conn.close();
  }
}

@Workers services bridge the gap between Baratine services and the outside, which usually are synchronous and blocking in nature. An @Workers service needs to be thread-safe, just like any other synchronous service. As such, it generally is not as fast as a single-threaded @Service under high concurrency. However, it does benefit from:

  1. inbox queuing for spiky loads, and
  2. can use @BeforeBatch and @AfterBatch to reduce the number of expensive operations.

@OnLookup

Baratine implements automatic partitioning based on the service URL. If Baratine does not know how to handle a particular URL, it will call the base service’s @OnLookup to return the child service instance to handle the call.

For example, a call to /counter/123 would invoke /counter‘s @OnLookup:

@Service("/counter")
public class CounterServiceImpl
{
  ...

  @OnLookup
  public Counter onLookup(String url)
  {
    return new Counter(url);
  }

  static class Counter {
    private long _count;

    public Counter(String url)
    {
    }

    @Modify
    public void addAndGet(long value, Result<Long> result)
    {
      _count += value;

      result.ok(value);
    }

    @OnLoad
    public void onLoad(Result<Void> result)
    {
      _store.get(_storeKey, result.of(count -> onLoadComplete(count));
    }

    private void onLoadComplete(Long count)
    {
      if (count != null) {
        _count = count.longValue();
      }
      else {
        _isValid = false;
      }
    }

    @OnSave
    public void onSave(Result<Void> result)
    {
      if (_isValid) {
        _store.put(_storeKey, _count,  result.of(x->null));
      }
      else {
        _store.remove(_storeKey, result.of(x->null));
      }
    }
  }
}

CounterServiceImpl returns a child Counter that Baratine will invoke incoming method calls on for that URL. The child is stored in an LRU and it may be evicted to reclaim memory. The child’s state should be saved and restored in the child’s @OnLoad and @OnSave.

Programmatic binding

Instead of using @Service to declare a service, you may do it programmatically with the Services:

Services manager = Services.current();

manager.newService()
       .service(new MyServiceImpl())
       .address("/my-service")
       .build();
  1. service() turns the POJO into a Baratine service
  2. address() assigns the URL to the service (so it can be looked up)

The URL /my-service is shorthand for local:///my-service. The bound service may be looked up, injected, and used just like any other Baratine service:

Services manager = Services.current();

MyApi proxy = manager.service("/my-service")
                     .as(MyApi.class);

Initialization in @OnInit

Services using other services often lookup inside an @OnInit method, because the Services is easily available from the context. Instead of passing around a Services during initialization, the service can discover its owning manager during any service call. Since the @OnInit method call is a service call, it has access to the context Services and ServiceRef.

import io.baratine.service.Services;
import io.baratine.service.Services;
import io.baratine.service.OnInit;

@OnInit
public void onInit()
{
  Services manager = Services.current();

  myApi = manager.service("/my-name")
                 .as(MyApi.class);
}

Persistence with @OnSave

Baratine provides annotations that signal to the service opportune times to save its internal state. Applications will generally use Vault Services persistence, which manages most of the boilerplate for loading and saving. The vault is built on top of @OnLoad and @OnSave.

When used by the vault, the service is saved using io.baratine.db.DatabaseService, or to BFS. The application may also save to an external database like MySQL.

  • @OnLoad: called when the instance is not resident in memory
  • @OnSave: called when an @Modify-annotated method has been called at least once

Your service may receive many @Modify calls but Baratine would call your @OnSave method only once per implicit batch (or once per journal roll-over if your service has a @Journal).

@Service("public:///counter")
public class CounterServiceImpl
{
  ...

  @Modify
  public void addAndGet(long value, Result<Long> result)
  {
    _value += value;

    result.ok(_value);
  }

  @OnLoad
  public void onLoad(Result<Boolean> result)
  {
    _store.get(_storeKey, count -> {
      if (count != null) {
        _count = count.longValue();
      }
      else {
        _isValid = false;
      }

      result.ok(true);
    });
  }

  @OnSave
  public void onSave(Result<Boolean> result)
  {
    if (_isValid) {
      _store.put(_storeKey, _count, () -> result.ok(true));
    }
    else {
      _store.remove(_storeKey, () -> result.ok(true));
    }
  }
}

The service should do the loading and saving asynchronously to prevent from blocking the main service thread. This means accepting a Result argument in your @OnLoad/@OnSave and passing it to the external services and completing it asynchronously.

You may also programmatically request that your @OnSave be called by calling ServiceRef.save():

ServiceRef ref = Services.currentService();
ref.save(Result.ignore());

// at this point, save request is queued in the inbox but not yet executed

Child service persistence

Child service persistence is nearly identical to regular (parent) service persistence. The only difference is that child services are stored in an LRU and may be evicted from memory; the child’s @OnSave and other life cycle events are called first before its evicted.

@Journal

@Journal is key to enabling Baratine’s reliable in-memory services. Since the journal records method calls that update the service state, it can be replayed on restart to recover the state.

The journal works with the @OnSave store to keep the number of replay messages down to an acceptable level. Without the checkpoint, the journal would need to replay every message since the service started. With the checkpoint, it only needs to replay messages since the last checkpoint.

Because services often have side-effects, and because a replayed message might duplicate a previously executed method, Baratine can inform the service when the replay is done, and when new messages are arriving. Side-effects will typically be disabled until the service is marked active.

Core journal

The core pattern for a reliable memory service includes @Journal to enable Baratine’s journaling, an @OnSave callback to save the service state to a checkpoint store, and an @OnLoad callback to load the service state from the checkpoint store.

A simple application might look like the following example. In the example, MyState is the service’s operational state, a plain Java object. For example, it might be a simulation model, or a map of shopping carts, or an auction. Since operational methods like myMethod work directly on the service model, the performance is as fast as the application code will run. The checkpoint store MyCheckpointStore might typically be a Baratine StoreService for performance.

Note that this example assumes no side-effects or idempotent side-effects. For applications with side-effects that should not be repeated, like sending mail, see the section on side-effects:

@Journal
@Service("/my-service")
public class MyService
{
  @Inject MyCheckpointStore _myStore;

  private MyState _myState;

  public String myMethod(String arg)
  {
    return _myState.doStuff(arg);
  }

  @OnLoad
  public void onLoad()
  {
    _myState = _myStore.loadState();

    if (_myState == null) {
      _myState = new MyState();
    }
  }

  @OnSave
  public void onSave()
  {
    _myStore.saveState(_myState);
  }
}

When the service starts, Baratine will call the @OnLoad callback telling the service to restore its state from the last checkpoint. The only required read from the callback store is from the single @OnLoad. Every following call either operates on memory or periodically writes to the checkpoint. Since writes don’t need to wait, this architecture avoids delays.

  1. @OnLoad - restores myState from the last checkpoint.
  2. myMethod (multiple) - replay from journal
  3. @OnActive - marker callback between replay and active
  4. myMethod (multiple) - active calls
  5. @OnSave (periodic) - periodic checkpoint
  6. myMethod (multiple) - more active calls

After the @OnLoad call, Baratine replays the saved method calls from the journal to restore the service state. To the service, the method calls look exactly like the original method calls. If a service has side-effects, it may be important to know that these method calls are replays, not originals. The distinction between the two is the reason for the @OnActive callback. If the service doesn’t care about the distinction, it doesn’t need to implement an @OnActive.

The Baratine restore model can be visualized like a complicated state machine, where the application’s model is the state, and the method calls are the state transitions.

Pre-shutdown:

s{n-1} - state after method call n-1
s{n} - checkpointed state
s{n+1} - state after method call n+1
s{n+2} - state after method call n+2
...
s{n+m} - state after method call n+m
shutdown/crash

Restore:

s’{0} - restored checkpoint state
s’{1} - state after replay 1
s’{2} - state after replay 2
...
s’{m} - state after replay m
s’{m+1} - state after active call 1

In the restore, the new state starts up with the same checkpointed state. So s{n} is the same as s'{0}. Since each replayed method acts the same on the restored state as the original method did for the original state, the final restored state s'{m} is the same as the original final state s{n+m}. In other words, the application starts in the same state as it left off; it’s fully recovered.

Side-effects

Since some services have side-effects, like sending email, a journalled service needs to distinguish replay methods from active methods. The @OnActive callback marks that transition.

If the side-effect service (or a facade) has an activate method, the service can activate the side-effect in the @OnActive callback. That pattern keeps the mainline code simple. Only the side-effect facade needs to know about the activation.

In the following example, the MyMailer might either be the mail service itself, or an application-specific facade that implements an isActive switch on the mailer methods. MyMailer might either default to inactive, as in this example, or it could be disabled either in the constructor or @PostConstruct or in the @OnInit callback. During the replay, calls to send a duplicate mail will short-circuit. After the activation, calls to send mail will go through:

public class MyService
{
  @Inject MyMailer _mail;

  ...

  public void myMethod(String arg)
  {
    _mail.sendMail(arg);
  }

  @OnActive
  public void onActive()
  {
    _mail.setActive(true);
  }
}

The mail facade might look something like:

public class MyMailer
{
  private RealMailer _delegate = ...;

  private boolean _isActive = false;

  public void setActive(boolean isActive)
  {
    _isActive = isActive;
  }

  public void sendMail(String arg)
  {
    if (_isActive) {
      _delegate.sendMail(arg);
    }
  }
}

Compare and contrast

It’s probably instructive to compare the Baratine reliable memory service model with a more traditional database or ORM model to see the differences more clearly.

The service’s own memory model is used for the mainline method calls in Baratine. There are no persistence calls to load the model, or to save the model, either directly or hidden by bytecode proxies. It’s all plain Java calls. While the persistence calls still exist, they’re split into the @OnLoad and @OnSave lifecycle calls. The mainline code isn’t tangled up with them. This plain application model opens the opportunity for simpler or more efficient models, for models designed around the problem, not the persistence framework. Potentially, this can lead to simpler code.

A journalled Baratine service must be aware of replayed methods, methods that are called twice for the same event. The replay and activation can be encapsulated, but for some people it will be a new programming model.

Database reads can be eliminated in a journalled memory service, except for a single checkpoint read on startup. Since reads are a major source of thread blocking, eliminating them allows for fast single-threaded calculation, and eliminates the delays while waiting for a read to complete. While caches can reduce read delays for an ORM application, the blocking on a cache miss generally forces a multithreaded design. Cache and ORM code also generally require the mainline method to call through the cache or ORM APIs, tangling the application’s model with the cache/ORM model. While caches and ORM frameworks do a good job of minimizing the overhead, it still exists. Compressing the layers into the application’s own model is generally simpler.

Database writes are automatically batched and compressed. The journal maintains the state reliably without requiring a write for each update. Writes can be batched at checkpoint time, and often eliminated when updating the same state before writing.

The hidden benefit is the synergy between modern CPUs and their caches and the Baratine single-threaded event method model. When a Baratine service processes a batch of methods, the CPU caches are pre-filled with the code and data to process the new call. These hidden cache hits save the CPU from loading from main heap memory. Under heavy load, meaning more method batching, a Baratine service can become more efficient, because the code and data caches are running at full capacity. Plus, the advantage of avoiding the overhead of locking for both performance and code complexity.

Embedding: creating services programmatically

Baratine can wrap services in any JVM context to solve problems like async requests, singleton services and streaming data for TCP or logging. Because clients use Java interfaces, and services are implemented as a Java class, the embedded Baratine service integrates well with existing applications.

  • asynchronous decoupling: handing off tasks to a dedicated service
  • singleton services: shared state for multiple clients
  • streaming: multiple clients like chat or logging writing to a single stream like websockets or a file

Use the Services newManager method to create an embedded service manager.

import io.baratine.service.Services;

void init()
{
  // create a new Baratine manager
  Services manager = Services.newManager()
                                         .build();

  // create a ServiceRef and client proxy for the service
  MyApi myClient = manager.newService()
                          .service(new MyServiceImpl())
                          .build()
                          .as(MyClient.class));

  // my-client calls use application APIs
  myClient.mySend("my-message");
  String myResult = client.myCall("my-arg");
  myClient.myAsyncCall("my-arg", x-> { System.out.println(x); });
}

The result is a client using the application’s own API that queues messages to the inbox to a single-threaded service implementation. Calls are plain Java calls. Methods with a return value will wait for a response using an internal future. Methods returning void like logging or streaming will send a fire-and-forget message to the service.

../../_images/service-inbox-outbox.png

Threading is managed by the created Services. When the inbox has messages, a thread will attach to the service and process the messages.

  • Client calls are thread-safe. Multiple producers add new method calls to the inbox. The service itself is single-threaded.
  • Blocking calls like TCP/websocket writes are allowed to the service without blocking the clients, unless the clients are waiting for a result.
  • Efficiency increases under heavy load because using the same service thread improves CPU caching, and because services can batch. Processing the second call is faster than the first.