Baratine on GitHub

In-Memory Primer

In-memory doesn’t mean operating without persistence; it means operating primarily from memory and using the disk as a backup. Because the critical path never has to touch disk, in-memory services advertise and guarantee orders of magnitude better performance than anything out there. There is an emergence of other in-memory systems, but they’re all geared towards offline map-reduce or streaming applications. Baratine is the first to bring easy and reliable in-memory computing to real-time web services.

How In-Memory Works

In-memory needs to be reliable. This means when the service restarts or the power gets cut off, the service needs to be able to resume from where it left off without losing data. On the surface, this sounds easy but it’s an incredibly hard problem.

At the bare minimum, in-memory requires a journal log and a store. These basic principles aren’t all that too different from those behind databases. Working in-memory means that the backing store is rarely touched for most operations. Instead, requests are written to a log for replayability in the event of a crash. To be fast, the journal must be append-only, i.e. new entries are appended to the end of the log. Occasionally, the in-memory data is checkpointed, which means the data is saved to the store and the journal cleared. When a service starts up, it loads data from the store into memory and starts replaying requests from the journal to bring it back up to its latest state.

To be super fast, in-memory requires batching. Without batching, a service would need to stop, wait, and block for new requests. Waiting for requests one at a time is incredibly slow because of multi-threading inefficiencies. Batching solves this issue by bundling multiple requests into one payload. For batching to work, requests must be ordered, sequential, and queued. This is accomplished by using a queue. Requests are placed onto a queue and a reactive system processes requests off the queue with a single actor/thread. Batching combined with a journal makes in-memory fast, efficient, safe, and reliable.

Benchmarks

For most applications, the bottleneck by far is the database. In-memory eliminates this bottleneck by moving the database out of the critical path. Applications should see huge performance improvements. Here is a benchmark that does one update per request:

../_images/in-memory-vs-servlet-same-network-benchmark.png

The numbers above are for a database that is on the same network. Baratine’s lead is even more pronounced if:

  • the database is on a different network, or
  • the database is heavily-loaded

Baratine is much faster because it doesn’t need to touch the database on each and every request - it just uses its data that is residing in memory.

Baratine In-Memory Real-Time Web Services

Baratine is the first platform to make it easy to build reliable in-memory real-time web services. This means Baratine handles batching, replaying, and persistence largely behind-the-scenes away from the developer. In Baratine, a real-time in-memory web service is less than 25 lines of code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
@Asset
@Service
public class BookStore {
  private HashMap<String,Book> _bookMap = new HashMap<>();

  @Get
  public void getBook(@Query("title") String title, RequestWeb request) {
    Book book = _bookMap.get(title);
    request.ok(book);
  }

  @Get @Post
  @Modify
  public void addBook(@Query("title") String title, @Query("author") String author, RequestWeb request) {
    Book book = _bookMap.putIfAbsent(title, new Book(title, author));
    request.ok(book == null);
  }

  public static void main(String[] args) throws Exception {
    Web.include(BookStore.class);
    Web.go(args);
  }
}

The service above is in-memory, batching, journaled, persistent, reactive, and real-time. The @Modify tells Baratine to request a save of the asset.

  • It is in-memory because it uses an in-memory HashMap.
  • It is batching because it is a reactive @Service backed by a queue.
  • It is persistent because it is an @Asset. In Baratine, an asset is persisted automatically. By default, Baratine saves the fields of the asset to an embedded, fully reactive document database named Kraken.
  • It is reactive because it is a reactive @Service where a single thread handles requests off of the queue. Requests can be completed immediately or asynchronously at a later time.
  • It is real-time because it is RPC and there is no significant delay between requests. Virtually all requests are served immediately while a handful are waiting in the queue behind a checkpoint, which is completed super quickly (usually a few milliseconds).

Lifecycle Hooks

Baratine provides lifecycle annotation hooks on methods to demystify the behind-the-scenes magic.

  • @Modify: marks a method as modifying asset state, requests a save. There may be many @Modify methods called before a save.
  • @OnLoad: called to load the initial data, called once and only once during the lifetime of the service. This method may block until it completes.
  • @OnSave: called to save the data, called many times during the lifetime of the service. This method may block until it completes.
  • @BeforeBatch: called before a batch of messages are processed from the queue.
  • @AfterBatch: called after a batch of messages have been processed from the queue.
  • @OnInit: called when the service is initializing.
  • @OnActive: called when the service has been initialized, the journal has been replayed, and the service is ready to accept new requests.
  • @OnDestroy: called when the service is shutting down.
  • @OnDelete: called when the asset is being deleted.

Manual Persistence

To control persistence, you would use @OnLoad and @OnSave. The example below saves the data to a MySQL database:

Note

The example below uses JdbcService, which is only available in 1.0.1+.

@Asset
@Service
public class BookStoreOnLoad {
  @Inject @Service("jdbc:///foo")
  private JdbcService _jdbc;

  private HashMap<String,Book> _bookMap = new HashMap<>();

  @Get
  public void getBook(@Query("title") String title, RequestWeb request) {
    Book book = _bookMap.get(title);
    request.ok(book);
  }

  @Get @Post
  @Modify
  public void addBook(@Query("title") String title, @Query("author") String author, RequestWeb request) {
    Book book = _bookMap.putIfAbsent(title, new Book(title, author));
    request.ok(book == null);
  }

  @OnLoad
  public void onLoad(Result<Void> result) {
    _jdbc.query(result.then(this::doLoad), "SELECT * FROM test LIMIT 1");
  }

  private Void doLoad(JdbcResultSet rs) {
    for (JdbcRowSet row : rs) {
      String title = row.getString(0);
      String author = row.getString(1);

      Book book = new Book(title, author);
      _bookMap.put(title, book);
    }

    return null;
  }

  @OnSave
  public void onSave(Result<Void> result) {
    _jdbc.query(result, this::doSave);
  }

  private Void doSave(Connection conn) throws Exception {
    String sql = "INSERT INTO test VALUES(?, ?)"
                 + "ON DUPLICATE KEY UPDATE title = ?, author = ?";

    PreparedStatement stmt = conn.prepareStatement(sql);

    for (Book book : _bookMap.values()) {
      stmt.setString(1, book.getTitle());
      stmt.setString(2, book.getAuthor());
      stmt.setString(3, book.getTitle());
      stmt.setString(4, book.getAuthor());

      stmt.executeUpdate();
    }

    return null;
  }

  public static void main(String[] args) throws Exception {
    Web.property("jdbc:///foo.url", "jdbc:mysql://localhost:3306/testDb");

    Web.include(BookStoreOnLoad.class);
    Web.go(args);
  }
}

Managing Batching

Baratine places incoming requests into a queue. When a service retrieves requests from its queue, it asks for a large batch. The service will work through the entire batch without stopping or interruptions. This means:

  • it does not ask the queue again
  • it executes the requests one-by-one in one go
  • lifecycle annotations are not called until the batch is done
  • excellent CPU cache utilization because the service uses a single thread to process the batch

Batching is a performance feature of Baratine that makes real-time requests low-latency and extremely fast. It is implicit and transparent to both the service and the developer. Nevertheless, the service can choose to be notified when a batch starts and finishes with @BeforeBatch/@AfterBatch. One possible reason for doing so is to reach sublime levels of efficiency:

private LogService _log;
private ArrayList<String> _logMessages = new ArrayList<>();

@BeforeBatch
public void beforeBatch(Result<Boolean> result) {
  // do nothing
}

public void doSomething(String value, Result<Boolean> result) {
  _logMessages.add("doSomething: " + value);
  ...
  result.ok(true);
}

@AfterBatch
public void afterBatch(Result<Boolean> result) {
  _log.logList(_logMessages);
  _logMessages.clear();
}

Instead of calling the LogService on each request, the above service calls it once per batch by storing messages in a list and sending the list at the end of the batch.

Differences Versus @OnSave/@OnBatch

For most cases, @BeforeBatch/@AfterBatch are equivalent to @OnLoad/@OnSave. Most services would only care about @OnLoad/@OnSave. @BeforeBatch/@AfterBatch is useful for setting a thread-context. Baratine manages threads and there is no guarantee that the same thread will be used across different batches. A service would use @BeforeBatch/@AfterBatch to store data that depends on ThreadLocal and clear it when the batch is finished.

Managing Replays

When an abrupt shutdown occurs, there may still be requests in the journal. Baratine will need to replay those requests upon the next service startup. Baratine guarantees that requests will be executed at least once. Some requests that may have been executed right before shutdown will be executed again during the replay. Therefore, a service may want to know when it is in replay mode.

A replay happens after @OnLoad, but before @OnActive. A service would know when it is in replay mode by checking to see if @OnActive has been called. If it is still in replay mode, a service may choose to process requests differently, like choosing not to send out emails during a replay to prevent duplicate emails from going out.

@Asset
@Service
public class EmailService {
  private boolean _isReplay = true;

  @OnActive
  public void onActive(Result<Void> result) {
    _isReplay = false;
  }

  public void sendEmail(Result<Void> result) {
    if (_isReplay) {
      return;
    }

    ...
  }
}