History

While Kafkorama is designed for real-time messaging, some messages are retained in memory for a configurable period of time, as described below.

Snapshot Messages

For each subject X, every node in a Kafkorama cluster maintains a snapshot message — the most recent message received with the retained property set to true. This applies to both QoS levels: Standard Message Delivery and Guaranteed Message Delivery.

The table below illustrates how the snapshot message for the subject /Stocks/NYSE/IBM evolves as new messages are received:

When a client connects or reconnects to a node in the Kafkorama cluster and subscribes to a subject, that node first sends the current snapshot message (if available). Subsequently, the client receives any new real-time messages for that subject as soon as they are published, including messages originating from Apache Kafka®.

Historical Messages

For clusters using Guaranteed Message Delivery, each node in the cluster maintains a history of recent messages for each subject. If a client experiences a failure and reconnects, it automatically retrieves missed messages from this history. These recovered messages are marked with the recovered message type (see the Messages section).

Additionally, the client libraries provide a method to request a number of historical messages before receiving real-time updates:

void subscribeWithHistory(
  List<String> subjects, 
  int numberOfHistoricalMessages
)

If the REST Client Interface is enabled, you can retrieve a snapshot message for a subject via a HTTP Request as shown here.