Core Parameters

The core parameters of the Kafkorama Gateway are described below.

LicenseKey

The license key consists of a sequence of numbers and letters. The license keys are issued by Kafkorama either for evaluation, development, or production usage of Kafkorama. To obtain a license key, please contact Kafkorama at support@kafkorama.com.

Memory

The memory size defined by this parameter can be expressed in megabytes, MB.

For example, to allocate 512 megabytes for Kafkorama, configure the parameter Memory as follows:

Memory = 512 MB

In a production environment it is recommended to use at least 8192 MB (i.e. 8 GB) or more depending on the load of data on the server and how many simultaneous clients will be connected.

Listen

The format of each address of the list defined by this parameter is either IP Address:Port or DNS Name:Port.

For example, to listen for clients on the port 80 of the address 192.168.1.1, use 192.168.1.1:80. Likewise, to listen for clients on the port 8800 of the address push.example.com, use push.example.com:8800. IPv6 addresses must be enclosed in square brackets as follows [2001:db8::a00:20ff:fea7:ccea]:80.

If you specify an address without a port, the default port 80 will be used.

By specifying an IP address, Kafkorama will bind only to that IP address. The wildcard address (*) will enable Kafkorama to bind to all available IP addresses of the machine.

ListenEncrypted

The same conventions applies as for the Listen parameter, except the fact that if you specify an address without a port, the default port 443 will be used.

KeyStore

The keystore file must be configured using absolute paths. For example:

The keystore must contain a SSL certificate for each address used in the configuration of the following parameters:

How to add a certificate to the keystore

Suppose the following DNS address push.example.com resolves to the IP address 192.168.1.1, and vice-versa, the IP address 192.168.1.1 resolves to the DNS address push.example.com.

If an address appears in the configuration of any of the parameters used to define encrypted connections, then the keystore file must contain an SSL certificate for that address. If the address is specified by its DNS name, say push.example.com, then its certificate entry in the keystore must have as alias the string push.example.com. If the address is specified by its IP address, say 192.168.1.1, then its certificate entry in the keystore must have as alias the string 192.168.1.1.

To create a certificate for an address in the keystore, there are two possibilities: either use a self-signed certificate for that address or use a certificate signed by a Certificate Authority (CA).

In the next two subsections we assume that the keystore file is named mykeystore.jks and the address for which a certificate should be added in the keystore is specified either by DNS name as push.example.com or by IP address as 192.168.1.1.

Adding a self-signed certificate to the keystore

Run one of following depending on how the address was specified in the configuration file:

You will be asked to set a password for the keystore if the file mykeystore.jks does not exist, or to enter the keystore password if the keystore file already exists and contains other certificate entries. This password must be used to configure the parameter KeyStorePassword.

Adding a CA-signed certificate to the keystore

To securely configure your Kafkorama Gateway for the domain push.example.com, you’ll need to configure the Kafkorama Gateway using something like:

ListenEncrypted = push.example.com:443
KeyStore = /my/path/mykeystore.jks
KeyStorePassword = mypassword

where

• ListenEncrypted: Specifies the domain and port (443 is standard for HTTPS) on which the server listens for encrypted connections
• KeyStore: Indicates the absolute path to your JKS file
• KeyStorePassword: Provides the password set for your keystore

Create a Java KeyStore (JKS):

• File Name: Choose a name for your keystore, e.g., mykeystore.jks
• Password: Set a password for your keystore, e.g., mypassword
• Certificate Entry: Ensure the keystore contains an entry with the alias push.example.com

To create this JKS keystore with the specified file name, password, and entry alias, you will work with several key files as follows:

• Private Key (push.example.com.key): This file contains the private key associated with your domain.
• Certificate Signing Request (CSR) (push.example.com.csr): This file is generated using your private key and contains information about your domain and organization. The CSR is sent to a Certificate Authority (CA) to request a signed certificate.
• Signed Certificate (push.example.com.crt): After validating your CSR, the CA provides this file. It contains your public key, domain information, and the CA’s digital signature, verifying the authenticity of your domain.
• Intermediate Certificate (intermediary.crt): Many CAs use intermediate certificates to enhance security.

STEP 1: Prepare the Certificate Chain (If Applicable):

If the Certificate Authority provided you with an Intermediate Certificate (intermediary.crt) in addition to the Signed Certificate (push.example.com.crt), then you will have to chain your Signed Certificate with the Intermediary Certificate. If you have not received an Intermediary Certificate, then just skip this step.

To chain your Signed Certificate with the Intermediate Certificate, append the Intermediate Certificate (intermediary.crt) and the Certificate Signing Request (CSR) (push.example.com.csr) to the Signed Certificate (push.example.com.crt) as follows:

cat intermediary.crt push.example.com.csr >> push.example.com.crt

STEP 2: Convert the Signed Certificate to PKCS#12 Format:

To convert the Signed Certificate to PKCS#12 format use the following command:

openssl pkcs12 -export \
  -in push.example.com.crt \
  -inkey push.example.com.key \
  -out push.example.com.pkcs12 \
  -name push.example.com \
  -passout pass:mypassword

This will produce the file push.example.com.pkcs12

STEP 3: Import into JKS keystore with specified alias:

Import the pkcs#12 certificate obtained in the previous step to the keystore, assigning a specific alias. To import the certificate with the alias push.example.com from the PKCS#12 file and assign it a new alias push.example.com in the JKS keystore, execute:

keytool -importkeystore \
  -srckeystore push.example.com.pkcs12 \
  -srcstoretype PKCS12 \
  -srcstorepass mypassword \
  -srcalias "push.example.com" \
  -destkeystore mykeystore.jks \
  -deststoretype JKS \
  -deststorepass mypassword \
  -destalias "push.example.com"

This will produce the keystore file mykeystore.jks that you will use in the parameter KeyStore as shown above.

You can verify that you JKS keystore contains the specified alias with the command:

keytool -list -keystore mykeystore.jks -storepass mypassword

Using a Wildcard CA-signed certificate

This guide details the process of configuring two instances of the Kafkorama Gateway (push1.example.com and push2.example.com) using a single wildcard SSL certificate (*.example.com) to enable secure communications.

Configure the first Kafkorama Gateway instance like:

ListenEncrypted = push1.example.com:443
KeyStore = /my/path/mykeystore.jks
KeyStorePassword = mypassword

and in the second Kafkorama Gateway instance like:

ListenEncrypted = push2.example.com:443
KeyStore = /my/path/mykeystore.jks
KeyStorePassword = mypassword

where

• ListenEncrypted: Specifies the domain and port (443 is standard for HTTPS) on which the server listens for encrypted connections
• KeyStore: Indicates the absolute path to your JKS file
• KeyStorePassword: Provides the password set for your keystore

Create a Java KeyStore (JKS):

• File Name: Choose a name for your keystore, e.g., mykeystore.jks
• Password: Set a password for your keystore, e.g., mypassword
• Certificate Entry: Ensure the keystore contains two entries with the aliases push1.example.com and push2.example.com

To create this JKS keystore with the specified file name, password, and entry aliases, you will work with several key files as follows:

• Private Key (wildcard.example.com.key): This file contains the private key associated with your domain.
• Certificate Signing Request (CSR) (wildcard.example.com.csr): This file is generated using your private key and contains information about your domain and organization. The CSR is sent to a Certificate Authority (CA) to request a signed certificate.
• Signed Certificate (wildcard.example.com.crt): After validating your CSR, the CA provides this file. It contains your public key, domain information, and the CA’s digital signature, verifying the authenticity of your domain.
• Intermediate Certificate (intermediary.crt): Many CAs use intermediate certificates to enhance security.

STEP 1: Prepare the Certificate Chain (If Applicable):

If the Certificate Authority provided you with an Intermediate Certificate (intermediary.crt) in addition to the Signed Certificate (wildcard.example.com.crt), then you will have to chain your Signed Certificate with the Intermediary Certificate. If you have not received an Intermediary Certificate, then just skip this step.

To chain your Signed Certificate with the Intermediate Certificate, append the Intermediate Certificate (intermediary.crt) and the Certificate Signing Request (CSR) (wildcard.example.com.csr) to the Signed Certificate (wildcard.example.com.crt) as follows:

cat intermediary.crt wildcard.example.com.csr >> wildcard.example.com.crt

STEP 2: Convert the Signed Certificate to PKCS#12 Format:

To convert the Signed Certificate to PKCS#12 format use the following command:

openssl pkcs12 -export \
  -in wildcard.example.com.crt \
  -inkey wildcard.example.com.key \
  -out wildcard.example.com.pkcs12 \
  -name wildcard.example.com \
  -passout pass:mypassword

This will produce the file wildcard.example.com.pkcs12

STEP 3: Import into JKS keystore with specified aliases:

Import the pkcs#12 certificate obtained in the previous step to the keystore, assigning a specific alias for each Kafkorama Gateway instance. This process allows you to reference each certificate individually within your Kafkorama Gateway instances.

To import the certificate with the alias wildcard.example.com from the PKCS#12 file and assign it a new alias push1.example.com in the JKS keystore, execute:

keytool -importkeystore \
  -srckeystore wildcard.example.com.pkcs12 \
  -srcstoretype PKCS12 \
  -srcstorepass mypassword \
  -srcalias "wildcard.example.com" \
  -destkeystore mykeystore.jks \
  -deststoretype JKS \
  -deststorepass mypassword \
  -destalias "push1.example.com"

Repeat this command for each desired alias, changing the -destalias parameter accordingly. For instance, to add another alias push2.example.com:

keytool -importkeystore \
  -srckeystore wildcard.example.com.pkcs12 \
  -srcstoretype PKCS12 \
  -srcstorepass mypassword \
  -srcalias "wildcard.example.com" \
  -destkeystore mykeystore.jks \
  -deststoretype JKS \
  -deststorepass mypassword \
  -destalias "push2.example.com"

This will produce the keystore file mykeystore.jks with two aliases push1.example.com and push2.example.com that you will use in the parameter KeyStore of each of two Kafkorama instances as shown above.

You can verify that you JKS keystore contains the specified aliases with the command:

keytool -list -keystore mykeystore.jks -storepass mypassword

KeyStorePassword

Configure this parameter with the password defined when you created the keystore file. The keystore file is created when adding the first certificate entry to it as explained in the description of the KeyStore parameter.

Monitor

The currently available monitoring options are JMX, HTTP, and Prometheus. Use comma separated values to configure multiple monitoring options. For example, to configure JMX and Prometheus, use the following configuration:

Monitor = JMX, Prometheus

MonitorUsername

MonitorPassword

MonitorJMX.Listen

The format of the address used by this parameter is the same as the format of the Listen parameter.

The jconsole utility that is freely available with OpenJDK can be used for JMX monitoring. There are also many commercial tools that can be used for JMX that provide enhanced functionality like dashboards and database persistence.

hostname -i

MonitorJMX.Authentication

This parameter can have two values: true or false. If set on true, then, in order to access the JMX monitoring, you should use the monitoring user name specified with the MonitorUsername parameter and the monitoring password specified with the MonitorPassword parameter.

MonitorJMX.Encryption

This parameter can have two values: true or false. If set on true, then a JMX client will connect to the JMX monitoring through a SSL/TLS encrypted connection. The use of an encrypted connection is especially recommended for JMX remote monitoring from an insecure network, including Internet.

If this parameter is set on true, then the address used by the parameter MonitorJMX.Listen must have a certificate entry in the keystore file defined by the parameter KeyStore. The certificate entry of the keystore must be created with the alias jmx as explained in the description of the KeyStore parameter.

Secure JMX monitoring over insecure networks

In this subsection it is assumed that the JMX client is jconsole. For other JMX clients you can also use the steps below, but you may have to adapt the configuration to suit your specific JMX client.

Supposing the file name of the keystore defined by the KeyStore parameter is mykeystore.jks, and supposing the keystore mykeystore.jks includes a certificate entry for the address used in the configuration of the parameter MonitorJMX.Listen having as alias jmx, use the following steps to establish a secure JMX connection:

1. Create a truststore for the JMX client. The truststore is a special keystore that can verify the trusted SSL certificates. The trustsore will be used by the JMX client. Supposing that the file name used for the truststore is mytruststore.jks, run the following two commands to create the truststore:

   keytool -export \
   -alias jmx \
   -keystore mykeystore.jks \
   -rfc -file temp.cer
   
   keytool -import \
   -alias jmx \
   -file temp.cer \
   -keystore mytruststore.jks
   

   For the first command, use the password of the keystore as defined by the KeyStorePassword parameter. For the second command, use a new password, say mytruststore-password.

2. Generate a new keystore for the JMX client having a certificate entry with the alias jmx as follows:

   keytool -genkeypair \
   -alias jmx \
   -keyalg RSA \
   -validity 3600 \
   -keystore clientkeystore.jks
   

   where clientkeystore.jsk is the name of the new keystore. For this command, use a new password, say clientkeystore-password.

3. To securely connect to the JMX monitoring service of Kafkorama, execute the following command:

   jconsole \
   -J-Djavax.net.ssl.keyStore=clientkeystore.jks \
   -J-Djavax.net.ssl.keyStorePassword=clientkeystore-password \
   -J-Djavax.net.ssl.trustStore=mystruststore.jks \
   -J-Djavax.net.ssl.trustStorePassword=mytruststore-password
   

MonitorHTTP.Listen

The format of the address used by this parameter is the same as the format of the Listen parameter.

Accessing the HTTP Monitoring Service

Supposing you have the following monitoring related configuration:

Monitor = HTTP
MonitorUsername = admin
MonitorPassword = pass
MonitorHTTP.Listen = push.example.com:4000
MonitorHTTP.Authentication = true
MonitorHTTP.Encryption = false

Then you can access the HTTP monitoring using the following URL:

http://push.example.com:4000/stats?username=admin&password=pass

Opening the URL above will produce an output with the following format:

<fieldname1>:<value1> <fieldname2>:<value2> ... <fieldnameN>:<valueN>

where each field corresponds to one of the following stats:

• Average
• Standard Deviation
• Maximum

applied to one of the following metrics:

• The number of connected clients
• The number of subjects
• The number of connections per second
• The number of disconnections per second
• The number of incoming messages per second received from publishers
• The number of outgoing messages per second sent to clients
• The number of incoming bytes per second received from publishers
• The number of outgoing bytes per second sent to clients

for one of the following periods of time:

• Current
• Since startup
• Last minute, last 5 minutes, and last 15 minutes
• Last hour, last 5 hours, and last 15 hours
• Last day, last 5 days, and last 15 days
• Last month, last 5 months, and last 15 months

XML and JSON Output Format

You can also retrieve data in XML and JSON format. Append a view parameter to the URL above with the value xml or json. For example, to retrieve the monitoring data in XML format use:

http://push.example.com:4000/stats?username=admin&password=pass&view=xml

Filters

You can filter the monitoring data by indicator and/or stats and/or period of time. To do so, add to the URL above one or more of the GET parameters listed in filters table below.

For example, to retrieve the maximum number of concurrent clients in the last 15 minutes, use:

http://push.example.com:4000/stats?username=admin&password=pass&indicator=ConnectedSessions&statistic=MAX&period=Last.15.Minute

Another example. To retrieve the average for all indicators for all available periods of time, use:

http://push.example.com:4000/stats?username=admin&password=pass&statistic=AVG

Secure HTTP monitoring over insecure networks

To securely monitor Kafkorama over an insecure network such as Internet, configure

MonitorHTTP.Encryption = true

See the description of the MonitorHTTP.Encryption parameter for more details, then access the HTTP monitoring service using over https as follows:

https://push.example.com:4000/stats?username=user&password=pass

MonitorHTTP.Authentication

This parameter can have two values: true or false. If set on true then, in order to access the HTTP monitoring, you should use the monitoring user name specified with the MonitorUsername parameter and the monitoring password specified with the MonitorPassword parameter.

MonitorHTTP.Encryption

This parameter can have two values: true or false. If set on true, then an HTTP client will connect to the HTTP monitoring through a SSL/TLS encrypted connection. The use of an encrypted connection is especially recommended for HTTP remote monitoring from an insecure network, including Internet.

If this parameter is set on true, then the address used by the parameter MonitorHTTP.Listen must have a certificate entry in the keystore file defined by the parameter KeyStore.

MonitorPrometheus.Listen

The format of the address used by this parameter is the same as the format of the Listen parameter.

MonitorPrometheus.Authentication

This parameter can have two values: true or false. If set on true, then, in order to access the Prometheus monitoring, you should use the monitoring user name specified with the MonitorUsername parameter and the monitoring password specified with the MonitorPassword parameter.

MonitorPrometheus.Encryption

This parameter can have two values: true or false. If set on true, then a Prometheus client will connect to the Prometheus monitoring through a SSL/TLS encrypted connection. The use of an encrypted connection is especially recommended for Prometheus remote monitoring from an insecure network, including Internet.

If this parameter is set on true, then the address used by the parameter MonitorPrometheus.Listen must have a certificate entry in the keystore file defined by the parameter KeyStore. The certificate entry of the keystore must be created with the alias prometheus as explained in the description of the KeyStore parameter.

LogFolder

If not configured, Kafkorama will use the default folder logs relative to the directory path used to start Kafkorama Server.

The log folder can be configured using absolute paths as follows:

LogLevel

The following levels are available:

• TRACE (most verbose)
• DEBUG
• INFO (recommended for production use)
• WARN
• ERROR (least verbose)

LogRotateLimit

The value of this parameter can be expressed in gigabytes, megabytes, and kilobytes by using the following postfixes: GB for gigabytes, MB for megabytes, or KB for kilobytes.

If the log file reaches the size provided by this parameter, then Kafkorama will automatically create a new log file. The previous log files are preserved on disk up to the number of log files defined by the LogRotateFileCount parameter.

LogRotateTime

The value of this parameter can be expressed in minutes (m), hours (h), days (D), weeks (W), months (M), or years (Y).

For example, in order to record the logs in a separate file every day use:

LogRotateTime = 1 D

To record the logs in a separate file every 4 hours use:

LogRotateTime = 4 h

The previous log files are preserved on disk up to the number of log files defined by the LogRotateFileCount parameter.

This parameter takes precedence over the parameter LogRotateLimit. Therefore, if this parameter is configured, then the parameter LogRotateLimit is ignored.

LogRotateFileCount

If the number of log files produced by log rotation defined by the parameters LogRotateTime or LogRotateLimit reaches the value of this parameter, then the oldest log file is removed whenever a new log file is created such that the total number of logs files will not exceed the value of this parameter.

DocumentRoot

This parameter is optional. If not supplied, the default folder html relative to the directory path used to start the Kafkorama is used.

ClusterPassword

The servers of a Kafkorama cluster communicate with each other. The password defined by this parameter is used to prevent unauthorized interference into the inter-cluster communication.

ClusterDeliveryMode

This parameter is used to define the quality of the service of the Kafkorama cluster. The possible values are:

Using Standard Message Delivery, in the case of a fail-over reconnection, the client will get the latest retained message available for each of its subscribed subjects.

Using Guaranteed Message Delivery, in the case of a fail-over reconnection, the client will get not only the latest retained message available for each of its subscribed subjects, but also any message received by the cluster during the fail-over period for the subscribed subjects of the client.

ClusterMemberListen

Each server of the Kafkorama cluster communicates with the other members of the cluster, including with itself, along TCP/IP connections. This parameter represents the address used to listen for incoming connections from the other cluster members, including from itself.

The format of the address specified by this parameter is the same as the format of the address specified by the Listen parameter.

ClusterMemberListen = push.example.com:8801

ClusterMembers

Each cluster member is specified by its inter-cluster listen address defined by the parameter ClusterMemberListen.

For developing and testing purposes, you could deploy the Kafkorama cluster on the single machine. For example:

ClusterMembers = 192.168.1.1:7701, 192.168.1.1:8801, 192.168.1.1:9901

For production deployments, you should ideally deploy the Kafkorama cluster such that each of its servers will run on a different machine. For example:

ClusterMembers = 192.168.1.1:8801, 192.168.1.2:8801, 192.168.1.3:8801

ClusterSeedMemberCount

For clustering using Guaranteed Message Delivery (see ClusterDeliveryMode), this parameter defines the number of seed nodes to be used for the cluster. The possible values for this parameter are 3 and 5.

For most cases, it is sufficient to use the default value of this parameter.

See the clustering section to learn more about seed nodes before configuring this parameter.

ClusterMemberCoordinationFolder

If not configured, the default folder coordination_logs relative to the directory path used to start the Kafkorama Gateway will be used.

The coordination log folder can be configured using absolute paths as follows:

Entitlement

To define which users of your application have access to which subjects, you can define one of the following Entitlement types:

• None allows any client to subscribe to and publish on any subject.
• Basic allows any client to subscribe to any subject. However, publication is allowed only from the clients which authenticate with the token defined by the parameter EntitlementAllowToken.
• JWT allows clients to subscribe and publish based on JWT tokens using the Kafkorama JWT Authorization add-on. To start using JWT authorization with Kafkorama, please read the JWT Authorization add-on documentation.
• Custom allows you to define your own entitlement rules. You can use the Server Extensions API to build an extension for Kafkorama to authorize users to subscribe to or publish on certain subjects.

EntitlementAllowToken

This parameter applies only if you configure the Entitlement parameter as Basic. See the semantics of this entitlement token in the definition of the entitlement type Basic in the description of the parameter Entitlement.