Clients

Kafka is available to external clients over SASL_SSL using SCRAM-SHA-512.

Credentials

To connect to Kafka, each client will need to download their own connection credentials from the /api/v1/kafka/client endpoint.

Diagram

Get as .zip File

If you include an Accept header of application/octet-stream, you will be given a binary stream to download as a .zip file.

Request
curl -X 'GET' \
  https://localhost/api/v1/kafka/client \
  -H 'accept: application/octet-stream' \                 (1)
  -H 'Authorization: Basic YWRtaW46eUNITHBDaG12dDRuTVUwNWpaZTZBbGl0'  (2)
1 Downloads a .zip file.
2 Can use Basic (username/password) or Bearer token.

If you get a 401 Unauthorized response, you will need an administrator to grant you access to Kafka.

This will provide you with a df-kafka.zip file containing:

  • df-kafka.zip - Client certificate needed for the connection.

  • kcat.conf - A pre-configured configuration file for testing your connection with kcat (a lightweight Kafka CLI client).

Your client credentials are in the kcat.conf as sasl.username and sasl.password.

kcat.conf
# Usage: kcat -b kafka-bootstrap.localhost:443 -C -t test-topic -F kcat.conf
ssl.ca.location=df-kafka.crt
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.username=9dfb067a-fd75-4b50-a0ea-8654487babd6
sasl.password=evqp2h0mt3l9h1lkpph0

Get as JSON

If you include an Accept header of application/json, you will be given a more programmatically friendly response.

Request
curl -X 'GET' \
  https://localhost/api/v1/kafka/client \
  -H 'accept: application/json' \                   (1)
  -H 'Authorization: Basic YWRtaW46eUNITHBDaG12dDRuTVUwNWpaZTZBbGl0'  (2)
1 Ask for JSON.
2 Can use Basic (username/password) or Bearer token.

If you get a 401 Unauthorized response, you will need an administrator to grant you access to Kafka.

This will provide you with a JSON response containing the same details as the .zip (above).

{
  "bootstrapServers": "kafka-bootstrap.localhost:443",
  "certificate": "-----BEGIN CERTIFICATE-----\n<certification-body>\n-----END CERTIFICATE-----\n",
  "sasl": {
    "mechanism": "SCRAM-SHA-512",
    "username": "8a81001a-28c0-4257-a2de-7f247e42c10e",
    "password": "o5t0v8pz2etego0apqav"
  }
}

Connection Test

To test your client credentials, you can run kcat with the kcat.conf configuration file provided in the df-kafka.zip download.

In the kcat.conf file you will find an example usage in the comment at the top of the file. Copy that line and run it in your terminal in the same directory containing the kcat.conf file.

kcat -b kafka-bootstrap.localhost:443 -C -t test-topic -F kcat.conf

If successful, you should see output similar to:

% Reading configuration from file kcat.conf
% Reached end of topic test-topic-u [0] at offset 0

The offset will vary depending on the current size of the test-topic topic.