API

Data is sent to Phirestream through its REST-based API. Any tool capable of making HTTP calls can be a client to Phirestream. Open source SDKs are available for integrating Phirestream in custom applications.

Phirestream implements a subset of the Apache Kafka’s REST API interface. Any general purpose HTTP client such as curl or client SDKs that implements this interface can be used with Phirestream.

Performance Considerations

The API methods listed below both take an optional query parameter called async. This optional parameter specifies whether the message producing to the Kafka topic will be blocking. Set async to true to not wait for confirmation the messages were produced. This will give higher throughput but will not provide indication if a message fails to be produced to the Kafka topic. Any errors encountered while producing messages will be written to Phirestream’s log.

Phirestream API

Producing messages for redaction to an Apache Kafka topic

Method	Endpoint	Description
`POST`	`/api/topics/{topic}`	Producing messages for redaction to an Apache Kafka topic

Body

The JSON-encoded messages to publish.

Path Parameters

topic - The Apache Kafka topic to produce the message to.

Query Parameters

async - Whether or not to wait for the message to be produced to the topic after redaction. Defaults to false.
profile - The name of the filter profile to apply. If not specified, the topic name is used as the name of the filter profile.
context - The redaction context.
id - The document ID.

Example Request

An example request using curl is shown below. In this request we are publishing a single message to Phirestream. The records property is an array so multiple messages can be included in a single request.

curl -X POST https://localhost:8080/topics/default \
  -H 'Content-Type: application/vnd.kafka.json.v2+json' \
  -d '{"records": [{"key": "key-1","value": "George Washington was president."}]}'

Example Response

{
   "key_schema_id":null,
   "value_schema_id":null,
   "offsets":[
      {
         "partition":2,
         "offset":100
      },
      {
         "partition":1,
         "offset":101
      }
   ]
}

If we look at the content of mytopic on the Apache Kafka broker we will see a single message with the value {{{REDACTED-entity}}} was president. Phirestream received the request, processed the person’s name as sensitive information, and published the modified data to Apache Kafka.

Produce messages for redaction to an Apache Kafka topic partition

Method	Endpoint	Description
`POST`	`/api/topics/{topic}/partitions/{partition_id}`	Producing messages for redaction to an Apache Kafka topic partition