API
Data is sent to Phirestream through its REST-based API. Any tool capable of making HTTP calls can be a client to Phirestream. Open source SDKs are available for integrating Phirestream in custom applications.
Phirestream implements a subset of the Apache Kafka’s REST API interface. Any general purpose HTTP client such as curl or client SDKs that implements this interface can be used with Phirestream.
Performance Considerations
The API methods listed below both take an optional query parameter called async. This optional parameter specifies whether the message producing to the Kafka topic will be blocking. Set async to true to not wait for confirmation the messages were produced. This will give higher throughput but will not provide indication if a message fails to be produced to the Kafka topic. Any errors encountered while producing messages will be written to Phirestream’s log.
Phirestream API
Producing messages for redaction to an Apache Kafka topic
Method | Endpoint | Description |
---|---|---|
|
| Producing messages for redaction to an Apache Kafka topic |
Body
The JSON-encoded messages to publish.
Path Parameters
topic
- The Apache Kafka topic to produce the message to.
Query Parameters
async
- Whether or not to wait for the message to be produced to the topic after redaction. Defaults to false.profile
- The name of the filter profile to apply. If not specified, the topic name is used as the name of the filter profile.context
- The redaction context.id
- The document ID.
Example Request
An example request using curl is shown below. In this request we are publishing a single message to Phirestream. The records property is an array so multiple messages can be included in a single request.
Example Response
If we look at the content of mytopic
on the Apache Kafka broker we will see a single message with the value {{{REDACTED-entity}}}
was president. Phirestream received the request, processed the person’s name as sensitive information, and published the modified data to Apache Kafka.
Produce messages for redaction to an Apache Kafka topic partition
Method | Endpoint | Description |
---|---|---|
|
| Producing messages for redaction to an Apache Kafka topic partition |
Body
The JSON-encoded messages to publish.
Path Parameters
topic
- The Apache Kafka topic to produce the message to.partition_id
- The partition to produce the message(s) to.
Query Parameters
async
- Whether or not to wait for the message to be produced to the topic after redaction. Defaults to false.profile
- The name of the filter profile to apply. If not specified, the topic name is used as the name of the filter profile.context
- The redaction context.id
- The document ID.
An example request using curl is shown below. In this request we are publishing a single message to Phirestream. The records property is an array so multiple messages can be included in a single request.
If we look at the content of mytopic
on the Apache Kafka broker we will see a single message with the value {{{REDACTED-entity}}}
was president. Phirestream received the request, processed the person’s name as sensitive information, and published the modified data to Apache Kafka.