Version: Next

Create a Trigger

PUT/­api/­v1/­datasets/­<organisation>/­<dataset_name>/­triggers
Permissions required: Triggers admin, View labels
curl -X PUT 'https://reinfer.io/api/v1/datasets/org1/collateral/triggers' \
-H "Authorization: Bearer $REINFER_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"trigger": {
"comment_filter": {
"user_properties": {
"number:Spend": {
"maximum": 100000,
"minimum": 100
},
"number:Transactions": {
"one_of": [
1
]
},
"string:Country": {
"one_of": [
"uk",
"de"
]
}
}
},
"description": "Used by ACME RPA to create tickets for disputes.",
"model": {
"label_thresholds": [
{
"name": [
"Some Label"
],
"threshold": 0.37
},
{
"name": [
"Another Label"
],
"threshold": 0.46
},
{
"name": [
"Parent Label",
"Child Label"
],
"threshold": 0.41
}
],
"version": 8
},
"name": "dispute",
"title": "Collateral Disputes"
}
}'

A trigger defines a stream of comments from a dataset that can be used to "trigger" downstream actions in an automated process. It enables persistent, stateful iteration through comments in a dataset, with predicted labels and entities computed using a pinned model.

Once created, the fetch and advance methods can be used to iterate through comments.

NameTypeRequiredDescription
namestringyesAPI name for the trigger, used in URLs. Must be unique within a dataset and must match [A-Za-z0-9-_]{1,256}.
titlestringnoOne-line human-readable title for the trigger.
descriptionstringnoA longer description of the trigger.
modelModelnoIf specified, comments fetched from this trigger will contain predictions from a pinned model.
comment_filterCommentFilternoIf specified, comments not matching the filter will not be returned.

Where Model has the following format:

NameTypeRequiredDescription
versionintegeryesA model version that has been pinned via the Models page.
label_thresholdsarray<LabelThreshold>noIf set, only values matching the given label_thresholds are returned. If not set, all labels and all prediction values will be returned.

Where LabelThreshold has the following format:

NameTypeRequiredDescription
namearray<string>yesThe name of the label to be returned, formatted as a list of hierarchical labels. For instance, the label "Some Label" will have the format ["Some Label"], and the label "Parent Label > Child Label" will have the format ["Parent Label", "Child Label"].
thresholdnumberyesThe confidence threshold to use for the label (a number between 0.0 and 1.0). The label will only be returned for a comment if its prediction is above this threshold.

Where CommentFilter has the following format:

NameTypeRequiredDescription
user_propertiesUserPropertyFilternoA filter that applies to the user properties of a comment. For more on user properties, see Comment structure.

The UserPropertyFilter is a map of user property name to filter. String properties may be filtered to values in a set ({"one_of": ["val_1", "val_2"]}). Number properties may be filtered either to values in a set ({"one_of": [123, 456]}) or to a range ({"minimum": 123, "maximum": 456}).

Get a Trigger

GET/­api/­v1/­datasets/­<organisation>/­<dataset_name>/­triggers/­<trigger_name>
Permissions required: View labels, View triggers
curl -X GET 'https://reinfer.io/api/v1/datasets/org1/collateral/triggers/dispute' \
-H "Authorization: Bearer $REINFER_TOKEN"

Delete a Trigger

DELETE/­api/­v1/­datasets/­<organisation>/­<dataset_name>/­triggers/­<trigger_name>
Permissions required: Triggers admin, View labels
curl -X DELETE 'https://reinfer.io/api/v1/datasets/org1/collateral/triggers/dispute' \
-H "Authorization: Bearer $REINFER_TOKEN"

Fetch Comments from a Trigger

POST/­api/­v1/­datasets/­<organisation>/­<dataset_name>/­triggers/­<trigger_name>/­fetch
Permissions required: Consume triggers, View labels, View sources
curl -X POST 'https://reinfer.io/api/v1/datasets/org1/collateral/triggers/dispute/fetch' \
-H "Authorization: Bearer $REINFER_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"size": 8
}'

Once a trigger is created, it can be queried to fetch comments and their predicted labels and entities. Below are some important aspects to keep in mind when fetching comments from a trigger.

Comment Queue

The comments are fetched in the order in which they have been uploaded to Re:infer, starting from the current position of the trigger.

When a trigger is created, its initial position is set to be equal to its creation time. If needed, you can set the trigger to a different position (either forwards or backwards in time) using the reset endpoint.

In the image above, if you fetch comments after the trigger was created (1), you will get emails C and D. If you fetch comments after the trigger was moved forwards in time (2), you get email D, and if you fetch comments after the trigger was moved backwards in time (3), you get emails B, C and D. Note that the place of the email in the comment queue is determined by the time it was uploaded to Re:infer, rather than the time it was originally sent or received.

Advancing Your Position in the Queue

Since the trigger will always return comments starting from its current position, you need to explicitly advance to the next position after each fetch request by using the advance endpoint. This way the API guarantees at-least-once processing of all comments - if your application fails while processing a batch, it will pick up the same batch on restart. (Note that since an application can successfully process a comment but fail at the advance step, it is important to handle seeing a comment multiple times).

Depending on your application design, you can choose between advancing the trigger once for the whole batch (using the batch's sequence_id contained in the response), or advancing it for each individual comment (using the comment's sequence_id contained in the response).

Comment Filter

If a comment_filter was specified when creating the trigger, comments not matching the filter will not be included in the results, but will still count towards the requested size, so you may see responses where all of size comments are filtered out, leading to an empty results array. In the example below, we request a batch of 8 comments, all of which are filtered out.

{
"filtered": 8,
"results": [],
"sequence_id": "qs8QcHIBAADJ1p3W2FtmBB3QiOJsCJlR",
"status": "ok"
}

To prevent this from happening, you can set the optional max_filtered parameter, which prevents filtered comments from counting towards the requested size.

Request Format

NameTypeRequiredDescription
sizenumberyesThe number of comments to fetch for this trigger. Will return fewer if reached end of triggered batch or if comments are filtered out according to trigger filter. Max value is 1024.
max_filterednumbernoConvenience parameter for Triggers with a Comment Filter. When provided, up to max_filtered filtered comments will not count towards the requested size. This is useful if you expect a large number of comments to not match the filter. Has no effect on Triggers without a Comment Filter. Max value is 1024.

Response Format

The response contains a batch of comments (of at most size comments). If the trigger was configured with a pinned model version, the response also contains predicted labels and entities for each comment. Refer to the corresponding sections to learn more about the format of the Comment structure and predicted labels and entities.

NameTypeDescription
statusstringok if the request is successful, or error in case of an error. See the Overview to learn more about error responses.
filterednumberNumber of comments that were filtered out according to a comment filter. If the trigger was created without a filter, this number will always be 0.
sequence_idstringThe batch sequence ID. Used to acknowledge processing of this batch and advance trigger to the next batch.
resultsarray<Result>An array containing result objects.

Where Result has the following format:

NameTypeDescription
commentCommentComment data. For a detailed explanation, see Comment structure section.
sequence_idstringThe comment's sequence ID. Used to acknowledge processing of this comment and advance trigger to the next comment.
labelsarray<Label>An array containing predictions for this comment. For a detailed explanation, see the Prediction structure section.
entitiesarray<Entity>An array containing extracted entities for this comment. An array containing entitites. For a detailed explanation, see the Prediction structure section.

Advance a Trigger

POST/­api/­v1/­datasets/­<organisation>/­<dataset_name>/­triggers/­<trigger_name>/­advance
Permissions required: Consume triggers, View labels
curl -X POST 'https://reinfer.io/api/v1/datasets/org1/collateral/triggers/dispute/advance' \
-H "Authorization: Bearer $REINFER_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"sequence_id": "qs8QcHIBAADJ1p3W2FtmBB3QiOJsCJlR"
}'

Each fetch request returns a sequence_id which represents the position it has fetched up to. Passing that same sequence_id to the advance api will make sure that next time a fetch is performed on the trigger it will start from this position. You can advance to the next batch by using the current batch's sequence_id. Alternatively, you can advance to the next comment by using the current comment's sequence_id.

Since an application can successfully process a comment but fail at the advance step, it is important to handle seeing a comment multiple time on the client application side.

NameTypeRequiredDescription
sequence_idstringyesThe sequence ID to advance the trigger to.

Reset a Trigger

POST/­api/­v1/­datasets/­<organisation>/­<dataset_name>/­triggers/­<trigger_name>/­reset
Permissions required: Consume triggers, View labels
curl -X POST 'https://reinfer.io/api/v1/datasets/org1/collateral/triggers/dispute/reset' \
-H "Authorization: Bearer $REINFER_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"to_comment_created_at": "2020-06-03T16:05:00"
}'

A trigger can be reset to move its position backwards or forwards in time, either to repeat previously returned comments or to skip comments. The timestamp used to reset a trigger refers to the time the comments were uploaded (i.e. the comment's created_at property, rather than its timestamp property).

NameTypeRequiredDescription
to_comment_created_atstringyesA ISO-8601 timestamp.

The response will contain the sequence_id corresponding to the new Trigger position.

Tag an Exception

PUT/­api/­v1/­datasets/­<organisation>/­<dataset_name>/­triggers/­<trigger_name>/­exceptions
Permissions required: Edit verbatims, View triggers
curl -X PUT 'https://reinfer.io/api/v1/datasets/org1/collateral/triggers/dispute/exceptions' \
-H "Authorization: Bearer $REINFER_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"exceptions": [
{
"metadata": {
"type": "No Prediction"
},
"uid": "18ba5ce699f8da1f.abcdef0123456789"
},
{
"metadata": {
"type": "Wrong Prediction"
},
"uid": "18ba5ce699f8da1f.0123456789abcdef"
}
]
}'

This endpoint allows you to tag comments as exceptions in the platform, so that a model trainer can review and label them in order to improve the model. We recommend to tag the comments for which the model returned no predictions, and comments for which the model returned wrong predictions. (For help with designing the exception handling flow, please check the Integration Guide).

NameTypeRequiredDescription
exceptionsarray<Exception>yesA list of exceptions.

Where Exception has the following format:

NameTypeRequiredDescription
uidstringyesThe uid of the comment that should be tagged as exception.
metadataMetadatayesAn object containing exception metadata.

Where Metadata has the following format:

NameTypeRequiredDescription
typestringyesThe exception type will be available as a filter property in the Re:infer UI. The value can be an arbitrary string. Please choose a short, easy-to-understand string such as "No Prediction" and "Wrong Prediction".