Skip to main content

CLI Batch Upload

The CLI allows you to upload comments (including pre-labelled comments) in batch. In addition to importing data into Re:infer in those cases where a live connection is not required, it can be used to upload pre-existing training data into Re:infer, or to overwrite existing comments or labels in Re:infer.

Setup#

Download the CLI tool for your platform (the CLI tool is available for Linux, Mac, and Windows).

Configure the command line tool:

re config add --name reinfer --endpoint https://reinfer.io/

You will be asked to paste your API token:

I A new context `reinfer` will be created.
* Enter API token [none]: MYSUPERSECRETTOKEN
W Be careful, API tokens are stored in cleartext in /home/username/.config/reinfer/contexts.json.
I New context `reinfer` was created.

Preparing Data#

The CLI expects data in JSONL format (also called newline-delimited JSON), where each line is a JSON value. Many tools will be able to export JSONL files out-of-the-box. Please let us know at support@reinfer.io if you have any questions.

Each line in the JSONL file represents a comment object. Each comment object should have at least a unique ID, a timestamp, and a piece of text, but can have other fields such as metadata. You can see the full list of available fields in the Comment reference.

Each line in the JSONL file should have the following format (only required fields shown). (Note that this is shown with indentation for readability, but should be all on one line in your file.)

{
"comment": {
"id": "<unique id>",
"timestamp": "<timestamp>",
"messages": [
{
"body": {
"text": "<text of the comment>"
}
}
]
}
}

If you would like to upload labels alongside comments, you can include them like so (same as above, this is shown with indentation for readability, but should be all on one line in your file):

{
"comment": {
"id": "<unique id>",
"timestamp": "<timestamp>",
"messages": [
{
"body": {
"text": "<text of the comment>"
}
}
]
},
"labelling": {
"assigned": [
{
"name": "<Your Label Name>",
"sentiment": "<positive|negative>"
},
{
"name": "<Another Label Name>",
"sentiment": "<positive|negative>"
}
]
}
}

Uploading Data#

Uploading Comments#

The command below will upload comments to the specified source. We recommend to upload comments into a new empty source, as it makes rolling back easier if something went wrong - you just delete the source.

re create comments \
--source <org_name/source_name> \
--file <file_name.jsonl>

If you want to update existing comments, you should specify the --overwrite flag. The comments will be overwritten based on the comment.id field. We recommend that you make a backup copy of the source before updating comments in order to be able to recover the original comments if something goes wrong.

Uploading Comments with Labels#

If you would like to upload labels together with your comments, you should specify a dataset into which the labels should be uploaded. The dataset should be connected to the source before you start uploading.

re create comments \
--source <org_name/source_name> \
--dataset <org_name/dataset_name> \
--file <file_name.jsonl>

You can overwrite labels on existing comments by specifying the --overwrite flag. Note that this will replace existing labels with new labels (not add existing labels to new labels). We recommend that you make a backup copy of the dataset before overwriting labels in order to be able to recover the original labels if something goes wrong.