Batches
The batch section allows you to create collections of records that will be, later on, inserted into the platform.
Overview
As we can see in the screenshot above, this overview contains all batches that have been set up in our account.
A batch can have one of the following statuses:
| Status | Description |
|---|---|
| DRAFT | The initial status of any batch when we create it |
| READY | The batch is ready to be processed |
| PROCESSING | The batch is being processed |
| FINISHED | The batch has been successfully processed |
| ERROR | There was a problem while processing the batch |
Batch information
If you click on the config button of one of the batches, you will see the possible options that can be performed.
The common actions are:
| Type | Description |
|---|---|
| Edit/View | This will take us to the wizard process |
| Clone | Generates a copy of the batch |
| Run | Changes the status of the batch to READY. Once this is done, the batch cannot be modified anymore - Only in Draft status |
| Delete | Deletes the batch - Only in Draft/Finished/Error statuses |
New batch
When we click on the “Start Wizard” button, we are redirected to the wizard process to create a new batch. Creating a new batch is divided into different steps:
- Information: here we provide some information about our batch process: a name and an optional description
- Upload format: in this part, we define the format of the payload that we are going to upload. This format can
be
JSON,XMLorCSV. - Select interface/flow: here we select the interface that we will use for mapping the data and the flow which the
data will be inserted into. This interface must contain the scope
batch. For more information about interfaces, please visit this link. - Upload file: in this step, we can either upload the file that contains our payload or paste it directly into the editor.
- File names: in this part, we can specify custom names for the input and output files.
- Overview: in the last step, we can see an overview of all data that has been entered in the previous steps. Also, we can launch our batch process by clicking on the “Run” button.
Location of batches
It is possible to access anytime all batches that have been created in the account. These are available within the “batch” folder of the file manager. For further information about the file manager, please refer to this post.
Batch version 2
This new version improves the way batches are inserted in Flowize, so now you can send bigger files with more records and also, they will be inserted faster. Version 2 also allows you to send the batch file as a file in the request, sending it as form-data.
v1 and v2 Comparison
| File type | File size | Records | v1 processing time | v2 processing time |
|---|---|---|---|---|
| JSON | 20MB | 96000 | 05h 10m 51s | 12m 21s |
| JSON | 40MB | 192000 | 09h 10m 51s | 17m 28s |
- In designer, only the first 20 records of the batch will be inserted and executed.
- Currently, v2 works only for JSON and CSV interfaces
How to use it?
To use it, go to your interface details, and in the API endpoints, you will see a v2 link. Just use this link instead of the v1 link . The other process is all the same.
Header (optional)
When you are making a POST request for the batch process to start, you can send the header 'X-Batch-Status: READY'
in your request to instantly start the batch process.
Batch report
In the batch report, you can find details about the batch process.
You can see when the batch started, when it finished and how long did it take.
| Name | Description |
|---|---|
| Created | When Flowize received the batch |
| Ready | Set to begin the start process |
| Started | The time when the process started |
| Finished | The time when the process finished |
Once the process is finished, Flowize will automatically create 4 files for you.
| File Name | Description |
|---|---|
| input | Input file will store the data that you send via POST request. |
| output | Output file will show the list of UUIDs of data records. |
| process | Process log will show you a detailed report for each of the records. |
| error | The error log will appear only if an error occurs, and it will show the details of the error. |
Send a batch request with a file via postman.
For the batch v2, you also are able to attach a JSON or CSV file in your request as form-data.
Click on body > form-data > file > select the file
