Running Processing Jobs
Pipeline processing is a processing step within a curation project: a set of operators turns data into structured outputs. It isn't a top-level navigation module, and it isn't required for every type of data.
| Data | Pipeline required? |
|---|---|
| RAW (Robot-free Ego, Robot-free UMI) | Required. RAW has no robot state, so it must go through a pipeline to produce structured data before it can be curated. |
| HDF5 (Station Lite, MOS) | Optional. HDF5 includes robot state and can be curated directly; you can still run a pipeline to enrich it. |
Where: The Pipeline button in the data workbench header opens a full-page dialog.
Dialog Layout
| Area | Contents |
|---|---|
| Top: operator management | Upload operators and choose output operator types |
| Middle: pipeline chain | A three-stage view: prerequisites → operator execution → post-processing |
| Bottom right | The Start Processing button |
Output Operator Types
| Type | Typical outputs |
|---|---|
| SLAM | Camera or device trajectory and pose sequence |
| Vision | Structured visual results such as object detection, segmentation, or hand keypoints |
| Depth | Depth maps estimated from stereo or monocular video |
| Tactile | Aligned tactile sensor signals and features |
| Multi-operator | A chain combining any of the above |
You can select multiple types; types you don't select produce no outputs.
Steps
- In the workbench, confirm you're in the right project and click Pipeline.
- In Operator Management, choose the output operator types for this run.
- Check the Pipeline Chain: prerequisites are met, and the operator order and parameters are correct.
- Click Start Processing in the lower-right corner. The pipeline status changes to Processing.
- You can leave the page. A status bar at the top of the workbench shows progress and log count, and you can reopen the dialog at any time.
- When processing completes, the structured outputs are written to the asset pool and the original RAW data is kept. Export HDF5 and Export LeRobot then appear in the workbench.
Permissions
| Action | R&D engineer | FAE | Data collector |
|---|---|---|---|
| Upload operators | ✓ | — | — |
| Configure operator parameters | ✓ | Read-only | — |
| Start pipeline processing | ✓ | ✓ | Not visible |
When anyone other than an R&D engineer opens the dialog, they can see the chain and the selected operators, but the parameters are read-only and a message indicates that R&D engineer permission is required.
Uploading Operators
- In Operator Management, an R&D engineer clicks Upload Operator and selects an operator package.
- The platform validates the package layout and descriptor file. If validation passes, the operator is added to the project's operator library.
- If an operator with the same name exists, you are asked to overwrite it or rename the new one.
Operator libraries are isolated by project. For the package layout and interface contract, see Developing Operators.
Execution Rules
| Rule | Description |
|---|---|
| Asynchronous | The job starts processing as soon as it is submitted and doesn't block the page; you can leave |
| Batch-level result | The job completes only when every required operator in the batch succeeds (gate G2) |
| Failure marks the whole batch | If any required operator fails, the whole batch is marked failed, with the failed step, logs, and a retry option. Partial success is never treated as success. |
| Run history kept | Retries apply only to failed steps; the original run record is kept |
| Source data preserved | Original RAW data is never replaced; outputs are stored separately |
How Long Processing Takes
Processing time depends on batch size and the number of operators. Refer to the progress and logs shown on the page.
A successful pipeline doesn't mean the data passes acceptance
A successful run only means the outputs were generated. Their quality still needs labeling and human review in the workbench.
Pipeline States
| State | Meaning |
|---|---|
| Not Processed | No pipeline has been submitted |
| Processing | Submitted and running |
| Completed | Every required operator succeeded and outputs were ingested |
| Failed | A required operator failed; check the logs and retry |

