Skip to content

Running Processing Jobs ​

Pipeline processing is a processing step within a curation project: a set of operators turns data into structured outputs. It isn't a top-level navigation module, and it isn't required for every type of data.

DataPipeline required?
RAW (Robot-free Ego, Robot-free UMI)Required. RAW has no robot state, so it must go through a pipeline to produce structured data before it can be curated.
HDF5 (Station Lite, MOS)Optional. HDF5 includes robot state and can be curated directly; you can still run a pipeline to enrich it.

Where: The Pipeline button in the data workbench header opens a full-page dialog.

Dialog Layout ​

AreaContents
Top: operator managementUpload operators and choose output operator types
Middle: pipeline chainA three-stage view: prerequisites → operator execution → post-processing
Bottom rightThe Start Processing button

Output Operator Types ​

TypeTypical outputs
SLAMCamera or device trajectory and pose sequence
VisionStructured visual results such as object detection, segmentation, or hand keypoints
DepthDepth maps estimated from stereo or monocular video
TactileAligned tactile sensor signals and features
Multi-operatorA chain combining any of the above

You can select multiple types; types you don't select produce no outputs.

Steps ​

  1. In the workbench, confirm you're in the right project and click Pipeline.
  2. In Operator Management, choose the output operator types for this run.
  3. Check the Pipeline Chain: prerequisites are met, and the operator order and parameters are correct.
  4. Click Start Processing in the lower-right corner. The pipeline status changes to Processing.
  5. You can leave the page. A status bar at the top of the workbench shows progress and log count, and you can reopen the dialog at any time.
  6. When processing completes, the structured outputs are written to the asset pool and the original RAW data is kept. Export HDF5 and Export LeRobot then appear in the workbench.

Permissions ​

ActionR&D engineerFAEData collector
Upload operators✓——
Configure operator parameters✓Read-only—
Start pipeline processing✓✓Not visible

When anyone other than an R&D engineer opens the dialog, they can see the chain and the selected operators, but the parameters are read-only and a message indicates that R&D engineer permission is required.

Uploading Operators ​

  1. In Operator Management, an R&D engineer clicks Upload Operator and selects an operator package.
  2. The platform validates the package layout and descriptor file. If validation passes, the operator is added to the project's operator library.
  3. If an operator with the same name exists, you are asked to overwrite it or rename the new one.

Operator libraries are isolated by project. For the package layout and interface contract, see Developing Operators.

Execution Rules ​

RuleDescription
AsynchronousThe job starts processing as soon as it is submitted and doesn't block the page; you can leave
Batch-level resultThe job completes only when every required operator in the batch succeeds (gate G2)
Failure marks the whole batchIf any required operator fails, the whole batch is marked failed, with the failed step, logs, and a retry option. Partial success is never treated as success.
Run history keptRetries apply only to failed steps; the original run record is kept
Source data preservedOriginal RAW data is never replaced; outputs are stored separately

How Long Processing Takes ​

Processing time depends on batch size and the number of operators. Refer to the progress and logs shown on the page.

A successful pipeline doesn't mean the data passes acceptance

A successful run only means the outputs were generated. Their quality still needs labeling and human review in the workbench.

Pipeline States ​

StateMeaning
Not ProcessedNo pipeline has been submitted
ProcessingSubmitted and running
CompletedEvery required operator succeeded and outputs were ingested
FailedA required operator failed; check the logs and retry

NexCore is evolving rapidly; the latest version prevails.NexCore Docs · Embodied AI Skill Learning Platform by Lumos Robotics