Skip to content

Data Formats ​

NexCore uses two sets of formats at different stages:

StageFormatsDetermined by
Storage (ingestion)HDF5 / RAWSet automatically by device model
Export (dataset versions)HDF5 / LeRobotChosen at export

About fields and dimensions

The layouts below are the platform's typical structures. The actual number of cameras, joint dimensions, and sampling rates depend on the device configuration; the metadata in each file (attrs / info.json) is authoritative.

Storage Format: HDF5 (Teleoperation Devices) ​

Each episode recorded on Station Lite or MOS is a single .hdf5 file. All streams are aligned to a common timeline, typically at 30 Hz.

text
episode_000123.hdf5
├── attrs                       # File-level metadata
│   ├── data_id                 # Platform data ID
│   ├── task_id                 # Source collection task ID
│   ├── device_model            # Device model, e.g. "Station Lite"
│   ├── device_sn               # Device serial number
│   ├── fps                     # Alignment rate, e.g. 30
│   └── collected_at            # Recording start time (ISO 8601)
├── timestamp                   # (T,) float64, Unix seconds
├── observations
│   ├── qpos                    # (T, D) float32, joint positions
│   ├── qvel                    # (T, D) float32, joint velocities
│   ├── effort                  # (T, D) float32, joint torques (if supported by the device)
│   ├── eef_pose                # (T, 2, 7) float32, end-effector poses for both arms [x, y, z, qx, qy, qz, qw]
│   ├── gripper                 # (T, 2) float32, gripper opening for both arms, normalized to [0, 1]
│   └── images
│       ├── cam_high            # (T,) variable-length bytes, JPEG-compressed frames
│       ├── cam_left_wrist
│       └── cam_right_wrist
├── action                      # (T, D) float32, teleoperation commands (same dimensions as qpos)
└── base_action                 # (T, 2) float32, base linear / angular velocity (MOS only)
DeviceJoint dimension DCamera configuration
Station Lite14 (6 joints + 1 gripper per arm)1 overhead + 2 wrist
MOS14 (dual arm) + 2-dimensional base1 head + 2 wrist

Reading Example ​

python
import h5py

with h5py.File("episode_000123.hdf5", "r") as f:
    print(dict(f.attrs))
    qpos = f["observations/qpos"][:]          # (T, D)
    action = f["action"][:]                   # (T, D)
    first_frame = f["observations/images/cam_high"][0]  # JPEG bytes
    print(qpos.shape, action.shape, len(first_frame))

Storage Format: RAW (Robot-Free Devices) ​

Each episode recorded on Robot-free Ego or Robot-free UMI is a directory of raw sensor data, without robot state.

text
session_20261020_103015/
├── manifest.json               # Session metadata and file list
├── video/
│   ├── left.mp4                # Left camera video
│   └── right.mp4               # Right camera video
├── imu.csv                     # Raw IMU data
├── frame_timestamps.csv        # Device timestamp for each frame
└── calib/
    └── stereo.yaml             # Stereo intrinsics and extrinsics

Example manifest.json:

json
{
  "data_id": "6af1c2d3e4b5a69788123456",
  "task_id": "20261020103000_ORG01",
  "device_model": "NexUMI",
  "device_sn": "NU-2609-0012",
  "started_at": "2026-10-20T10:30:15+08:00",
  "duration_sec": 142.6,
  "streams": [
    { "name": "left", "type": "video", "file": "video/left.mp4", "fps": 30 },
    { "name": "right", "type": "video", "file": "video/right.mp4", "fps": 30 },
    { "name": "imu", "type": "imu", "file": "imu.csv", "rate_hz": 200 }
  ]
}

RAW data must go through pipeline processing. Pipeline outputs such as SLAM trajectories, depth, and hand keypoints are written to the asset pool in structured formats, tagged with the pipeline job ID, and the original RAW data is kept.

Export Format: HDF5 ​

An HDF5 export produces one file per sample, with the same structure as the stored HDF5, processed according to the curation instructions:

  • items marked for exclusion are not exported;
  • frames in excluded ranges are trimmed;
  • if keep ranges are defined, only frames within them are exported.

Dataset directory:

text
dataset_<name>_v3/
├── manifest.json               # Export manifest
├── episode_000000.hdf5
├── episode_000001.hdf5
└── ...

Export Format: LeRobot ​

Organized to the LeRobot dataset specification:

text
dataset_<name>_v3/
├── meta/
│   ├── info.json               # fps, feature definitions, total samples and frames
│   ├── episodes.jsonl          # Length and task for each sample
│   ├── tasks.jsonl             # Task descriptions
│   └── stats.json              # Statistics for each feature
├── data/
│   └── chunk-000/
│       ├── episode_000000.parquet
│       └── ...
├── videos/
│   └── chunk-000/
│       ├── observation.images.cam_high/episode_000000.mp4
│       └── ...
└── nexcore_manifest.json       # NexCore export manifest
LeRobot featureHDF5 field
observation.stateobservations/qpos
actionaction
observation.images.<camera>observations/images/<camera>
timestampSeconds from the start of the sample

Export Manifest ​

Every dataset version includes an export manifest that always records each sample's source, labels, notes, and ranges.

json
{
  "dataset_id": "6b02a1f0c3d4e5f60718293a",
  "name": "pick_place_bimanual",
  "version": 3,
  "parent_version": 2,
  "format": "HDF5",
  "project": { "id": "6afe...", "name": "pick_place_curation", "version": 5 },
  "exported_at": "2026-10-20T15:42:10+08:00",
  "counts": { "included": 100, "normal": 99, "abnormal": 1 },
  "episodes": [
    {
      "file": "episode_000000.hdf5",
      "data_id": "6af1c2d3e4b5a69788123456",
      "source": "collection",
      "review_status": "REVIEWED",
      "annotations": [
        { "note": "black frames", "start_frame": 360, "end_frame": 600 }
      ],
      "segments": [
        { "action": "EXCLUDE", "start_frame": 360, "end_frame": 600 }
      ]
    }
  ],
  "failures": [
    { "data_id": "6af1c2d3e4b5a69788123499", "reason": "Channel cam_left_wrist missing" }
  ]
}
FieldDescription
sourceSample source: collection / inference_feedback / operation_feedback (continuous operation data) / pipeline (pipeline output)
review_statusPENDING (pending review) / REVIEWED (reviewed)
annotationsAnomaly labels; note is the label note
segmentsRange instructions; action is KEEP or EXCLUDE
countsnormal + abnormal = included

NexCore is evolving rapidly; the latest version prevails.NexCore Docs · Embodied AI Skill Learning Platform by Lumos Robotics