Data Formats
NexCore uses two sets of formats at different stages:
| Stage | Formats | Determined by |
|---|---|---|
| Storage (ingestion) | HDF5 / RAW | Set automatically by device model |
| Export (dataset versions) | HDF5 / LeRobot | Chosen at export |
About fields and dimensions
The layouts below are the platform's typical structures. The actual number of cameras, joint dimensions, and sampling rates depend on the device configuration; the metadata in each file (attrs / info.json) is authoritative.
Storage Format: HDF5 (Teleoperation Devices)
Each episode recorded on Station Lite or MOS is a single .hdf5 file. All streams are aligned to a common timeline, typically at 30 Hz.
episode_000123.hdf5
├── attrs # File-level metadata
│ ├── data_id # Platform data ID
│ ├── task_id # Source collection task ID
│ ├── device_model # Device model, e.g. "Station Lite"
│ ├── device_sn # Device serial number
│ ├── fps # Alignment rate, e.g. 30
│ └── collected_at # Recording start time (ISO 8601)
├── timestamp # (T,) float64, Unix seconds
├── observations
│ ├── qpos # (T, D) float32, joint positions
│ ├── qvel # (T, D) float32, joint velocities
│ ├── effort # (T, D) float32, joint torques (if supported by the device)
│ ├── eef_pose # (T, 2, 7) float32, end-effector poses for both arms [x, y, z, qx, qy, qz, qw]
│ ├── gripper # (T, 2) float32, gripper opening for both arms, normalized to [0, 1]
│ └── images
│ ├── cam_high # (T,) variable-length bytes, JPEG-compressed frames
│ ├── cam_left_wrist
│ └── cam_right_wrist
├── action # (T, D) float32, teleoperation commands (same dimensions as qpos)
└── base_action # (T, 2) float32, base linear / angular velocity (MOS only)| Device | Joint dimension D | Camera configuration |
|---|---|---|
| Station Lite | 14 (6 joints + 1 gripper per arm) | 1 overhead + 2 wrist |
| MOS | 14 (dual arm) + 2-dimensional base | 1 head + 2 wrist |
Reading Example
import h5py
with h5py.File("episode_000123.hdf5", "r") as f:
print(dict(f.attrs))
qpos = f["observations/qpos"][:] # (T, D)
action = f["action"][:] # (T, D)
first_frame = f["observations/images/cam_high"][0] # JPEG bytes
print(qpos.shape, action.shape, len(first_frame))Storage Format: RAW (Robot-Free Devices)
Each episode recorded on Robot-free Ego or Robot-free UMI is a directory of raw sensor data, without robot state.
session_20261020_103015/
├── manifest.json # Session metadata and file list
├── video/
│ ├── left.mp4 # Left camera video
│ └── right.mp4 # Right camera video
├── imu.csv # Raw IMU data
├── frame_timestamps.csv # Device timestamp for each frame
└── calib/
└── stereo.yaml # Stereo intrinsics and extrinsicsExample manifest.json:
{
"data_id": "6af1c2d3e4b5a69788123456",
"task_id": "20261020103000_ORG01",
"device_model": "NexUMI",
"device_sn": "NU-2609-0012",
"started_at": "2026-10-20T10:30:15+08:00",
"duration_sec": 142.6,
"streams": [
{ "name": "left", "type": "video", "file": "video/left.mp4", "fps": 30 },
{ "name": "right", "type": "video", "file": "video/right.mp4", "fps": 30 },
{ "name": "imu", "type": "imu", "file": "imu.csv", "rate_hz": 200 }
]
}RAW data must go through pipeline processing. Pipeline outputs such as SLAM trajectories, depth, and hand keypoints are written to the asset pool in structured formats, tagged with the pipeline job ID, and the original RAW data is kept.
Export Format: HDF5
An HDF5 export produces one file per sample, with the same structure as the stored HDF5, processed according to the curation instructions:
- items marked for exclusion are not exported;
- frames in excluded ranges are trimmed;
- if keep ranges are defined, only frames within them are exported.
Dataset directory:
dataset_<name>_v3/
├── manifest.json # Export manifest
├── episode_000000.hdf5
├── episode_000001.hdf5
└── ...Export Format: LeRobot
Organized to the LeRobot dataset specification:
dataset_<name>_v3/
├── meta/
│ ├── info.json # fps, feature definitions, total samples and frames
│ ├── episodes.jsonl # Length and task for each sample
│ ├── tasks.jsonl # Task descriptions
│ └── stats.json # Statistics for each feature
├── data/
│ └── chunk-000/
│ ├── episode_000000.parquet
│ └── ...
├── videos/
│ └── chunk-000/
│ ├── observation.images.cam_high/episode_000000.mp4
│ └── ...
└── nexcore_manifest.json # NexCore export manifest| LeRobot feature | HDF5 field |
|---|---|
observation.state | observations/qpos |
action | action |
observation.images.<camera> | observations/images/<camera> |
timestamp | Seconds from the start of the sample |
Export Manifest
Every dataset version includes an export manifest that always records each sample's source, labels, notes, and ranges.
{
"dataset_id": "6b02a1f0c3d4e5f60718293a",
"name": "pick_place_bimanual",
"version": 3,
"parent_version": 2,
"format": "HDF5",
"project": { "id": "6afe...", "name": "pick_place_curation", "version": 5 },
"exported_at": "2026-10-20T15:42:10+08:00",
"counts": { "included": 100, "normal": 99, "abnormal": 1 },
"episodes": [
{
"file": "episode_000000.hdf5",
"data_id": "6af1c2d3e4b5a69788123456",
"source": "collection",
"review_status": "REVIEWED",
"annotations": [
{ "note": "black frames", "start_frame": 360, "end_frame": 600 }
],
"segments": [
{ "action": "EXCLUDE", "start_frame": 360, "end_frame": 600 }
]
}
],
"failures": [
{ "data_id": "6af1c2d3e4b5a69788123499", "reason": "Channel cam_left_wrist missing" }
]
}| Field | Description |
|---|---|
source | Sample source: collection / inference_feedback / operation_feedback (continuous operation data) / pipeline (pipeline output) |
review_status | PENDING (pending review) / REVIEWED (reviewed) |
annotations | Anomaly labels; note is the label note |
segments | Range instructions; action is KEEP or EXCLUDE |
counts | normal + abnormal = included |

