Egocentric Data Collection
Egocentric data collection doesn't involve a robot. The demonstrator wears or holds a capture device and performs the task by hand. The devices are lightweight and can be taken into far more real-world environments. Data is stored as RAW and must be processed by a data pipeline into structured data before it can be curated or used for training.
Supported Devices
| Model | Form factor | What is captured |
|---|---|---|
| Robot-free Ego | Head-mounted first-person stereo camera | Left and right video, IMU, frame timestamps, stereo calibration |
| Robot-free UMI | Handheld capture device | Stereo video, IMU, frame timestamps, calibration |
See Data Formats for the RAW directory layout.
Prerequisites
| Requirement | Details |
|---|---|
| Device registered | The device model and SN are in the device inventory |
| Valid calibration | Stereo calibration ships with the device and is updated during periodic recalibration |
| Collection task | A collection task bound to the device exists |
| Operators ready | The project's operator library contains the operators needed for processing. See Developing Operators. |
Collecting Data
- The collector claims the task in Collection Tasks and clicks Start Collecting.
- Wear or hold the device and make sure both hands and the target objects are fully in view.
- Finish the demonstration and end the episode; the data uploads automatically.
- For long continuous sessions, use segmented upload: the recording is split into 5-minute segments and uploaded while recording, minimizing loss if the session is interrupted.
Capture Guidelines
| Guideline | Details |
|---|---|
| Keep hands in view | Keep both hands and the objects in frame throughout the key parts of the task |
| Avoid rapid head turns | Fast head or device motion causes motion blur, which degrades SLAM and keypoint results |
| Stable lighting | Avoid strong backlighting and frequent changes in brightness |
| Sufficient texture | Textured surroundings help visual localization; accuracy drops near plain, uniform walls |
| Hold still before starting | Keep the device still for about one second before each episode to aid initialization |
After Collection: Pipeline Processing
Egocentric data is ingested with status Awaiting Pipeline and can't be selected for curation directly. The processing path is:
text
RAW ingested (Awaiting Pipeline) → Create curation project → Pipeline processing (SLAM / vision / depth, etc.) → Outputs ingested (Ready to Curate) → Curation and exportFor the full walkthrough, see Tutorial A: From Raw Data to Annotated Data.
Common Issues
| Issue | Resolution |
|---|---|
| Items can't be selected | RAW data must be processed by a pipeline first |
| SLAM trajectory drifts | Check for fast motion or low-texture scenes during capture; recollect if needed |
| Keypoints are lost | Check whether the hands left the frame or were occluded |

