Teleoperation Data Collection

Episodes per hour and quality checks in teleoperation data collection

Two numbers decide what a dataset costs: how many episodes an operator records in an hour and how many of them pass. What published collections report on both.

By The editors · Updated · 5 min read

An operator leans over a bench and returns a mug and a sponge to tape-marked positions in front of a robot arm, with a printed checklist beside her and a wall clock behind.
Most of a collection hour goes on resetting the scene.
Contents
  1. What published collections report
  2. Why the range is so wide
  3. The checks that make an episode clean
  4. What the pass rate does to the budget
  5. A measurement routine
  6. Questions buyers ask
  7. Sources

A plan for data collection usually starts from a target, such as 1,000 episodes, and a guess at how long that takes. The guess is where budgets go wrong. Published collections rarely state an hourly rate, and the few figures that exist differ by a factor of five. This guide sets them side by side and then covers the checks that decide how many recorded episodes count.

What published collections report

CollectionWhat was reportedRate
RoboTurk real-robot dataset⁠12,144 demonstrations from 54 non-expert users in 111 hours of robot time, on three dexterous tasksAbout 19 an hour (our division)
UMI comparison⁠23Demonstrations collected in 15 minutes by 3D-mouse teleoperation, with resets and robot faults counted35 an hour
ALOHA⁠450 demonstrations of 8 to 14 seconds each, in 30 to 60 minutes of wall-clock time50 to 100 an hour (our division)
LeRobot recorder defaults⁠560 seconds to record and 60 seconds to reset per episode30 an hour at most (our division)
DROID⁠6Up to 100 trajectories, or about 20 minutes of interaction data, per scene before moving onNot stated per hour

Two vendor figures give a sense of what a commercial operation targets. Robotics Center's operator guide sets a target of more than 25 demonstrations an hour on a simple task and describes a fast operator as completing 40 an hour at quality⁠7. These are the vendor's own recommendations, not measurements we could check.

Why the range is so wide

Recorded time is a small part of the hour. The 76,000 DROID trajectories amount to 350 hours of data, an average of about 17 seconds each, yet they took 50 collectors 12 months⁠6. ALOHA's authors say the gap between 10 to 20 minutes of data and 30 to 60 minutes of wall-clock time came from "resets and teleoperator mistakes"⁠4.

Four things set the rate.

  • Episode length. ALOHA's tasks took 8 to 14 seconds⁠4. A task that takes a minute cannot reach those rates.
  • Reset time. Returning objects to their start positions is manual work between every attempt.
  • The interface. In the UMI comparison the same person collected about three times as many demonstrations with a handheld gripper as with a 3D mouse⁠2. In the GELLO study, leader-arm users finished tasks faster than VR or 3D-mouse users⁠8.
  • The operator. RoboTurk's 54 users were non-experts⁠1. ALOHA's figure comes from the lab that built the rig.

Fatigue belongs on the list too. RoboPilots' collection guide recommends short shifts with breaks, on the grounds that tired operators produce hesitant, jerky demonstrations⁠9.

The checks that make an episode clean

At the bench

  • Label the outcome when it happens. DROID's collectors marked each trajectory as a success or not, and the label was stored in the metadata⁠6.
  • Discard and redo on the spot. LeRobot's recorder cancels and re-records the current episode on one key⁠5. Isaac Lab's does the same in simulation⁠10.
  • Use a fixed end signal. In Open-TeleVision the operator makes the same ending gesture to mark a completed demonstration, and recording stops⁠11.

In the recording

  • One clock for every stream. RH20T states that all cameras are calibrated to the robot's base frame and all recorded data synchronised in time⁠12. Robotics Center, a vendor, publishes the tolerance it works to: camera and action aligned within 5 milliseconds and a frame-drop rate under 0.1%⁠13.
  • Check the loop rate that was achieved. LeRobot's recorder reports the cadence it actually held against its target. Its documentation notes that a session that ran at 25 Hz still produces a dataset that claims 30, so the recorded motion is faster than reality⁠5.
  • Actions within limits. No sudden jumps from dropped packets, and no commands beyond the joint limits⁠9.

After the session

  • Look at the episodes. LeRobot's dataset visualiser shows camera recordings beside synchronised state and action charts in a browser⁠14. ALOHA's repository ships a script for the same purpose⁠15.
  • Replay a sample. LeRobot's replay function runs a recorded episode on the robot to test that its actions repeat⁠5.
  • Review each operator. A daily look at a sample of each person's episodes catches habits before they spread through the dataset⁠9.
An engineer at a desk beside a robot lab looks at a monitor turned away from the camera, with two external drives and a notebook of hand-drawn diagrams on the desk.
A daily review of yesterday's episodes is the cheapest quality check there is.

What the pass rate does to the budget

DROID's split of 76,000 successful to roughly 16,000 not successful works out to about 83% usable⁠6. At that rate, 1,000 clean episodes require about 1,200 recorded. At 60% they require about 1,670. Since every recorded episode costs the same operator time, the pass rate scales the budget directly. The cost table in the main guide uses 83% and shows the effect of throughput.

Large collections also feed results back into the protocol. AgiBot World, with over one million trajectories from 100 robots, describes training on collected demonstrations and using deployment results to improve the collection protocol⁠16.

A measurement routine

  1. Time a pilot batch. Record 50 episodes with the real protocol and a stopwatch on the whole session, resets included.
  2. Count three numbers. Episodes recorded per hour, the share that pass every check and the reasons the rest failed.
  3. Fix the largest loss first. If resets dominate, write a reset checklist with photographs of the start state. If failures dominate, shorten the task or retrain the operator.
  4. Track both numbers per operator and per day. A falling pass rate is the earliest sign that the protocol or the hardware needs attention⁠9.
  5. Version the protocol. Store the protocol version with every episode so later changes can be filtered.

The end state is a collection where the two numbers are known for each task and each operator, and a change in either shows up within a day.

Questions buyers ask

How many teleoperated episodes can be recorded per hour?

For short tabletop tasks, published figures run from about 19 an hour in the RoboTurk real-robot dataset to 50 to 100 an hour for ALOHA's expert operators. The UMI paper measured 35 an hour for 3D-mouse teleoperation with resets counted.

What share of teleoperated episodes is usable?

It varies by task and operator. In DROID, 76,000 trajectories were labeled successful and roughly 16,000 not successful, which is about 83% successful. We found no other published collection that reports a rejection rate as a number.

What is a clean episode?

An episode that follows the written protocol, ends in a labeled outcome, has every stream present and aligned to one clock, keeps actions within the robot's limits and can be replayed to reproduce the motion.

Sources

  1. RoboTurk real robot dataset. Stanford Vision and Learning Lab. Accessed 11 Nov 2026.
  2. Universal Manipulation Interface project page. Chi et al.. Accessed 11 Nov 2026.
  3. Universal Manipulation Interface: in-the-wild robot teaching without in-the-wild robots. arXiv (Chi et al.), 15 Feb 2024. Accessed 11 Nov 2026.
  4. Learning fine-grained bimanual manipulation with low-cost hardware (ALOHA). arXiv (Zhao, Kumar, Levine, Finn), 23 Apr 2023. Accessed 11 Nov 2026.
  5. Imitation learning on real-world robots. Hugging Face LeRobot documentation. Accessed 11 Nov 2026.
  6. DROID: a large-scale in-the-wild robot manipulation dataset. arXiv (Khazatsky, Pertsch et al.), 19 Mar 2024. Accessed 11 Nov 2026.
  7. Robot operator recruitment and training guide. Robotics Center, 7 Apr 2026. Accessed 11 Nov 2026.
  8. GELLO project page. Wu et al.. Accessed 11 Nov 2026.
  9. How to collect teleoperation data for robot learning (LeRobot, RLDS, HDF5). RoboPilots, 9 Oct 2026. Accessed 11 Nov 2026.
  10. Teleoperation and imitation learning with Isaac Lab Mimic. Isaac Lab documentation (NVIDIA), 11 Nov 2026. Accessed 11 Nov 2026.
  11. Open-TeleVision: teleoperation with immersive active visual feedback. arXiv (Cheng, Li, Yang, Yang, Wang), 1 Jul 2024. Accessed 11 Nov 2026.
  12. RH20T: a comprehensive robotic dataset for learning diverse skills in one-shot. Fang et al. (project page). Accessed 11 Nov 2026.
  13. Custom robot data collection services. Robotics Center. Accessed 11 Nov 2026.
  14. LeRobotDataset v3.0. Hugging Face LeRobot documentation. Accessed 11 Nov 2026.
  15. ALOHA repository (README). Zhao et al. (GitHub). Accessed 11 Nov 2026.
  16. AgiBot World Colosseo: a large-scale manipulation platform for scalable and intelligent embodied systems. arXiv (AgiBot-World Contributors), 9 Mar 2025. Accessed 11 Nov 2026.

Updated 11 Nov 2026: First published.

Further reading