Teleoperation Data Collection

Field guide · Updated quarterly

Teleoperation data collection: a working guide for 2027

What a robotics team needs to decide before it records its first demonstration, with the published numbers on rigs, throughput, quality checks and formats, and a worked cost per clean episode.

By The editors · Updated · 11 min read

An operator at a plywood bench holds two small leader arms while two robot arms across the bench place a block in a tray, watched by a colleague with a clipboard.
A demonstration is one attempt at one task, recorded from every camera and joint.
Contents
  1. The problem with a first collection
  2. How this guide was built
  3. Step 1: choose the rig for the task
  4. Step 2: write the protocol before recording
  5. Step 3: know what an hour produces
  6. Step 4: check quality before training
  7. Step 5: pick the file format
  8. What a clean episode costs
  9. In-house or hired operators
  10. The path to a first dataset
  11. Questions buyers ask
  12. Sources

In brief

Teleoperation data collection means a person drives the robot through a task while cameras, joint states and commands are recorded as one episode. The rig is the smaller cost. In a worked example with sourced inputs, a US operator on a $24,000 bimanual kit produces a clean episode for about $1.30 to $3.50, depending on throughput, and the person is at least three quarters of the hourly cost. Three decisions set the result: the interface, the written protocol and who operates.

  • If you want trained operators without hiring them: RoboPilots. Remote pilots trained on your robot, a published starting price of $12 an hour and a one-week trial that measures clean episodes per operator hour.

The problem with a first collection

A team that has a robot and a task and wants a policy needs demonstrations on its own hardware. Public datasets help with pre-training, but they were recorded on other robots, with other cameras. The largest of them show how much labour is involved. DROID took 50 data collectors 12 months to record 76,000 trajectories, or 350 hours of interaction⁠1. Google's RT-1 dataset of over 130,000 episodes was collected with a fleet of 13 robots over 17 months⁠2. Physical Intelligence reports pre-training its π0 model on over 10,000 hours of robot data⁠3.

Most teams need far less than that to start, but they face the same four unknowns, and each one changes the budget. The first is the rig, which sets what the operator can do and what the dataset records. The second is the operator, who sets how many episodes an hour produces and how consistent they are. The third is quality control, which decides how many of those episodes are worth training on. The fourth is the file format, which decides whether the training job can read them at all.

The papers that describe collection rigs rarely report what an hour of collection produces, and the companies that sell collection rarely publish a price. This guide gathers the figures that are public and combines them into one number a team can plan with, the cost of a clean episode.

How this guide was built

Every figure below comes from a research paper, a project page, official documentation, a maker's product page, a dated job listing or an official statistic, opened on the access date shown in the source list. Research results describe one lab's robot and task. Prices and pay ranges are as published on the day we read them. The rules are on the methodology page.

Step 1: choose the rig for the task

A teleoperation rig has two halves: the device in the operator's hands and the sensors that record. The device decides what kind of motion the dataset contains.

InterfacePublished costWhat it recordsEvidence
Leader arm (kinematic replica)GELLO: under $300 in parts per device⁠4. SO-101 leader and follower: $229.88 in parts⁠5Joint positions of the leader, which become joint-space actionsHighest average success in the GELLO user study: 0.92, against 0.72 for VR and 0.63 for a 3D mouse⁠4
Complete bimanual kitALOHA: under $20,000 to build⁠6. Trossen Stationary AI: $23,995.95⁠7Two leader-follower pairs, four cameras, 50 Hz in ALOHA⁠6880 to 90% success on fine tasks from about ten minutes of demonstrations⁠6
VR headsetApple Vision Pro: from $3,699⁠9Head and hand poses, mapped to the robot; stereo video back to the operatorOpen-TeleVision runs its loop at 60 Hz⁠10; DROID used a Quest 2 headset with controllers⁠1
Handheld gripper, no robotUMI: $73 gripper plus $298 camera and accessories⁠11Wrist-camera video and gripper motion, later transferred to a robot111 demonstrations an hour, against 35 for teleoperation with a 3D mouse⁠12

The pattern in the evidence is that devices which mirror the robot's own joints give the cleanest manipulation data, headsets suit robots the operator has to see through, and handheld tools trade robot-specific data for speed. The rig guide covers each in detail, including simulation.

One rule holds across all of them. Pick one interface per task family and keep it fixed, because mixing interfaces in one dataset changes the distribution of actions in ways that are hard to see later⁠13.

Step 2: write the protocol before recording

An episode is one attempt at one task, from a defined start state to success or failure. The protocol is the document that makes one operator's episode comparable with another's. It states the reset state, the variation allowed in object placement and lighting, the success criterion and what the operator does when something goes wrong⁠13.

Published guidance is consistent on how to start. Hugging Face's LeRobot documentation suggests at least 50 episodes for a first grasp-and-place task, with 10 episodes per object location, fixed cameras and consistent grasping, and warns against adding variation too quickly⁠14. ALOHA recorded 50 demonstrations per task⁠6. Mobile ALOHA used 50 per task and raised success by co-training with existing data⁠15.

An operator leans over a bench and returns a mug and a sponge to tape-marked positions in front of a robot arm, with a printed checklist beside her.
Resetting the scene between attempts takes more of the hour than the demonstrations do.

Step 3: know what an hour produces

Throughput is the number most plans guess. The published figures for short tabletop tasks are these.

SourceSettingEpisodes per hour
RoboTurk real-robot dataset⁠1654 non-expert users, 2,144 demonstrations in 111 robot-hoursAbout 19 (our division)
LeRobot recorder defaults⁠1460 seconds to record, 60 seconds to reset30 at most (our division)
UMI paper⁠1211Cup arrangement by 3D mouse, resets and robot faults counted35
ALOHA⁠650 demonstrations in 30 to 60 minutes of wall-clock time50 to 100 (our division)

The spread is wide because the tasks differ, and because resets dominate. ALOHA's 50 demonstrations amounted to 10 to 20 minutes of recorded data but took 30 to 60 minutes "because of resets and teleoperator mistakes"⁠6. Longer tasks, mobile robots and humanoids fall well below these figures. No published source gives a rate for another team's task, which is why a measured trial is the first thing to buy or run.

Step 4: check quality before training

Not every recorded episode is usable. DROID's collectors marked each trajectory as successful or not. The dataset counts 76,000 successful episodes, and roughly 16,000 more were labeled not successful⁠1. That is about 83% usable, in a large and well-run academic collection.

The checks that separate a clean episode from a recorded one are short to list. Every stream is present for every frame, with one clock and no gaps. Actions stay within the robot's limits with no jumps from dropped packets. Outcome labels are spot-checked against video. A sample of episodes is replayed on the robot and reproduces the motion⁠1314. The throughput and quality guide sets out each check with the tools that run it.

Step 5: pick the file format

Four formats cover most robot-learning work. LeRobotDataset v3.0 stores states, actions and timestamps in Parquet and camera frames in MP4⁠17. RLDS stores episodes of steps and is the format of the Open X-Embodiment collection⁠1819. HDF5 is the single-file layout used by robomimic and by ALOHA's recorder⁠2021. MCAP is a container for timestamped log data and is the default log format in ROS 2, by its maintainers' account⁠22. The formats guide compares them and covers conversion.

What a clean episode costs

The cost of a clean episode is the cost of an hour of collection divided by the usable episodes that hour produces:

cost per clean episode = (operator cost per hour + rig cost per hour) ÷ (episodes per hour × pass rate)

The sourced inputs

InputValueSource
Operator wage, US listings in 2026$25 an hour (Physical Intelligence), $28 to $36 (1X), $30 to $32 (Dyna Robotics), $32 (Figure)Company job listings⁠23242526
Employer cost on top of the wageWages are 70.0% of private-industry compensation, so cost is the wage ÷ 0.70US Bureau of Labor Statistics, June 2026⁠27
Hired remote operatorFrom $12 an hour, a published starting price; the rest is quotedRoboPilots⁠28
Bimanual kit$23,995.95Trossen Stationary AI⁠7
Low-cost arm pair$229.88 in partsSO-101 bill of materials⁠5
Episodes per hour19, 35 or 50RoboTurk, UMI paper and ALOHA, from the table above⁠16126
Pass rate83%DROID: 76,000 successful of about 92,000 recorded⁠1

Two inputs are our assumptions and are not sourced: the rig is written off over 2,000 hours of collection, about one year of single shifts, and a hired remote operator is paired with one in-house person who resets scenes for four stations.

The worked table

ScenarioCost per hourAt 19 an hourAt 35 an hourAt 50 an hour
In-house operator at $30 an hour, $24,000 kit$42.86 + $12.00 = $54.86$3.48$1.89$1.32
In-house operator at $30 an hour, SO-101 arm pair$42.86 + $0.11 = $42.97$2.72$1.48$1.04
Hired remote operator at the $12 starting price, $24,000 kit, shared in-house resetter$12.00 + $12.00 + $10.71 = $34.71$2.20$1.19$0.84

Each cell is the hourly cost divided by the clean episodes in that hour: 15.8, 29.1 and 41.5 at an 83% pass rate.

How to read it

Three things follow from the arithmetic. First, throughput moves the result more than any price does. Going from 19 to 50 episodes an hour cuts the cost by 62% in every row. Second, the person is most of the cost. With the $24,000 kit, the operator is 78% of the hourly figure; with the low-cost arms, almost all of it. Third, the hired row depends on two things the table cannot know: the rate actually quoted for the task, which may be above the starting price, and whether the task needs someone on site to reset it.

The table leaves out what the methodology lists: engineering time for the recording pipeline, recruiting and training, supervision, facilities, robot repairs, storage and labeling. Those are why datasets sold by the demonstration cost far more than these figures. Robotics Center publishes $2,500 for a 20-demonstration pilot and $8,000 for 100, which is $125 and $80 a demonstration, with task design, setup and quality checks included⁠29.

The model is an illustration and not a quote. A team should replace our inputs with its own after a measured week.

In-house or hired operators

The last decision is who sits at the rig. RoboPilots' guide to buying teleoperation describes four models: in-house operators, staffing or contract operators, software-only platforms where the customer brings the people, and managed services that supply trained operators, coverage and the data pipeline together⁠30.

In-house suits a small fleet whose engineers sit beside the operators. The costs beyond the wage are recruiting, training and shift planning. One vendor's guide estimates that certifying a new operator costs about $3,000 in training time⁠31. Tesla's 2024 listing for data collection operators described three eight-hour shifts, which is what it takes to keep robots recording around the clock⁠32.

Hiring suits a team whose constraint is people and management time. Remote operation adds network delay, which matters for contact-rich tasks: by RoboPilots' rule of thumb, direct manipulation feels close to natural under roughly 100 to 150 milliseconds of round trip and becomes hard above about 300⁠33. Its free latency calculator estimates the round trip between two cities⁠34. The in-house or hired guide works through the comparison.

The path to a first dataset

  1. Write the protocol. Start state, allowed variation, success criterion and the failure procedure, on one page.
  2. Fix the interface and the cameras. One device per task family, cameras that do not move.
  3. Record a pilot batch of about 50 episodes. Time it. Count episodes per hour and the share that pass the checks.
  4. Load, replay and train. Confirm the files load into the training code and a replay reproduces the motion.
  5. Compute the cost per clean episode. Use the measured throughput and pass rate in the formula above.
  6. Decide who operates at scale. Compare an in-house hire with a quoted service on the same number.
  7. Collect where the policy fails. Let evaluation on held-out setups choose the next batch.

The end state is a collection that runs like a production line. A known number of clean episodes arrives each week in the format the training job reads. Each one is labeled by task and outcome and comes from operators who follow one protocol. The cost per clean episode is measured each week, so adding a station or a shift becomes a matter of arithmetic.

If the constraint is operators. RoboPilots trains remote pilots on your robot and starts with a one-week trial that measures clean episodes per operator hour. See how RoboPilots works

Questions buyers ask

What is teleoperation data collection?

It is the recording of a person driving a robot through a task. Each attempt is saved as an episode containing the camera frames, the robot's joint states and the commands sent, aligned in time. Imitation-learning and vision-language-action models are trained on these episodes.

How many demonstrations does a robot need?

It depends on the task and the model. ALOHA recorded 50 demonstrations per task and reached 80 to 90% success on fine bimanual tasks. Hugging Face's LeRobot documentation suggests at least 50 episodes for a first pick-and-place task. Generalist models use far more; Physical Intelligence reports pre-training on over 10,000 hours of robot data.

How many episodes can one operator record in an hour?

Published figures for short tabletop tasks range from about 19 to 100 an hour. The RoboTurk real-robot dataset averaged about 19 demonstrations per robot-hour, the UMI paper measured 35 an hour for teleoperation with a 3D mouse including resets, and ALOHA's 50 demonstrations took 30 to 60 minutes of wall-clock time.

What does a teleoperated episode cost?

In our worked example a clean episode costs roughly $1.30 to $3.50 when a US operator at $30 an hour collects on a $24,000 kit. Lab-collected datasets sold by the demonstration are priced far higher; Robotics Center publishes $2,500 for 20 demonstrations, which includes task design, setup and quality checks.

Which data format should I record in?

Record in the format your training code reads. LeRobotDataset stores tabular signals in Parquet and video in MP4 and suits PyTorch and Hugging Face workflows. RLDS is the episode format used by the Open X-Embodiment collection. HDF5 is the simple one-file layout used by robomimic and ALOHA. MCAP is the default log format in ROS 2.

Is a VR headset or a leader arm better for data collection?

For arm manipulation, the GELLO study found that a kinematic replica of the arm gave higher success rates than VR controllers or a 3D mouse across five tasks with 12 participants. VR is the usual choice when the operator must look around, as with humanoids and mobile robots.

Should failed episodes be kept?

Keep them, labeled. The DROID collectors marked every trajectory as successful or not; about 16,000 were labeled not successful, released with the dataset and excluded from its headline count of 76,000.

Should we hire operators or use a service?

Teams with one or two robots and engineers beside the bench usually collect in-house. Hiring a service makes sense when the constraint is people, shifts or management time. The two cost models differ mainly in the hourly cost of the operator, so compare them on cost per clean episode after a short trial.

Sources

  1. DROID: a large-scale in-the-wild robot manipulation dataset. arXiv (Khazatsky, Pertsch et al.), 19 Mar 2024. Accessed 11 Nov 2026.
  2. RT-1: Robotics Transformer for real-world control at scale (project page). Google (Brohan et al.), 13 Dec 2022. Accessed 11 Nov 2026.
  3. π0: a vision-language-action flow model for general robot control. arXiv (Black et al., Physical Intelligence), 31 Oct 2024. Accessed 11 Nov 2026.
  4. GELLO: a general, low-cost, and intuitive teleoperation framework for robot manipulators. arXiv (Wu, Shentu, Yi, Lin, Abbeel), 22 Sep 2023. Accessed 11 Nov 2026.
  5. SO-ARM100 and SO-101: bill of materials (README). TheRobotStudio (GitHub). Accessed 11 Nov 2026.
  6. Learning fine-grained bimanual manipulation with low-cost hardware (ALOHA). arXiv (Zhao, Kumar, Levine, Finn), 23 Apr 2023. Accessed 11 Nov 2026.
  7. Stationary AI robotic machine learning kit. Trossen Robotics. Accessed 11 Nov 2026.
  8. ALOHA project page. Zhao et al.. Accessed 11 Nov 2026.
  9. Buy Apple Vision Pro. Apple. Accessed 11 Nov 2026.
  10. Open-TeleVision: teleoperation with immersive active visual feedback. arXiv (Cheng, Li, Yang, Yang, Wang), 1 Jul 2024. Accessed 11 Nov 2026.
  11. Universal Manipulation Interface: in-the-wild robot teaching without in-the-wild robots. arXiv (Chi et al.), 15 Feb 2024. Accessed 11 Nov 2026.
  12. Universal Manipulation Interface project page. Chi et al.. Accessed 11 Nov 2026.
  13. How to collect teleoperation data for robot learning (LeRobot, RLDS, HDF5). RoboPilots, 9 Oct 2026. Accessed 11 Nov 2026.
  14. Imitation learning on real-world robots. Hugging Face LeRobot documentation. Accessed 11 Nov 2026.
  15. Mobile ALOHA: learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv (Fu, Zhao, Finn), 4 Jan 2024. Accessed 11 Nov 2026.
  16. RoboTurk real robot dataset. Stanford Vision and Learning Lab. Accessed 11 Nov 2026.
  17. LeRobotDataset v3.0. Hugging Face LeRobot documentation. Accessed 11 Nov 2026.
  18. RLDS: reinforcement learning datasets (README). Google Research (GitHub). Accessed 11 Nov 2026.
  19. Open X-Embodiment repository. Google DeepMind (GitHub). Accessed 11 Nov 2026.
  20. robomimic: dataset overview. robomimic documentation. Accessed 11 Nov 2026.
  21. ALOHA repository (README). Zhao et al. (GitHub). Accessed 11 Nov 2026.
  22. MCAP: a container file format for multimodal log data. Foxglove. Accessed 11 Nov 2026.
  23. Robot Operator, San Francisco (job listing). Physical Intelligence, 6 Aug 2026. Accessed 11 Nov 2026.
  24. Operator, Data Collection (Swing Shift), San Carlos (job listing). 1X Technologies, 9 Sep 2026. Accessed 11 Nov 2026.
  25. Data Collection Operator, Redwood City (job listing). Dyna Robotics, 8 May 2026. Accessed 11 Nov 2026.
  26. Humanoid Robot Pilot, San Jose (job listing). Figure. Accessed 11 Nov 2026.
  27. Employer Costs for Employee Compensation, June 2026. US Bureau of Labor Statistics, 9 Sep 2026. Accessed 11 Nov 2026.
  28. Robot teleoperation pricing. RoboPilots. Accessed 11 Nov 2026.
  29. Custom robot data collection services. Robotics Center. Accessed 11 Nov 2026.
  30. Robot teleoperation as a service: how it works and what it costs. RoboPilots, 11 Nov 2026. Accessed 11 Nov 2026.
  31. Robot operator recruitment and training guide. Robotics Center, 7 Apr 2026. Accessed 11 Nov 2026.
  32. Tesla is paying up to $48 per hour for data collection operators. Business Insider (Grace Kay), via AOL, 19 Aug 2024. Accessed 11 Nov 2026.
  33. Teleoperation latency: how much delay each robot task tolerates. RoboPilots, 31 Oct 2026. Accessed 11 Nov 2026.
  34. Teleoperation latency calculator. RoboPilots, 31 Oct 2026. Accessed 11 Nov 2026.

Updated 11 Nov 2026: First published.

Further reading