DXTR turns one phone recording of a person doing a task into object-centric trajectories that are re-solved per robot and replayed in simulation against the object’s tolerance tube before they ship. Fifteen answers below, in the same terms the pipeline uses. If something is not covered, email [email protected].
What is object-centric retargeting?
Object-centric retargeting treats the object’s 6-DoF trajectory as the goal of a demonstration, not the human’s joint angles. From one phone capture DXTR recovers the path the object took through the task, and then each robot solves its own grasp and its own joint motion to move that object along that same path. The object is never rescaled to make a body fit.
How is this different from teleoperation?
Teleoperation records one specific robot being driven by a person, so the episode is bound to that robot and costs an operator’s time per episode. DXTR records a person doing the task with their own hands on a phone, and re-solves the result per robot body — one capture yields trajectories for every embodiment in the registry. The trade is honest: teleoperation gets feasibility for free because a real robot did the motion, while DXTR has to recover the geometry and then prove feasibility in simulation before anything ships.
Which robots does DXTR support?
Six embodiments are retargeted today: Franka Panda, UR5e, PR2, MyCobot 280, Unitree G1 and SO-101. Each one chooses its own grasp on the reconstructed object and keeps its arm clear of the reconstructed scene. Adding a robot means adding a URDF and a gripper spec to the registry — it does not mean re-collecting the data.
What does the phone actually record?
The DXTR Eye iOS app records RGB video at 1920×1440, per-frame LiDAR depth, and ARKit camera pose while a person performs the task. It runs eyes-free: audio coaching and simple hand gestures start, stop and cancel each take, so the collector never touches the screen mid-demonstration. It requires an iPhone Pro or Pro Max (or an iPad Pro) with a LiDAR sensor.
What does “verified or refused” mean?
Every retargeted episode is replayed in simulation against the object’s tolerance tube before it ships. If the object follows its path inside the tube, the episode passes. If it does not — the reach is short, no grasp holds across the whole contact interval, the arm cannot clear the scene — the episode ships an explicit infeasibility certificate naming the sub-interval and what blocked it, instead of a silently rescaled trajectory that looks clean and is not. This is sim-verified at the kinematic tier: verified or refused.
What formats does the data ship in?
LeRobot datasets per robot — the format the pi0, ACT and Diffusion Policy stacks read — and MCAP for the raw multimodal record, which opens in Foxglove. Both are downloadable from the portal or pulled from the keyed public API at api.dxtr.co.
Who is DXTR for?
Robotics labs and data teams that need manipulation demonstrations for a particular embodiment and would rather not stand up a teleoperation rig or a capture fleet to get them. There are two ways in: record with the Eye app as a collector, or license retargeted trajectories for your robot — or the pipeline that produces them from a human capture.
How do I get data for my robot?
Tell us the robot and the tasks and we will scope a capture — [email protected]. If you would rather drive it from a script, the public API at api.dxtr.co commissions captures, reviews what comes back, and pulls robot-ready exports; the OpenAPI document is at /v1/openapi.json and the written docs are at app.dxtr.co/docs.
What devices does the Eye app need?
An iPhone Pro or Pro Max with a LiDAR sensor, or an iPad Pro. Capture uses LiDAR depth and ARKit scene data, so it will not start on a non-LiDAR device — a regular iPhone will not work. The app is distributed through TestFlight; you will need Apple’s free TestFlight app installed first. Camera and microphone access are required to record a demonstration; location is never used and there is no advertising or tracking.
What kinds of tasks can be captured?
The collection catalog currently holds 200 phone-capturable tasks across 19 categories — everyday manual work of the pick, place, pour, open and transport kind. The practical filter is whether the task is something a person can do with their own hands, in a room, with a phone mounted at eye level. Customers can also describe their own task in the portal — optionally AI-drafted — and broadcast it to approved collectors.
How accurate is the retargeting?
The published figure is a 5 mm p95 end-effector tracking residual on free transport for Franka and UR5e. Read that precisely: it is a solver residual — how closely the IK solution tracks its commanded targets — not ground-truth accuracy measured against a physical robot. The number that speaks to task success is the simulation replay against the object’s tolerance tube, which is pass or refuse rather than a residual.
Can I add my own robot?
That is the shape of the system: a robot enters the registry as a URDF plus a gripper specification — maximum width, finger geometry, end-effector frame, reach — and existing captures are re-solved for it. Nothing is re-collected, because the goal stored per episode is the object’s trajectory rather than any particular arm’s motion. Tell us the robot and the tasks and we will scope it: [email protected].
Does the data contain collector identity?
No. Customers receive episodes and derived datasets under a pseudonymous collector alias — never a name, email or contact details — and aliases are masked per organisation, so the same collector cannot be correlated across customers. Collectors agree not to record identifiable bystanders, faces, voices or personal documents. The full detail is in the Privacy Policy.
What are the limits — what does this not do?
Worth stating plainly. Contact forces are inferred, not measured — a phone has no force sensor, so if your task turns on force control this is not a substitute for a teleoperation rig. Quality depends on object pose recovery and metric reconstruction, so a task where the object is never clearly visible is a hard case. Frames where nothing moves — a press, a wipe, a reach to a switch — have no object trajectory to follow and fall back to wrist imitation, labelled as such. Verification is at the kinematic tier: it establishes that the object followed its path in simulation, not that a physical robot reproduced it. Articulated objects, and bimanual and mobile-base support, are the next tiers on the same certificate machinery rather than shipped capability.
How does DXTR relate to UMI, EgoMimic, DexPilot and AnyTeleop?
Those are the hand-copying lineage, and they are good work: DexPilot and AnyTeleop retarget a tracked human hand onto a robot for teleoperation; EgoMimic and UMI-style handheld rigs collect from a person directly. What they share is that the human’s motion is the thing being transferred, which is why a five-finger grasp or a human wrist path degrades as the bodies diverge — UMI narrows that gap by making the recording device gripper-shaped, at the cost of a hardware fleet. DXTR takes the other branch: track what the object did, and let each robot choose its own grasp and path to reproduce it. That transfers across bodies and is checkable in simulation, but it is blind to frames where nothing moves — which is why the shipped policy is a hybrid of both contracts, with every frame labelled by which rule it obeyed.