Pick and Place · Franka FR3 · v1.0
Bin-to-bin pick and place of household objects on a Franka FR3 with a Robotiq 2F-85 gripper (DROID).
π₀.₅
Wooden spoons
8 of 8 items in 99 s
GR00T
Wooden spoons
8 of 8 items in 89 s
ACT
Wooden spoons
8 of 8 items in 89 s
SmolVLA
Wooden spoons
7 of 8 items in 156 s
| # | Model | Runs | Reliability @ 120 s | Speed @ 120 s |
|---|---|---|---|---|
| % items placed | items/min | |||
| Loading the benchmark… | ||||
Reliability is the percent of items placed, each within the time limit of the previous item. Speed is items per minute: 60 over the mean time of those items. Models share a rank when chance can explain the difference between their time-to-success curves. ± is half the 95% range.
How reliable is it at each time limit?
The percent of items placed within each time limit of the previous item, with its 95% range. Items still under way when a run stopped count up to that point. An item placed together with the previous one counts at 0 s. Drag the marker to rank the table at another limit.
More charts
When do the successes come?
How fast the Reliability curve rises at each second: the share of items placed per second.
Is it faster, or more reliable?
Each line is a model at every time limit from 1 s to 120 s: its Speed in items/min across, its Reliability up. The dot is the current time limit, and the dashed lines are its 95% ranges. The axis ends at 15 items/min, so the quickest successes at a short limit fall beyond it.
Do two models really differ?
Episodes
Each episode and its own result, the newest first.