Smart people recognize each other – science proves it
39 by 01-_- | 28 comments on Hacker News.
Monday, 6 April 2026
Sunday, 5 April 2026
Saturday, 4 April 2026
Friday, 3 April 2026
Thursday, 2 April 2026
Wednesday, 1 April 2026
Tuesday, 31 March 2026
New top story on Hacker News: Show HN: PhAIL – Real-robot benchmark for AI models
Show HN: PhAIL – Real-robot benchmark for AI models
7 by vertix | 7 comments on Hacker News.
I built this because I couldn't find honest numbers on how well VLA models [1] actually work on commercial tasks. I come from search ranking at Google where you measure everything, and in robotics nobody seemed to know. PhAIL runs four models (OpenPI/pi0.5, GR00T, ACT, SmolVLA) on bin-to-bin order picking – one of the most common warehouse operations. Same robot (Franka FR3), same objects, hundreds of blind runs. The operator doesn't know which model is running. Best model: 64 UPH. Human teleoperating the same robot: 330. Human by hand: 1,300+. Everything is public – every run with synced video and telemetry, the fine-tuning dataset, training scripts. The leaderboard is open for submissions. Happy to answer questions about methodology, the models, or what we observed. [1] Vision-Language-Action: https://ift.tt/Op5o1Im
7 by vertix | 7 comments on Hacker News.
I built this because I couldn't find honest numbers on how well VLA models [1] actually work on commercial tasks. I come from search ranking at Google where you measure everything, and in robotics nobody seemed to know. PhAIL runs four models (OpenPI/pi0.5, GR00T, ACT, SmolVLA) on bin-to-bin order picking – one of the most common warehouse operations. Same robot (Franka FR3), same objects, hundreds of blind runs. The operator doesn't know which model is running. Best model: 64 UPH. Human teleoperating the same robot: 330. Human by hand: 1,300+. Everything is public – every run with synced video and telemetry, the fine-tuning dataset, training scripts. The leaderboard is open for submissions. Happy to answer questions about methodology, the models, or what we observed. [1] Vision-Language-Action: https://ift.tt/Op5o1Im
Monday, 30 March 2026
Sunday, 29 March 2026
Saturday, 28 March 2026
Subscribe to:
Comments (Atom)