Assistant Professor of CS @nyuniversity.
I like robots!
Lerrel Pinto
Loading...
This project, which combines hardware design with learning-based controllers was a monumental effort led by @anyazorin.bsky.social and Irmak Guzey. More links and information about RUKA are below:
Website: ruka-hand.github.io
Assembly Instructions: ruka.gitbook.io/instructions
Point Policy uses sparse key points to represent both human demonstrators and robots, bridging the morphology gap. The scene is hence encoded through semantically meaningful key points from minimal human annotations.
We just released RUKA, a $1300 humanoid hand that is 3D-printable, strong, precise, and fully open sourced!
The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.
Is there a word for the feeling when you want to cheer for the other team?
When life gives you lemons, you pick them up.
(trained with robotutilitymodels.com)
The overall algorithm is simple:
1. Extract key points from human videos.
2. Train a transformer policy to predict future robot key points.
3. Convert predicted key points to robot actions.
The robot behaviors shown below are trained without any teleop, sim2real, genai, or motion planning. Simply show the robot a few examples of doing the task yourself, and our new method, called Point Policy, spits out a robot-compatible policy!
This project was an almost solo effort from @haldarsiddhant.bsky.social. And as always, this project is fully opensourced.
Project page: point-policy.github.io
Paper: arxiv.org/abs/2502.20391
What would you love to know about #robot learning and decision making?
Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments.
Send me your questions for Lerrel: robottalk.org/ask-a-question/
How is AI helping robots to generalise their skills to unfamiliar environments? 🤖 🏠
In the latest episode, I chatted to Prof. Lerrel Pinto (@lerrelpinto.com) from New York University about #robot learning and decision making.
Available wherever you get your podcasts: linktr.ee/robottalkpod
