Figure Status Update - OpenAI Speech-to-Speech Reasoning

Channel:Figure · Length:2:35 · Published:2024-03-13 · Views:3090625

Watch on YouTube

Summary

A short status-update demo from Figure showing its humanoid robot using OpenAI-powered speech-to-speech reasoning to converse and perform tasks.

Why HN recommends it

Mostly cited as an example of the state of LLM-controlled robots, with some skepticism that it is a polished demo video.

LLMs & generative AI · Robotics

Subtitles (excerpt)

hey figure one what do you see right now I see a red apple on a plate in the center of the table a drying rack with cups and a plate and you standing nearby with your hand on the table great can I have something to eat sure thing great can you explain why you did what you just did while you pick up this trash on it so I gave you the Apple because it's the only uh edible item I could provide you with from the table great so based on the scene right now where do you think the dishes in front of you go next the dishes on the table like that plate and cup are likely to go into the drying rack next great can you put them there of course [Music] all right so how do you think you did I I think I did pretty well the apple found its new owner the trash is gone and the table Weare is right where it belongs I agree thanks figure one you're welcome if there's anything else you need help with just let me [Music] know

On Hacker News

> For starters, human minds do more than just language. These days, so do the AI (even though these ones are still called LLMs, which I think is now a bad name even though I continue to use the term without always being mindful of this). > I've yet to see an humanoid robot controlled by an LLM that can tackle difficult physical problems like riding a bike or skiing in for the first time real-time… — ben_w, 2025-01
[LINK] specifically their speech demo video (which is, of course, a demo video) [VIDEO] [LINK] and [LINK] are undoubtedly using such models. It's an area of active research, eg [LINK] [LINK] [LINK] [LINK] — fragmede, 2024-12
> Let’s say you gave o1 an API to control a good robot. Could it throw a football? Maybe. > Could it accomplish even the most basic tasks? Definitely: [VIDEO] — xvector, 2024-10
> The other isn't even being worked on in any serious fashion. It is being worked on, it's just hard: [LINK] [VIDEO] I'm expecting a minimum of 5-10 years to pass between real-time vision AI getting good enough for level-5 autonomous cars and the same stuff fitting into the power envelope available to this kind of domestic robot. — ben_w, 2024-07
Robotics is on the cusp of total reinvention. Almost everything that it took to make robots work is about to become archaic. The core ideas in old robotics will become a framework for the guardrails for new robotics, but this will widely become a solved problem that few need to think about Stanford Aloha [LINK] MIT [LINK] Figure/OpenAI [VIDEO] Tesla Optimus [LINK] NVIDIA Groot, [LINK] — paulsutter, 2024-06

Related videos