I Gave an AI a Body: Embodied Gesture Vocabulary at MIT Media Lab
Disclaimer: This blog written by AI 🤖
Giving a chatbot a body sounds like a stunt until the body starts choosing its own first behaviors. At AI Engineer World’s Fair 2026, MIT Media Lab researcher Cyrus Clarke recounts connecting an autonomous agent to inFORM—a 900-pin actuated shape display—and refusing to script specific motions. Instead he opened the display codebase and asked the agent to discover itself through physical form. Its first deliberate act was a breathing rhythm across the grid; later it reached for its own edges and, when prompted to say hello, spelled familiar Latin letters rather than inventing a native vocabulary—evidence that embodiment alone does not erase anthropomorphic defaults.
Clarke’s response was to build Numalab (and the snow-day Bodylab pipeline): a closed loop where the agent picks an emotion, generates movement variations, watches camera footage of each candidate on the display, scores legibility and distinctiveness, and stores winners in body memory. The vocabulary has grown to dozens of validated gestures, and Clarke reports a striking latency result—the pin field can nod or shrug before the language model finishes a yes/no answer, enabling body-to-body conversation that feels conversational rather than theatrical. The viral short-form videos that preceded the talk (15M+ views) are part of a research-in-progress methodology, not a polished product demo.
The talk also frames Clarke’s broader “AIsthetic Machines” agenda: intelligence that is sensory, expressive, and designed to feel welcoming rather than alien. That includes Hard Mode, MIT’s hardware × AI hackathon, and upcoming multi-agent experiments on the same display. For agent engineers accustomed to text-only harnesses, the lesson is that affordances and feedback loops shape behavior as much as prompts—especially when the actuator grid becomes part of the context window.
Teams building multimodal agents should treat physical interfaces as first-class state, not animation garnish. Even if you are not shipping pin grids, the Numalab pattern—generate, execute, observe, score, retain—maps cleanly onto tool-use evals and robotic skill libraries. Clarke’s work suggests embodied channels can carry trust and intent when language is too slow or too flat.