dimentary/llm-robotics-playground

LLM Robotics Playground 🤖

A library of experiments using frontier LLMs/VLMs for robotics tasks, currently focusing on GPT-6 Astra.

Each experiment includes a MuJoCo environment, controller code, and a recorded run. Current tasks cover cable untangling, mobile manipulation, writing, and drawing.

| Experiment | Preview | Demo | | --- | --- | --- | | Headphone untangling | | Two robot arms open two tangled cable regions. | | Robo spider | | A six-legged robot carries and places two objects using two arms. | | Fibonacci writing | | A humanoid writes a Python program on a whiteboard. | | Dove drawing | | An articulated hand draws a dove with a pencil. |

Watch or download the demos.

Setup

Tested on macOS Apple Silicon with Python 3.14 and MuJoCo 3.12.0. Rendering needs graphics support. Other platforms have not been verified.

Install uv, then clone the repository and install its dependencies:

git clone https://github.com/dimentary/llm-robotics-playground.git
cd llm-robotics-playground
uv sync --locked
uv run python check.py

uv handles Python 3.14 and the project environment. Pick an experiment and follow its README to run it. New results go in its outputs/ folder, which Git ignores. The experiments share the robot models in assets/.

Replay a demo

Download a recorded run without rerunning the simulation:

uv run python fetch.py robo-spider
cd experiments/robo-spider
uv run python validate.py
uv run python render.py --preview

To try another demo, replace robo-spider with its folder name and follow that experiment's replay instructions. Downloads are checked against recordings.json. If replay files already exist in outputs/, move them aside before downloading again.

How the experiments work

The first experiments used GPT-6 Astra in Codex to build the environments and write the robot-control code. I guided the task setup and gave feedback along the way. The controllers read positions and contacts from the simulator. The code here runs locally without calling a model API. Each experiment's README explains its setup and what the demo shows.

License

Original code is MIT licensed. Robot models retain their upstream licenses. The dove artwork and artwork-derived assets are excluded from the MIT grant; see the drawing credits.