Robot Darkroom

Guests take a picture at a photo booth outside a glass room. Inside, an arm carries a 1 m strip of ultraviolet LEDs; swept 2 mm from a sheet of photochromic paper, the strip prints the picture in violet, and it fades back to white within 10–20 s. Between prints the arm dances with the strip lit, and it notices the people outside the glass.

Behind it is a show system I built for one arm, light and an audience: the room as an Isaac Sim twin to rehearse in, a generated library of clips, and a state machine that composes them as the show runs, with TouchDesigner and the people in the room as inputs. Where Motion-Core plays keyframed animation across several arms, this composes a show live.

The room

It ran through the evening of Todos Bien @ Volvox on 10.02, on its own. The ceiling light follows the show, dimming for each scan.

Drawing with light

The pigment fades faster than one pass can write a picture, so a scan is eight passes, down and up, slowing from 0.3 to 0.15 m/s with the LEDs lit both ways: about 51 s. TouchDesigner writes the image column by column as the strip moves.

Behaviours

Between prints the arm plays from a library of 70 short clips at four positions: it peeks, nods, bounces, shies away, wipes the air. None of them is keyframed: they are generated as a library (below), and the show strings them along an energy arc, never the same one twice in a row.

Noticing people

A depth camera watches the space in front of the glass. Step onto the spot or wave, and the arm leaves what it was doing, turns to you, follows you along the glass, and nods goodbye when you go. On the real arm it has engaged 9 times over four runs with no unsafe pose.

How it runs

The room, as a twin

The lab was measured with the arm itself: its tool touched to the floor and walls, its own kinematics agreeing with the controller to about 1.1 mm. A RoomPlan scan filled in the rest, aligned to the probed walls within 0.47°. The room is kept as a USD stage that Isaac Sim loads as it is, lit to match the room from photographs, so a render reads like the place.

Every clip is checked against this room before it plays: the arm as capsules fitted to its meshes, against the measured floor, walls and furniture. A clip that comes too close fails the build.

Rehearsal in Isaac Sim

The whole show runs in Isaac Sim before it runs on the arm, and the light runs with it. The UV strip was modelled on the tool before it was ever mounted, so the twin shows where each LED sits as the arm moves, what TouchDesigner's animation looks like along it, how a scan writes the paper, and how the ceiling light plays against all of it, live, as TouchDesigner sends it. The cable for the strip is simulated too, to find where it can be clipped to the arm without binding. The full library replayed with 0 contacts and a worst tracking error of 0.96°.

The scan's layout was chosen here too, from where the guests would stand.

A library, not a timeline

Nothing in the show is keyframed end to end. The clips are generated in Houdini with the Houdini Robot Toolkit: gestures aimed at a guest at one position, and at another, dance phrases built from Laban's efforts (punch, glide, float, wring and the rest), each labelled afterwards by what it measures as, not only by what it was asked to be.

Composing the show live

A build compiles them into 85 segments (70 idle clips at four positions, the moves between them, and the scan) and checks every one against the room. Profiling cut the build from 33 min to 141s, with the output identical to the bit.

At run time a state machine picks the next clip, weighted, never twice in a row, along an energy arc over 3 min, and finishes the running clip before a scan or a guest takes over. The arm gets one continuous ServoJ stream at 125 Hz.

Clips are timed against the arm's measured accelerations, 300 deg/s² on the first three joints and 600 on the wrist, so they play at their designed length. Timed for velocity alone, an earlier export of the same curve would have played 8.54× slower than drawn.

At the party it ran the evening unattended. When the arm halted on its wrist singularity 3.4 h in (about once in 317 crossings), the runtime cleared the fault, made a checked move to where it was going, and the show went on.

The arm, inside TouchDesigner

The runtime tells TouchDesigner where the arm is and what it is doing over OSC: the scan position, the strip's distance to the paper, the beat of the current clip. TouchDesigner answers with the strip's LEDs and the room's ceiling light. The arm becomes one more thing in a scene the studio already lights and cues.