Previous post: Grown QWOP Walker Next post: Analytic vs Learned Inverse Kinematics

A Character That Follows Language Into Action

A small 3D world where typed goals become queued moves.

Overview

A language system can play many roles when given context. This page is my attempt at understanding Janus’s simulators theory

How It Works

Goal → language plan → tool list → queue → character → 3D scene

The page sends a full snapshot with each request: position, rotation, and object list. The server builds a prompt from that snapshot and returns tools. The client runs tools one by one. A failure in one step logs and continues with the next.

Six Tools

The character knows six moves and no more:

  • move_forward in meters
  • move_backward in meters, done as turn around then walk forward
  • rotate_left in degrees
  • rotate_right in degrees
  • go_to a named object, with auto path
  • pick_up a named object, with auto path and removal

Limits are fixed: 0.5–5 m per walk, 15–180° per turn. Unknown tools drop. Out-of-range numbers drop. Extra fields strip. Text numbers drop. A well-formed call to a missing object passes the shape check and fails at run time with Object "<name>" not found or not available.

Examples:

{ "tool": "go_to", "target": "red_cube" }
[
{ "tool": "rotate_left", "degrees": 90 },
{ "tool": "move_forward", "meters": 2 }
]

Scene

The room is 24 × 24 units. It holds 5 cubes: red, blue, green, yellow, purple. Each cube keeps its color and name. The names enter the prompt, so the plan can point at them.

Walk speed is 1.5 units/s. Turn speed is 120°/s. Arrival counts at 0.5 units. Named moves use path steering through the navmesh; stepped walks use arrive steering. Turns use timed rotation. Spawn points use random choice, so each load starts fresh. Each command still runs in order.

Animation has four clips: idle, walk, left turn, right turn. Fades run 0.3 s through the mixer.

Chat

The input sits beside the 3D view. Enter sends. The send button sends. Input locks while a reply runs. Each tool logs one line with run marks for start, pass, and fail. Unknown tools skip on the client.

Hide the widget mid-turn and rotation snaps to its target so the queue never stalls.

Checks

The tool surface has a golden set in src/__tests__/agent-eval.test.ts. It runs in CI and passes 22 cases.

Group Cases Check
Valid singles 8 Each tool at and inside its bounds parses
Multi-step 6 Order holds across turn-then-walk and go-then-pick
Adversarial 6 Unknown tools, bad ranges, and empty targets drop with a clear note
Field hygiene 2 Extra fields strip, text numbers drop

Two choices shape the checks. The parser checks shape and ranges only. Scene truth stays at run time. Rebuilt tool objects carry known fields only.

The harness grades parsing and checks, not plans. It proves bad output never reaches motion. It does not grade whether the plan is smart.

Next Step

Implement agentic loops. Add more characters with separate views and goals in the same room. Then watch for shared patterns: queuing, avoidance, turn-taking. The base for that work is here: fixed tools, checked inputs, serial runs.

References