Writing / Engineering notes

8 GitHub Tools for Making Videos With Claude Code or Codex

Play the talking character I made with Codex and try two examples for each of the eight tools that bring a video to life.

Read as Markdown

Press play on the robot above. “Piper gave me this voice. Rhubarb tells my mouth when to move.” I asked Codex to write that line, voice it, animate the character, and export an MP4. Listen once, then watch the mouth on the word “move.”

Watch the full walkthrough on YouTube

1. Piper gives the character a voice

Play Lessac and Ryan saying the same sentence. Listen to how each voice handles “meteorological.” These are Piper’s official samples. For the robot at the top of the page, I used Lessac to turn one line of text into a WAV file on my own computer. Piper repository · Voice samples and model information

Lessac
Ryan

With the Lessac model and its configuration file in the current directory, this produces the character’s voice track.

bash
echo 'Piper gave me this voice. Rhubarb tells my mouth when to move.' > line.txt
piper -m en_US-lessac-high.onnx -f line.wav < line.txt

2. Rhubarb Lip Sync turns speech into mouth shapes

Choose a voice below and press play. The letter beside the robot is the mouth shape Rhubarb assigned to that instant in the recording. Pause on an open vowel, then drag the audio playhead and watch the shape change. Rhubarb Lip Sync repository

Inspect the cue timings for Lessac or Ryan.

For the character’s own line, Rhubarb produced 32 cues. Each cue has a start time, an end time, and a letter. My drawing code uses that letter to choose a mouth. Here’s the command for analyzing your own recording with the script as a hint.

bash
rhubarb -r phonetic -f json -d line.txt -o mouth-cues.json line.wav

3. Remotion puts the drawings on a timeline

Scrub the spring title below one frame at a time, then switch to the second title. Both are React compositions. In the character scene, I used the same frame clock to choose a mouth drawing and start the audio. Remotion repository · Licensing

This is the cue lookup from the character scene. The narration starts 0.2 seconds into that scene, so the lookup accounts for those frames. Shape X is the resting mouth.

jsx
const frame = useCurrentFrame();
const {fps} = useVideoConfig();
const time = (frame - Math.round(0.2 * fps)) / fps;
const cue = cues.mouthCues.find(
  c => time >= c.start && time < c.end
)?.value ?? 'X';

4. FFmpeg makes the file you can share

Play the widescreen pan and the vertical crop. Both started as still photos. FFmpeg scales or crops the photograph, draws the title, adds fades, and encodes the frames as H.264. FFmpeg repository

Widescreen pan
Vertical crop

The first render and the revision

The first robot was already in sync. At viewing size, I could barely see its mouth move. I asked Codex to make the character bigger and more expressive. Compare the two clips at about three seconds. The revised drawing has distinct teeth and tongue shapes, and both clips use the same voice recording and cue timings.

First render
Revised render

5. Anime.js animates a headline or a chart

Switch between the letters and the data bars. Both examples use a stagger, a small delay between the start of each element’s animation. In the chart, each bar grows from its baseline. Its height carries the value and the animation controls controlslist="nodownload" how quickly it gets there. Anime.js repository

js
import {animate, stagger} from 'animejs';

animate('.real-bars i', {
  scaleY: [0.06, 1],
  duration: 1500,
  delay: stagger(120),
  ease: 'outElastic(1,.7)',
});

6. canvas-sketch draws moving backgrounds

Compare the contour field with the orbital drawing. The first sends waves across horizontal lines. The second draws each line around a center point. Both examples redraw the strokes as time advances. Pause either one to see the individual paths. canvas-sketch repository

7. gpu-io simulates dye in motion

Both previews start with three patches of color. On each step, the shader samples the previous frame slightly upstream to find the color at the new position. Switch to Swirling dye to see what a stronger rotation does to the same starting pattern. These examples use WebGL on your device. gpu-io repository

8. Three.js adds light and depth

The torus knot has a smooth surface. Switch to the faceted shape and watch how sharply the highlights change across its faces. The scene uses the same pair of directional lights, which makes the difference easy to see. Three.js repository

Watch the walkthrough

For a first project, give your agent one five-second title card to build in Remotion and export with FFmpeg. Once the timing feels right, add a line of Piper narration. Rhubarb becomes useful when you have a character whose mouth needs to follow that line.

Watch the walkthrough on YouTube or jump to the character comparison at 4:21.

Media credits