I wanted the effect you have seen in TouchDesigner reels: you wave a hand at the camera and the whole image behaves like the surface of mercury — ripples spreading from your fingertips, your face smearing into chrome, the room melting behind you.
TouchDesigner is the obvious tool. But the entire thing is four ideas stacked on top of each other, and all four are a handful of NumPy and OpenCV calls. So I built it in Python instead. It runs at 77 fps at 960×540.
What it does: a live webcam filter that turns the frame into a rippling liquid-metal surface you can touch with your hands — wave propagation, refraction, specular chrome highlights, ink contours, and a feedback loop that makes gestures smear and melt over seconds.
Stack: Python, NumPy, OpenCV, MediaPipe Hands. No GPU, no shaders, no node graph. Roughly 400 lines.
Here is how each layer works.
The whole simulation is a 2D height field — one float per pixel, at half the resolution of the video. Every frame you update it with the discretised wave equation:
That is it. Each point gets pulled toward the average of the points around
it, and the − previous term is what makes it
overshoot and oscillate instead of just blurring toward
flatness. Multiply the result by a damping factor slightly under 1.0 and
the ripples lose energy and eventually die.
In code, the “average of neighbours, times two” is a convolution kernel whose weights sum to 2.0:
Most implementations of this use only the four orthogonal neighbours.
Including the diagonals costs nothing — it is the same single
filter2D call — and the ripples come out noticeably
rounder. With a 4-tap kernel you can see the square grid in the
wavefronts.
damping is the personality knob. At 0.955 you get a pool:
rings spread, ring, and settle. At 0.9965 energy barely leaves, ripples
pile up on top of each other and the surface never resolves. That is the
melt.
The height field is invisible. What you actually see is what it does to the pixels underneath it — and real refraction bends light by the slope of the surface, not its height. A flat spot passes the image straight through no matter how high it sits; a steep wall shoves it sideways.
So: take the gradient, use it as a displacement map, and let
cv2.remap do the rest.
strength is “how many pixels a unit of slope displaces
you.” Around 26 it reads as water. Around 100 the image tears apart
into ribbons.
Two details that matter more than they look:
Build the meshgrid once. np.meshgrid for a
960×540 frame allocates two 2 MB float arrays. Doing that 30 times a
second is pure waste. Allocate base_x/base_y at
startup and write into preallocated map_x/map_y
with cv2.multiply(..., dst=) and
cv2.add(..., dst=).
Simulate at half resolution. The height field lives at 480×270 and only the gradient gets upscaled to frame size before the remap. Water is smooth — you lose nothing visually, and you quarter the cost of the most expensive part of the loop.
The mercury quality is not in the warping. Warping alone gives you a funhouse mirror — recognisable, but flat and plasticky. What sells liquid metal is specular lighting, and the gradient you already computed is the surface normal.
Dot it with a light direction and add the result to the pixels:
Slopes tilted toward the light go white, slopes tilted away go black. That is the whole thing — and because it is driven by the same field that is doing the warping, the highlights track the deformation perfectly. Free, essentially.
I added one more term: an ink pass that darkens the steepest slopes, which produces the black contour lines you see outlining the blobs in liquid-metal footage.
This is the one that turns a nice ripple effect into the thing I was actually after, and it is a one-liner.
Instead of warping the fresh camera frame each time, blend the previous warped output back in as the source:
At fb = 0, every frame is warped once from clean video —
ripples on glass. At fb = 0.86, each frame is warping
something that has already been warped, over and over. Distortion
compounds. A gesture from two seconds ago is still smeared across the
frame, being re-bent by whatever you are doing now. That is the melting.
It self-corrects, too: when the water goes flat, output ≈ input, and the feedback loop converges back to the live frame within about fifteen frames. So it melts under your hands and reheals when you stop.
MediaPipe Hands gives you 21 landmarks per hand at maybe 8 ms a frame on CPU. Run it on a 320px-wide copy of the frame — you do not need detail to find a hand, and full-res detection roughly triples the cost.
The naive integration is: for each landmark, stamp a dimple at that position. It looks wrong. You get a hand-shaped dent that follows you around like a cursor, not something touching water.
Two fixes make it feel physical.
Draw the stroke, not the point. At 30 fps a fast hand moves 40+ pixels between frames. Stamping circles gives you a dotted line of separate splashes. Instead, draw a line from where the landmark was last frame to where it is now:
Now a swipe leaves a continuous wake.
Scale amplitude by speed. A hand resting in frame should barely dimple the surface; a fast swipe should gash it. One multiplier:
That single line is most of the difference between “an effect is happening near my hand” and “I am touching this.”
One bug worth flagging: I originally keyed the per-hand position history by
MediaPipe’s detection index. When the detection order flips between
frames — which it does — hand 0 becomes hand 1, and every
landmark reports a velocity equal to the distance between your two hands.
You get a random explosion across the frame. Key on the handedness label
("Left" / "Right") instead.
First full run of the melt preset rendered a completely black frame. Not glitchy — black.
The cause is specific to running the wave sim at damping 0.9965. Energy barely leaves the system, and my hands were pumping energy in every frame, so the height field ratcheted upward until it hit the safety clip at ±6. Once it is clipped everywhere, everything is a plateau at maximum slope. The ink pass, which darkens steep slopes, dutifully darkened every pixel in the frame.
The fix is not a harder clip — that just makes the plateau arrive sooner. It is a governor on total energy: measure the RMS of the field, and if it is above target, scale the whole thing down.
The ** 0.35 matters — it makes the correction gentle,
spread over several frames, so you do not hear the sim “duck”
every time you move fast. Scaling both buffers matters too: they encode
position and implicitly velocity, and touching only one shoves
energy back in through the side door.
Result: a surface that churns indefinitely and never saturates. Exactly the behaviour I wanted from damping ≈ 1, without the mud.
Measured on synthetic frames, effect only:
| Resolution | ms/frame | fps |
|---|---|---|
| 960×540 | 12.9 | 77 |
| 1280×720 | 25.5 | 39 |
MediaPipe adds roughly 8 ms. 960×540 leaves comfortable headroom for
30 fps capture, which is why it is the default. Everything is
cv2 calls on contiguous float32 arrays — there is no
Python loop over pixels anywhere, and there is no GPU involved at all.
2 · average(neighbours) − previous,
times a damping factor — one filter2D, and the
diagonals are free.
cv2.remap.
The core insight is that a wave field, a gradient, and a
remap get you most of the way to effects that look like they
needed a node graph and a GPU. The remaining 20% is feedback and velocity
— and those are one line each.