Rain falling down the webcam feed is a five-minute effect: give some particles a downward velocity, draw a line for each, respawn them at the top when they leave the frame. Nobody is impressed.
The version that is interesting is rain that knows you are there. Drops fall past the background untouched, and the instant one crosses onto your shoulder it bursts into a spray of droplets that fly outward, arc back down under gravity, and leave a wet mark that runs down you afterwards.
That sounds like it needs depth sensing or a physics engine. It needs neither. It is one boolean array per frame and a rising-edge test.
What it does: a live webcam filter where rain collides with your silhouette — per-hit splash bursts with their own gravity, and a fading blue-tinted wetness trail that smears downward so water appears to run off you between impacts.
Stack: Python, NumPy, OpenCV, MediaPipe Image Segmenter. No GPU, no physics library, no depth camera. Around 400 lines.
MediaPipe’s ImageSegmenter running the
selfie_segmenter.tflite model takes a frame and returns a
confidence mask — a float per pixel, roughly “how likely is
this a person.” Threshold it and you have the only piece of world
knowledge the whole effect needs:
The model is about 250 KB and runs on CPU. Note
segment_for_video rather than segment —
the video mode takes a timestamp and keeps temporal state, so the mask
stays stable frame to frame instead of flickering at the edges.
Everything after this point is just particles asking that array questions.
Checking whether a drop is on your body is a lookup. Because the mask is
indexed [row, column] and particle positions are floats,
you cast and clip:
Fancy-indexing with two arrays samples all several-hundred drops in a single operation. There is no Python loop over particles.
But “is this drop on the body” is the wrong question. A drop that has travelled two pixels into your arm is still on the body, and if you spawn a splash whenever the answer is true, every drop spawns splashes on every frame it spends inside you. You get a permanent fog of spray, and the particle pool drains in about three frames.
What you want is the transition — the frame where a drop goes from background to body. So keep last frame’s answer and compare:
That single & ~previous is the difference between fog
and impact. It is the same rising-edge trick you would use to debounce a
button, applied to a few hundred particles at once.
A hit then does two things: spawn a splash burst at that position, and mark the drop for respawn at the top of the frame. Which means the same boolean drives both:
One subtlety worth writing down. When you store the history, mask out the drops you just respawned:
Skip that & ~respawn and a drop that hit your body
carries on_body_prev = True into its new life at the top of
the frame. If it happens to respawn above your head, its first frame
reads as “already on the body,” no rising edge fires, and it
silently passes straight through you. Roughly one drop in six goes
ghost.
There are no particle objects anywhere. A drop is not a class instance;
it is index i across eight parallel NumPy arrays —
rx, ry, rvx, rvy,
rlen, and so on. Drops are never created or destroyed, only
recycled:
Splashes need the same treatment but they genuinely come and go, so they
get an salive boolean and spawning means claiming free
slots:
A splash gets a random angle, a speed between 40 and 220 px/s, and a deliberately upward-biased vertical velocity so droplets fly up off the impact before gravity pulls them back — which is what real water does when it hits a surface:
The -np.abs(...) is the whole trick. A plain
sin would send half the droplets downward into your chest,
which reads as the drop passing through you rather than off you.
Splashes alone look like sparks. Water has memory — it stays on you and runs down. That is a single-channel float buffer the size of the frame, which impacts add into:
np.add.at rather than wet_trail[yi, xi] += 0.9
matters here: plain fancy-index assignment does not accumulate when the
same pixel appears twice in the index arrays, so two drops landing on
one pixel would count as one. add.at is the unbuffered
version that adds every occurrence.
Then each frame the buffer decays, and a fraction of every row is copied two rows down — a one-directional blur that reads as dripping:
Rendering it is a blue lift plus a slight darkening, so wet areas look glossy rather than merely tinted:
The darkening is doing more work than the tint. Adding brightness alone makes you look painted; taking a little light away where the water sits makes it look like a surface.
First real run, immediate crash:
The traceback points at the respawn line, but nothing is wrong there.
respawn arrives already 2D, and the cause is four
statements upstream: this version of MediaPipe returns the confidence
mask as (H, W, 1), not (H, W). With a trailing
channel axis, mask[yi, xi] returns shape
(n, 1) instead of (n,) — so every
downstream boolean inherits the extra axis and eventually tries to index
a 1D array with it.
Worth internalising as a shape bug rather than an indexing bug: NumPy propagated a wrong shape silently through four operations before anything complained, and it complained in the wrong place. The fix is at the source, guarded so it still works on versions that return 2D:
The download helper looked reasonable:
The guard starts with if model_path and ..., but
model_path is None unless you pass
--model explicitly. So on the default path the cache check
short-circuits to false and the model is re-downloaded on
every single run — while the code immediately below
demonstrates it knew the default location all along. Resolve the path
first, then check it:
A cache that silently never hits is worse than no cache, because the only symptom is slowness you attribute to something else.
This is the interesting one. Turning the density up made the effect look worse — more streaks in the air, less spray coming off the body.
The splash pool was a fixed 1500 particles while the drop pool was
adjustable up to 2000. Each hit wants 7 droplets, and at any moment a
good fraction of your drops are in contact with you, so at high density
demand runs to several thousand. And _spawn_splash returns
early when no slots are free:
So the pool sat permanently exhausted. Every additional raindrop competed for slots that were already gone, and because the drop was still marked for respawn on hit, the extra rain contributed nothing but cost. The failure mode was not a crash or a slowdown — it was the effect quietly getting less impressive as I turned it up.
The fix is to make the pool a function of the rain, not a constant:
Twelve slots per drop covers the worst case with margin. And since
density is adjustable at runtime, set_density has to grow
the pool too — otherwise the bug simply comes back the moment you
press +.
While there: the +/- keys stepped by a flat
20, so getting from 160 drops to 1500 was 67 presses. Proportional steps
make the same range eight:
160 → 226 → 315 → 435 → 597 → 815 → 1110 → 1508.
Simulation and render only, on synthetic frames with a body-shaped mask:
| Drops | Live splashes | ms/frame | fps |
|---|---|---|---|
| 160 | 370 | 4.9 | 204 |
| 600 | 1249 | 7.8 | 128 |
| 1500 | 3481 | 13.7 | 73 |
| 3000 | 7011 | 22.6 | 44 |
Segmentation adds a fixed cost on top, so 600–1500 drops is the band that still holds 30 fps capture comfortably. That is genuinely heavy rain.
One honest note, since the project README claims everything is
vectorized: the simulation is, but the renderer is not. Drawing streaks
and droplets is a Python loop with a cv2.line or
cv2.circle call per particle, and past about a thousand
drops that loop — not the physics — is the bottleneck. The
scaling in the table is essentially linear in particle count for exactly
that reason. Batching the streaks into a single
cv2.polylines call is the obvious next win.
polylines call, and stop paying per-particle Python
overhead.
mask[yi, xi].
now & ~previous), not the
state, or every drop splashes on every frame it is inside you.
True and pass through you invisibly.
alive boolean; recycle indices,
never allocate per particle.
-np.abs(...) so spray
comes off the surface rather than through it.
np.add.at for the wetness buffer — plain
fancy-index += drops duplicate pixels.
The reason this effect works is that segmentation collapses a hard problem — where is the person, in 3D, this frame — into an array lookup. Once your world model is a boolean grid you can index, particles interacting with a human body costs about as much as particles interacting with nothing.