Making Rain Splash Off Your Body in Real Time

Overview

Rain falling down the webcam feed is a five-minute effect: give some particles a downward velocity, draw a line for each, respawn them at the top when they leave the frame. Nobody is impressed.

The version that is interesting is rain that knows you are there. Drops fall past the background untouched, and the instant one crosses onto your shoulder it bursts into a spray of droplets that fly outward, arc back down under gravity, and leave a wet mark that runs down you afterwards.

That sounds like it needs depth sensing or a physics engine. It needs neither. It is one boolean array per frame and a rising-edge test.

1. Your body is a boolean array

MediaPipe’s ImageSegmenter running the selfie_segmenter.tflite model takes a frame and returns a confidence mask — a float per pixel, roughly “how likely is this a person.” Threshold it and you have the only piece of world knowledge the whole effect needs:

The model is about 250 KB and runs on CPU. Note segment_for_video rather than segment — the video mode takes a timestamp and keeps temporal state, so the mask stays stable frame to frame instead of flickering at the edges.

Everything after this point is just particles asking that array questions.

2. Collision is a rising-edge test, and that detail is the effect

Checking whether a drop is on your body is a lookup. Because the mask is indexed [row, column] and particle positions are floats, you cast and clip:

Fancy-indexing with two arrays samples all several-hundred drops in a single operation. There is no Python loop over particles.

But “is this drop on the body” is the wrong question. A drop that has travelled two pixels into your arm is still on the body, and if you spawn a splash whenever the answer is true, every drop spawns splashes on every frame it spends inside you. You get a permanent fog of spray, and the particle pool drains in about three frames.

What you want is the transition — the frame where a drop goes from background to body. So keep last frame’s answer and compare:

That single & ~previous is the difference between fog and impact. It is the same rising-edge trick you would use to debounce a button, applied to a few hundred particles at once.

A hit then does two things: spawn a splash burst at that position, and mark the drop for respawn at the top of the frame. Which means the same boolean drives both:

One subtlety worth writing down. When you store the history, mask out the drops you just respawned:

Skip that & ~respawn and a drop that hit your body carries on_body_prev = True into its new life at the top of the frame. If it happens to respawn above your head, its first frame reads as “already on the body,” no rising edge fires, and it silently passes straight through you. Roughly one drop in six goes ghost.

3. Fixed pools, never allocation

There are no particle objects anywhere. A drop is not a class instance; it is index i across eight parallel NumPy arrays — rx, ry, rvx, rvy, rlen, and so on. Drops are never created or destroyed, only recycled:

Splashes need the same treatment but they genuinely come and go, so they get an salive boolean and spawning means claiming free slots:

A splash gets a random angle, a speed between 40 and 220 px/s, and a deliberately upward-biased vertical velocity so droplets fly up off the impact before gravity pulls them back — which is what real water does when it hits a surface:

The -np.abs(...) is the whole trick. A plain sin would send half the droplets downward into your chest, which reads as the drop passing through you rather than off you.

4. Wetness is one float buffer, decayed and smeared

Splashes alone look like sparks. Water has memory — it stays on you and runs down. That is a single-channel float buffer the size of the frame, which impacts add into:

np.add.at rather than wet_trail[yi, xi] += 0.9 matters here: plain fancy-index assignment does not accumulate when the same pixel appears twice in the index arrays, so two drops landing on one pixel would count as one. add.at is the unbuffered version that adds every occurrence.

Then each frame the buffer decays, and a fraction of every row is copied two rows down — a one-directional blur that reads as dripping:

Rendering it is a blue lift plus a slight darkening, so wet areas look glossy rather than merely tinted:

The darkening is doing more work than the tint. Adding brightness alone makes you look painted; taking a little light away where the water sits makes it look like a surface.

5. Three bugs

The mask had a third dimension

First real run, immediate crash:

The traceback points at the respawn line, but nothing is wrong there. respawn arrives already 2D, and the cause is four statements upstream: this version of MediaPipe returns the confidence mask as (H, W, 1), not (H, W). With a trailing channel axis, mask[yi, xi] returns shape (n, 1) instead of (n,) — so every downstream boolean inherits the extra axis and eventually tries to index a 1D array with it.

Worth internalising as a shape bug rather than an indexing bug: NumPy propagated a wrong shape silently through four operations before anything complained, and it complained in the wrong place. The fix is at the source, guarded so it still works on versions that return 2D:

The model cache never hit

The download helper looked reasonable:

The guard starts with if model_path and ..., but model_path is None unless you pass --model explicitly. So on the default path the cache check short-circuits to false and the model is re-downloaded on every single run — while the code immediately below demonstrates it knew the default location all along. Resolve the path first, then check it:

A cache that silently never hits is worse than no cache, because the only symptom is slowness you attribute to something else.

Heavier rain produced fewer splashes

This is the interesting one. Turning the density up made the effect look worse — more streaks in the air, less spray coming off the body.

The splash pool was a fixed 1500 particles while the drop pool was adjustable up to 2000. Each hit wants 7 droplets, and at any moment a good fraction of your drops are in contact with you, so at high density demand runs to several thousand. And _spawn_splash returns early when no slots are free:

So the pool sat permanently exhausted. Every additional raindrop competed for slots that were already gone, and because the drop was still marked for respawn on hit, the extra rain contributed nothing but cost. The failure mode was not a crash or a slowdown — it was the effect quietly getting less impressive as I turned it up.

The fix is to make the pool a function of the rain, not a constant:

Twelve slots per drop covers the worst case with margin. And since density is adjustable at runtime, set_density has to grow the pool too — otherwise the bug simply comes back the moment you press +.

While there: the +/- keys stepped by a flat 20, so getting from 160 drops to 1500 was 67 presses. Proportional steps make the same range eight:

160 → 226 → 315 → 435 → 597 → 815 → 1110 → 1508.

Performance

Simulation and render only, on synthetic frames with a body-shaped mask:

Segmentation adds a fixed cost on top, so 600–1500 drops is the band that still holds 30 fps capture comfortably. That is genuinely heavy rain.

One honest note, since the project README claims everything is vectorized: the simulation is, but the renderer is not. Drawing streaks and droplets is a Python loop with a cv2.line or cv2.circle call per particle, and past about a thousand drops that loop — not the physics — is the bottleneck. The scaling in the table is essentially linear in particle count for exactly that reason. Batching the streaks into a single cv2.polylines call is the obvious next win.

What I would do next

• Splash along the surface normal. Take the gradient of the mask at the impact point and bias the burst away from your body instead of spraying isotropically. Water hitting your shoulder should go sideways, not straight up.
• Occlusion. Drops currently render in front of you even when they are behind you. The mask already says which pixels are the body — gating the streak draw against it would put rain behind the subject for free.
• Wetness that accumulates. The trail decays uniformly. Letting it saturate on the shoulders and head, where rain actually lands, would build up soaked patches over time.
• Batch the renderer into one polylines call, and stop paying per-particle Python overhead.

Recap

• A thresholded segmentation mask is a boolean array; collision detection against your body is mask[yi, xi].
• Test the rising edge (now & ~previous), not the state, or every drop splashes on every frame it is inside you.
• Mask respawned particles out of that history, or recycled drops inherit a stale True and pass through you invisibly.
• Fixed pools with an alive boolean; recycle indices, never allocate per particle.
• Bias splash velocity upward with -np.abs(...) so spray comes off the surface rather than through it.
• Use np.add.at for the wetness buffer — plain fancy-index += drops duplicate pixels.
• Darkening sells wetness more than blue tint does.
• A wrong array shape from a library can propagate silently and crash four operations later, in code that is correct. Fix shapes at the boundary.
• Size a particle pool relative to what feeds it. A pool that is permanently exhausted degrades quality with no error and no slowdown.

The reason this effect works is that segmentation collapses a hard problem — where is the person, in 3D, this frame — into an array lookup. Once your world model is a boolean grid you can index, particles interacting with a human body costs about as much as particles interacting with nothing.