Preface: Yapping…

I’m having a headache, I'm not gonna lie. May in the bay area wasn’t supposed to be this cold, at least I didn’t think so. Between the  pressure change, and the rain, the windy-ness, I’ve been spending more than 12 hours in bed each day. No amount of sleep was enough. I am tired, hungry, yet with no appetite, stressed for no reason, cold, always sneezing and with a light headache. I feel tight, I feel sore. I’m not sure if it's the age catching up to me, or I’m just being overly sensitive. 

Last night, I made a few songs with Sono. I wrote a few lines of lyrics, and asked Claude to rearrange them for me, then I made some style notes, and put it all into Sono. The results were astounding. Sono doesn’t allow you to reference existing artists directly, so all you have to do is to describe what you want to the best of your abilities. They got pretty close to what I wanted, or better than what I had in mind actually. I never made music, but after a few rounds, I too, started having opinions about how I want to fine tune the beats, instruments, singing style, which is not possible with generic prompts, so I think I might get my hands on some mixing equipment eventually. Time to put my baby keyboard skills in use. 

This is liberating, this experience being able to use AI-empowered music production tools with just describing what I want. It makes everything so much easier and makes everything a lot less daunting. I was not scared to experiment, and like a kid in a candy shop, I was quite excited. I’m getting lots of criticism about how I use AI in my creative process, people asking about if original materials are credited or paid for since they are used to train the models. Idk, i don’t really care about anything out in the open on the internet, in the public space, to be used by the model to train. They are technically already exposed to any possibilities of the future the moment you decide to make it public. People aren’t mad that their things are used “without consent”, making it public is consent. They are mad that AI learned from them, but turned out to be better than them. How can you be excited about something that could destroy humanity? Contrary to popular belief, I think only humanity can destroy humanity.

But anyways, back to the movies we were talking about. I found a documentary on Disney+ called “Light and Magic”(Light & Magic | Official Trailer | Disney+). It features people’s interviews, including George Lucas and everything else. It’s possibly easier to watch that than read my blog, though the two cover mostly different topics. The Abyss is this movie that James Cameron made prior to Titanic. He must have been really into deep sea diving related stories, first The Abyss, then Titanic, later Way of Water Avatar. It is true that most directors spend all their lives making different versions of the same story, since our lives are just different versions of the same story. And that was Cameron’s story, water, deep sea, spectacular tales. Before working on The Terminator 2, ILM had already worked on The Abyss for James Cameron, during which they did the water tentacle thingy, which was not as difficult as T-1000, which was like a fluid metal. This is more difficult because for water, the reflections and diffractions of light don’t have to be as accurate as a smooth and perfectly reflective material like T-1000. It still required writing and building mathematical equations into the engines, but it was okay if details were off a little bit. But not for T-1000. 

Water Tentacles: Second Take 

To simulate the water tentacles digitally, ILM needed software that could answer two separate questions: What shape are the water tentacles at any given moment? Given that shape, what does it look like photographically, with correct water optics? These are two completely different technical problems that had to be solved independently and then combined. Shape is a geometry problem. Appearance is a rendering problem. In 1989 neither had been solved for a fluid organic object at feature film quality. A rigid object can be defined by a fixed set of vertices. The geometry doesn't change. You model it once and render it from different angles. We call these rigid bodies. A fluid object has no fixed geometry. Its shape is continuously changing according to physical laws. To represent a fluid object digitally you need a mathematical description not of a fixed shape but of a shape-changing process.

The approach ILM used was parametric surface modeling — specifically a mathematical construct called a swept surface or generalized cylinder. The basic idea is this: define a central spine — a curve running through the center of the water tentacles from base to tip. Then define a cross-sectional profile — the shape of the water tentacles's cross-section at any point along the spine. The surface of the water tentacles is generated by sweeping that cross-section along the spine. The spine was defined as a spline curve — a mathematical curve defined by a small number of control points, with the curve smoothly interpolating between them. Move a control point and the spine bends. The software then recalculated the surface geometry for every frame. This gave the animators a manageable set of parameters to control — the positions of a handful of control points — that produced complex, organic-looking surface geometry. Instead of manually positioning thousands of vertices, the animator moved a few control points and the mathematics generated the resulting shape.

This is rather straight-forward in the mathematical sense, nothing confusing there. However, defining the shape is one problem, and making the shape change over time in a way that looks physically plausible is another. Real fluid motion is governed by the Navier-Stokes equations — a set of partial differential equations that describe how the velocity, pressure, and density of a fluid change over time and space. Solving Navier-Stokes fully for a complex fluid simulation is computationally prohibitive even today. In 1989 it was completely impossible in real time. ILM didn't solve Navier-Stokes, they find a way to approximated it, which is brilliant. The key approximation was to treat the water tentacles not as a fully simulated fluid but as a procedurally animated object: an object whose motion was generated by mathematical procedures designed to produce fluid-like behavior rather than by physically accurate simulation. Sinusoidal wave functions were applied along the spine to produce undulating motion — the water tentacles' characteristic rippling movement as it rises and moves through the air. The amplitude and frequency of these waves were parameters the animators could adjust. Higher frequency produced faster, tighter ripples. Lower amplitude produced gentler, more relaxed movement.

Noise functions — specifically Perlin noise, developed by Ken Perlin at NYU in 1983 — were layered on top of the wave functions to introduce organic irregularity. Pure sinusoidal waves look mathematical and regular. Adding Perlin noise breaks the regularity in a way that mimics the natural turbulence of real fluid. Perlin noise is a specific type of gradient noise that produces smooth, continuous random variation — it avoids the harsh discontinuities that simpler random functions produce, which would look wrong on a smooth water surface. The combination of wave functions for the large-scale motion and Perlin noise for the small-scale irregularity produced movement that read as genuinely fluid without requiring a full physical simulation.

The face morphing was the sequence's most technically demanding component and the one that had never been attempted before. As mentioned previously, the starting point was digitizing Mary Elizabeth Mastrantonio's face. It's worth mentioning here again, with the knowledge we gathered from the last post, that ILM used a structured light scanning system. It was a projector casting a grid of lines onto the face, with cameras capturing how those lines deformed across the facial geometry.

The deformation pattern allowed the software to calculate the three-dimensional position of points across the face's surface. The scanning technology in 1989 was primitive. The resolution was low by any subsequent standard: the scanned face had significantly fewer polygons than a contemporary face scan would produce. The raw scan data required extensive manual cleanup by technical artists who went in and corrected artifacts, filled gaps in the data, and refined the geometry to produce a usable model. I'm pasting a photo here that's more modern day technology, not what ILM did back then, but it gives you the general idea.

The resulting digital face model was then mapped onto the water tentacle's surface using a technique called free-form deformation, or FFD. FFD works by embedding the object being deformed — in this case the face geometry — inside a lattice of control points. Moving the lattice control points deforms the embedded object in a smooth, continuous way. ILM used FFD to gradually deform the water tentacles surface toward the face geometry — the water surface slowly acquiring the topology of Mastrantonio's face as the water tentacles approached her. The transition needed to feel like the water was genuinely taking on the face's shape rather than the face being pasted onto the water. This required careful timing of the FFD deformation relative to the water tentacles's movement and the water surface animation. The face had to emerge gradually, with the water surface still visibly present and moving even as it resolved into recognizable features. Once the geometry was defined and animated, the rendering system had to produce an image that looked like actual water — translucent, refractive, reflective, with a surface tension that created a specific kind of edge.

The renderer ILM used was an early version of Renderman — the same software they would extend for T2 two years later. Light passing through water bends its direction changes because water has a different refractive index than air. The amount of bending is described by Snell's Law: n₁ sin θ₁ = n₂ sin θ₂, where n₁ and n₂ are the refractive indices of the two media and θ₁ and θ₂ are the angles of the light ray relative to the surface normal. For water, n ≈ 1.33, meaning light bends significantly as it enters and exits the water surface. The shader implemented Snell's Law to calculate the refracted direction of light rays passing through the water tentacles. At the water surface, some light is reflected rather than refracted. The proportion reflected versus transmitted depends on the angle of incidence and is described by the Fresnel equations. At grazing angles, when you look at water nearly edge-on, almost all light is reflected, giving water its mirror-like quality at those angles. At steep angles, looking straight down into water,  most light passes through. The shader implemented the Fresnel equations to calculate the blend between reflected and refracted light at each point on the surface.

When light refracts through the curved water surface, it focuses and defocuses, creating the characteristic rippling light patterns visible on surfaces near water. These caustic patterns were calculated and projected onto nearby surfaces in the scene. Where the water meets air at the water tentacles's boundary, surface tension creates a specific optical behavior: a bright highlight at the edge where light refracts through the curved surface boundary. This was added as a post-process pass on the rendered geometry, calculating the edge curvature and adding the appropriate highlight. Creating a water tentacle that’s believable basically requires observing all qualities of water while interacting with light and its environment, and add as many qualities and as many passes to it as possible, for the difference between the imitate and the real thing indistinguishable. We call this “photo-real”. 

The water tentacles sequence runs approximately seventy-five seconds in the finished film. At twenty-four frames per second that's roughly 1,800 frames. Each frame required the geometry simulation, the surface animation, the shader calculation for refraction and reflection, and the compositing with distortion mapping. On the SGI workstations ILM was using in 1989, which was Silicon Graphics IRIS workstations running at a few MIPS,  each frame took significant compute time. The total compute time for the sequence was measured in weeks of continuous processing across multiple machines. ILM had a small render farm that ran continuously through the production of the sequence. Each of these became standard tools in the VFX pipeline. Every water simulation, every fire, every smoke effect in modern cinema descends from the mathematical approach ILM developed for the water tentacles.  It’s actually pretty cool.

🔥