I.

Introduction

Special Effects or SFX, the use of specialized techniques to create images and scenes that would be impossible, dangerous, or prohibitively expensive to film using standard methods of cinematography.

From the moment moving pictures were invented in the last years of the 19th century, film-makers sought ways to exploit the medium’s unique photochemical and mechanical properties in order to dazzle and deceive the viewer.

Early experimenters quickly realized that film did not have to be shot continuously, and that during filming the camera could be stopped, some aspect of the scene changed, and then filming resumed. The result would be an apparently spontaneous and inexplicable on-screen event. This basic trick, known as “stop-action”, was used to create what is considered the first-ever special effects shot in the early short film The Execution of Mary Queen of Scots (1895). To create the illusion of a royal beheading, an actress knelt down before being replaced by a dummy whose head was chopped off.

The master of early film effects was the French magician George Méliès, who pioneered many of the basic techniques that would be used and improved upon over the next century. Méliès made hundreds of short films using methods such as double exposure (exposing the negative in the camera more than once in order to film multiple objects—such as the same person appearing twice in the same image), stop-action, forced perspective (the building of models and sets with an exaggerated perspective—allowing apparently huge scenes to be built within a small studio), painted environments, models, and special effects make-up. They included the first great special effects science fiction film, Voyage dans la Lune (1902; A Trip to the Moon).

“Special effects” became a term broadly used to describe any production technique that deviated from ordinary methods of live-action film-making. Several distinct areas of special effects production have developed over the years leading to a general consensus within the film production industry as to the terms that are used to describe each discipline.

II.

Special Effects

Although “special effects” is commonly, and confusingly, used to describe any aspect of effects production, the term is generally used within the film industry to refer to any process or illusion that is performed “live” in front of the camera during filming of the main live action. This can include the creation of full-scale fires and explosions, bullet hits, smoke, rain, collapsing buildings, action props, and crashing vehicles. Such effects are also called “physical effects” or “mechanical effects”.

A branch of physical effects, special effects make-up, includes the design and creation of sculpted make-up, wounds, masks, and remotely controlled mechanical puppets, or “animatronics”, that are filmed on set along with the live-action performers.

III.

Visual Effects

The term “visual effects” is used to describe any form of additional manipulation that is made to the image after it has been photographed. It can also be used to describe the creation and filming of scenery, characters, and action using special techniques that can only be achieved separately from the main live-action filming. This could include the building and filming of models, and the creation and animation of fantasy characters.

The possibilities of visual effects were once constrained by the optical and photochemical limitations of the analogue medium of film. Such techniques, relied upon until the mid-1990s, were known as optical visual effects.

However, the rapid development of computing and digital imaging technology has made the production of previously unthinkable imagery available to even modest productions. Such techniques have now entirely replaced traditional optical processes, and are known as digital visual effects.

IV.

Travelling Mattes and Compositing

Historically, one of the greatest challenges in visual effects cinematography has been to find ways of combining elements filmed at different times and locations in order to produce a single, seamless shot. For example, being able to combine footage of an actor filmed in a Hollywood studio with stock footage of the pyramids would save the time and expense of sending an entire production crew to Egypt.

Early methods of combining separately filmed images involved exposing the original camera negative to multiple images in order to create a combination, or composite. Early examples of this technique are seen in the Edwin S. Porter film The Great Train Robbery (1903). In one scene a robbery takes place inside a telegraph office while a train can be seen passing outside the window. This was achieved by filming the telegraph office in a studio and masking or “matteing” off the area of the frame occupied by the window, thus leaving that portion of the negative unexposed. The camera was then taken to a railway, a mask or “matte” applied to the already exposed area of the frame, and a passing train photographed into the unexposed window area of the frame. Two separate exposures filmed in two locations were therefore combined on one piece of film to produce a single composite shot.

Film-makers soon searched for ways of convincingly combining several images without having to apply masks to the frame during filming. Furthermore, they wanted combined elements to be able to travel around the frame, rather than stay in one fixed place, as in the window example, above.

The most effective early method was the Williams Process, patented by Hollywood cinematographer Frank Williams in 1918. Though relatively crude, this method established the basic principles that remain at the heart of all matte photography techniques to this day.

The process required actors who were to be placed into other environments to be filmed in front of a plain white (or black) background in the studio. The developed negative was then copied using high-contrast film, which turned parts of the image into either solid black or clear “white” areas. In the case of an actor walking around the screen, this would result in an image that was clear except for a solid black silhouette or “male matte” of the moving actor. The image would next be copied to produce a negative “female matte”, in which the area of the actor was clear but the surrounding frame was black. Pre-filmed footage of background scenery such as the pyramids, for example, would then be sandwiched with the male matte of the actor and re-photographed onto a new negative. This would allow the pyramid image to be copied onto the new piece of film except in the area occupied by the male matte.

The footage of the actor was combined with the female matte and also photographed onto the negative. This time the area already exposed to the pyramid scenery would be covered by the black area of the matte and be prevented from further exposure, while the image of the actor would be copied into the uncovered, previously unexposed, area of the negative. When developed the result would be a scene in which a studio-filmed actor appears to walk in front of the pyramids.

This method of male and female counter-mattes, allowing elements such as actors to move around within the frame, is known together with its subsequent optical and digital variations as travelling matte photography.

Various methods for creating travelling mattes were developed over the years, the greatest challenges coming with the development of colour photography. From the 1950s onwards most travelling mattes were created by filming actors in front of blue screens. Several methods of blue screen travelling matte photography were developed, each using a complex combination of coloured filters to separate performers, models, or animated characters from the blue background and create the male and female mattes needed to place them into new environments. When elements could not be filmed in front of a blue screen they could even be painstakingly drawn around and isolated by hand—a process called rotoscoping.

The process of combining various pieces of footage using mattes is called compositing, and until the era of digital visual effects production this was achieved using a machine called an optical printer—the single most important piece of equipment at the heart of every visual effects studio. The first sophisticated optical printers were built by Linwood Dunn in the early 1930s. They consisted of a high quality projector that beamed the image from one strip of developed film into a camera, which re-photographed it onto a new negative. By using a system of beam-splitting prisms, several projectors could beam their images into the camera simultaneously, allowing a number of elements and their mattes to be combined at once. The most sophisticated optical printers were built in the 1980s, allowing up to four images to be accurately combined in each “pass”; the negative film in the camera could then be rewound and exposed to another four images, and so on. Some of the most ambitious optically composited images are seen in Return of the Jedi (1983, George Lucas) for which the effects company Industrial Light and Magic sometimes combined over a hundred images of spaceships, scenery, and animation, and their associated travelling mattes, to produce a single brief shot.

Several alternative techniques to the laborious process of creating travelling mattes were developed from the 1930s onwards. Most popular of these was rear projection, in which actors were filmed performing in front of a translucent screen onto which pre-filmed footage was projected from behind. This technique was most frequently used when studio-bound actors needed to look as if they were driving on the open road.

V.

Digital Compositing

Digital technology has enabled film-makers to combine multiple images in extraordinarily sophisticated and realistic ways, easily achieving shots that would be unthinkable using an optical printer.

Scenes to be digitally composited are usually first photographed on film and then scanned in order to create a digital version that can be manipulated on a computer. Increasingly feature films are shot digitally, recording images as digital information that can be played directly into a computer. Once the digital visual effects have been completed, however, most films are still printed back onto celluloid film for distribution to cinemas.

To create travelling mattes digitally the computer is simply instructed to remove the blue screen part of an image, and the colour will automatically be isolated and a set of male and female mattes produced. In fact, any colour can now be used as a background—green being the most common. Even if a character has not been filmed in front of a colour screen there is a range of sophisticated semi-automated methods of isolating an object and producing mattes. Many characters or objects in a scene may now be generated by computer, and these will automatically have a set of mattes created for them as part of the production process.

Once all the elements have been prepared they can be composited, with the operator building them up in layers on the screen. Digital compositing allows the manipulation of every aspect of the scene; size, shape, position, colour, shadows, reflections, and the interaction of every element can be endlessly tweaked until the desired image is achieved.

Digital technology allows filmed images to be manipulated in countless ways, making film production much more efficient. For example, scenes filmed in the spring can be made to look like autumn by turning green foliage brown. Period films can have modern features such as overhead power cables and television aerials erased from a scene after filming—saving the expense of hiding or physically removing them during location filming.

VI.

Animation

The basic principle at the heart of moving pictures is the ability to view still images shown at the speed of 24 frames per second, thus creating the illusion of continuous movement. Animation takes advantage of the fact that each frame in a sequence can be photographed individually. By manipulating objects incrementally between the photographing of each frame, animators are able to give inanimate objects the appearance of spontaneous movement when the finished film is projected at normal speed.

There are two broad categories of animation. Two-dimensional (2D) animation involves the photography of flat artwork such as drawings or paintings to produce what is often called a cartoon. Less well known is the practice of animating 2D artwork to create visual effects for live-action films. Elements such as sparks, lightning, gunfire, and laser bolts can all be drawn manually and photographed one frame at a time before being added to live-action images. These traditionally painstaking and laborious processes are now performed digitally.

Three-dimensional (3D) animation, on the other hand, involves manipulating the physical position of dimensional objects such as puppets and models from frame to frame. This technique, known as stop-motion or stop-frame animation, is best known for producing the performance of fantastic creatures in films such as King Kong (1933) or contemporary characters such as the Nick Park creations Wallace and Gromit. In these cases intricately engineered puppets with articulated joints are animated and filmed within miniature sets, or placed into real environments through a variety of compositing methods. The acknowledged master of stop-motion film animation is Ray Harryhausen, who created the fantasy characters in films such as Jason and the Argonauts (1963) and Clash of the Titans (1981).

3D animation has been greatly affected by the digital revolution and many spectacularly realistic objects, from dinosaurs to spaceships and even clouds of dust or raging seas, are now created and animated entirely within the computer.

3D computer animation, also known as computer-generated imagery, or CGI, is produced by first creating models that have their size, shape, and appearance defined within a 3D space inside the computer. A simple object such as a cube would be created by defining the relative position of its eight corner points, or vertices, in terms of x, y, and z coordinates. The vertices are then connected by lines called segments, resulting in a wire mesh box. The box will have six square sides, each of which is made up of two triangular polygons. By assigning a colour or a texture to each side of the cube, much like covering a box with gift-wrapping paper, it can be made to look like it was made from stone, wood, metal, or almost any other material.

There are many methods of creating objects within 3D space. Simple “primitives” such as cubes or spheres can be called up at the touch of a button before being further refined. More complex shapes such as creatures have to be “sculpted” using the virtual equivalent of clay, or laser-scanned into the computer from an actual model that has been created by an artist using real clay.

Digital models of characters can be filled with the computer equivalent of a skeleton, complete with joints, muscles, skin, and fur, all programmed to move and react in a realistic manner.

Digital characters and objects are animated much like physical models, with the animator using the mouse to move them a fraction at a time in a process called key frame animation. The greatest difference between this and traditional stop motion is that the animator can scroll a timeline back and forth to preview the performance of a character while still working on it, allowing almost endless revision. Characters with a human-like physique can even have their movements imported directly from those of real actors. This technique, called performance capture, was used to produce the movement of the giant ape in King Kong (2005, Peter Jackson) and the characters of Beowulf (2007, Robert Zemeckis).

CGI is increasingly used for the creation of abstract forms such as water, dust, and snow. In such cases tiny particles are programmed to perform according to complex algorithms written by programmers who have studied the natural world. The animator sets up parameters such as the size and speed of the particles, how they react when they touch each other, and how they are affected by other elements such as wind or gravity. The final animated sequence is then calculated automatically by the computer in a process called procedural animation.

Characters, models, and environments created in the computer can be lit much like objects in the real world using an array of adjustable “virtual lights” and are filmed using a “virtual camera” that can be animated to move like a real camera. Once everything is satisfactory the computer calculates the final appearance of a scene, one frame at a time, in a processor-intensive process called rendering. Rendered scenes can then be composited with other elements before being edited into the final film.

VII.

Matte Painting

Matte painting, the art of combining live-action footage with painted scenery, is a method of creating locations that would be too expensive, time consuming, or impossible to film in any other way.

The concept of matte painting predates cinema itself. Early stills photographers discovered that they could alter the look of a location by placing a sheet of glass in front of the camera and painting additional scenic details onto the glass. The final photograph, shot through the glass, would contain a combination of real background scenery and the painted foreground additions.

The pioneer of this technique for moving pictures was the American filmmaker Norman O. Dawn. Dawn perfected the so-called “glass shot” technique, using it to add “restored” bell towers and roofs to dilapidated religious buildings in the documentary film California Missions (1909). For feature films, glass shots were typically used to add additional storeys to single-storey sets—for example, the lower walls of a castle would be constructed in a studio and the battlements and turrets added using a glass shot.

Glass shots were cumbersome, time-consuming, and always had to be created at the time and location of the original filming. To address these limitations, Dawn developed a more flexible system that allowed footage filmed on location to have painted areas added at a later date. Used from 1911 onwards, original negative matte painting required the area of a frame that was to have a painting added to it to be masked off during original photography. The result was an exposed area of negative containing the live action and an unexposed area to which a painting would later be added. The painted scenery was then created in an art studio and photographed into the unexposed area of the original negative. When developed this negative was therefore a first-generation combination of location live action and studio painting. Until the digital era, the original negative technique was used to produce the highest quality matte painting shots, the master of the art being Albert Whitlock who created photo-realistic matte painting shots for films such as The Birds (1963, Alfred Hitchcock), Earthquake (1974), and The Hindenburg (1975, Robert Wise).

A number of alternative methods for combining live action and painted scenery were developed over the years. The most commonly used of these was rear projection matte painting, in which pre-filmed live-action footage was projected into a scene painted on glass using a projector placed behind the painting and then filmed with a camera placed in front of it. A typical use of this technique might be to paint a space-age city scene and then add live-action footage of actors into the windows of the buildings.

Today, matte paintings are created digitally. Scenery can be painted directly into the computer or created by assembling and cloning real photographs using software such as Photoshop. When the image is complete, live action can be seamlessly composited into it, along with additional elements such as smoke, rain, or moving clouds.

The greatest limitation of traditional matte painting was that the camera could not move during a shot because the lack of any parallax movement between near and far objects would reveal the two-dimensional nature of the painted image. Digital matte paintings, however, allow the camera to pan across or even travel through the painted scenery. This is done by creating a 3D matte painting. For example, a shot requiring the camera to move through Dickensian London would first be designed by the film’s art director. Blueprints for each building would then be used to build full-scale sets. Typically only the lower parts of buildings are built, creating an environment for actors to perform in. The blueprints would then be used by a visual effects company to create digital models that corresponded exactly to their real-life counterparts. These are often painted by cloning photographic textures derived from the actual location sets, ensuring that the two halves look exactly the same. The live-action element of a scene, in which actors interact with the bottom half of the buildings, can then be filmed with a moving camera. Next, these camera movements are replicated by a virtual camera within the computer, which is able to move in exactly the same way while “filming” the top-half of the 3D matte-painted models. After this, the live-action footage is composited with the computer-generated footage—real world and digital scenery meshing together perfectly as the camera moves around it.

VIII.

Miniatures

Like matte paintings, miniatures can be used to create locations and objects that would be too expensive or impossible to film in any other way. Providing the miniature is well built and carefully filmed, audiences will never know that they are not looking at a full-scale version of the object in question.

An early and ingenious way to use miniatures was to hang them in front of the camera so that they lined up with partially built full-scale sets in the background—similar to a traditional glass shot. The advantage of using a hanging miniature rather than a painting was that it could be engineered to move. A fine example can be seen in Ben-Hur (1925), in which the lower part of the Circus Maximus is an enormous set populated by thousands of extras, while the top levels are a miniature filled with tiny, jostling puppets.

Models are frequently used when an environment or object needs to be destroyed, it being safer and cheaper to topple, crash, or explode something in miniature. In such cases model-makers and cinematographers take considerable effort to plan the best scale at which to build a model and how to film it so that it looks the proper size when seen on screen. The destruction of miniatures is typically filmed at very high frame rates, slowing down the action so that a two-second miniature explosion will look like an enormous, long-lasting inferno when seen on the big screen.

Today models can be built either as real-world constructs or, increasingly, as computer-generated environments and objects. Often real and CG models will be combined in one shot. Key to the meshing of such elements is the use of motion control photography in which the movement of both real and virtual cameras can be programmed, scaled up and down, and repeated precisely so that elements produced in different ways will fit together convincingly in the final composite. For example, a camera might need to replicate the same movement when filming a real-world miniature city, a computer-generated spaceship that will land in the city, and the blue screen performance of actors pretending to climb out of the spaceship.

IX.

The Future of Special Effects

The jump from optical to digital effects techniques has seen an extraordinary boom in visual effects production, allowing images of previously unimaginable complexity to become commonplace. The greatest challenges facing visual effects producers are now ones of cost and efficiency, making the creation of spectacular imagery cheaper and faster. Artistically, the greatest obstacle remains the generation of entirely convincing synthetic human characters of a quality that could perform, unnoticed, alongside genuine human actors. Such an endeavour is undoubtedly achievable from a technical point-of-view, although whether animators can conjure a convincingly nuanced and emotive human performance from little more than a mouse and some computer code remains to be seen.