Film Dynamics (2003)
Exploring time and dynamics in video.
In the 1970s I had borrowed the book “Expanded Cinema” by Gene Youngblood at the public library. I renewed this loan again and again, but finally returned the book. Sadly, I never bought the book, but thanks to the Vasulka Archive you can now download this book as a PDF file
The images in the book made a big impression on me in the 70s and I have always wanted to experiment with something looking like them. One of the first ideas that came to my mind was the early film by the Australian Len Lye and Canadian Norman Mclaren. In particular, Mclaren’s short animations “Lines Horizontal” and “Lines Vertical” (1961/62) fascinated me deeply. Both Lye and Mclaren made film by drawing directly upon the film with pen and ink. By doing so they escaped the boundaries of the single frame and thereby the discontinued time of stop-frame animation.
After having worked with short video originating from one single image, I became interested in using motion footage as the input for my videos. And I was beginning to think of video as a 3D object in space.
Film as a 3D space/time-object has been very creatively explored by the German group Art+Com in their project “Invisible Shape of Things Past” (1995-2001). They collected short documentary films about Berlin, and the films were then digitised and positioned inside a VR model of the city of Berlin. The individual frames were lined up along the path of the camera. The movements of the (real) recording camera formed virtual 3D shapes consisting of frames. A camera moving down the pavement created a simple elongated box. More compound motions created complex shapes.
My plan was much simpler. I imagined video as a 3D object. The individual 2D frames constitute the vertical and horizontal axis, and the series of successive images being the third axis. Time substituting for depth. A video sequence becomes a tunnel of images. When watching the sequence, you are moving through movie-space at a speed of 25frames/second from end to end in the tunnel.
I began to hack some code. I had no intentions of making the fastest or “smartest” program, but needed something I could experiment with. It should take all the single images in a video sequence – 1500 per minute - and construct a virtual volume filled with pixels (or “voxels” as volumetric pixels are called).
As most programming languages are so similar, programmers often use a kind of pseudo-code to describe ideas or algorithms. You can then translate the pseudo-code into your preferred programming language. The pseudo-code of my program could be something as simple as this:
y=image_height/2;
for (frame = startframe; frame
{
loadimage(frame);
for (x=0; x
{
voxel(x,y,frame) = read_pixel (x,y);
store(voxel,x,y,frame);
}
}
Once I had the code running and the volume stored, the idea was to explore the dynamics in the video, by moving through the object along a different route. The simplest variation would be rotating the axis, passing through the video top-down.
However these volumes are huge. So I was heading for trouble. The full resolution of a DVD- or DV video is 720 x 576 (PAL format). Each minute of video would require manipulating 720x576x(25x60) voxels, that’s more than 620 million voxels to keep track of at a time.
Instead I store lots of still-images of the movement through the video-object. It is almost like the good old days of computer-based art. I conceptualise an algorithm, write the code, press “run”, and go to bed. The next morning I explore the resulting images on my hard drive. The images on the left shows different results from my hacking.
Sharp horizontal shifts in the image indicate a cut in the movie – everything in the scene changes. The rectangular red area in the top image, originate from a scene where Monsieur Hulot (Jacques Tati) switch on a red lamp and a couple of seconds later switches it off again. The more organic swirling shapes – resembling forms from the works of Umberto Boccioni or Giacomo Balla - are actors moving around.
You can read more about slit-scan video here: An Informal Catalogue of Slit-Scan Video Artworks compiled by Golan Levin.
The images in the book made a big impression on me in the 70s and I have always wanted to experiment with something looking like them. One of the first ideas that came to my mind was the early film by the Australian Len Lye and Canadian Norman Mclaren. In particular, Mclaren’s short animations “Lines Horizontal” and “Lines Vertical” (1961/62) fascinated me deeply. Both Lye and Mclaren made film by drawing directly upon the film with pen and ink. By doing so they escaped the boundaries of the single frame and thereby the discontinued time of stop-frame animation.
After having worked with short video originating from one single image, I became interested in using motion footage as the input for my videos. And I was beginning to think of video as a 3D object in space.
Film as a 3D space/time-object has been very creatively explored by the German group Art+Com in their project “Invisible Shape of Things Past” (1995-2001). They collected short documentary films about Berlin, and the films were then digitised and positioned inside a VR model of the city of Berlin. The individual frames were lined up along the path of the camera. The movements of the (real) recording camera formed virtual 3D shapes consisting of frames. A camera moving down the pavement created a simple elongated box. More compound motions created complex shapes.
My plan was much simpler. I imagined video as a 3D object. The individual 2D frames constitute the vertical and horizontal axis, and the series of successive images being the third axis. Time substituting for depth. A video sequence becomes a tunnel of images. When watching the sequence, you are moving through movie-space at a speed of 25frames/second from end to end in the tunnel.
I began to hack some code. I had no intentions of making the fastest or “smartest” program, but needed something I could experiment with. It should take all the single images in a video sequence – 1500 per minute - and construct a virtual volume filled with pixels (or “voxels” as volumetric pixels are called).
As most programming languages are so similar, programmers often use a kind of pseudo-code to describe ideas or algorithms. You can then translate the pseudo-code into your preferred programming language. The pseudo-code of my program could be something as simple as this:
y=image_height/2;for (frame = startframe; frame
loadimage(frame);
for (x=0; x
voxel(x,y,frame) = read_pixel (x,y);
store(voxel,x,y,frame);
}
}
Once I had the code running and the volume stored, the idea was to explore the dynamics in the video, by moving through the object along a different route. The simplest variation would be rotating the axis, passing through the video top-down.
However these volumes are huge. So I was heading for trouble. The full resolution of a DVD- or DV video is 720 x 576 (PAL format). Each minute of video would require manipulating 720x576x(25x60) voxels, that’s more than 620 million voxels to keep track of at a time.
Instead I store lots of still-images of the movement through the video-object. It is almost like the good old days of computer-based art. I conceptualise an algorithm, write the code, press “run”, and go to bed. The next morning I explore the resulting images on my hard drive. The images on the left shows different results from my hacking.
Sharp horizontal shifts in the image indicate a cut in the movie – everything in the scene changes. The rectangular red area in the top image, originate from a scene where Monsieur Hulot (Jacques Tati) switch on a red lamp and a couple of seconds later switches it off again. The more organic swirling shapes – resembling forms from the works of Umberto Boccioni or Giacomo Balla - are actors moving around.
Thanks to:
Morten Søndergaarde who edited the book/DVD"Get Real". There you can read more about my experiments. Published by Informations Forlag in the EU (ISBN 87-7514-110-8) and by George Braziller in USA (ISBN 0-8076-1564-4).You can read more about slit-scan video here: An Informal Catalogue of Slit-Scan Video Artworks compiled by Golan Levin.