Visuals pile up

Andrew Booker 2008-05-28 15:30:12

In less than a week, on 04 June 08, we will be at the Plough in Walthamstow. A week or so ago I put posters up saying that we will be doing back-projections. That, set in quick-drying concrete, is my deadline for getting the visuals stuff sorted. Am I gonna do it?? Thanks to a usefully timed bank holiday weekend just gone, I think so. For a round-up of where I'm up to, we should start at the end and work backwards.

I need at least two hours of visuals, on a DVD, probably in two one-hour episodes, to accompany us during a standard Plough gig where we have an interval in the middle. Two hours is a lot of footage.

It may be a lot of footage, but it doesn't necessarily need a lot of material. If modern minimalist one-musician performers can use audio loops everywhere, I'm going to use video loops. A video loop works well when the start and end points look as similar as possible, so you don't get a nasty jump between one frame and the next. That means you might take a while to notice there's a loop, and it's a great way of padding out the running time of a cool-looking piece.

I'm not using any computer-generated imagery. With the exception of some downloadable weather satellite images, I am using video I have shot myself. After a couple of months of thinking about the project, and carrying my phone video camera, I am amassing a load of different ideas and sources to stitch together into two big blobs. Most of it is either time-lapse stuff, where everything is sped up, or slow-motion pieces. For the time-lapse material, I have either been capturing frames from a webcam directly onto my laptop, or I've been shooting ordinary video onto a camcorder and copying one frame every 20 or 30 off the tape. For the slow motion ideas, I have been using every frame, even inserting cross-fade frames between one and the next.

It may sound a lot of work, but so far I've been finding it easy to organise everything around individual frame files. So my first job is to build up a folder of frames for each piece, with lots, sometimes thousands, of frames in each, and a little XML instruction file that tells my software what to do with them. Here are the different sources I'm using.

Downloaded weather satellite JPG images
I've been downloading these every day since the middle of March 08 from four viewpoints over the globe. These images will be the frames with which I'll compile a slow-motion video, maybe inserting one or more morphed cross-fade images (a pixel-by-pixel average) between each frame and the next. Some of the sources look cool inverted (photographic negative), others will at least benefit from some colour adjustment. I can do that kind of image processing in my own software at the pixel level in each frame, without having to remote-control Photoshop to do it for me, as I originally thought.

Still time-lapse images from the webcam
Like the weather pictures, these are ready-made frames going straight to AVI, maybe with effects. I spent quite a bit of time writing C++ code to gather these images reliably, but so far I've yet to make much use of it. I've taken the laptop out on the odd car journey, which it coped with very well, writing to its hard disk while bouncing around on the back seat for four hours. But the piece didn't come out very well. Fast-motion video like that needs to be more-or-less continuous. Motorways in daylight in early spring look pretty dull by anyone's taste. London roads at night look much better, but stop-start traffic makes poor material.

Time-lapse also works well from a fixed position, and I had grand ideas about capturing a whole day's sky movements from a good vantage point, but the weather and electrical practicalities have shot that one in the foot.

The main problem is being able to stick the camera in an interesting but safe place for a long period and be able to power the laptop. When it's shooting time-lapse images, my laptop consumes 2A. On what, heaven knows. The car can easily handle this if it's on the move, but if it's parked, the laptop will run the battery down in a few hours. I was thinking of using a solar charger, but the expensive equipment on show in my parked car is now starting to ker-ching! I may be able to put it indoors somewhere in the Liverpool St area. It's annoying that the job of simply taking one photo every five seconds does not itself need much power. I'm wondering if one day, when I'm really clever, I could write some Java code to run on my phone and save the images using its camera. Or get a phone that runs Windows, and I might be able most of the way there already.

Video shot using my phone
My Nokia N95 phone can shoot pretty good video at 640x480 pixels, though battery life limits videoing to small doses. It saves MP4 files onto a 4GB memory card. The compression to MP4 is a bit nasty, and the resolution is less than I want, but not much, and those are both outweighed by the convenience of working with files, which I can copy off my phone over a USB connection. Once on the laptop, the next step is to use a MOVAVI converter to convert the MP4s to uncompressed AVI. Being hand-held material, clips are typically only a couple of minutes, so uncompressed AVI is fine. From there, I use my own software to extract every frame. I will either use the frames as they are, or more likely slow the material down by inserting cross-fade morphs.

I guess it would be a lot quicker if I could read MP4s with my software, but life feels a bit short right now, and what I have works.

So far, using the phone has been the the most fun. It fits in my pocket. It saves to files. Anything interesting I see, I can film. More or less. Certain parts of Soho are probably off-limits, nor will I even take it out of my pocket in Tower Hamlets. Having pointed it upwards at the vertical construction skyline of the Dubai Marina a month or so ago, I find the same photogenic material in the city of London. Needs good weather though. Once taken, I can instantly review what I've shot, and because I'm going to be using it either at real-time speed or slower, I can pretty much tell straight away when it's going to look good, when then eggs me on to go and shoot some more stuff. Gathering time-lapse frames yields no such benefits. I can't see what it looks like until I've built an AVI out of it.

Video shot using my camcorder
This is the same process as using the phone, except it doesn't fit in my pocket, and records to DV tape, not files. But the resolution and image quality are better than the phone, and it's battery doesn't go flat after 20 minutes. I've used it in the car, and the other evening I videoed a sunset from a footbridge over the North Circular. Are we glamour at Improvizone, or what.

Back home, I connect the camcorder to my laptop with a firewire cable, and the fun ends. This bit has taken me weeks to get working satisfactorily. Yes, I did what Microsoft says, and used their Windows Movie Maker to record a compressed AVI from the video stream from the camcorder as it plays back the tape in real time. Rather than extend my code to read a compressed AVI (probably very easy, if I could be bothered), I used MOVAVI to decompress. After four hours, that gave me an AVI of around 113GB for an hour's video. That's OK, I have an external hard disk. Erm, except it was not OK, because Microsoft's own AVI libraries cannot read past the first 2GB. Into the recycle bin. Stupid idea anyway, I only wanted one frame per second.

Finally, I wrote some code to remote-control the camcorder and save individual frames at one-second intervals direct from the video stream coming down the firewire lead. Much better.

Even though the images have been saved best quality on the tape, there doesn't seem to be a way of dragging them off to a file without compression. I guess it has to be compressed, or the laptop recorder cannot write the file to disk fast enough as it reads off the tape. I suppose I could check whether Firewire is sufficiently fast to allow real-time transfer of uncompressed video. Probably not. But it would certainly be fast enough to give me one uncompressed frame every second. Sigh. Yawn.

So that's how I'm getting my material. From each of those four, I end up with a bunch of individual bitmap frames. I have a week to organise the whole lot into video.

The first thing I did when I started all of this, downloading weather pictures, was to make sure I could build them into an AVI. I got that working soon enough... but not quite. As I gathered more images, I realised my uncompressed AVI writer bolloxes up writing a file bigger than 2GB. You get the file. It just doesn't play. 2GB of uncompressed AVI is not very many minutes. I need two hours. I could get that from lots and lots of clips, but I want to do loops, and those should be able to run for at least 10 minutes if I want. Not looking good, was this.

Nor entirely my fault. This is Microsoft's own AVI library failing to write a file of their own format, because their basic file header uses a 32-bit number to store the file size. Now, if you take out your calculator and raise 2 to the power 32, you get 4G. So why can I only have 2GB? I suspect some daft pothead used a signed number, where bit 32 has negative value, so that as you increase above 2GB, the size goes negative. How many files do we know that are allowed to have negative size? Exactly. I would love to claim this is my own stupidity, because then I could correct it myself, but the file writing is all done inside Microsoft's AVI library.

Anyway 4GB or 2GB, to hell with uncompressed AVI. Right now I've got round the problem by writing compressed AVI (which suggests I should be able to read from it as well) using the MS MP4 version 2 codec. I have reduced the file size by a factor of 60-70, which means less than 2GB for a one-hour video, and it doesn't look too bad. I'm more concerned about the quality of the source material than the output, which is going to be squished onto a DVD, so it will look dismal anyway. My other option is to figure out how to use an OpenDML header in the AVI, that will let me have files bigger than 2GB. Something tells me I won't be able to use the MS libraries, but will have to put together the AVI myself. I'm OK for the moment, thanks...

What's left for next week?

I need a basic story board. What goes where, when, for how long etc.

I need to organise the looping bits into finished video pieces. Then I can work out how much video I've got. Right now I have no idea. 10 minutes? 45 minutes? Six hours? Dunno.

Another thing, I've never actually tried burning video onto a DVD before. Ha ha. I'm assuming I can figure out how to do that in less than one evening.

Finally, I ought really to put some wording on there that says Improvizone (dot com). And other stuff, maybe. For example, where I've used the weather images that are not my copyright, I should be a good boy and make sure the copyright owner is clearly visible. I've not tried writing stuff onto images before, but if I can crack that, I'll soon be auto-generating full resolution posters for Improvizone gigs. How about that for geek.

Music and effects for architects << | >> All eyes on the screen