So how does it work? A script exports every single frame from an MPEG movie into individual JPEG files. These are then collected together, base64 encoded and exported into a JSON file. The script then creates image objects for each frame and shows them all in succession. There is no support for sound.
First strategy was to create an Image object for each frame and render it on a canvas element using drawImage(). That worked fine and performance was nice (although Opera used a lot of CPU), but I figured I’d try just using a regular image tag and just change the src property to another data:uri each frame. The change was barely noticeably in Firefox and Safari and it ran a bit better in Opera, so I lost the canvas and stuck with plain old images.
Now, it seems that Firefox will eat up all the memory in the world if you keep throwing new data:uris at the same image tag, which led to another change, so for each frame a new Image object was created and saved for later and as the video played, the previous frame Image was replaced by the new Image object. That seemed to work, but introduced an annoying delay as all these Image objects were created before playing, so I ended up moving the Image creation to actual render cycle where it simply checks if the frame Image has already been created, and if not, creates it.
So this is totally impractical but who cares: it’s cool and a fun experiment. I wish I still had time for experiments like these!