Movies from OpenGL

I’ve got a few natty programs that draw stuff with OpenGL, I’d like to be able to record their output in some appropriate video format that I can show my friends, post on YouTube etc. Not quite sure how to do that, but it can’t be that difficult I’d have thought.

As as experiment, I’ll write it all up as I go along.

It seems that I can use the data in the frame buffer directly with glReadPixels:

http://www.opengl.org/sdk/docs/man/xhtml/glReadPixels.xml

so it should be easy to dump out frames as raw RGB values or whatever is required by the next step, ie. encoding to the actual movie format, AVI, mpeg4, whatever.

ffmpeg seems to be the tool for doing this. Let’s try some experiments: we can dump from X:

ffmpeg -f x11grab -s cif -r 25 -i :0.0+10,20 /tmp/out.mpg

but that has horrible encoding artefacts, trying with:

ffmpeg -qscale 5 -f x11grab -s cif -r 25 -i :0.0 /tmp/out.mpg

does rather better (qscale it seems goes from 1 to 31, lower is higher quality).

Apparently, ffmpeg is ‘deprecated’ and we should use avconf, but that seemed to fail dismally, it looked like it wasn’t synchronized with the frame buffer refreshes or something, I get all sorts of purple streaks and flashes, not nice.

avconv -f x11grab -s cif -r 25 -i :0.0 /tmp/out.mpg

maybe there’s an option for that too, but no idea what it might be.

Sticking to ffmpeg for the moment, after the usual Googling, this seems useful:

http://stackoverflow.com/questions/7238013/rawvideo-and-rgb32-values-passed-to-ffmpeg

Looks like something like:

ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb32 -s256x256 -i pipe:0 -f avi -vcodec foo.avi

might work.

I’ve got some raw video files around from my MMPS project: if you take a PPM type 6 file, and take off the header lines you end up with raw bytes, packed as 24 bit RGB values.

A bit of experimentation and I find all that is needed to make a rather boring video of the earth, contrary to Galileo, not moving, is:

(while true; do cat earth.raw; done) | ffmpeg -qscale 1 -f rawvideo -pix_fmt rgb24 -s 1000x500 -i pipe:0 foo7.avi

ffmpeg seems to work out that we need an avi wrapper for an avi file, so we don’t need -f avi and it uses its own mpeg4 encoder by default, so we don’t need -vcodec mpeg4, and the -f rawvideo covers -vcodec rawvideo. And another thing that I now know, but not knowing held me up a little: making an AVI file like this but with just one frame doesn’t work very well (at least Linux Movie Player won’t play the result).

Back to OpenGL, now I’ve got the video coding back end sorted I need to generate some real data. Since I’ll be piping the data into ffmpeg, I just need to dump out the frames to stdout, in the appropriate format. The OpenGL doc for the various formats is, as usual fairly baffling for a newbie like myself but this seems to work:

glFlush();
static size_t nbytes = 3*screen_width*screen_height;
static char* framedata = new char[nbytes];
glReadPixels(0,0,screen_width,screen_height,GL_RGB,GL_UNSIGNED_BYTE,framedata);
fwrite(framedata,1,nbytes,stdout);
glutSwapBuffers();

I’m not going to put this forward as model code: static variables, yuk, and there’s no protection against some damn fool resizing the window while we are ‘recording’, which won’t make anything very happy, and I’m not even checking the return value of fwrite or for errors in glReadPixels. We could do things entirely offline, but we might want to record some interactions and writing out each frame doesn’t seem to slow things down too much.

./poly2 1 1 1 1 | ffmpeg -qscale 1 -vcodec rawvideo -f rawvideo -pix_fmt rgb24 -s 800x640 -i pipe:0 poly.avi

I probably ought to change poly2 so it takes width and height on the command line rather than hardwired (but recompilation is so fast these days).

Next step, uploading to YouTube. Looking at:

http://support.google.com/youtube/bin/answer.py?hl=en&answer=1722171&topic=2888648&ctx=topic

it seems that I probably want 480p format, 854×480 as a sort of baseline. I still don’t understand about codecs, but looks like H.264 is the way to go. Now, what encoder should I use, looks like x264 or libx264 is the thing, but nothing is installed:

$ ffmpeg -codecs | grep 264
D V D h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
D V D h264_vdpau H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (VDPAU acceleration)

It seems I can decode, but not encode.

The rather helpful:

https://help.ubuntu.com/community/FFmpeg

suggests installing these extra packages:

sudo apt-get install libavcodec-extra-53 libavdevice-extra-53 libavfilter-extra-2 libavformat-extra-53 libavutil-extra-51 libpostproc-extra-52 libswscale-extra-2

and now I have:

$ ffmpeg -codecs | grep 264
D V D h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
D V D h264_vdpau H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (VDPAU acceleration)
EV libx264 libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10

Nice, and now:

./poly2 1 1 1 1 | ffmpeg -qscale 1 -vcodec rawvideo -f rawvideo -pix_fmt rgb24 -s 800x640 -i pipe:0 -vcodec libx264 foo3.avi

seems to do the trick.

Looking good, let’s try the YouTube recommended 854×480 dimensions; hmm, we’ve gone monochrome with rather prominent scan lines (reminded me of the old Dr Who tent at Glastonbury, hands up if anyone remembers that):

Screenshot from 2013-03-10 16:49:55

Looks like a data alignment problem – I’ve still got magic numbers in the code, but they seem to be the right ones – but checking OpenGL documentation, I find:

http://www.opengl.org/sdk/docs/man/xhtml/glPixelStore.xml

Looks like OpenGL is word-aligning the pixel rows, fine for 800 pixels, not so good for 854.

glPixelStorei(GL_PACK_ALIGNMENT,1)

should sort this out (note the ‘i’ for ‘integer’ on the end there), and indeed it does:

./poly2 1 1 1 1 | ffmpeg -f rawvideo -pix_fmt rgb24 -s 854x480 -i pipe:0 -vcodec libx264 foo264.avi

makes me a nice 20MB, 1 minute 17 sec video of some coloured balls writhing around in empty space. A nice background (and one that disguises the stuck pixel on my new laptop that I only notice watching this sort of thing) would be good.

Now for that YouTube account, I suppose I might as well use my Google account, anything for an easy life and it’s not like I’m uploading anything too embarrassing. OK, here we go: “We did not recognise the audio format for this file, but we will try to process it anyway. See this article on recommended formats for more information”, well, I didn’t include any audio, so fair enough I suppose (a soundtrack would be nice though, suggestions on a postcard).

Anyway, here it is, in all its glory: I give you “invert264”:

Perhaps I should come up with a catchier name, but that will do for now.