Monday, October 12, 2009

Live screencasting using ffmpeg

I reconnected with an old friend from home this summer. He's living out in Utah, and I was working in Boston at the time, but I tracked him down on Skype. Remembering that I still had his copy of Myst III: Exile from years ago, I confessed that I'd never played all the way through the game: without having much free time, it was hard to keep at the puzzles. “It's too bad,” I lamented, “that we can't play it together.”

But wait ... perhaps we could?

I'd been aware for some time that it's possible to stream video using ffserver and ffmpeg. Sadly, it's not a completely intuitive process, and until this point, I'd never had the motivation to figure out how to put all the parts together. But now, motivated by the possibility of being spoon-fed (just barely enough) hints to finally finish Exile, I resolved to figure out how to stream my desktop to Utah. After some initial success, and a fair bit of tweaking of codecs and parameters, and then hours of fighting with wine to get Exile running, I got everything working. Below, I provide a simple guide that should enable anyone else running Ubuntu to set up a similar live screencast. (How I got Exile to run acceptably under wine, though, is a story for another day.)

The general process of streaming with ffmpeg looks something like the following. First, you run an ffserver instance. It's configured with a number of feeds (usually local files), which provide audio and/or video, and output streams, which are hooked up to incoming connections over RTP/RTSP/HTTP.

Our problem is slightly more complicated, in that we need to feed live data to ffserver. The easiest way to do this is to use an FFM stream: we can configure ffserver with a feed that's sourced to an FFM file, and, once ffserver is running, fire up ffmpeg and tell it to dump live audio and video into the FFM file. Then, a buddy in Utah can connect to the correct output stream and see what we're streaming. Sounds great, right?

Now that you can see the big picture, it's not too hard to get things going. First things first: you'll need some packages:
sudo apt-get install ffmpeg \
    libavcodec-unstripped-52 \
    libx264-67 \
    libmp3lame0
Then you'll need to specify a configuration for ffserver with at least one feed and at least one output stream. I used libx264 for video encoding and libmp3lame for audio encoding, and found that the settings in the ffserver.conf below were more than generous enough for the available bandwidth:
Port 8090
BindAddress 0.0.0.0
MaxHTTPConnections 2000
MaxClients 1000
MaxBandwidth 1000
CustomLog -
NoDaemon

<feed exile.ffm>
  File /tmp/exile.ffm
  FileMaxSize 16M
</Feed>

<stream exile.asf>
  Feed exile.ffm
  Format asf

  AudioBitRate 64
  AudioChannels 1
  AudioSampleRate 22050

  VideoBitRate 128
  VideoBufferSize 400
  VideoFrameRate 15
  VideoSize 320x240

  VideoGopSize 12

  VideoHighQuality
  Video4MotionVector

  AudioCodec libmp3lame
  VideoCodec libx264
</Stream>
Put this configuration file wherever you like; then you can start ffserver and point it at the location you chose:
ffserver -f /path/to/your/ffserver.conf
The remaining piece of the puzzle is getting ffmpeg to capture audio and video from your desktop. Video is easy enough; you can tell ffmpeg to grab a chunk of the X11 display using -f x11grab -s [W]x[H] -r [R] -i 0.0+[X],[Y], where [W], [H], [X], and [Y] represent the width, height, x-offset, and y-offset, respectively, of your desired capture region. Additionally, [R] specifies the desired capture framerate, which should match the one you specify in your configuration file.

For audio, things could get a bit hairier — but, rather than mucking around with arcane ALSA magic in your .asoundrc or setting up JACK, I recommend taking advantage of PulseAudio here; though it is (perhaps deservedly — I caught it using 2.4 GB of memory in an unrelated incident earlier today) maligned by many, it really shines in this use case. You'll need pavucontrol, the very-helpful PulseAudio volume control, to make this work:
sudo apt-get install pavucontrol
You can then tell ffmpeg to record audio from a PulseAudio input channel by using -f alsa -i pulse. Adding to this the URL of the ffserver feed to which ffmpeg should be connecting, you have everything you need to invoke ffmpeg. For example, if you're using the ffserver.conf above, and you want to capture a 640x480 window at the upper-left corner of the screen, you'll want to invoke ffmpeg like this:
ffmpeg -f x11grab -s 640x480 -r 25 -i :0.0 \
       -f alsa -i pulse \
        http://localhost:8090/exile.ffm
With ffmpeg running, you'll then want to fire up pavucontrol and make sure that ffmpeg is connected to the monitor feed for your sound card, as shown in the screenshot below:



And that's it! You can tell your friend in Utah to point a streaming client, e.g. vlc, at your stream, and it should just work. Just remember to use your outward-facing IP address, and to make sure the port's open on your router or firewall, if any. For the ffserver.conf above, if my external IP address were 123.45.67.8, the URL I'd send to my friend would be http://123.45.67.8:8090/exile.asf.

There is one thing to note: if your friend's client takes a long time to start up, it may be because ffserver's default behavior is to connect clients to the live stream without a delay, which then causes many clients to pause for a number of seconds to fill up a buffer with incoming data as it becomes available. This can be accelerated somewhat by appending a ?buffer=X (where X is a number of seconds) param to the end of the URL; this causes ffserver to start the client on a position in the stream that's X seconds in the past, which enables the client to retrieve those X seconds as fast as bandwidth will allow, thus possibly reducing the time spent buffering. In experimenting, I also found that this cut back on the number of buffer underruns experienced on my friend's end.

6 comments:

  1. Hey, awesome post! Just one question, were you able to do in it realtime? I'm just testing via localhost for now and it's working great. I am planning on using it on our intranet so our transfer speed should be decent. There's always a few seconds delay even when connecting from localhost, I tried ?buffer=[3,10,30]. Thanks in advance, Vincent.

    ReplyDelete
  2. I, too, was unable to completely avoid the delay. Five to ten seconds seemed to be as close as I could get to realtime. Another weird thing to watch out for is that sometimes the audio can slip out of sync ... I'm not sure whether that problem was in the server or the client (VLC), in my case. Report back if you figure anything out!

    ReplyDelete
  3. Hey Devin..

    I am getting the foll Error :

    ffmpeg -f x11grab -s 640x480 -r 25 -i :0.0 -f alsa -i pulse http://localhost:8090/exile.ffm
    FFmpeg version SVN-r24577, Copyright (c) 2000-2010 the FFmpeg developers
    built on Jul 29 2010 18:11:55 with gcc 4.4.3
    configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-postproc --enable-pthreads --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libx264 --enable-libxvid --enable-x11grab
    libavutil 50.23. 0 / 50.23. 0
    libavcore 0. 1. 0 / 0. 1. 0
    libavcodec 52.84. 1 / 52.84. 1
    libavformat 52.77. 0 / 52.77. 0
    libavdevice 52. 2. 0 / 52. 2. 0
    libavfilter 1.26. 1 / 1.26. 1
    libswscale 0.11. 0 / 0.11. 0
    libpostproc 51. 2. 0 / 51. 2. 0
    [x11grab @ 0x9b85470] device: :0.0 -> display: :0.0 x: 0 y: 0 width: 640 height: 480
    [x11grab @ 0x9b85470] shared memory extension found
    [x11grab @ 0x9b85470] Estimating duration from bitrate, this may be inaccurate
    Input #0, x11grab, from ':0.0':
    Duration: N/A, start: 1280409845.986681, bitrate: 245760 kb/s
    Stream #0.0: Video: rawvideo, bgra, 640x480, 245760 kb/s, 25 tbr, 1000k tbn, 25 tbc
    [alsa @ 0x9b91c40] capture with some ALSA plugins, especially dsnoop, may hang.
    [alsa @ 0x9b91c40] Estimating duration from bitrate, this may be inaccurate
    Input #1, alsa, from 'pulse':
    Duration: N/A, start: 1280409845.967494, bitrate: N/A
    Stream #1.0: Audio: pcm_s16le, 44100 Hz, 1 channels, s16, 705 kb/s
    [buffer @ 0x9baa5b0] w:640 h:480 pixfmt:bgra
    [scale @ 0x9ba7270] w:640 h:480 fmt:bgra -> w:320 h:240 fmt:yuv420p flags:0xa0000004
    [libx264 @ 0x9ba8100] broken ffmpeg default settings detected
    [libx264 @ 0x9ba8100] use an encoding preset (vpre)
    Output #0, ffm, to 'http://localhost:8090/exile.ffm':
    Stream #0.0: Audio: libmp3lame, 22050 Hz, 1 channels, s16, 64 kb/s
    Stream #0.1: Video: libx264 (hq), yuv420p, 320x240, q=2-31, 128 kb/s, 1000k tbn, 15 tbc
    Stream mapping:
    Stream #1.0 -> #0.0
    Stream #0.0 -> #0.1
    Error while opening encoder for output stream #0.1 - maybe incorrect parameters such as bit_rate, rate, width or height

    ReplyDelete
  4. I bet the delay was caused by pulseaudio ..

    ReplyDelete
  5. Small typo: `-i 0:0+[X],[Y]` should be `-i 0.0+[X],[Y]` (with a dot in lieu of the colon). You have the right command a few lines down when you write out the whole thing as: `ffmpeg -f x11grab -s 640x480 -r 25 -i :0.0`. Great article though, thanks!

    ReplyDelete
    Replies
    1. Thanks for catching that. Hope things worked out for you -- I'd be surprised if there weren't other corrections/updates to be made, seeing as this post is nearly three years old.

      Delete