Using GStreamer to Stream All Audio Captured from Soundcard from One PC to Another

Jump to the solution

The Problem

I often work on multiple computers at a time. Additionally, I use headphones the majority of the time. Sometimes I would like to be able to stream all the sound (including system notifications) from one computer to another, allowing me to hear the system notifications from both machines on my headphones. The old-school way to accomplish this means running an audio cable from the output of one PC to the line in on the other. Although this works, the line-in device is usually low quality and noisy. Plus, what if the computers are far away? How about sending the audio digitally over the network? No need for a cable, assuming the computers are already networked!

The Solution - GStreamer

GStreamer is actually an amazing framework that allows you to create all sorts of media pipelines. In my opinion, it is easier to use than ffmpeg once you become familiar with the concepts of how to use it.

What is GStreamer and how does it work?

GStreamer uses a simple concept. You create pipes. Pipes have a source and a sink. A source generates content, and a sink consumes it. You can do fun things with pipes. Imagine that you have a soda fountain that takes in soda water for one source, soda flavoring from another, and spits out into cup sized increments. This process is very similar to the process of taking a video source and an audio source, combining them into a media stream, and then breaking it up in to packets to send it out over the network. Keep the idea of a pipe, with sources and sinks that you can chain together, in mind.

The trickiest part of GStreamer is that each source and each sink has its own set of input types or output types that it can produce or consume. The trick is matching up sinks to sources. If an mp3 encoding element takes in raw audio and spits out an mp3, you must ensure that something that can source raw audio is plugged in one side and something that sinks mp3 audio is plugged in the other. Makes sense, right? It is a little trickier in practice, just because there are so many different formats media can take.

The other piece of the puzzle - PulseAudio

PulseAudio is a sound proxy for Linux and other POSIX operating systems. It forms the bridge between applications and your soundcard. We'll take advantage of the fact that all the sound on our system is going through PulseAudio. We'll pull out all the audio, compress it, and send it to another computer to decompress and play.

Install pavucontrol and GStreamer

We need to tell PulseAudio that we want the input for gstreamer to be the system audio. The easiest way to do this is with the pavucontrol program.

$ sudo apt-get install pavucontrol gstreamer0.10-*

Building the pipeline - Capturing all sound card audio and playing it back

Server Side

Our initial source will be the pulsesrc element. We will be using GStreamer's test application for building streams, gst-launch-0.10. Add the -v option to get better debug information and see how GStreamer is trying to link your elements.

$ gst-launch-0.10 -v pulsesrc

Easy enough. Now, we add the next element, which is the audioconvert element. This just converts from the raw format coming out of the pulsesrc into a raw format that our mp3 encoder can use. Consider audioconvert as something like a simple joiner pipe fitting.

$ gst-launch-0.10 -v pulsesrc ! audioconvert

Next, we can convert the raw audio to mp3 to make conserve bandwidth. This is not necessary, if you are on, say, a gigabit network and don't care about piddly uncompressed audio. We will be using the lame encoder which is used EVERYWHERE! We will do something advanced and set the bitrate.

$ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192

Next, let's do something fun and pipe the audio out over UDP to some host. Note, GStreamer supports multicast addresses, so if you want to allow numerous hosts to join, you might want to look into that. Perhaps that is a topic for another day. Okay, now our pipe becomes

$ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192 ! udpsink port=3000 host=<some ip address>

There! Now you can run the command at this point. You should see a bunch of GStreamer information about it building the pipeline. Ensure that gst-launch is running and recording audio before moving to the next step.

Set PulseAudio to record all system sounds

Do not skip this step! Now you have to tell PulseAudio where GStreamer's input should come from. If you want, you can modify some config file buried somewhere... or you can just use pavucontrol.

Run pavucontrol

$ pavucontrol

Go to the 'Input Devices' tab and in the dropdown 'Show' select 'Monitors'

Here, ensure that the input channel is not muted. The mute button is on the top right. Next, navigate to the 'Recording' tab.

Here, you will want to select from the 'Record Stream from' dropdown, 'Monitor of Built-in Audio Analog Stereo'. Ensure that the channel is not muted. Once selected, you are all done setting up the server.

Client Side

At this point you should have the system audio encoded as an mp3 streaming over UDP on port 3000 to some host at <some ip address> or whatever address. On the client side, we must now take in that stream and spit it out the speakers. Sound hard? Not really. All you need is GStreamer on the client computer.

$ gst-launch-0.10 -v

First, because on the other computer we left off with a UDP sink, we now need to connect it to a UDP source on port 3000 on the other end

$ gst-launch-0.10 -v udpsrc port=3000

Now, instead of encoding raw audio into mp3, we need to decode the mp3 into raw audio. We can do this using the mad mp3 decoder

$ gst-launch-0.10 -v udpsrc port=3000 ! mad

Finally, pipe the raw audio out through PulseAudio

$ gst-launch-0.10 -v udpsrc port=3000 ! mad ! pulsesink

With that final command, you should now have all of the audio transmitted to the other computer. Just to note, the audio is unencrypted. If privacy, for some reason, is a concern, you could easily set up an SSH tunnel and forward the port, letting SSH do all the encryption for you.

Further thoughts

Encoding the audio into mp3 actually may use a portion of the server's CPU. Decoding is nowhere near as CPU intensive. You can sacrifice bandwidth for CPU usage by leaving out the audioconvert, lame, and mad elements. Just be warned, uncompressed audio might be around 1.5mbit. Not much on a wired network, but could be enough to cause problems on spotty wireless connections. The other thought is that you could use queue elements to buffer the audio, as well as use some sort of RTP scheme to improve playback quality of stream. RTP can be a little tricky, but it is designed to improve streaming audio and video over a network.

The final GStreamer solution

Server Side

$ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192 ! udpsink port=3000 host=<some ip address>

Don't forget to run pavucontrol to configure PulseAudio to monitor all incoming audio

Client Side

$ gst-launch-0.10 -v udpsrc port=3000 ! mad ! pulsesink

Alternatives - icecast and darkice

GStreamer is not the only solution to capture sound and spit it out to another location. Icecast is specifically a fully featured media server. You could create your own streaming media server using it, and provide playlists to your complete audio library. Darkice is an encoder that converts the output of your soundcard into mp3. Combined, darkice and icecast can provide a powerful web based streaming server. But, if all you want is to stream audio (even multicast) from one pc to the next, GStreamer is, in my opinion, a simpler solution.

Alternatives - Using PulseAudio directly

Trying to configure PulseAudio directly to stream audio from one computer to another turned out to be more difficult than I wanted. And I couldn't get it working in 15 minutes. So, I moved on to other things. But! It can be done. To be fair, if you got this working it would probably have the least overhead.