Tuesday, December 11, 2012

Using GStreamer to Stream All Audio Captured from Soundcard from One PC to Another

Jump to the solution

The Problem

I often work on multiple computers at a time. Additionally, I use headphones the majority of the time. Sometimes I would like to be able to stream all the sound (including system notifications) from one computer to another, allowing me to hear the system notifications from both machines on my headphones. The old-school way to accomplish this means running an audio cable from the output of one PC to the line in on the other. Although this works, the line-in device is usually low quality and noisy. Plus, what if the computers are far away? How about sending the audio digitally over the network? No need for a cable, assuming the computers are already networked!

The Solution - GStreamer

GStreamer is actually an amazing framework that allows you to create all sorts of media pipelines. In my opinion, it is easier to use than ffmpeg once you become familiar with the concepts of how to use it.

What is GStreamer and how does it work?

GStreamer uses a simple concept. You create pipes. Pipes have a source and a sink. A source generates content, and a sink consumes it. You can do fun things with pipes. Imagine that you have a soda fountain that takes in soda water for one source, soda flavoring from another, and spits out into cup sized increments. This process is very similar to the process of taking a video source and an audio source, combining them into a media stream, and then breaking it up in to packets to send it out over the network. Keep the idea of a pipe, with sources and sinks that you can chain together, in mind.

The trickiest part of GStreamer is that each source and each sink has its own set of input types or output types that it can produce or consume. The trick is matching up sinks to sources. If an mp3 encoding element takes in raw audio and spits out an mp3, you must ensure that something that can source raw audio is plugged in one side and something that sinks mp3 audio is plugged in the other. Makes sense, right? It is a little trickier in practice, just because there are so many different formats media can take.

The other piece of the puzzle - PulseAudio

PulseAudio is a sound proxy for Linux and other POSIX operating systems. It forms the bridge between applications and your soundcard. We'll take advantage of the fact that all the sound on our system is going through PulseAudio. We'll pull out all the audio, compress it, and send it to another computer to decompress and play.

Install pavucontrol and GStreamer

We need to tell PulseAudio that we want the input for gstreamer to be the system audio. The easiest way to do this is with the pavucontrol program.

$ sudo apt-get install pavucontrol gstreamer0.10-*

Building the pipeline - Capturing all sound card audio and playing it back

Server Side

Our initial source will be the pulsesrc element. We will be using GStreamer's test application for building streams, gst-launch-0.10. Add the -v option to get better debug information and see how GStreamer is trying to link your elements.

$ gst-launch-0.10 -v pulsesrc

Easy enough. Now, we add the next element, which is the audioconvert element. This just converts from the raw format coming out of the pulsesrc into a raw format that our mp3 encoder can use. Consider audioconvert as something like a simple joiner pipe fitting.

$ gst-launch-0.10 -v pulsesrc ! audioconvert

Next, we can convert the raw audio to mp3 to make conserve bandwidth. This is not necessary, if you are on, say, a gigabit network and don't care about piddly uncompressed audio. We will be using the lame encoder which is used EVERYWHERE! We will do something advanced and set the bitrate.

$ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192

Next, let's do something fun and pipe the audio out over UDP to some host. Note, GStreamer supports multicast addresses, so if you want to allow numerous hosts to join, you might want to look into that. Perhaps that is a topic for another day. Okay, now our pipe becomes

$ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192 ! udpsink port=3000 host=<some ip address>

There! Now you can run the command at this point. You should see a bunch of GStreamer information about it building the pipeline. Ensure that gst-launch is running and recording audio before moving to the next step.

Set PulseAudio to record all system sounds

Do not skip this step! Now you have to tell PulseAudio where GStreamer's input should come from. If you want, you can modify some config file buried somewhere... or you can just use pavucontrol.

Run pavucontrol

$ pavucontrol

Go to the 'Input Devices' tab and in the dropdown 'Show' select 'Monitors'

Here, ensure that the input channel is not muted. The mute button is on the top right. Next, navigate to the 'Recording' tab.

Here, you will want to select from the 'Record Stream from' dropdown, 'Monitor of Built-in Audio Analog Stereo'. Ensure that the channel is not muted. Once selected, you are all done setting up the server.

Client Side

At this point you should have the system audio encoded as an mp3 streaming over UDP on port 3000 to some host at <some ip address> or whatever address. On the client side, we must now take in that stream and spit it out the speakers. Sound hard? Not really. All you need is GStreamer on the client computer.

$ gst-launch-0.10 -v

First, because on the other computer we left off with a UDP sink, we now need to connect it to a UDP source on port 3000 on the other end

$ gst-launch-0.10 -v udpsrc port=3000

Now, instead of encoding raw audio into mp3, we need to decode the mp3 into raw audio. We can do this using the mad mp3 decoder

$ gst-launch-0.10 -v udpsrc port=3000 ! mad

Finally, pipe the raw audio out through PulseAudio

$ gst-launch-0.10 -v udpsrc port=3000 ! mad ! pulsesink

With that final command, you should now have all of the audio transmitted to the other computer. Just to note, the audio is unencrypted. If privacy, for some reason, is a concern, you could easily set up an SSH tunnel and forward the port, letting SSH do all the encryption for you.

Further thoughts

Encoding the audio into mp3 actually may use a portion of the server's CPU. Decoding is nowhere near as CPU intensive. You can sacrifice bandwidth for CPU usage by leaving out the audioconvert, lame, and mad elements. Just be warned, uncompressed audio might be around 1.5mbit. Not much on a wired network, but could be enough to cause problems on spotty wireless connections. The other thought is that you could use queue elements to buffer the audio, as well as use some sort of RTP scheme to improve playback quality of stream. RTP can be a little tricky, but it is designed to improve streaming audio and video over a network.

The final GStreamer solution

Server Side

$ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192 ! udpsink port=3000 host=<some ip address>

Don't forget to run pavucontrol to configure PulseAudio to monitor all incoming audio

Client Side

$ gst-launch-0.10 -v udpsrc port=3000 ! mad ! pulsesink

Alternatives - icecast and darkice

GStreamer is not the only solution to capture sound and spit it out to another location. Icecast is specifically a fully featured media server. You could create your own streaming media server using it, and provide playlists to your complete audio library. Darkice is an encoder that converts the output of your soundcard into mp3. Combined, darkice and icecast can provide a powerful web based streaming server. But, if all you want is to stream audio (even multicast) from one pc to the next, GStreamer is, in my opinion, a simpler solution.

Alternatives - Using PulseAudio directly

Trying to configure PulseAudio directly to stream audio from one computer to another turned out to be more difficult than I wanted. And I couldn't get it working in 15 minutes. So, I moved on to other things. But! It can be done. To be fair, if you got this working it would probably have the least overhead.

16 comments:

  1. Hi I am Sucheth,
    I have doubt after this following commands
    $ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192 ! udpsink port=3000 host=

    where should i run "pavucontrol" commandin the same terminal or should i open othe terminal and run this command...

    ReplyDelete
    Replies
    1. You can run pavucontrol from a new terminal or your launcher (alt+f2)

      Delete
  2. Hi,

    Thank you for the reply,
    please help me i am doing following steps
    1>I ran following command on Panda Board
    gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192 ! udpsink port=3000 host=192.168.XX.YY

    and other terminal executed pavucontrol and done the settings,

    In the client side that is in other system(192.168.XX.YY) i ran following command
    gst-launch-0.10 -v udpsrc port=3000 ! mad ! pulsesink

    i connected Headphone no sound came... could you plese tell me am i doing properly...

    Thank you
    Sucheth S L

    ReplyDelete
  3. You have to run pavucontrol on the machine that you want to be the source of the sound. If you don't have pavucontrol, you will have to configure pulse audio via text file, which I am not sure how to do.

    ReplyDelete
  4. Hi thank you for the reply,

    I tried but I am not able to hear any sound....
    could u please let me know in the following pipleline
    $ gst-launch-0.10 -v pulsesrc ! audioconvert ! lame bitrate=192 ! udpsink port=3000 host=

    is there any possibility of giving my source of audio and encode and in the client side i can decode and listen... please if you have any pipeline related to it please share me or any resource frome where i can get information..

    any tutorials or steps involved in streaming of audio from one pc to other through Network...?

    ReplyDelete
    Replies
    1. I still don't think that you have the your pulse audio input source set correctly. You might have the channel muted. Gotta check to make sure that the pusle source is not muted through either pavucontrol or alsamixer.

      Delete
  5. Hi,
    Some how i can stream the audio now using other pipeline....
    do you have any Link or source where i can get the pipelines for streaming Video.

    Thank you,
    Sucheth S L

    ReplyDelete
    Replies
    1. Glad you got it working! There are so many options for streaming video, I'm just going to have to point you to google. There are video media formats that support muxing video and audio into the same stream. What is your application?

      Delete
  6. hi,
    its kind of demo project so experimenting with audio and video streaming....

    as you said i Google it, i got some Gstreamer Pipelines for audio streaming and video streaming using Multicast UDP. in that i am able to stream Audio from the board to other Client PC little disturbance is there...

    but Video still not displaying if i stream

    ReplyDelete
    Replies
    1. What video source are you trying to use? Also, you might try one of the RTP elements to improve audio quality.

      Delete
  7. Server side:

    gst-launch filesrc location=/path/to/video.ext ! decodebin ! x264enc ! video/x-h264 ! rtph264pay pt=96 ! udpsink host=ip.remote.host port=5000 sync=true

    Client side:

    gst-launch udpsrc port=5000 ! rtph264depay ! video/x-h264 ! decodebin ! xvimagesink


    Hi for video i am using the above pipelines not working....

    ReplyDelete
    Replies
    1. Are you using the 'caps' property on the udpsrc? You have to use it when doing RTP. The caps is kind of complicated, but most examples should have it. Try the stream without RTP and then add it in. I often swap out the network elements with 'filesink' and try playing the recorded file in VLC or something just to check it is getting that far.

      Delete
  8. Hi,

    as you said i changed the Pipeline
    Server:
    gst-launch filesrc location=/path/to/video.ext ! decodebin ! x264enc ! video/x-h264 ! udpsink host=ip.remote.host port=5000 sync=true

    client:

    gst-launch udpsrc port=5000 ! video/x-h264 ! decodebin ! filesink location-test.mp4

    but test mp4 is empty... not recorded....:(

    ReplyDelete
  9. Hello!

    Thanks for the guidance! I have a problem though, when i run pavucontrol y get Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-3kAJ98Gs2Q: Connection refused, and therefore (i think) i cannot set the recording devices properly, cause it says "no application is currently recording audio".

    Do you know anything to get around this? thanks!

    ReplyDelete
  10. Hello, Very good information.
    I have an embedded platform with mic and speaker on it. its wifi device. I want to stream audio from one device to another. I cannot have the pulseaudio on this platform.
    How to link it with gstreamer without pulseaudio?

    ReplyDelete

Youtube Channel

Loading...