Android and Raspberry Pi Camera low/no latency via gstreamer

Article is inspired by video .

I will not go into detail what gstreamer is and how gstreamer works, because I assume you already know it. I assume you also you have a basic knowledge and experience with video streaming and RPi devices.

It’s very hard to measure and achieve low latency or even zero latency, so we need to focus on simplification and removing bottle necks in our gstreamer pipeline to get best performance.

Hardware used in experiment:

  1. Raspberry Pi device, model B/B+/B2 ( with Raspbian Wheezy or Jessie installed
  2. Raspberry Pi camera module (
  3. Samsung S4 with Android 4.4.2 installed

Software used:

  1. gstreamer1.0 (1.4.3)
  2. RaspberryPi Camera Viewer (


  1. Connect camera module and launch RPi. Please rememebr to enable your camera module (
  2. Install gstreamer 1.x on RPi
    sudo apt-get install gstreamer1.0
  3. Install RaspberryPi Camera Viewer ( on your Android device

Now you can try to stream video from your camera via gstreamer and it’s time to focus on latency. If you have already tried experiment with raspivid, netcat and mplayer, then you see how effective such combination can be. Mplayer has lowest latency itself, but mplayer is not a media framework you may want to use and integrate with your other systems. However, here is an experiment you may try to see results:

 raspivid -t 0 -hf -n -h 512 -w 512 -fps 15 -o - | nc 5001
 c:\nc111nt_rodneybeede\nc.exe -L -p 5001 | c:\mplayer-svn-36251\mplayer.exe -vo direct3d -fps 24 -cache 512 -

Where is your Laptop IP.

In my experiment with gstreamer I focused on achieving the same performance as I observed with mplayer. I started reducing bottle necks from network, then by pipeline simplification. Here, Netcat due to it’s simplicity has better performance than gstreamer’s tcpserversink and tcpclientsink. I was surprised, but UDP sinks/clients were even worst. I modified pipeline by adding small queue with size=1 to keep only one frame in the buffer and drop all the rest. Subsequent pipeline elements, like videorate and autovideosync is all we need.

In both examples below two IPs were used: is an Android device, is RPi.

Low latency solution – TCP version (where android device is a host):

  1. On Android device, open RaspberryPi Camera Viewer and run following pipeline:
    tcpserversrc host= port=5001 ! queue2 max-size-buffers=1 ! decodebin ! autovideosink sync=false


  2. On Raspberry Pi:
    raspivid -t 0 -hf -n -h 480 -w 640 -fps 15 -o - | nc 5001
  3. click play button on the center of preview window in RaspberryPi Camera Viewer

Low latency solution – TCP version (where RPi is a host):

  1. On Raspberry Pi:
    raspivid -t 0 -hf -n -h 480 -w 640 -fps 15 -o - | nc -l -p 5001
  2. On Android device, open RaspberryPi Camera Viewer and run following pipeline:
    tcpclientsrc host= port=5001 ! queue2 max-size-buffers=1 ! decodebin ! autovideosink sync=false


  3. click play button on the center of preview window in RaspberryPi Camera Viewer

To get even better performance, you can also play with videorate, like:

tcpserversrc host= port=5001 ! queue2 max-size-buffers=1 ! decodebin ! videorate ! video/x-raw,framerate=60/1 ! autovideosink sync=false

In theory, setting framerate bigger than input streams’ rate could help, but personally I did not see any difference. However, it’s worth to try and see results.

Network bandwidth is satisfied as well (nmon output):


Low latency solution – UDP version:

  1. On Android device, open RaspberryPi Camera Viewer and run following pipeline:
    udpsrc port=5000 ! queue2 max-size-buffers=1 ! decodebin ! autovideosink sync=false


  2. On Raspberry Pi (where is IP of your Android device):
    raspivid -t 0 -hf -n -h 480 -w 640 -fps 15 -o - | gst-launch-1.0 --gst-debug=3 fdsrc ! udpsink host= port=5000

    low latency udp

  3. click play button on the center of preview window in RaspberryPi Camera Viewer


I hope this article can help you with effective video streaming with minimal latency. I’m happy to discuss, so feel free to comment or contact with me.

How to compile boost for QT MSVC2012 x64

  • Download boost
  • open “Qt 5.2.1 64-bit for Desktop (MSVC 2012)” command prompt (or “VS2012 x64 Cross Tools Command Prompt”)
  • Execute .\bootstrap.bat
  • Execute b2 address-model=64 link=static toolset=msvc-11.0 --build-type=complete stage --with-filesystem --with-signals --with-system --with-thread --with-date_time --with-log
  • Conversion VC++ library to GNU (MinGW, GCC) format

    Sometimes you have to use other compiler than VC++ (ok, be honest, in most cases 🙂 ), but what to do when your libs are in VC++ format? Solution is very easy, and it will be shown below on MySQL library case.

    I assume you have mingw.

    Download mingw-utils:

    reimp.exe -d libmysql.lib
    dlltool.exe -k -d libmysql.def -l libmysql.a

    That’s all folks 🙂

    How to compile libjpeg using mingw

    Downlaod latest MinGW from sourceforge: and install it to c:\MinGW

    Download latest libjpeg sources from: and untar it to c:\MinGW\msys\1.0\home\

    Launch msys shell: c:\MinGW\msys\1.0\msys.bat
    cd /home//libjpeg-
    make install

    now in .libs folder we will have files: libjpeg.a, libjpeg.dll.a and libjpeg-7.dll

    How to make FreeTDS, unixODBC and QT working together

    UnixODBC and FreeTDS driver compialtion and configuration has been taken from the

    Here are instructions for getting tsql and isql to play nice together.
    Apt-getting doesn’t always get you all the tools you need to develop against SQLserver and Sybase.
    Gather some important packages:

    apt-get install libtool bison autotools-dev g++ build-essential tcsh unixodbc-dev tdsodbc

    Download, extract and compile FreeTDS driver:

    tar xzvf freetds-stable.tgz
    cd freetds-0.82/
    ./configure --prefix=/usr --sysconfdir=/etc --with-unixodbc=/usr --with-tdsver=8.0
     make && make install clean

    Create a file named tds.driver.template with the following contents:

    Description     = v0.82 with protocol v8.0
    Driver          = /usr/lib/

    Register the driver with ODBC

    odbcinst -i -d -f tds.driver.template

    On 64 bit Debian, I’ve run into the situation where freetds won’t compile against the distribution’s ODBC. To resolve this, it was necessary to compile and install a parallel ODBC and compile freetds against it.
    If this is a server, you may not have a GUI, so don’t try to compile with it.

    tar xvf unixODBC-2.2.14.tar
    cd unixODBC-2.2.14
    ./configure --enable-gui=no
    make install

    Default installation dir is: /usr/local/bin/. Now, follow the above instructions replacing the configure line for freetds with this:

    ./configure --prefix=/usr --sysconfdir=/etc --with-unixodbc=/usr/local --with-tdsver=8.0

    Possible errors:

    • SQLInstallDriverEx failed with Invalid install path
    • . Probably /usr/local/etc/odbcinst.ini file is missing, or use root account to install driver template.

    How to test unixODBC:

    odbcinst -s -q show available ODBC sources
    odbcinst -d -q show available ODBC drivers
    odbcinst -j show config
    isql -v datasourcename login password conenct using data source name

    How to test freetds:

    tsql -H -p 1433 -U sa -P password conenct using host name
    tsql -S datasourcename -U sa -P password conenct using data source name from the freetds.conf file

    Configure QT to work with unixODBC

    Note that QT application will use files:

    , but isql uses:

    The best solution is to create symbolic links and keep orginal files in one place.

    cd $QT/src/plugins/sqldrivers/odbc
    qmake "INCLUDEPATH+=/usr/local/unixODBC/include" "LIBS+=-L/usr/local/unixODBC/lib -lodbc"
    make install

    On Ubuntu x64 we need to modify following files manually:
    sudo vim /usr/local/include/sqltypes.h

    #ifdef ODBCINT64
    //typedef ODBCINT64     SQLBIGINT;
    typedef long    SQLBIGINT;
    #ifdef UODBCINT64
    //typedef UODBCINT64    SQLUBIGINT;
    typedef unsigned long   SQLUBIGINT;

    vim ../../../sql/drivers/odbc/qsql_odbc.cpp

    when connection fails, then try to update add LD_LIBRARY_PATH in Qt project settings
    LD_LIBRARY_PATH = /usr/lib:/usr/lib/odbc:/usr/local/lib:/usr/local/Trolltech/Qt-4.7.4/lib

    cd $QT/src/plugins/sqldrivers/tds
    qmake "INCLUDEPATH+=$QT/QtSources/4.8.1/include" "LIBS+=-L/usr/local/unixODBC/lib -lodbc"
    make install
    qDebug() << QSqlDatabase::drivers();

    (“QSQLITE”, “QODBC3”, “QODBC”)

    QSqlDatabase db = QSqlDatabase::addDatabase("QODBC3");
    if( ! )
    	qDebug() << "Connection errror:"  << db.lastError().text();
    	QSqlQuery query("SELECT country FROM artist", db);
    	while (
    		QString country = query.value(0).toString();
    		qDebug() << country;

    #Driver = /usr/lib/
    Driver = FreeTDSDriver
    Description = Northwind sample database
    Trace = Yes
    Server =
    Port = 4101
    Database = kplus
    UID = kplus

    cat /etc/odbcinst.ini
    Description = v0.82 with protocol v8.0
    Driver = /usr/lib/
    UsageCount = 2
    CPTimeout =
    CPReuse =

    host =
    port = 4101
    tds version = 5.0
    dump file = /var/log/freetds.log


    How to generate C++ .h file with current SVN revision number

    for /f "delims=: tokens=1,2" %%a in ('svn info') do (
      if "%%a"=="Revision" (
        set /a RELEASE_REVISION=%%b
    set SVNVERSIONFILE="svn_current_version.h"
    echo #endif >> %SVNVERSIONFILE%