Skip to content

MPEG1 Streaming

Marcus Oskarsson requested to merge gh-1f5c45f3/503/marcus-oscarsson/streaming into master

Created by: marcus-oscarsson

MPEG1 encoding and streaming displayed using JSMpeg.

Images are written as RGB24 to a v4l2 (Video 4 linux 2) loopback device (/dev/video0). The v4l2loopback kernel module needs to be loaded which will create the loopback device.

The video stream from /dev/video0 is encoded to MPEG1 by ffmpeg and written to a websocket that is read by JSMpeg.

There is a preliminary UI for selecting the video stream size but its not yet connected to the backend. It currently only does client side scaling, like before. There is a few second delay before the stream starts, we will look into if we can reduce this delay.

I've tried to get some performance data, see the graph with comments below

mpeg1-14fps-and-mjpeg-10fps

The red plot shows the original MJPEG stream and the green plot shows the MPEG1 stream. The camera was running in 14 FPS with a video size of 659x493 pixels and the same image (an empty loop was recorded for two minutes). Both plots are showing the TCP packages sent to the decoding client.

MJPEG

  • Only 10 FPS was achieved with the MJPEG stream which most likely means that our encoding process takes a little bit longer than the exposure period of the camera so we are dropping 4 frames per second.

  • Roughly 150000 bytes per second with MJPEG stream, the small dips in the plot corresponds to non full (last segment of the frame) TCP packages.

MPEG1

We can clearly see the key frames (the peaks) in the graph

  • The full 14 FPS received, no signs of bottlenecks within the encoding and streaming chain, We have not yet done any CPU performance measurements. It could perhaps be interesting. (The streaming and the encoding runs as separate processes)

  • Roughly 31500 Bytes per second are sent, the compression of the JPEG key frame is likely much harder in the MPEG1 stream and that would explain why the key frame size ( 40Kb) is so much smaller than the MJPEG frames (150 Kb). The differential encoding is clearly visible in the plot as the points between the peaks.

Comments We still need to perform some measurements on delay, the initial tests on the beamline shows no noticeable delay. The JSMpeg website http://jsmpeg.com/ claim that delays can be as low as 50ms, a claim which we might need to verify :)

The MPEG1 encoding process is clearly much more efficient both in terms of data rate and most likely CPU (that's at least what the dropped frames indicate). The MJPEG stream could probably benefit from faster and much harder JPEG compression.

We finally need to see to this behaves on the internet, especially in regards to delay. It otherwise seems like the MPEG1 stream would be the one to use.

Please let us know if you have any comments or questions !?

Marcus and Matias

Merge request reports