Spent the last couple of weeks reading and patching the current gstreamer Sourcecode 1.29.2. Issue in gstreamer gitlab

Our company needs the Sony imx335 camera with its Y10 pixelformat to work with gstreamer. Adding a new pixelformat is pretty straight forward: A definition here and there, a new case statement someplace else. After a day or two, my patch was running at 1 FPS on i.MX8M+.

Later came the realization that the pipeline was not zero-copy, due to the Vivante GPU rejecting the DRM_FORMAT_R16 DMA buffer format. I realized only later, that gstreamer is able to output the EGL supported DMA formats through a debug switch, so I wrote my own test program.

After mapping Y10 to DRM_FORMAT_GR88 instead - the only 16 bpp format for this case supported by the hardware, I got a red picture at 20 FPS. However correcting the image in the glcolorconvert element, slowed down the performance to around 7 FPS. Spent a couple of days puzzled and investigating the issue. It turned out that the pixel shader code in latest gstreamer, with its more general swizzle functions, is too slow for the Vivante GPU. Removing the swizzle function I achieved a performance of 30 FPS. I also turned off sync in the qml6glsink to maximize the throughput.

Upstreaming our patch to gstreamer as is, is not trivial because of our distinct use case. I’m certain that gstreamer wouldn’t adopt our solution, which isn’t as general as theirs. It goes to show however, that the current gstreamer code is not optimal and regressions creeped in when compared to version 1.22, and they obviously lost a lot of performance with gray videos on embedded hardware.

Last modified on 2026-07-20