Using GStreamer

Introduction

GStreamer is a pipeline-based multimedia framework that links together a wide variety of media processing systems to complete complex workflows. The pipeline design serves as a base to create many types of multimedia applications such as video editors, transcoders, streaming media broadcasters and media players. The AMD AMA Video SDK includes an enhanced version of GStreamer which can communicate with the hardware accelerated transcode pipeline in AMD devices.

It is not within the scope of this document to provide an exhaustive guide on the usage of GStreamer. Various resources can be found online, for example:

Overview of the GStreamer Plugins

The AMD AMA Video SDK provides the following GStreamer plugins for building hardware-accelerated video pipelines using AMA compatible devices:

  • ama_av1dec for av1 decoding

  • ama_av1enc for av1 encoding

  • ama_h264dec for h264 decoding

  • ama_h264enc for h264 encoding

  • ama_h265dec for h264 decoding

  • ama_h264enc for h265 encoding

  • ama_scaler for hardware accelerated scaling and color conversion

  • ama_download data download from device to host

  • ama_upload data upload from host to device

The following sections describe the options used with GStreamer to configure and use the various hardware accelerators available on AMA compatible devices.


Decoder Plugin

The ama_av1dec, ama_h264dec and ama_hevcdec plugins provide support for hardware-accelerated decoding using AMA compatible devices, for AV1, AVC and HEVC.

Decoder Inputs and Outputs

  • Input: 8 or 10 bits AV1, AVC or HEVC encoded stream, in OBU format for AV1 or byte-stream format for AVC or HEVC

  • Output: One of NV12_4L4, NV12_10LE_4L4, I420, NV12, I420_10LE, I420_10BE, ARGB, ABGR, RGBA, BGRA, RGB or BGR formats

Decoder Parameters

Property Name

Description
device
Numeric ID of an XMA device
Type: Integer
Default Value: -1 (auto)
Range -1 to 2147483647
low-latency
Whether to enable low latency decoding
Type: Boolean
Default Value: false

Decoder Example Pipelines

Refer to the Decoder Pipeline example for an illustration of how to use this plugin.

Encoder Plugin

The ama_av1enc, ama_h264enc and ama_h265enc plugins provide support for hardware-accelerated encoding using AMA compatible devices, for AV1, AVC and HEVC.

Encoder Inputs and Outputs

  • Input: NV12_4L4, NV12_10LE_4L4, I420, I420_10LE, I420_10BE, NV12 or NV12_10LE32

  • Output: One of AV1, AVC or HEVC 8-bit/10-bit encoded stream, in OBU format for AV1 and byte-stream for AVC and HEVC

Encoder Parameters

Property Name

Description
b-frames
Number of B-frames between two consecutive P-frames
Type: Integer
Range: -1 - 7, for AV1
Range: -1 - 3, for AVC and HEVC
Default: -1 (auto)
bitrate
Fixed output bitrate in bits per second
Type: Integer
Range: 0 - 4294967295
Default: 5000000
crf
If specified, enable CRF mode
Type: Boolean
Default: False
device
Numeric ID of an XMA device
Type: Integer
Range: -1 - 2147483647
Default: -1 (auto)
device-type
AV1 device type to use
Type: Enum
(0): any
(1): Type 1
(2): Type 2
Default: 1
forced-idr
Encode all intra frames as IDR
Type: Boolean
Default: True
gop-length
Maximum distance between two consecutive I frames
Type: Integer
Range: -1 - 2147483647
Default: -1 (auto)
idr-interval
The periodicity of IDR frames
Type: Integer
Range: 0 - 4294967295
Default: 4294967295
lookahead-depth
Number of frames in the future of the currently encoded frame that the encoder will analyze and take into account
Type: Integer
Range: 0 - 40
Default: -1 (auto)
max-bitrate
Use to limit encoding bitrate if you have not specified bitrate parameter
Type: Integer
Range: 0 - 35000000000
Default: -1 (auto)
max-qp
Maximum QP value allowed for the rate control
Type: Integer
Range: 0 - 51, for AVC and HEVC
Range: 0 - 255, for AV1
Default: 51, for AVC and HEVC
Default: 255, for AV1
min-qp
Minimum QP value allowed for the rate control
Type: Integer
Range: 0 - 51, for AVC and HEVC
Range: 0 - 255, for AV1
Default: 0
qp
Fixed quantization value. When constant quality is enabled, bitrate must be set to 0
Type: Integer
Range: -1 - 255
Default: -1 (disable)
qp-mode
QP control mode.
Type: Enum
(0): auto - Auto
(1): relative-load - Relative load (spatial AQ + temporal AQ)
(2): uniform - Uniform (AQ off)
rate-control
Rate control mode to use for encoding
Type: Enum
(-1): auto - Auto
(0): cqp - Constant QP
(1): cbr - Constant bitrate
(2): vbr - Variable bitrate
(3): cvbr - Constrained variable bitrate
slice
Which slice of the XMA device to use
Type: Integer
Range: -1 - 1
Default: -1
spatial-aq
Enable spatial AQ
Type: Boolean
Default: True
spatial-aq-gain
Spatial AQ gain scale factor
Type: Integer
Range: 0 -255
Default: 255
temporall-aq
Enable temporal AQ
Type: Boolean
Default: True
temporal-aq-gain
Temporal AQ gain scale factor
Type: Integer
Range: 0 -255
Default: 255
Default: 1
tier
Tier
Type: Boolean
(-1): auto - Auto
(0): main - Main
(1): high - High
preset
Encoder VQ-Density preset
Valid values: medium or slow. (default is medium)
(medium) Full density with medium VQ - default
(slow) High VQ at lower density
Consult the Tuning Video Quality section for more details on how to use this option.
Note that this option has no effects on AV1 device-type 1 encoding.
tune-metrics
Tune VQ for objective metrics
Type: Enum
(1): vq - Best visual quality
(2): psnr - Best PSNR
(3): ssim - Best SSIM
(4): vmaf - Best VMA
encoding-params-file
Dynamic Configuration File
Path to dynamic file location (default is empty)
Consult the Dynamic Encoder Parameters section for more details on how to use this option.

Encoder Example Pipelines

Refer to the Encoder Pipeline example for an illustration of how to use this plugin.


Scaler Plugin

The ama_scaler plugin provides support for hardware-accelerated resizing and cropping.

Scaler Inputs and Outputs

  • Input: NV12_4L4, NV12_10LE_4L4, I420, NV12, I420_10LE, I420_10BE, ARGB, ABGR, RGBA, BGRA, RGB, and BGR

  • Output: NV12_4L4, NV12_10LE_4L4, I420, NV12, I420_10LE, I420_10BE, ARGB, ABGR, RGBA, BGRA, RGB, and BGR

Scaler Parameters

Property Name

Description
crop
Crop the input frame before scaling.
Type: Boolean
Default Value: false
crop-height
Height of crop area. Must be < input height - crop-top and a multiple of 2
Type: Integer
Range: 0 - 2147483647
Default: 0
crop-left
Left position of crop area. Must be < input width and a multiple of 2
Type: Integer
Range: 0 - 2147483647
Default: 0
crop-top
Top position of crop area. Must be < input height and a multiple of 2
Type: Integer
Range: 0 - 2147483647
Default: 0
crop-width
** Width of crop area. Must be < input width - crop-left and a multiple of 2**
Type: Integer
Range: 0 - 2147483647
Default: 0
device
Numeric ID of an XMA device
Type: Integer
Range: -1 - 2147483647
Default: -1 (auto)

Scaler Example Pipelines

Refer to the Transcode with Multiple-Resolution Outputs example for an illustration of how to use this plugin.

Hardware Down and Up Loads

The ama_download and ama_upload plugins provide support for DMA transfer of data to and from the host, respectively. Note that transaction directions are with respect to the host.

Inputs and Outputs

  • Input: I420, NV12, I420_10LE, I420_10BE, ARGB, ABGR, RGBA, BGRA, RGB, BGR

  • Output: I420, NV12, I420_10LE, I420_10BE, ARGB, ABGR, RGBA, BGRA, RGB, BGR

DMA Parameters

Property Name

Description
device
Numeric ID of an XMA device
Type: Integer
Range: -1 - 2147483647
Default: -1 (auto)

Moving Data through the Pipeline

The GStreamer plugins included in the AMD AMA Video SDK take care of efficiently moving data through the video processing pipeline. For optimal performance, video buffers will be moved between the host and the appropriate device only if needed by plugins in the pipeline. This is known as "zero-copy".

  • The decoder plugin will only copy the output buffer from the device to the host if a downstream element (e.g. filesink) running on the host needs to access the decoded frame. This allows downstream hardware-accelerated plugins (e.g. scaler, encoder) running on the same AMD device to process the decoded frames without unnecessary data transfers.

  • The encoder plugin will only copy the input buffer from host to the device if it is allocated in user-space memory on the host or in a different device. This allows the encoder to process frames produced by upstream hardware-accelerated plugins (e.g. decoder, scaler) running on the same AMD device without unnecessary data transfers.

Working with Multiple Devices

By default (if no device identifier is specified) a job is submitted to device 0. When running large jobs or multiple jobs in parallel, device 0 is bound to run out of resources rapidly and additional jobs will error out due to insufficient resources.

By using the device option of the GStreamer plugins included in the AMD AMA Video SDK, the different functions (decoder, scaler, encoder) of a pipeline can be individually submitted to a specific device. This makes it easy and straightforward to leverage the entire video acceleration capacity of your system, regardless of the number of cards and devices.

Rebuilding GStreamer and Application Examples

Obtain the zipped version of SDK's Gstreamer code from ma35d_sdk_v1.0_gstreamer.zip. Unzip this file into ma35_gstreamer folder. This folder will contain the source code for AMA SDK plugins and a patch for Gstreamer.

You can rebuild the Gstreamer executable with optional plugins by following the instructions below.

  1. Install the prerequisite packages:

    sudo apt install ninja-build flex bison python3-pip unzip pkg-config libjansson-dev uuid-dev
sudo snap install cmake --classic
sudo pip3 install meson ninja

  2. Unzip the downloaded zip file:

    unzip ma35d_sdk_v1.0_gstreamer.zip -d ma35_gstreamer

  3. Get Gstreamer:

    git clone https://github.com/GStreamer/gstreamer.git
cd gstreamer
git checkout f6e672f27fcac06af58d7ac995551893d36944ce

  4. Apply the patch to Gstreamer:

    cp ../ma35_gstreamer/ama_gstreamer.patch .
git apply ama_gstreamer.patch

  5. Build Gstreamer and AMA SDK plugins:

    cd ../ma35_gstreamer
ln -s ../gstreamer/subprojects ./
export C_INCLUDE_PATH=/opt/amd/ama/ma35/include/xma/:/opt/amd/ama/ma35/include/xrm/:/opt/amd/ama/ma35/include/xrm_interface/:$C_INCLUDE_PATH
export LIBRARY_PATH=/opt/amd/ama/ma35/lib/:$LIBRARY_PATH
meson setup build
cd build
ninja

  6. Install Gstreamer and plugins:

    ninja install