Attention

This version of the SDK has been superseded by the latest release of the SDK.

Using FFmpeg

This page documents how to use FFmpeg with the AMD AMA Video SDK.

Introduction

FFmpeg is an industry standard, open source, widely used utility for handling video. FFmpeg has many capabilities, including encoding and decoding of all video compression formats, encoding and decoding of audio, encapsulating, and extracting audio, and video from transport streams, and many more. The AMD AMA Video SDK includes an enhanced version of FFmpeg, version n4.4, which communicates with the hardware accelerated transcode pipeline.

It is not within the scope of this document to provide an exhaustive guide on the usage of FFmpeg. Various resources can be found online, for example:

• https://www.ffmpeg.org/documentation.html

• http://howto-pages.org/ffmpeg/

The following sections describe the options used with FFmpeg to configure the various hardware accelerators available on an MA35 device.

Example Commands

A simple FFmpeg command for accelerated transcoding with the AMD AMA Video SDK will look like this one:

ffmpeg -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i infile.mp4 -c:v hevc_ama -b:v 1000K -r 60 -f mp4 -y transcoded.mp4

There are many other ways in which FFmpeg can be used to leverage the video transcoding features of an MA35 device. Examples illustrating how to run FFmpeg for encoding, decoding, and transcoding with or without ABR scaling, video composition and ML are included in FFmpeg tutorials.

General FFmpeg Options

General FFmpeg Options

Options	Descriptions
-i	Input video URI.
-c:v	Specify the video codec This option must be set for any video stream processed on an MA35 device. Valid decode values are vp9_ama, h264_ama, hevc_ama, and av1_ama for VP9, H.264, HEVC, and AV1, respectively. Valid encode values are h264_ama, hevc_ama, and av1_ama for H.264, HEVC, and AV1, respectively.
-s	The frame size (WidthxHeight). For example 1920x1080 or 3840x2160.
-f	The container format.
-r	The frame rate in fps (Hz).
-filter_complex	Used for frame filtering operation.
-hwaccel	Global option used to specify on which MA35 device the FFmpeg job should run. Consult Using Explicit Device IDs for more details on how to use this option. Valid values are positive integers in [0..N-1] range. Default is device 0.
-vf	Specify DMA direction. Valid options are hwdownload and hwupload, for DMA transfer of data to and from the host, respectively. Note that transaction directions are with respect to the host.

Note

Detailed help for encoder, decoder and scaler is obtained through FFmpeg extended help, e.g. ffmpeg -h encoder=h264_ama, ffmpeg -h decoder=hevc_ama or ffmpeg -h filter=scaler_ama.

The following conventions are used in the following tables:

1. When parameters can be specified as both numerical and string values, they are noted as:

   (<number>) <string>
   

   Conversely, when this is not the case, options are noted with Roman numerals as:

   <Roman numeral>) <option>
   

2. auto or -1 note that values are selected by the SDK to optimally meet the specified constraints.

Video Decoding

For the complete list of features and capabilities of the AMD AMA Video SDK decoder, refer to Release Notes chapter of the documentation.

The table below describes all the options for the AMD AMA Video SDK decoder.

Video Decoding Options

Options	Descriptions
-out_fmt	Decoder output pixel format Valid global values for pixel format: nv12, p010le, packed10, rgbp, yuv420p, and yuv420p10le. Note that an identical pixel format must be specified as value to format option, within -vf. See Decode Only for an example.
-low_latency	Enable/Disable single frame level decode When enabled, the decoding latency of streams not containing any B frames is reduced. This option has no impact on streams that include B frames. Valid values: 0 or 1. (default is 0) (0) disable - default (1) enable

Video Encoding

For the complete list of features and capabilities of the AMD AMA Video SDK encoder, refer to Release Notes chapter of the documentation.

The table below describes all the options for the AMD AMA Video SDK encoder.

Video Encoding Options

Options	Descriptions
-b:v	Specify the video bitrate You can specify this in Mb or Kb. For example -b:v 1M or -b:v 1000K.
-max_bitrate	Maximum bitrate Valid values: 0 to 3.5e+10 You may want to use this to limit encoding bitrate if you have not specified a -b:v bitrate Default value -1 implies unspecified.
-g	GOP size Valid values: 0 to UINT32_MAX Default value is -1, implying 2[s] gop duration, i.e., twice the frame rate
-level	Encoding level restriction Valid values for H.264: 1, 1.1, 1.2, 1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4, 4.1, 4.2, 5, 5.1, 5.2 6, 6.1, 6.2 Valid values for HEVC: 1, 2, 2.1, 3, 3.1, 4, 4.1, 5, 5.1,5.2, 6, 6.1, 6.2 Valid values for AV1: 2, 2.1, 3, 4, 4.1, 5, 5.1, 5.2, 5.3 6, 6.1, 6.2, 6.3 Default value is auto.
-profile	Set the encoding profile Valid values for H.264: baseline, main, high Valid values for HEVC: main, main-intra, main-10 and main-10-intra Valid values for AV1: main Default value is auto.
-tier	Set the encoding tier (AV1 and HEVC only) Valid values: -1, 0, 1 (-1) auto (0) main (1) high Default value is auto
-bufsize	Rate control (VBV) buffer size Valid values: -1, 400M [bits] (default is -1) (-1) For auto selection This option is ignored unless: 1) lookahead_depth is 0 (ULL) 2) control_rate is CBR 3) Encoder is set to one of h264_ama, hevc_ama or av1_ama Type 2 Value 0 implies strict ULL, whereas, larger values imply relaxed ULL. See Latency Adjustment.
-crf	Constant Rate Factor Valid values: 0, 1 (default is 0) (0) disable - default (1) enable This option uses -qp for its quality assignment. Note that this option ignores all other options that may impose restrictions on qp values. These include -min_qp, -max_qp, -b:v, and -control_mode.
-qp	Quantization Parameter Valid values for H.264 and HEVC 0-51 Valid values for AV1 0-255 Default value -1 implies not used. Note that if this option is used, it implies Constant QP for -control_rate option.
-bf	Number of B frames Valid values for H.264, HEVC and type-2 AV1: 0-3 Valid values for type-1 AV1: 0-7 Default auto selection value is -1.
-lookahead_depth	Number of frames to lookahead for qp maps Valid values: -1 to 47+Maximum number of B frames -bf Lookahead depth of 0 implies ULL mode Default auto selection value, based on frame rate, is -1.
-spatial_aq	Enable spatial AQ Valid values: auto, disable or enable (-1) auto (0) disable (1) enable Default value is auto. When qp_mode is uniform, this parameter is ignored and treated as disable.
-spatial_aq_gain	Scale for spatial AQ gain Valid values: 0 to 255 (255) For auto selection Default value for H264, HEVC and AV1 is 64 Default value for type-1 AV1 is 72
-temporal_aq	Enable temporal AQ Valid values: auto, disable or enable (-1) auto (0) disable (1) enable Default value is auto. When qp_mode is uniform, this parameter is ignored and treated as disable.
-temporal_aq_gain	Scale for temporal AQ gain Valid values: 0 to 255 (255) For auto selection Default value for H264, HEVC and AV1 is 80 Default value for type-1 AV1 is 0
-qp_mode	QP control mode Valid values: auto, relative_load and uniform. (0) auto for best VQ (1) relative_load implying both spatial and temporal AQ (2) uniform meaning no AQ Default value is auto.
-control_rate	Set the rate control mode Valid values: -1 to 4 (-1) auto (0) const_qp for Constant QP (1) cbr for Constant Bitrate (2) vbr for Variable Bitrate (3) cvbr for Capped Variable Bitrate (4) cabr for content adaptive bit-rate See RC Mode Table for details. Default value is auto.
-min_qp	Minimum QP value allowed for rate control Valid values for H264 and HEVC: 0 to 51 Valid values for AV1: 0 to 255 Default auto selection value is -1. This option has no effect when -control_rate is set to Constant QP (0).
-max_qp	Maximum QP value allowed for rate control Valid values for H264 and HEVC: 0 to 51 Valid values for AV1: 0 to 255 Default auto selection value is -1.
-qp_i_offset	QP Offset for I Frames Valid values: -51 to 51 Auto selection mode is -1. Default 0.
-qp_b_offset	QP Offset for B or P Frames Valid values: -51 to 51 Auto selection mode is -1. Default 0. This option has no effect when -control_rate is set to Constant QP (0).
-forced_idr	Force insertion of IDR frames Valid values: 0, 1 (default is 1) (0) disable (1) enable - default Encodes all the intra frames as IDR
-slice	Select a slice to run an encode job Valid values if xrmd is not used are: 0, 1 (default is 0). If xrmd is active and this option is not set, xrmd will make the slice selection.
-tune_metrics	Enable tuning video quality for the best objective metrics This option optimizes the encoder for the selected tuning metric. See Tuning Video Quality for details. Valid values: 1 to 4 (default is 1) (1) vq - default (2) psnr (3) ssim (4) vmaf
-type	Applicable to AV1 encoders only Valid values: 1 or 2. i) 1 - default ii) 2 Default value is 1. Consult the AV1 Selection section for more details on how to use this option.
-preset	Encoder VQ-Density preset Valid values: fast, medium or slow. (default is medium) (fast) Allows for increase in total throughput from 4kp60 to 4kp75 , when 2 or more channels are used. Does not apply to AV1 -type 1 encoding. Not applicable under ULL setting. Note lower VQ than medium preset is to be expected. (medium) Full density with medium VQ - default (slow) High VQ at lower density. Applicable to 10 bit contents only. Consult the Tuning Video Quality section for more details on how to use this option.
-dynamic_gop	Enable dynamic GOP Valid values: -1, 0 or 1. (-1) auto (0) disabled (1) enable Default value is auto.
-dynamic_params_file	Dynamic configuration file Path to dynamic file location (default is empty) Consult the Dynamic Encoder Parameters section for more details on how to use this option.
-cores	AV1 Type 1 Cores Valid values: 1 or 2 i) 1 - default, with minimum resolution of 176x144 or 144x176, and aggregate rate of 4kp60 ii) 2 - minimum resolution of 720x720, with with aggregate rate of 4kp120, in ULL mode
-latency_ms	Look ahead depth in milliseconds Valid values: 0 to 60000 Default auto selection value, based on frame rate, is -1.
-no_bll	Disable low latency B frames Valid value: 1 Value 1 ensures that the generated GOP has a homogeneous structure. See GOP Composition.

The following table describes relevant parameters that are applicable to each RC mode

RC Mode Options

RC Mode	Allowed Options
const_qp	Constant QP -qp <QP>
cbr	Constant Bit Rate -b:v <Bit Rate> (ABR mode) -b:v <Bit Rate> -max_bitrate <Bit Rate> (CBR mode) -b:v <Bit Rate> -bufsize 0 ULL mode
vbr	Variable Bit Rate -b:v <Bit Rate>
cvbr	Constrained Variable Bit Rate -b:v <Bit Rate> [-min_qp <QP> | -max_qp <QP> | -max_bitrate <max_bitrate>]
crf	Constant Rate Factor -crf 1 -qp <QP>
cabr	Content Adaptive Bit Rate -b:v <Bit Rate>

Video Scaling

The AMD AMA Video SDK hardware scaler is leveraged in FFmpeg by using the scaler_ama complex filter and the FFmpeg filter graph syntax. This section describes the options of the scaler_ama complex filter.

scaler_ama

Filter implementing the AMD AMA Video SDK ABR multiscaler. Takes one input and up to 16 output streams. The complete list of options is described below.

Multiscale Filter Options

Options	Description
outputs	Specify the number of scaler outputs Valid values are integers between 1 and desired number of outputs.
out_res	Specify ABR ladder rungs <1>*<16>(WxH|Rate|Format) List of up to 16 tuples of form (Resolution|Rate|Pixel Format). Note that no spaces are allowed before or after |. Valid values for resolution: 176x144 to 3840x2160 - default 1920x1080 Valid values for frame rate: half and full - default full, where full refers to the incoming frame rate and half to the half of the incoming frame rate. Note that the first entry in this option must be full rate. Valid global values for pixel format: nv12, p010le, packed10, rgbp, yuv420p, and yuv420p10le
crop	Specify the crop area Filter chain of form out_res=(WXH|Rate|Pixel Format):crop(W|H|x|y), where x and y specify the coordinates of crop on 0,0 top-left coordinate system.
enable_pipeline	Enable pipelining to improve performance Valid values: 0 and 1 (default 1) (0) disable (1) enable - default Note that enabling this option adds 2 frames worth of latency to the pipeline.

2D Engine GPU

The AMD AMA Video SDK 8-bit 2D GPU is leveraged in FFmpeg by using the 2d_ama complex filter and the FFmpeg filter graph syntax. This section describes the options of the 2d_ama complex filter.

2d_ama

Filter implementing the AMD AMA Video SDK 2D GPU. Performs various 2D operations such as color space conversion, tiling, etc., via the accelerated engine without relying on the host CPU.

2D Engine GPU Options

Options	Descriptions
inputs	Specify number of inputs Valid values 1 to INT_MAX Default is 2
processor	Specify 2D GPU Valid values are: i) rotate for multiples of 90 degree rotations ii) csc for color space conversion iii) subsample for pixel subsampling iv) overlay for video overlay v) tile for tiling operation Default is none
core_id	Specify execution core Valid values 0 to 1 Default is 0
shortest	Specify multi-video exit condition Force termination when the shortest input terminates Valid values true or false Default false
w	Specify the width of video Valid values 0 to 7680 Default is sum of input widths
h	Specify the height of video Valid values 0 to 7680 Default is sum of input heights
	Specify layout options Specified as NxM, where N*M <=32

In the following table if default value is not specified, it implies none.

GPU Specific Options

Options	Descriptions
rotate	Specify rotation options rotation valid values are 90, 180 and 270
csc	Specify color space conversion options csc valid values are rgb2yuv and yuv2rgb
subsample	Specify 4:2:2 to 4:2:0 conversion No options
overlay	Specify overlay options x horizontal position 0 to 7680 y vertical position 0 to 7680 Note that inputs to this plugin must have the same frame rate. Use -r to enforce this.
tile	Specify tiling operation No options

Video Machine Learning

This section describes various video ML complex filters of AMA SDK that are utilized using FFmpeg filter graph syntax. Note that supported ROI models are 1080p in landscape mode, 720p in both landscape and portrait modes.

Inference Engine

The AMD AMA Video SDK ML inference engine is leveraged in FFmpeg by using the ml_ama complex filter.

ml_ama

Filter implementing the AMD AMA Video SDK ML inference engine that executes against a supplied rgbp input and detects face or text classes. Its output is used by downstream components in the pipeline.

Inference Engine Options

Options	Descriptions
model	Specify engine model Valid value roi
model_args	Model specific arguments Valid values for roi: type=face and type=text i) type=text - for text detection VQ improvements ii) type=face - for face detection VQ improvements
core_id	Specify engine number Valid values: range -1 to 3 or none i) If xrmd is enabled, this will be selected automatically ii) If xrmd is disabled, and core_id is set to -1, then one of 0 to 3 cores will be selected ii) Otherwise, value in 0-3 range will be selected
nb_inputs	Specify number of inputs Valid value: 1 Default 1

ROI Scaler

The AMD AMA Video SDK ROI scaler is leveraged in FFmpeg by using software plugin roi_scale_ama complex filter.

roi_scale_ama

Filter implementing the AMD AMA Video SDK ROI scaler engine that executes against a supplied rgbp input and output of ml_ama. This engine generates metadata used by downstream encoder within the pipeline.

Scaler Options

Options	Descriptions
cutoff	Specify cutoff threshold Valid value range 0-100000 Default 5000 0-100% range is mapped to 0-100000, e.g., 5.67% is represented as 5670, and is used as shut off threshold for detected ROI regions. That is, if the percentages of detected ROI regions is above cutoff, then ROI directed encoding is disabled..
strength	Specify probability-to-QP conversion gain factor Valid values are low, normal, and high Default is normal Providing high value generates higher QP modulation that signals to encoder to spend more bits on the associated block.
nb_inputs	Specify number of inputs Valid values: range 2 to 4 Default 2
roi_map_type	Specify probability-to-QP mapping Valid values are dqp, roi, dqp_and_roi Default dqp
static_map_file	Specify static map file Valid value is the path to rgbp map file. The size of this file is ceil of width and height of incoming video divided by 16, in each dimension. This file contains values in 0-255 range, for each channel, and servers as the lower boundary of probability-to-QP mapping, i.e., maximum of values in this file and ml_ama is used for QP mapping

ROI Overlay

The AMD AMA Video SDK ROI overlay is leveraged in FFmpeg by using software plugin roi_overlay_ama complex filter.

roi_overlay_ama

Filter implementing the AMD AMA Video SDK ROI overlay that overlays shaded probability rectangles on detected classes.

Overlay Options

Options	Descriptions
skip_frames	Specify overlay period Valid value range 0-255 The value toggles overlay rectangles after the specified number of frames.
highlight_threshold	Specify activation threshold Valid value range 0-255 Default 128

See ML Based Face ROI and ML Based Text ROI for sample usage.

Rebuilding FFmpeg

Obtain the zipped version of SDK's FFmpeg source code, from ma35d_sdk_v1.1.2_ffmpeg.zip. Unzip this file into ma35_ffmpeg folder. This folder contains the entire source code for the FFmpeg executable, which is a fork of the main FFmpeg GitHub (release 5.1.2) with AMD plugins patches applied. Due to licensing restrictions, the FFmpeg executable included in the Video SDK is enabled with the AMD AMA Video SDK plugins only.

You can rebuild the FFmpeg executable with optional plugins by following the instructions below. Additionally, comprehensive instructions for compiling FFmpeg can be found on the FFmpeg wiki page.

1. Make sure libhugetlbfs-dev, nasm and yasm are installed on your machine.

2. Unzip the downloaded zip file:

   unzip ma35d_sdk_v1.1.2_ffmpeg.zip
   

and navigate to root folder of the unzipped file.

1. Execute the following:

   ./configure --disable-alsa --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-libxcb-shape --disable-xlib --disable-libmfx --disable-vaapi --disable-vulkan --prefix=$PWD --disable-stripping --enable-shared --enable-vpe --enable-libxrm --extra-libs="-L/opt/amd/ama/ma35/lib -lvpi -lroi_scale -llog_ama -lxrm -lxrm_interface -lhugetlbfs -lpthread -lstdc++ -lm -ldl -lrt  /opt/amd/ama/ma35/lib/libxrm.so.1.6.0" --extra-cflags="-I/usr/include -I$PWD -I/opt/amd/ama/ma35/include/xrm_interface/ -I/opt/amd/ama/ma35/include/xrm/ -DSUPPORT_OSAL" --extra-ldflags="-Wl,-rpath=$PWD/libavutil:$PWD/libavfilter:$PWD/libavcodec:$PWD/libavformat:$PWD/libavdevice:$PWD/libswscale:$PWD/libswresample:$PWD:/opt/amd/ama/ma35/lib/"
   

2. Add any other plugins such as libx264 to the above configure line. (See FFmpeg Compilation Guide for details.) A list of configure options is printed by running configure --help.

3. Type make -j to build FFmpeg. GNU Make 3.81 or later is required.

4. Type make install to install all binaries and libraries you built.