Using FFmpeg

This page documents how to use FFmpeg with the AMD AMA Video SDK.

Introduction

FFmpeg is an industry standard, open source, widely used utility for handling video. FFmpeg has many capabilities, including encoding and decoding of all video compression formats, encoding and decoding of audio, encapsulating, and extracting audio, and video from transport streams, and many more. The AMD AMA Video SDK includes an enhanced version of FFmpeg, version n4.4, which communicates with the hardware accelerated transcode pipeline.

It is not within the scope of this document to provide an exhaustive guide on the usage of FFmpeg. Various resources can be found online, for example:

• https://www.ffmpeg.org/documentation.html

• http://howto-pages.org/ffmpeg/

The following sections describe the options used with FFmpeg to configure the various hardware accelerators available on an MA35 device.

Example Commands

To obtain the FFmpeg version used in this SDK, issue the following command:

ffmpeg -version

A simple FFmpeg command for accelerated transcoding with the AMD AMA Video SDK will look like the following:

ffmpeg -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i infile.mp4 -c:v hevc_ama -b:v 1000K -r 60 -f mp4 -y transcoded.mp4

There are many other ways in which FFmpeg can be used to leverage the video transcoding features of an MA35 device. Examples illustrating how to run FFmpeg for encoding, decoding, and transcoding with or without ABR scaling, video composition and ML are included in FFmpeg tutorials.

To

General FFmpeg Options

General FFmpeg Options

Options	Descriptions
-i	Input video URI.
-c:v	Specify video or image codec This option must be set for any input processed on an MA35 device. Valid video decode values are vp9_ama, h264_ama, hevc_ama, and av1_ama for VP9, H.264, HEVC, and AV1, respectively. Valid video encode values are h264_ama, hevc_ama, and av1_ama for H.264, HEVC, and AV1, respectively. Valid image decode value is jpeg_ama for JPEG images. Valid image encode values are jpeg_ama, ljpeg_ama, and av1_ama for JPEG, lossless JPEG and AV1 images. See Image Encoding Options for details.
-s	The frame size (WidthxHeight). For example 1920x1080 or 3840x2160.
-f	The container format.
-r	The frame rate in fps (Hz).
-filter_complex	Used for frame filtering operation.
-hwaccel	Global option used to specify on which MA35 device the FFmpeg job should run. Consult Using Explicit Device IDs for more details on how to use this option. Valid values are positive integers in [0..N-1] range. Default is device 0.
-vf	Used for simple filtering operations, e.g., hwdownload and hwupload, for DMA transfer of data to and from the host, respectively. Note that transaction directions are with respect to the host.

Note

Detailed help for encoder, decoder and scaler is obtained through FFmpeg extended help, e.g. ffmpeg -h encoder=h264_ama, ffmpeg -h decoder=hevc_ama or ffmpeg -h filter=scaler_ama.

Note

Using -stream_loop option along with jpeg_ama, results in the following benign error message:

[ERROR] AMA.VPI.DEC.jpeg_ama@0x5555b71aa980:{492302} Can't find valid pts info

The following conventions are used in the following tables:

1. When parameters can be specified as both numerical and string values, they are noted as:

   (<number>) <string>
   

   Conversely, when this is not the case, options are noted with Roman numerals as:

   <Roman numeral>) <option>
   

2. auto or -1 note that values are selected by the SDK to optimally meet the specified constraints.

Video Decoding

For the complete list of features and capabilities of the AMD AMA Video SDK decoder, refer to Release Notes chapter of the documentation.

The table below describes all the options for the AMD AMA Video SDK decoder.

Video Decoding Options

Options	Descriptions
-resize	Decoder down scaler Valid global values are of the for WidthxHeight. Note that this parameter must be specified before the intended input.
-out_fmt	Decoder output pixel format Valid global values for pixel format: nv12, p010le, packed10, rgbp, yuv420p, and yuv420p10le. Note that an identical pixel format must be specified as value to format option, within -vf. See Decode Only for an example.
-low_latency	Enable/Disable single frame level decode When enabled, the decoding latency of streams not containing any B frames is reduced. This option has no impact on streams that include B frames. Valid values: 0 or 1. (default is 0) (0) disable - default (1) enable

Video Encoding

For the complete list of features and capabilities of the AMD AMA Video SDK encoder, refer to Release Notes chapter of the documentation.

The table below describes all the options for the AMD AMA Video SDK encoder.

Video Encoding Options

Options	Descriptions
-b:v	Specify the video bitrate You can specify this in Mb or Kb. For example -b:v 1M or -b:v 1000K.
-max_bitrate	Maximum bitrate Valid values: 0 to 3.5e+10 You may want to use this to limit encoding bitrate if you have not specified a -b:v bitrate Default value -1 implies unspecified.
-g	GOP size Valid values: 0 to UINT32_MAX Default value is -1, implying 2[s] gop duration, i.e., twice the frame rate
-level	Encoding level restriction Valid values for H.264: 1, 1.1, 1.2, 1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4, 4.1, 4.2, 5, 5.1, 5.2 6, 6.1, 6.2 Valid values for HEVC: 1, 2, 2.1, 3, 3.1, 4, 4.1, 5, 5.1,5.2, 6, 6.1, 6.2 Valid values for AV1: 2, 2.1, 3, 4, 4.1, 5, 5.1, 5.2, 5.3 6, 6.1, 6.2, 6.3 Default value is auto.
-profile	Set the encoding profile Valid values for H.264: baseline, main, high Valid values for HEVC: main, main-intra, main-10 and main-10-intra Valid values for AV1: main Default value is auto.
-tier	Set the encoding tier (AV1 and HEVC only) Valid values: -1, 0, 1 (-1) auto (0) main (1) high Default value is auto
-bufsize	Rate control buffer size flag Valid values: 0, or non-zero (default is 1) (0) Disable relaxed ULL (non-zero) Enable relaxed ULL This option is ignored unless: 1) lookahead_depth is 0 (ULL) 2) control_rate is CBR Value 0 implies strict ULL, whereas, other values imply relaxed ULL. See Latency Adjustment.
-crf	Constant Rate Factor Valid values: -1 to 63 (-1) default Note that this option ignores all other options that may impose restrictions on qp values. These include -min_qp, -max_qp, -b:v, and -control_mode.
-qp	Quantization Parameter Valid values for H.264 and HEVC 0-51 Valid values for AV1 0-255 Default value -1 implies not used. Note that if this option is used, it implies Constant QP for -control_rate option.
-bf	Number of B frames Valid values for H.264, HEVC and type-2 AV1: 0-3, with default 3. Valid values for type-1 AV1: 0-7, with default 7 for FPS greater than 30, and 3 otherwise. Default auto selection value is -1.
-lookahead_depth	Number of frames to lookahead for qp maps Valid values: -1 to 47+Maximum number of B frames -bf Lookahead depth of 0 implies ULL mode Default auto selection value, based on frame rate, is -1.
-spatial_aq	Enable spatial AQ Valid values: auto, disable or enable (-1) auto (0) disable (1) enable Default value is auto. When qp_mode is uniform, this parameter is ignored and treated as disable.
-spatial_aq_gain	Scale for spatial AQ gain Valid values: 0 to 255 (255) For auto selection Default value for H264, HEVC and AV1 is 64 Default value for type-1 AV1 is 72
-temporal_aq	Enable temporal AQ Valid values: auto, disable or enable (-1) auto (0) disable (1) enable Default value is auto. When qp_mode is uniform, this parameter is ignored and treated as disable.
-temporal_aq_gain	Scale for temporal AQ gain Valid values: 0 to 255 (255) For auto selection Default value for H264, HEVC and AV1 is 80 Default value for type-1 AV1 is 0
-qp_mode	QP control mode Valid values: auto, relative_load and uniform. (0) auto for best VQ (1) relative_load implying both spatial and temporal AQ (2) uniform meaning no AQ Default value is auto.
-control_rate	Set the rate control mode Valid values: -1 to 6 (-1) auto (0) const_qp for Constant QP (1) cbr for Constant Bit Rate (2) vbr for Variable Bit Rate (3) cvbr for Capped Variable Bit Rate (4) cabr DEPRECATED Content Adaptive Bit Rate (5) crf RESERVED (6) mlcae Experimental Machine Learning Content Aware Encoding See RC Mode Table for details. Default value is auto.
-cabr	CABR parameters Valid values: i) enable ii) disbale iii) auto - The same as disable iv) vq_offset =x, where x is in -63 - 63 range. If not specified, vq_offset =0. See CABR for details.
-min_qp	Minimum QP value allowed for rate control Valid values for H264 and HEVC: 0 to 51 Valid values for AV1: 0 to 255 Default auto selection value is -1. This option has no effect when -control_rate is set to Constant QP (0).
-max_qp	Maximum QP value allowed for rate control Valid values for H264 and HEVC: 0 to 51 Valid values for AV1: 0 to 255 Default auto selection value is -1.
-qp_i_offset	QP Offset for I Frames Valid values: -51 to 51 Auto selection mode is -1. Default 0.
-qp_b_offset	QP Offset for B or P Frames Valid values: -51 to 51 Auto selection mode is -1. Default 0. This option has no effect when -control_rate is set to Constant QP (0).
-forced_idr	Force insertion of IDR frames Valid values: 0, 1 (default is 1) (0) disable (1) enable - default Encodes all the intra frames as IDR
-slice	Select a slice to run an encode job Valid values if xrmd is not used are: 0, 1 (default is 0). If xrmd is active and this option is not set, xrmd will make the slice selection.
-tune_metrics	Enable tuning video quality for the best objective metrics This option optimizes the encoder for the selected tuning metric. See Tuning Video Quality for details. Valid values: 1 to 4 (default is 1) (1) vq - default (2) psnr (3) ssim (4) vmaf
-type	Applicable to AV1 encoders only Valid values: 1 or 2. i) 1 - default ii) 2 Default value is 1. Consult the AV1 Selection section for more details on how to use this option.
-preset	Encoder VQ-Density preset Valid values: fast, medium or slow. (default is medium) (fast) Allows for increase in total throughput from 4kp60 to 4kp75 , when 2 or more channels are used. Does not apply to AV1 -type 1 encoding. Not applicable under ULL setting. Note lower VQ than medium preset is to be expected. (medium) Full density with medium VQ - default (slow) High VQ at lower density. (For AV1 Type 1, improvements are observed in 10 bit contents only.) Note that presets have no impact on AV1 Type 2. Consult the Tuning Video Quality section for more details on how to use this option.
-dynamic_gop	Enable dynamic change in the number of B frames Valid values: -1, 0 or 1. (-1) auto (0) disabled (1) enable Default value auto is enabled. When this option is enabled, the number of B frames is always less or equal to default or user-specified one. See -bf.
-dynamic_params_file	Dynamic configuration file Path to dynamic file location (default is empty) Consult the Dynamic Encoder Parameters section for more details on how to use this option.
-cores	AV1 Type 1 Cores Valid values: 1 or 2 i) 1 - default, with minimum resolution of 176x144 or 144x176, and aggregate rate of 4kp60 ii) 2 - minimum resolution of 720x720, with with aggregate rate of 4kp120
-latency_ms	Look ahead depth in milliseconds Valid values: 0 to 60000 Default auto selection value, based on frame rate, is -1.
-no_bll	Disable low latency B frames Valid value: 1 Value 1 ensures that the generated GOP has a homogeneous structure. See GOP Composition.

The following table describes relevant parameters that are applicable to each RC mode

RC Mode Options

RC Mode	Allowed Options
const_qp	Constant QP -qp <QP>
cbr	Constant Bit Rate -b:v <Bit Rate> [-bufsize X] (Averaged over span of IDRs)
vbr	Variable Bit Rate -b:v <Bit Rate>
cvbr	Capped Variable Bit Rate -b:v <Bit Rate> followed by any combination of [-min_qp <QP> | -max_qp <QP> | -max_bitrate <max_bitrate>]
crf	Constant Rate Factor -crf <Value> followed by any combination of [-max_qp <QP> | -max_bitrate <max_bitrate>]
auto	Auto Mode -qp <QP> implies const_qp mode. -b:v <Bit Rate> followed by -bufsize <value> implies cbr mode. -b:v <Bit Rate> implies vbr mode. -b:v <Bit Rate> followed by any combination of [-min_qp <QP> | -max_bitrate <max_bitrate>] implies cvbr mode. -crf <Value> followed by any combination of [-max_qp <QP> | -max_bitrate <max_bitrate>] implies crf mode.

The table below describes all the options for the AMD AMA Video SDK encoder that are applicable to images only.

Image Encoding Options

Options	Descriptions
-slice	See -slice
-quality	Specify JPEG encoding quality Valid values: 1 to 100. Larger values indicate higher quality. Default 80.
-still_picture	Specify AV1image format AVIF Valid value: 1

2D Engines

This section describes different hardware accelerated 2D processing engines that are available in AMA SDK. These processing engines are leveraged in FFmpeg by using the FFmpeg filter graph syntax.

In the following table, it is assumed:

1. 0,0 coordinates refer to top-left.

2. On each device, 2D engines, other than scaler_ama, can execute on either of core 0 or 1, which is specified through core_id [0|1] option, where default is 0. Note that to reach full density, load balancing between the 2 cores must happen, manually,

3. Where applicable, output video rendition resolution can be specified by w and h parameters, which are in 0 to 7680 range. Exceptions are scaler_ama, where w and h are not applicable, and drawbox_ama, where w and h are as noted below.

4. [Video 1] ... [Video N] sequence refers to each one of the N video inputs, e.g., [0:v].

2D Accelerator Engines

Engine Name	Description	Options
scaler_ama	ABR multiscaler Takes one input and generates up to 16 output streams.	outputs specifies the number of scaler outputs. Valid values are integers between 1 and desired number of outputs. out_res specifies ABR ladder rungs <1>*<16>(WxH|Rate|Format). List of up to 16 tuples of form (Resolution|Rate|Pixel Format). Note that no spaces are allowed before or after |. Valid values for resolution: 176x144 to 3840x2160 - default 1920x1080 Valid values for frame rate: half and full - default full, where full refers to the incoming frame rate and half to the half of the incoming frame rate. Note that the first entry in this option must be full rate. Valid global values for pixel format: nv12, p010le, packed10, rgbp, yuv420p, and yuv420p10le This plugin can be combined with other plugins, without any degradation in pixel throughput.
crop_ama	Video crop Takes size, coordinates and rendition resolution.	b_left sets the crop box x min co-ordinates (from 0 to INT_MAX) (default 0) b_top sets the crop box y min co-ordinates (from 0 to INT_MAX) (default 0) b_w sets the crop box width (from 10 to INT_MAX) (default 10) b_h sets the crop box height (from 10 to INT_MAX) (default 10)
pad_ama	Video pad Takes size, coordinates, rendition resolution and pad color.	p_left sets the pad box x min co-ordinates (from 0 to INT_MAX) (default 0) p_top sets the pad box y min co-ordinates (from 0 to INT_MAX) (default 0) p_w sets the pad box width (from 10 to INT_MAX) (default 10) p_h sets the pad box height (from 10 to INT_MAX) (default 10) color sets the color of the box (default "black")
drawbox_ama	Draw box Takes size, coordinates, line thickness and color.	color sets the color of the box (default "black") thickness sets box thickness (from 2 to 16) (default 2) x sets box x min co-ordinates (from 0 to INT_MAX) (default 0) y sets box y min co-ordinates (from 0 to INT_MAX) (default 0) w sets box width (from 0 to INT_MAX) (default 10) h sets box height (from 0 to INT_MAX) (default 10)
rotate_ama	Rotate frame Takes angle, and rendition resolution.	angle one of 0, 90, 180 or 270
colorspace_ama	Color space conversion Converts between RGB and YUV.	csc i) rgb2yuv converts all supported rgb formats to yuv420p ii) yuv2rgb converts from yuv420p input to bgra format
subsample_ama	Chroma subsampling Converts from 422 to 420
overlay_ama	Video overlay Overlays 1 picture over another Inputs are specified as [Background][Overlay]overlay_ama	shortest forces termination when the shortest input terminates (default false) x overlay's x_position (from 0 to INT_MAX) (default 0) y overlay's y_position (from 0 to INT_MAX) (default 0) out_fmt sets the YUV output format [optional] (default ""). See -out_fmt. alpha sets src alpha value (from 0 to 1) (default 1)
tile_ama	Video tile Maps input streams to specified layout Inputs are specified as [Video 1]...[Video N]tile_ama, in raster order.	inputs sets number of inputs (from 1 to 32) (default 2) shortest forces termination when the shortest input terminates (default false) layout sets grid size [Max 32 inputs] (default "2x2")
compositor_ama	Video compositions Composites inputs together The compositor supports up to 64 inputs. Each input will be "blit" in a user defined z order on top of a solid background color with specified output width and height. The compositor supports dynamic changes to any of the input properties, as well as allowing inputs to be enabled/disabled. The dynamic behavior has been implemented in FFmpeg using the sendcmd option which allows passing in a file that can script the dynamic behavior. For asynchronous property changes, commands can also be send through a message queue. See Composition for usage sample.	Compositor parameters can be categorized as global modifiers, dynamic options, input transformers, and output descriptors: Global modifiers apply to all the inputs: inputs_enabled sets the inputs to enable / disable (mask) from (I64_MIN to I64_MAX) (default -1) eof_action the action to take when an input encounters EOF (from 0 to 2) (default repeat) (0) disable disables inputs that reach EOF. End once all input have reached EOF (1) repeat repeats the last frame of the input that reaches EOF (2) endall ends the output stream once the first input reaches EOF Dynamic options are only applicable when dynamic changes are required: inputs sets number of inputs (from 1 to 64) (default 1) Input transformers are n-tuples that transform their respective inputs: input_params sets input parameter initial values. Example: input_params=(res=1920x1080|ypos=1080|alpha=0.8). input sets current input to which parameters apply. in_crop input crop dimensions (such as 640x360) in_x input's x position in_y input's y position out_res destination's resolution (such as 1920x1080) out_x destination's x position out_y destination's y position alpha Alpha blend value zorder zorder of the input angle one of 0, 90, 180, 270 flip flip the input. Values: v, h (vertical, horizonal) border_color the color of the border in A8R8G8B8 format border_inner_size the inside border thickness in pixels border_outer_size the outside border thickness in pixels Output descriptor define the output frame: out_res sets the output resolution out_fmt sets the yuv output format [optional] (default "") enable_background enables / disables the background (default false) background sets the background color (default "black")

Note

For plugins that accept multiple video streams, i.e., compositor_ama, tile_ama, and overlay_ama, only a single audio track can be added the each output video stream. To select the desired audio, use the -map x:a -c:a copy syntax, where x refers to the selected audio stream number, e.g., -map 1:a -c:a copy.

Video Machine Learning

This section describes various video ML complex filters of AMA SDK that are utilized using FFmpeg filter graph syntax. Note that supported ROI models are 1080p in landscape mode, 720p in both landscape and portrait modes.

Inference Engine

The AMD AMA Video SDK ML inference engine is leveraged in FFmpeg by using the ml_ama complex filter.

ml_ama

Filter implementing the AMD AMA Video SDK ML inference engine that executes against a supplied rgbp input and detects face or text classes. Its output is used by downstream components in the pipeline.

Inference Engine Options

Options	Descriptions
model	Specify engine model Valid value roi
model_args	Model specific arguments Valid values for roi: type=face and type=text i) type=text - for text detection VQ improvements ii) type=face - for face detection VQ improvements
core_id	Specify engine number Valid values: range -1 to 3 or none i) If xrmd is enabled, this will be selected automatically ii) If xrmd is disabled, and core_id is set to -1, then one of 0 to 3 cores will be selected ii) Otherwise, value in 0-3 range will be selected
nb_inputs	Specify number of inputs Valid value: 1 Default 1
inference_period	Specify frame skip period Valid value: 1-64 Default 1

ROI Scaler

The AMD AMA Video SDK ROI scaler is leveraged in FFmpeg by using software plugin roi_scale_ama complex filter.

roi_scale_ama

Filter implementing the AMD AMA Video SDK ROI scaler engine that applies ml_ama ROI map to the input video. This engine generates metadata used by downstream encoder within the pipeline.

Scaler Options

Options	Descriptions
cutoff	Specify cutoff threshold Valid value range 0-100000 Default 5000 0-100% range is mapped to 0-100000, e.g., 5.67% is represented as 5670, and is used as shut off threshold for detected ROI regions. That is, if the percentages of detected ROI regions is above cutoff, then ROI directed encoding is disabled..
strength	Specify probability-to-QP conversion gain factor Valid values are low, normal, and high Default is normal Providing high value generates higher QP modulation that signals to encoder to spend more bits on the associated block.
nb_inputs	Specify number of inputs Valid values: range 2 to 4 Default 2
roi_map_type	Specify probability-to-QP mapping Valid values are dqp, roi, dqp_and_roi Default dqp
static_map_file	Specify static map file Valid value is the path to rgbp map file. The size of this file is ceil of width and height of incoming video divided by 16, in each dimension. This file contains values in 0-255 range, for each channel, and servers as the lower boundary of probability-to-QP mapping, i.e., maximum of values in this file and ml_ama is used for QP mapping

ROI Overlay

The AMD AMA Video SDK ROI overlay is leveraged in FFmpeg by using software plugin roi_overlay_ama complex filter.

roi_overlay_ama

Filter implementing the AMD AMA Video SDK ROI overlay that overlays shaded probability rectangles on detected classes.

Overlay Options

Options	Descriptions
skip_frames	Specify overlay period Valid value range 0-255 The value toggles overlay rectangles after the specified number of frames.
highlight_threshold	Specify activation threshold Valid value range 0-255 Default 128

See ML Based Face ROI and ML Based Text ROI for sample usage.

Other AMA Filters

Other AMA Filters

Filter Name	Description	Options
hwupload_ama	AMA device upload Takes the target device number as an argument and upload data to it.	device specifies the device number. Valid values are integers between 0 and available number of devices minus 1.
hwdownload_ama	AMA host download Takes a argument to enable or disable pipelining	pipeline enables pipelining Note that while enabling pipelining increases the throughput; however, it introduces 2 frames worth of latency.

Rebuilding FFmpeg

Obtain the zipped version of SDK's FFmpeg source code, from ma35d_sdk_v1.2.1_ffmpeg.zip. Unzip this file into ma35_ffmpeg folder. This folder contains the entire source code for the FFmpeg executable, which is a fork of the main FFmpeg GitHub (release 6.1.1) with AMD plugins patches applied. Due to licensing restrictions, the FFmpeg executable included in the Video SDK is enabled with the AMD AMA Video SDK plugins only.

You can rebuild the FFmpeg executable with optional plugins by following the instructions below. Additionally, comprehensive instructions for compiling FFmpeg can be found on the FFmpeg wiki page.

1. Make sure libhugetlbfs-dev, nasm and yasm are installed on your machine.

2. Unzip the downloaded zip file:

   unzip ma35d_sdk_v1.2.1_ffmpeg.zip
   

and navigate to root folder of the unzipped file.

1. Execute the following:

   ./configure --disable-alsa --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-libxcb-shape --disable-xlib --disable-libmfx --disable-vaapi --disable-vulkan --prefix=$PWD --disable-stripping --enable-shared --enable-vpe --enable-libxrm --extra-libs="-L/opt/amd/ama/ma35/lib -lvpi -lroi_scale -llog_ama -lxrm -lxrm_interface -lhugetlbfs -lpthread -lstdc++ -lm -ldl -lrt  /opt/amd/ama/ma35/lib/libxrm.so.1.6.0" --extra-cflags="-I/usr/include -I$PWD -I/opt/amd/ama/ma35/include/xrm_interface/ -I/opt/amd/ama/ma35/include/xrm/ -DSUPPORT_OSAL" --extra-ldflags="-Wl,-rpath=$PWD/libavutil:$PWD/libavfilter:$PWD/libavcodec:$PWD/libavformat:$PWD/libavdevice:$PWD/libswscale:$PWD/libswresample:$PWD:/opt/amd/ama/ma35/lib/"
   

2. Add any other plugins such as libx264 to the above configure line. (See FFmpeg Compilation Guide for details.) A list of configure options is printed by running configure --help.

3. Type make -j to build FFmpeg. GNU Make 3.81 or later is required.

4. To build the example folder, type make examples.

5. Type make install to install all binaries and libraries you built.