FFmpeg Tutorials

This page provides tutorials on how to use FFmpeg with the AMD AMA Video SDK. The complete reference guide for the FFmpeg version included in the AMD AMA Video SDK can be found here.

The tutorials break down the commands, starting with simple decode, scale and encode pipelines. The tutorials end with different varieties of full transcode pipelines.

Device Selection

As noted in Video Codec Unit section, each card is made of 2 devices, and by default all processing tasks are delegated to the first device in the systems. (See mautil to find out how many devices are available, in a runtime environment.) To override this behavior, default and non-default devices must be set, explicitly. To do so within a FFmpeg command, the default device is set by using FFmpeg's hardware acceleration framework, i.e., ffmpeg -hwaccel ama  -hwaccel_device /dev/ama_transcoder0 ..., and can be further refined, by delegating subtasks to non-default devices by using hwupload_ama. See Multi-device Transcode for an example.

Environment Setup

  1. The AMD AMA Video SDK version of FFmpeg, Gstreamer and XMA applications can be found in the /opt/amd/ama/ma35/bin folder of your system. If this folder is not present, install the required packages:

    See On Premises

  2. Configure the environment to use the AMD AMA Video SDK. This a mandatory step for all applications:

    source /opt/amd/ama/ma35/scripts/setup.sh

The setup script exports important environment variables and ensures proper execution environment for AMD AMA Video SDK.

Sourcing the setup script should be performed each time you open a new terminal on your system. This is required for the environment to be correctly configured.


Simple FFmpeg Examples

Some of the examples read or write RAW files from disk (encode-only or decode-only pipelines). There is a chance that due to the massive bandwidth required for operating on these RAW files, you will notice a drop in FPS; this is not due to the AMD AMA Video SDK but the disk speeds. We recommend reading/writing from /dev/shm which is a RAM disk.

In the following sections, description of command line options is done in an accumulative manner, i.e., previously described options are not explained further.

Decode Only

This example accepts a clip that is already encoded in H.264, and will decode the file into a RAW format and save it to disk.

Command Line:

ffmpeg -y -hwaccel ama -c:v h264_ama -out_fmt nv12 -i <INPUT> \
 -vf hwdownload_ama,format=nv12 -f rawvideo /tmp/dec_out.nv12

Explanation of the flags:

  • ffmpeg

    • The ffmpeg application, which is provided by AMD, and moved to the top of the PATH when you sourced the setup.sh script

  • hwaccel ama

    • Instructs FFmpeg to use accelerated plugins provided by AMD AMA Video SDK

  • -out_fmt nv12

    • Specifies nv12 output format for the decoded video. Note that this option has to be specified twice: 1) To convert from the internal buffer format to nv12 in the decoder and 2) To convert when transferring to the host.

  • -f rawvideo

    • This signifies that the video is in a raw format, without container or other metadata/information about the clip

  • -c:v h264_ama

    • Declares the decoder's codec for video (as opposed to, e.g., audio -c:a ac3) is the hardware-accelerated H.264 decoder

  • -i <INPUT>

    • The input file to be transcoded

  • -vf hwdownload_ama

    • Internally, the decoder operates on AMD AMA Video SDK type buffers to improve performance. To convert back to a host-buffer, you must execute this filter.

  • -y

    • Enable overwrite without prompting the user if they're sure

  • /tmp/dec_out.yuv

    • The decoder will save the file to the path above

Encode Only

This example accepts a RAW 1080p60 clip in YUV420 format. It will pass the clip to the encoder to produce an AV1 encoded MP4 output with a target bitrate of 5Mbps and saves it to disk. The command uses the default VQ setting. See FFmpeg Video Quality for details.

Command Line:

ffmpeg -re -hwaccel ama -f rawvideo -s 1920x1080 -framerate 60 -i <INPUT> -vf "hwupload_ama" -c:v av1_ama -b:v 5M -f mp4 -y sn1_av1.mp4

Explanation of the flags:

  • -re

    • Flag to maintain the target frame rate

  • -s 1920x1080

    • Since there is no container or metadata in a RAW clip, the user must define the input clip's resolution/size. This example states the input is 1080p

  • -framerate 60

    • Again, without metadata, the encoder requires the framerate of the incoming stream

  • -pix_fmt yuv420p

    • The color space of the encoder is by default yuv420p. this example is defining the input clip as being this same color space

  • -f mp4

    • Sets the output video container to MP4

  • -b:v 5M

    • The target bitrate of the encoded stream. 8M signifies a target bitrate of 8 Megabits per second. You can also use 8000K or 8000000.

  • -c:v av1_ama

    • Declares the encoder's codec for video (as opposed to audio -c:a ...) is the hardware-accelerated AV1 encoder

  • /tmp/enc_out.mp4

    • Save the output in the above path.

4:2:2 10 Bit Conversion

To encode YUV, 4:2:2, 10 bit pixel format to YUV, 4:2:0, 8 bit use the following command:

Command Line:

ffmpeg -hwaccel ama -i <INPUT>  -vf "format=yuv420p,hwupload_ama" -c:v h264_ama -b:v 8M <OUTPUT>

Explanation of the flags:

  • -vf "format=yuv420p,hwupload_ama"

    • Instructs the pipeline to upload and convert the input video as yuv420p.

Constant Rate Factor (CRF) Mode

The following examples demonstrate the usage of the -crf flag and impact of its value on the quality of the encoded streams.

High Quality Encoding

Command Line:

ffmpeg -re -hwaccel ama -f rawvideo -s 1920x1080 -framerate 60 -i <INPUT> -vf "hwupload_ama" -c:v av1_ama -crf 0  -f mp4 sn1_crf_hq.mp4

Explanation of the flags:

  • -crf 0

    • Enables the highest quality -crf mode

Low Quality Encoding

Command Line:

ffmpeg -re -hwaccel ama -f rawvideo -s 1920x1080 -framerate 60 -i <INPUT> -vf "hwupload_ama" -c:v av1_ama -crf 63 -f mp4 sn1_crf_lq.mp4

Explanation of the flags:

  • -crf 63

    • Sets the encoded AV1 stream to lowest CRF quality

Insert IDR

This example shows how to insert IDR frames at preassigned frame positions. See -force_key_frames for details:

ffmpeg -hide_banner -loglevel info -hwaccel ama -hwaccel_device /dev/ama_transcoder0  -re -f lavfi -i testsrc=duration=10:size=1920x1080:rate=60,format=yuv420p \
-f rawvideo  -vf "hwupload_ama"  -c:v h264_ama  -g 30 -forced_idr 0  -force_key_frames 'expr:gte(t,n_forced*2)' -b:v 8M -f mp4 -y idr.mp4

Explanation of the flags:

  • forced_idr 0

    • Disables setting all I frames as IDR

  • -force_key_frames 'expr:gte(t,n_forced*2)'

    • Inserts IDR frames every 2 seconds

To confirm IDR insertion, observe the output of the following command:

ffprobe -hide_banner -loglevel error -select_streams v:0 -show_entries frame=best_effort_timestamp_time,pict_type,key_frame -select_streams v -of csv=p=0 -unit  idr.mp4 | grep I
1,0.000000 s,I
0,0.533333 s,I
0,1.000000 s,I
0,1.533333 s,I
1,2.000000 s,I
0,2.533333 s,I
0,3.000000 s,I
0,3.533333 s,I
1,4.000000 s,I
...

Note that every 2 seconds key_frame flag is set to 1, indicating that the corresponding I frame is IDR.

Basic Transcode

This example takes an H.264 clip and transcodes it to HEVC at the bitrate of 8Mbps. The output is written into /tmp/h264_to_hevc.

Command Line:

ffmpeg -y -hwaccel ama -c:v h264_ama -i <INPUT> \
 -c:v hevc_ama -b:v 6M  -f rawvideo  /tmp/h264_to_hevc.hevc

Decode Only Into Multiple-Resolution Outputs

This example decodes an existing H.264 file and then scales it into multiple resolutions as defined below. It will not re-encode them, but save the RAW outputs to disk under /tmp/_scale<resolution>.yuv

Command Line:

 ffmpeg -y -hwaccel ama \
-c:v h264_ama  -out_fmt nv12 -i <INPUT>  \
-filter_complex "scaler_ama=outputs=4:out_res=(1920x1080|full|nv12)(1280x720|full|nv12)(720x480|full|nv12)(360x240|full|nv12) [a][b][c][d]; \
                 [a]hwdownload_ama,format=nv12[a1];[b]hwdownload_ama,format=nv12[b1];[c]hwdownload_ama,format=nv12[c1];[d]hwdownload_ama,format=nv12[d1]" \
-map '[a1]' -f rawvideo -pix_fmt nv12 -y /tmp/scale_1080p.yuv \
-map '[b1]' -f rawvideo -pix_fmt nv12 -y /tmp/scale_720p.yuv  \
-map '[c1]' -f rawvideo -pix_fmt nv12 -y /tmp/scale_480p.yuv \
-map '[d1]' -f rawvideo -pix_fmt nv12 -y /tmp/scale_240p.yuv

Explanation of the flags:

  • -filter_complex

    • The FFmpeg -filter_complex flag allows combining multiple filters together using a graph-like syntax. This example uses the scaler_ama to create 4 output resolutions from the input stream.

    • The scaler_ama filter configures the hardware-accelerated scaler to produce 4 output resolutions (1920x1080, 1280x720, 720x480, and 360x240). For each output, the width, height, frame rate and pixel format are defined as arguments to out_res option as in (WidthxHeight|Rate|Format). The 4 outputs are transfered to the host as a1, b1, c1 and d1 respectively.

  • -map "[ID]"

    • Selects an output of the filter graph. The flags that follow apply to the selected stream.

  • /tmp/scale_<resolution>.yuv

    • Save the output files to the paths listed

Encode Only Into Multiple-Resolution Outputs

This example takes a raw 1080p60 YUV file, scales it down to different resolutions and frame rates, encodes each of the scaled streams to different formats and saves them to disk under /tmp/<encode format>_<resolution>p<frame rate>.mp4

Command Line:

ffmpeg -y -hwaccel ama -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -framerate 60 -i <INPUT> \
-filter_complex "hwupload_ama,scaler_ama=outputs=4:out_res=(1920x1080|full)(1280x720|half)(720x480|half)(360x240|half) [a][b][c][d]" \
-map '[a]' -c:v hevc_ama -b:v 6M -f mp4 -y /tmp/hevc_1080p60.mp4 \
-map '[b]' -c:v av1_ama  -b:v 2M -f mp4 -y /tmp/av1_720p30.mp4 \
-map '[c]' -c:v h264_ama -b:v 1M -f mp4 -y /tmp/h264_480p30.mp4 \
-map '[d]' -c:v av1_ama  -b:v 1M -f mp4 -y /tmp/av1_240p30.mp4

Explanation of the flags:

  • -filter_complex "hwupload_ama...

    • Specifies that video clip gets upload to the device.

  • -filter_complex "...scaler_ama=outputs=4:out_res=...(1280x720|half)...

    • Declares scaler output of given resolution and rate. See scaler_ama for more details.

  • -c:v [hevc_ama | av1_ama | h264_ama]

    • Declares various encode types HEVC, AV1 and H.264, respectively.

  • /tmp/encode-format_resoultion.encode-format

    • Saves the output clips to the location listed

Transcode with Multiple-Resolution Outputs

This example implements a complete transcoding pipeline on an 1080p60 H.264 input. It decodes the input stream, scales it to 4 different resolutions and encodes scaler's outputs into various formats and saves them to disk under /tmp/<encoder format>_<resolution>.mp4

Command Line:

ffmpeg -y -hwaccel ama \
-c:v h264_ama -i <INPUT> \
-filter_complex "scaler_ama=outputs=4:out_res=(1920x1080)(1280x720)(720x480)(360x240) [a][b][c][d]" \
-map '[a]' -c:v hevc_ama -b:v 6M -f mp4 -y /tmp/hevc_1080p.mp4 \
-map '[b]' -c:v av1_ama  -b:v 2M -f mp4 -y /tmp/av1_720p.mp4 \
-map '[c]' -c:v h264_ama -b:v 1M -f mp4 -y /tmp/h264_480p.mp4 \
-map '[d]' -c:v av1_ama  -b:v 1M -f mp4 -y /tmp/av1_240p.mp4

Transcode in ULL Mode

This example implements an ultra low latency transcode pipeline. For more details refer Tuning Transcode Latency.

Command Line:

ffmpeg  -hide_banner -loglevel error -y -hwaccel ama -low_latency 1 -c:v h264_ama -i <INPUT> \
-c:v hevc_ama -b:v 6M -lookahead_depth 0 -f rawvideo /tmp/h264_to_hevc.hevc

Explanation of the flags:

  • -low_latency 1

    • Enables low latency decoding mode.

  • -lookahead_depth 0

    • LA size of 0 triggers ultra low latency encoding mode.

8kp30 Transcode

This example shows how to construct an AV1 Type-1 8kp30 transcode pipeline.

Command Line:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i <INPUT> -c:v -b:v 25M av1_ama -cores 2 -y -f mp4 <OUTPUT>

Explanation of the flags:

  • -cores 2

    • Uses both cores for AV1 Type-1 encoding

Multi-slice Transcode

This example shows how to transcode a 8kp30 input into a 4kp30 AVC and a 4kp30 HEVC.

Command Line:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -resize 3840x2160 -i <INPUT> -filter_complex "split=2[a][b]" \
-map "[a]" -c:v h264_ama -b:v 32M -slice 0 -f mp4 <OUTPUT 1> \
-map "[b]" -c:v hevc_ama -b:v 24M -slice 1 -f mp4 <OUTOUT 2>

Explanation of the flags:

  • -resize 3840x2160

    • Uses decoder's downscaling capability

  • -slice 0 and 1

    • Explicitly targets both slices, for 2x4kp60 encoding. Note that explicit slice assignment is only required if xrmd is not enabled.

Multi-device Transcode

This example demonstrates hwdownload_ama ability to execute encoding or 2D processing tasks, on an arbitrary device.

Command Line:

ffmpeg  -hwaccel ama -hwaccel_device /dev/ama_transcoderX -c:v h264_ama -out_fmt yuv420p -i <INPUT> -filter_complex "hwdownload_ama,hwupload_ama=device=Y" -c:v h264_ama -b:v 8M -f mp4 <OUTPUT>

Explanation of the flags:

  • hwdownload_ama

    • Downloads the decoded stream from device X to the host

  • hwupload_ama=device=Y

    • Uploads the decoded stream from the host to device Y for encoding

Double Density Example with xrmd

This example demonstrates an example on how to achieve 4x4kp60 density on a single device

Command Line:

ffmpeg -hide_banner -hwaccel ama -c:v h264_ama -i <4Kp6 H.264 file> \
 -filter_complex "split=4[a][b][c][d]" \
 -map "[a]" -c:v av1_ama -b:v 16M -f rawvideo -y /tmp/out_3840x2160_0.av1 \
 -map "[b]" -c:v hevc_ama -b:v 24M -f rawvideo -y /tmp/out_3840x2160_0.hevc \
 -map "[c]" -c:v av1_ama -b:v 16M -f rawvideo -y /tmp/out_3840x2160_1.av1 \
 -map "[d]" -c:v hevc_ama -b:v 24M  -f rawvideo -y /tmp/out_3840x2160_1.hevc

Explanation of the flags:

  • -split

    • Splits input into several identical outputs.

Double Density Example without xrmd

This example demonstrates how to achieve 4x4kp60 density on a single device, by explicitly allocating tasks on slices 0 and 1. Note in this example xrmd service has to be stopped.

Command Line:

ffmpeg -hide_banner -hwaccel ama -c:v h264_ama -i <4Kp6 H.264 file> \
 -filter_complex "split=4[a][b][c][d]" \
 -map "[a]" -c:v av1_ama -b:v 16M  -slice 0 -f rawvideo -y /tmp/out_3840x2160_0.av1 \
 -map "[b]" -c:v hevc_ama -b:v 24M -slice 0 -f rawvideo -y /tmp/out_3840x2160_0.hevc \
 -map "[c]" -c:v av1_ama -b:v 16M -slice 1 -f rawvideo -y /tmp/out_3840x2160_1.av1 \
 -map "[d]" -c:v hevc_ama -b:v 24M -slice 1 -f rawvideo -y /tmp/out_3840x2160_1.hevc

Full Double Density

This example demonstrates how to achieve full density, by distributing transcode pipelines, between 2 devices.

Command Line:

for i in `seq 1 16`; do
 ffmpeg -y -nostdin -hwaccel ama -hwaccel_device /dev/ama_transcoderX -re -c:v h264_ama -i <30 FPS INPUT> -c:v <TYPE 2 ENCODER> -b:v 5M -lookahead_depth 10 -f null /dev/null > out_T2_0_$i.log 2>&1 &
 ffmpeg -y -nostdin -hwaccel ama -hwaccel_device /dev/ama_transcoderX -re -c:v h264_ama -i <30 FPS INPUT> -c:v av1_ama -b:v 5M -lookahead_depth 10 -f null /dev/null > out_av1_0_$i.log 2>&1 &
 ffmpeg -y -nostdin -hwaccel ama -hwaccel_device /dev/ama_transcoderY -re -c:v h264_ama -i <30 FPS INPUT> -c:v <TYPE 2 ENCODER> -b:v 5M -lookahead_depth 10 -f null /dev/null > out_T2_1_$i.log 2>&1 &
 ffmpeg -y -nostdin -hwaccel ama -hwaccel_device /dev/ama_transcoderY -re -c:v h264_ama -i <30 FPS INPUT> -c:v av1_ama -b:v 5M -lookahead_depth 10 -f null /dev/null > out_av1_1_$i.log 2>&1 &
done

, where X and Y represent 2 different devices. <30 FPS INPUT> is a 30 FPS encoded video input. (The above example assumes H264 encoded input.), and <TYPE 2 ENCODER> is any one of AV1 Type-2, H264 or HEVC encoder. Note that by default av1_ama uses Type-1 AV1 encoder.

The above example transcodes 64 simultaneous 1080p30 pipelines. Note that if XRMD is disabled, then -slice parameter must be set explicitly.

JPEG Image Decoding

The following example demonstrates how to decode a series of JPEG images into a AVC video.

Command Line:

ffmpeg -framerate 60 -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v jpeg_ama -i <FILE PRFIX>_%03d.jpg  \
-r 60 -c:v h264_ama -b:v 8M -f mp4 <OUTPUT>

, where <FILE PRFIX>_%03d.jpg indicates the name pattern of the JPEG images.

JPEG Image Encoding

The following example demonstrates how to encode a video stream into a series of JPEG images.

Command Line:

ffmpeg -y -hwaccel ama -s:v 1920x1080 -pix_fmt yuv420p -i <YUV INPUT> -vf "fps=1,hwupload_ama" -c:v jpeg_ama -quality 90 <FILE PRFIX>_%04d.jpg

, where -quality specifies JPEG encoding quality.

AV1 Image Encoding

The following example demonstrates how to encode a video stream into a series of AVIF images.

Command Line:

ffmpeg -hide_banner -loglevel info -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -f lavfi -i testsrc=duration=10:size=1920x1080:rate=1,format=yuv420p  -vf "hwupload_ama" -c:v av1_ama  -still_picture 1 test_avif_%03d.avif

, where -still_picture specifies AV1 image encoding.

2D Processing

This section provides examples on usage of various 2D Processing utilities. See 2D Engine for details.

Cropping

The following example demonstrates a 720P crop at offset (320, 180).

Command Line:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -re -out_fmt yuv420p -c:v h264_ama -i <INPUT> \
-filter_complex "crop_ama=b_left=320:b_top=180:b_w=1280:b_h=720 [b]" -map "[b]" -c:v h264_ama -b:v 4M -f mp4 <OUTPUT>

Padding

The following example demonstrates padding a 720P clip into a 1080p.

Command Line:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i <720P INPUT> \
-filter_complex "pad_ama=p_left=320:p_top=180:p_w=1920:p_h=1080:color=pink" -c:v h264_ama -b:v 8M -f mp4 <1080P OUTPUT>

Video Rotation

The following example demonstrates how to transcode a flip-over operation

Command Line:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-c:v h264_ama -i <INPUT> -filter_complex "rotate_ama=angle=180[c]" -map "[c]" -c:v hevc_ama -b:v 6M -f mp4 <OUTPUT>

, where INPUT and OUTPUT are any one of supported encoded files. Note that only rotations which are multiples of 90 degrees are supported.

Color Space Conversion

The following example demonstrates how to convert from RGB to YUV:

Command Line:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -s 1920x1080 -pix_fmt bgra -i <INPUT> -filter_complex "[0:v]hwupload_ama,colorspace_ama=csc=rgb2yuv,hwdownload_ama" -pix_fmt yuv420p -f rawvideo <OUTPUT>

The following example demonstrates how to encode a RGBA video file :

Command Line:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -f rawvideo -s 1920x1080 -pix_fmt bgra -i <INPUT> -filter_complex "[0:v]hwupload_ama,colorspace_ama=csc=rgb2yuv" -c:v hevc_ama -b:v 6M -f mp4 <OUTPUT>

Similarly, the following example demonstrates YUV to planar RGB conversion:

Command Line:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i <INPUT> -filter_complex "colorspace_ama=csc=yuv2rgb,hwdownload_ama" -pix_fmt bgra -f rawvideo -y <RGBP OUTPUT>

Chroma Subsampling

The following example demonstrates how to convert from 4:2:2 pixel format to 4:2:0:

Command Line:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -s 1920x1080 \
-pix_fmt yuv422p -i <INPUT> -filter_complex "hwupload_ama,subsample_ama,hwdownload_ama" -pix_fmt yuv420p -f rawvideo <OUTPUT>

, where graphic processor is set for pixel subsampling operation.

The following example demonstrates how to transcode a 4:2:2 pixel format video to 4:2:0 using both the host, for decoding, and an AMA AMD compatible card, for subsampling and encoding:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -i <INPUT> -filter_complex "hwupload_ama,subsample_ama" -c:v h264_ama -b:v 8M -f mp4 <OUTPUT>

, where graphic processor is set for pixel subsampling operation.

Picture In Picture (PIP)

The following example demonstrates how to create a PIP video:

Command Line:

ffmpeg -y -hide_banner  -hwaccel ama -hwaccel_device /dev/ama_transcoder0  \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 1> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 2> \
-filter_complex "[1:v]scaler_ama=outputs=1:out_res=(720x480|yuv420p)[p]; \
                 [0:v][p]overlay_ama=x=0:y=0[x]" \
-map "[x]" -c:v h264_ama -frames 1000 -b:v 8M -f mp4 -y <OUTPUT>

, where graphic processor is set for scaling and overlay operations, with INPUT 1 set as the background.

Tiling

The following example demonstrates how to create a tiling video:

Command Line:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 1> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 2> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 3> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 4> \
-filter_complex "[0:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[0]; \
                 [1:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[1]; \
                 [2:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[2]; \
                 [3:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[3]; \
                 [0][1][2][3]tile_ama=layout=2x2:inputs=4:w=1920:h=1080[r]"
-map "[r]" -c:v h264_ama -b:v 15M -f mp4 -y <OUTPUT>

, where graphic processor's is set for scaling and tiling operations, with INPUT 1 ... INPUT 4 set as tiles.

Thumbnailing

The following example shows how to generate JPEG thumnails from a video asset.

Command Line:

ffmpeg -hwaccel ama -c:v h264_ama -i <INPUT> -vf "select='eq(pict_type\,I)', scaler_ama=outputs=1:out_res=(144x144|full)" -vsync vfr -c:v jpeg_ama out_%03d.jpg

JPEG thumbnails are named from out_000.jpg through out_999.jpg.

Alternatively, down scaling can be done through the decoder.

Command Line:

ffmpeg -hwaccel ama -c:v h264_ama -resize 144x144 -i <INPUT> -vf "select='eq(pict_type\,I)'" -vsync vfr -c:v jpeg_ama out_%03d.jpg

Composition

The following example demonstrates how to create a static composition.

Command Line:

/opt/amd/ama/ma35/bin/ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 1> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 2> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 3> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 4> \
-filter_complex "[0:v][1:v][2:v][3:v]\
compositor_ama=inputs=4:out_res=1920x1080:enable_background=1:background=black:\
input_params=\
(960x540|angle=90|out_x=8|out_y=8|zorder=1)\
(960x540|angle=90|out_x=408|out_y=208)\
(960x540|angle=90|out_x=808|out_y=408|zorder=1)\
(960x540|angle=90|out_x=1208|out_y=608|border_inner_size=2)[co];\
[co]sendcmd=f=composite_static.txt[c]" \
-map "[c]" -c:v av1_ama -b:v 5M -vframes 570 -f mp4 composite_test.mp4

, where composite_static.txt contains:

00:00:01.000 compositor_ama 'background=pink';
00:00:03.000 compositor_ama 'background=green:input=1:border_inner_size=4:border_outer_size=2:border_color=blue';
00:00:06.000 compositor_ama 'input=2:border_inner_size=4:border_outer_size=2:border_color=brown:input=0:border_inner_size=10:border_outer_size=10';
00:00:08.000 compositor_ama 'input=3:flip=h:zorder=2:out_res=1280x540:in_y=1082';
00:00:10.000 compositor_ama 'input=0=flip=v';
00:00:12.000 compositor_ama 'angle=90';
00:00:15.000 compositor_ama 'inputs_enabled=2';
00:00:16.000 compositor_ama 'inputs_enabled=3';
00:00:17.000 compositor_ama 'zorder=0'

This example starts by overlapping 3 scaled and rotated input clips, with a black background. The static composition file, then directs the pipeline to change the background color, z order, boarder line,etc. at prescribed intervals.

The compositor can also be executed in a dynamic manner, where different properties can be adjusted interactively, through ZeroMQ protocol.

Command Line:

/opt/amd/ama/ma35/bin/ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 1> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 2> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 3> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 4> \
-filter_complex "[0:v][1:v][2:v][3:v]\
compositor_ama=inputs=4:out_res=1920x1080:enable_background=1:background=black:\
input_params=\
(960x540|angle=90|out_x=8|out_y=8|zorder=1)\
(960x540|angle=90|out_x=408|out_y=208)\
(960x540|angle=90|out_x=808|out_y=408|zorder=1)\
(960x540|angle=90|out_x=1208|out_y=608|border_inner_size=2)[co]; \
[co]zmq=bind_address=tcp\\\://127.0.0.1\\\:5555[c]" \
-map "[c]" -c:v h264_ama -b:v 5M  -vframes 570 -f mpegts udp://MC_IP:MC_PORT??pkt_size=1316

To interactively change the compositor's properties, create the following Python script:

#!/usr/bin/python3
import zmq
import sys

context = zmq.Context()
Url=sys.argv[1]
socket = context.socket(zmq.REQ)
socket.connect(Url)
cmd = sys.stdin.read()
socket.send_string(cmd)
recv = socket.recv()
print(f"Received reply [{recv.decode()}]")

, and update properties as follows, e.g. the following changes the background to pink:

printf "compositor_ama background=pink" |  ./zmq_send.py "tcp://localhost:5555"

To observe the interactive updates, on a machine within the same multicast network, issue the following command:

ffplay udp://MC_IP:MC_PORT

, where MC_IP and MC_PORT are multicast address and multicast ports, respectively.

ML Processing

This section provides examples on usage of ROI for both face and text VQ enhancement.

ML Based Face ROI

The following example demonstrates how to perform ROI face encoding on an AMA AMD compatible card:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i <INPUT> \
-filter_complex "scaler_ama=outputs=3:out_res=(1280x720|yuv420p)(1280x720|yuv420p)(1280x720|rgbp)[def][ori][a]; \
                 [a]ml_ama=model=roi:model_args=type=face[mlop];[ori][mlop]roi_scale_ama[res1]" \
-map [res1] -c:v hevc_ama -b:v 300K -f mp4 <OUTPUT 1> \
-map [def] -c:v hevc_ama -b:v 300K -f mp4 <OUTPUT 2>

, where ML engine is set for face ROI enhancement, and the resulting enhanced image, res1, is encoded.

ROI Gain Factor

The following example demonstrates the impact of probability-to-QP mapping gain factor:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i  <INPUT> \
   -filter_complex "split=outputs=5 [plain][ml0][l][n][h]; \
                             [ml0] scaler_ama=outputs=1:out_res=(1280x720|rgbp) [ml1]; \
                             [ml1] ml_ama=model=roi:model_args=type=face [ml2]; \
                             [ml2] split=outputs=3 [ml3_1][ml3_2][ml3_3]; \
                             [l][ml3_1] roi_scale_ama=strength=low [ml_l]; \
                             [n][ml3_2] roi_scale_ama=strength=normal [ml_n]; \
                             [h][ml3_3] roi_scale_ama=strength=high [ml_h]" \
     -map '[plain]' -c:v h264_ama -b:v 300k -f mp4 <OUTPUT1> \
     -map '[ml_l]' -c:v h264_ama -b:v 300k -f mp4 <OUTPUT2> \
     -map '[ml_n]' -c:v h264_ama -b:v 300k -f mp4 <OUTPUT3> \
     -map '[ml_h]' -c:v h264_ama -b:v 300k -f mp4 <OUTPUT4>

Here <OUTPUTx>, for x={1-4}, represent encode streams without ML, low, normal and high gain factors, respectively.

ROI Ladder Transcoding

The following example demonstrates how to perform ROI transcoding at different resolutions:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i <INPUT> \
 -filter_complex "split=2[s_org][s_ml]; \
                  [s_org]scaler_ama=outputs=4:out_res=(1920x1080)(1280x720)(720x480)(360x240) [sa][sb][sc][sd]; \
                  [s_ml]scaler_ama=outputs=1:(1280x720|rgbp)[ml_in]; \
                  [ml_in]ml_ama=model=roi:model_args=type=face[ml_out]; [ml_out]split=4[ml_out_sa][ml_out_sb][ml_out_sc][ml_out_sd]; \
                  [sa][ml_out_sa]roi_scale_ama[roi_a]; \
                  [sb][ml_out_sb]roi_scale_ama[roi_b]; \
                  [sc][ml_out_sc]roi_scale_ama[roi_c]; \
                  [sd][ml_out_sd]roi_scale_ama[roi_d]" \
-map [roi_a] -c:v av1_ama -b:v 300K -f rawvideo <OUTPUT1> \
-map [roi_b] -c:v av1_ama -b:v 300K -f rawvideo <OUTPUT2> \
-map [roi_c] -c:v av1_ama -b:v 300K -f rawvideo <OUTPUT3> \
-map [roi_d] -c:v av1_ama -b:v 300K -f rawvideo <OUTPUT4>

Note that scale engine is invoke twice. Once for scaling the outputs and second time to perform pixel conversion.

ROI Face Detection

The following example demonstrates inference engine's ability to detect faces:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama  -i <INPUT> \
-filter_complex "scaler_ama=outputs=2:out_res=(1280x720|yuv420p)(1280x720|rgbp)[ori][mlip]; \
                 [mlip]ml_ama=model=roi:model_args=type=face[mlop]; \
                 [ori][mlop]roi_scale_ama=roi_map_type=roi[rso]; \
                 [rso]hwdownload_ama, roi_overlay_ama[res1]" \
-map [res1] -f rawvideo -pix_fmt yuv420p -f rawvideo <OUTPUT>

, where software plugin roi_overlay_ama overlays transparent rectangular boxes around all detected faces.

ML Based Text ROI

The following example demonstrates how to perform ROI text encoding on an AMA AMD compatible card:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama  -i <INPUT> \
-filter_complex "scaler_ama=outputs=3:out_res=(1280x720|yuv420p)(1280x720|yuv420p)(1280x720|rgbp)[def][ori][a]; \
                 [a]ml_ama=model=roi:model_args=type=text[mlop];[ori][mlop]roi_scale_ama[res1]" \
-map [res1] -c:v hevc_ama -b:v 200K -f mp4  <OUTPUT 1> \
-map [def] -c:v hevc_ama -b:v 200K -f mp4  <OUTPUT 2>

, where ML engine is set for text ROI enhancement, and the resulting enhanced image, res1, is encoded.