New Feature Preview¶

Overview ¶

This section describes new enhanced features, which are meant for demonstrative proposes and are only supported in FFmpeg. Note that although these features are not meant to be used in production environments; however, as they mature, they will become fully qualified as production ready. The following table delineates the list of available options:

Video Processing Accelerators¶
Accelerator Core	Functions/Supported Models	Notes
2D Processor	Video Rotation, Color Space Conversion, Chroma Subsampling, Picture In Picture and Tiling.	Picture In Picture and Tiling require the same frame rate on all inputs to the `2d_ama` plugin. To cascade multiple 2D processor cores, pixel format `yuv420p` must be set, explicitly.
Machine Learning	ML Based Face ROI and ML Based Text ROI models	Supported ROI models are 1080p in landscape mode, 720p in both landscape and portrait modes. Maximum supported instances of ml_ama plugin (ROI model) per device is limited to 4.

In the following sections, description of command line options is done in an accumulative manner, i.e., previously described options are not explained further.

Video Rotation ¶

The following example demonstrates how to transcode a flip-over operation on an AMA AMD compatible card:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-c:v h264_ama -out_fmt yuv420p  -i <INPUT> \
-filter_complex "[0:v]2d_ama=inputs=1:processor=rotate:rotation=180[c]" \
-map "[c]" -c:v hevc_ama -f mp4 <OUTPUT>

, where 2d_ama is graphic processor's plugin, set for rotation operation; INPUT and OUTPUT are any one of supported encoded files. Note that only rotations which are multiples of 90 degrees are supported.

Color Space Conversion ¶

The following example demonstrates how to convert from RGB to YUV on an AMA AMD compatible card:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -s 1920x1080 \
-pix_fmt bgra -i <INPUT> \
-filter_complex "[0:v]hwupload[a];[a]2d_ama=inputs=1:processor=csc:csc=rgb2yuv[c];[c]hwdownload" \
-pix_fmt yuv420p -f rawvideo <OUTPUT>

, where graphic processor is set for color conversion operation.

The following example demonstrates how to encode a RGBA video file on an AMA AMD compatible card:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -f rawvideo -s 1920x1080 \
-pix_fmt bgra -i <INPUT> \
-filter_complex "[0:v]hwupload[a];[a]2d_ama=inputs=1:processor=csc:csc=rgb2yuv[c]" \
-map "[c]" -c:v hevc_ama -f mp4 <OUTPUT>

Similarly, the following example demonstrates YUV to planar RGB conversion:

ffmpeg -y  -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-c:v h264_ama -i <INPUT> \
-filter_complex "scaler_ama=outputs=1:out_res=(1920x1080|rgbp)[a];[a]hwdownload,format=rgbp[a1]" \
-map '[a1]' -f rawvideo -y <RGBP OUTPUT>

Chroma Subsampling ¶

The following example demonstrates how to convert from 4:2:2 pixel format to 4:2:0 on an AMA AMD compatible card:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -s 1920x1080 \
-pix_fmt yuv422p -i <INPUT> \
-filter_complex "[0:v]hwupload[a];[a]2d_ama=inputs=1:processor=subsample[c];[c]hwdownload" \
-pix_fmt yuv420p -f rawvideo <OUTPUT>

, where graphic processor is set for pixel subsampling operation.

The following example demonstrates how to transcode a 4:2:2 pixel format video to 4:2:0 using both the host, for decoding, and an AMA AMD compatible card, for subsampling and encoding:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-i <INPUT> \
-filter_complex "[0:v]hwupload[a];[a]2d_ama=inputs=1:processor=subsample[c]" \
-map [c] -c:v h264_ama -b:v 5M -f mp4 <OUTPUT>

, where graphic processor is set for pixel subsampling operation.

Picture In Picture (PIP)¶

The following example demonstrates how to create a PIP video on an AMA AMD compatible card:

ffmpeg -y -hide_banner  -hwaccel ama -hwaccel_device /dev/ama_transcoder0  \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 1> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 2> \
-filter_complex "[1:v]scaler_ama=outputs=1:out_res=(720x480|yuv420p)[p];[0:v][p]2d_ama=inputs=2:processor=overlay:core_id=0:x=0:y=0[x]" \
-map "[x]" -c:v h264_ama -frames 1000 -b:v 15M -f mp4 -y <OUTPUT>

, where graphic processor is set for scaling and overlay operations, with INPUT 1 set as the background.

Tiling ¶

The following example demonstrates how to create a tiling video on an AMA AMD compatible card:

ffmpeg -y -hide_banner -hwaccel ama -hwaccel_device /dev/ama_transcoder0 \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 1> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 2> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 3> \
-c:v h264_ama -out_fmt yuv420p -i <INPUT 4> \
-filter_complex "[0:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[0];[1:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[1];[2:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[2];[3:v]scaler_ama=outputs=1:out_res=(1280x720|yuv420p)[3];[0][1][2][3]2d_ama=layout=2x2:inputs=4:processor=tile[c];  [c]scaler_ama=outputs=1:out_res=(1920x1080|yuv420p)[r]" \
-map "[r]" -c:v h264_ama -b:v 15M -f mp4 -y <OUTPUT>

, where graphic processor's is set for scaling and tiling operations, with INPUT 1 ... INPUT 4 set as tiles.

ML Region Of Interest (ROI)¶

The following sub-sections describe the current capabilities of the ML based ROI engine. This engine is capable of processing 4 independent 720p30 video streams in parallel for face and text detections.

ML Based Face ROI ¶

The following example demonstrates how to perform ROI face encoding on an AMA AMD compatible card:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama -i <INPUT> \
-filter_complex "scaler_ama=outputs=3:out_res=(1280x720|yuv420p)(1280x720|yuv420p)(1280x720|rgbp)[def][ori][a];[a]ml_ama=model=roi:model_args=type=face[mlop];[ori][mlop]roi_scale_ama[res1]" \
-map [res1] -c:v hevc_ama -b:v 300K -f mp4 <OUTPUT 1> \
-map [def] -c:v hevc_ama -b:v 300K -f mp4 <OUTPUT 2>

, where ML engine is set for face ROI enhancement, and the resulting enhanced image, res1, is encoded.

ROI Face Detection ¶

The following example demonstrates inference engine's ability to detect faces:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama  -i <INPUT> \
-filter_complex "scaler_ama=outputs=2:out_res=(1280x720|yuv420p)(1280x720|rgbp)[ori][mlip];[mlip]ml_ama=model=roi:model_args=type=face[mlop]; [ori][mlop]roi_scale_ama=roi_map_type=roi[rso]; [rso]hwdownload, roi_overlay_ama[res1]" \
-map [res1] -f rawvideo -pix_fmt yuv420p -f rawvideo <OUTPUT>

, where software plugin roi_overlay_ama overlays transparent rectangular boxes around all detected faces.

ML Based Text ROI ¶

The following example demonstrates how to perform ROI text encoding on an AMA AMD compatible card:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v h264_ama  -i <INPUT> \
-filter_complex "scaler_ama=outputs=3:out_res=(1280x720|yuv420p)(1280x720|yuv420p)(1280x720|rgbp)[def][ori][a];[a]ml_ama=model=roi:model_args=type=text[mlop];[ori][mlop]roi_scale_ama[res1]" \
-map [res1] -c:v hevc_ama -b:v 200K -f mp4  <OUTPUT 1> \
-map [def] -c:v hevc_ama -b:v 200K -f mp4  <OUTPUT 2>

, where ML engine is set for text ROI enhancement, and the resulting enhanced image, res1, is encoded.