New Feature Preview

Overview

This section describes new enhanced features, which are meant for demonstrative proposes and are only supported in FFmpeg. Note that although these features are not meant to be used in production environments; however, as they mature, they will become fully qualified as production ready.

Machine Learning Content Aware Encoding (MLCAE)

MLCAE is a novel rate control mode designed to optimize frames' encoding based on their complexity. By leveraging the statistics generated by the lookahead module, the ML algorithm updates the encoder's configuration and achieves significant bitrate savings without compromising perceived video quality.

The current implementation of MLCAE enhances the CBR mode by dynamically adjusting the bitrate according to the frame's predicted complexity. This feature optimizes the encoder's performance, leading to more efficient video encoding. The MLCAE algorithm monitors the lookahead module's stats and dynamically adjusts the encoder's parameters based on the predicted frame complexity. These adjustments ensure that the encoder config is optimally set in real-time, resulting in consistent video quality and bitrate savings throughout the entire video stream.

The following example demonstrates how MLCAE gets invoked:

ffmpeg -hide_banner -loglevel info -hwaccel ama -hwaccel_device /dev/ama_transcoder0  -re -c:v h264_ama -i <INPUT>  -c:v h264_ama -control_rate mlcae -b:v 5M -f mp4 -y <OUTPUT>

Note

For the best results, when using mlcae mode, encode bit rate should be above 3 Mbps. Also note that for this release, only h264_ama encoder, at 1080p portrait or landscape orientation, is supported.