Tuning Latency¶

Latency tuning allows for trade off among end to end delay, video bit rate, GOP composition, etc. Interactive applications that require low latency, can do so by setting relevant parameters in the video pipeline. It is understood that lowering the latency comes at the cost of increased bit rate, for a given visual quality.

Decoder Latency ¶

Decoding latency can be reduced by enabling the -low_latency decoder option in FFmpeg.

Encoder Latency ¶

AMD AMA Video SDK encoder performs multi-objective optimization with set constraints on bit rate, GOP topology, visual quality measures, etc. As such, it may be tuned to achieve a compromise between latency and quality or be specialized to ultra low latency.

Guidelines on Encoder Latency Configuration¶

Encoding latency can also be reduced by trading off compression rate or visual quality. The following table lists the encoder options which can used to reduce to that effect.

Encode Options	Notes
Look Ahead Depth	For best visual quality, it is recommend to let the buffer depth be determined automatically. If it is found that the selected depth adds unacceptable delay, then this option can be set explicitly. Supported range is 0 - 46+number of B frames.
Number of B frames	It is understood that for every inserted B frame there will be a frame period delay.

See Encoding Compatibility Matrix combination of allowable parameters.

Automatic Look Ahead Depth Calculation¶

The default VQ optimized look ahead buffer depths are:

800 ms for 8 bit

600 ms for 10 bit

Ultra Low Latency (ULL) Mode¶

Ultra Low Latency (ULL) encoding is enabled by setting -lookahead_depth flag to 0.

Notes

In ULL encoding mode, frames are always processed in display order. As such, this mode is not compatible with B frames. Furthermore, only Constant Quantization Parameter (CQP) and Constant Bit Rate (CBR) options are allowed. See -control_rate.

Latency Adjustment¶

The overall latency can be further tuned by adjusting the -bufsize parameter. This parameter allows for tuning strict and relaxed ULL modes. Both relaxed ULL and strict ULL modes have the lowest achievable encoding latency, in AMD AMA SDK. Strict ULL has lower transmission latency than relaxed ULL, by restricting frame size variations at the expense of lower VQ. Relaxed ULL has better VQ than strict ULL, by allowing larger frame size variations. Such variations may results in higher transmission latency, depending on network bandwidth.

FFmpeg Latency Measurements ¶

Latency measurements can be obtained by enabling -latency_logging. The following is a sample transcode, HEVC to AVC, pipeline:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v hevc_ama -out_fmt nv12 -i <INPUT> -filter_complex "scaler_ama=outputs=1:out_res=(1920x1080)" -c:v h264_ama -frames 50 -lookahead_depth 0 -latency_logging enable -f rawvideo -b:v 6M /tmp/hevc_to_scale_to_h264.h264

...
2023-05-25 12:27:01.028958 NOTICE [Channel] Latency @ output frame  24:  0 frames, 4.5 ms; 152.79 fps = (24-0)/(157.1 ms)
2023-05-25 12:27:01.194255 NOTICE [Channel] Latency @ output frame  49:  0 frames, 4.5 ms; 151.24 fps = (49-24)/(165.3 ms)
frame=   50 fps=0.0 q=-0.0 latency= 12ms Lsize=     189kB time=00:00:00.83 bitrate=1858.8kbits/s speed=2.27x
...

Note that latency reports are printed every 25^th frame. In the case noted above, the end to end latency is 12 [ms].