Tuning Latency

Latency tuning allows for trade off among end to end delay, video bit rate, GOP composition, etc. Interactive applications that require low latency, can do so by setting relevant parameters in the video pipeline. It is understood that lowering the latency comes at the cost of increased bit rate, for a given visual quality.

Decoder Latency

Decoding latency can be reduced by enabling the -low_latency decoder option in FFmpeg. The low latency decoding mode is not compatible with B frames. If the input streams contains B-frames, the -low_latency option should not be used.

Encoder Latency

AMD AMA Video SDK encoder performs multi-objective optimization with set constraints on bit rate, GOP topology, visual quality measures, etc. As such, it may be tuned to achieve a compromise between latency and quality or be specialized to ultra low latency.

Guidelines on Encoder Latency Configuration

Encoding latency can also be reduced by trading off compression rate or visual quality. The following table lists the encoder options which can used to reduce to that effect.

Encode Options

Notes

Look Ahead Depth
For best visual quality, it is recommend to let the buffer depth be determined automatically. If it is found that the selected depth adds unacceptable delay, then this option can be set explicitly.

Number of B frames
It is understood that for every inserted B frame there will be a frame period delay.

Automatic Look Ahead Depth Calculation

The VQ optimized look ahead buffer depth is the summation of the following 3 components:

  1. The fixed 4 frames depth

  2. The number of selected B frames

  3. Linear function of frame rate:

    1. 8 bit: 0.7xframe rate

    2. 10 bit: 0.4xframe rate

As an example, the assigned depth of an 8-bit, AV1p30, with IPBBB sub gop, stream is 4+3+21=28 frames.

The valid range of look ahead buffer depth is between 9+Number of B frames and 46+Number of B frames, except for AV1 Type-1, where:

  1. For FPS below 30, the range is between 12+Number of B frames and 49+Number of B frames.

  2. For FPS above 30, the range is between 16+Number of B frames and 53+Number of B frames.

Ultra Low Latency (ULL) Mode

Ultra Low Latency (ULL) encoding is enabled by setting -lookahead_depth flag to 0.

Notes

  • In ULL encoding mode, frames are always processed in display order. As such, this mode is not compatible with B frames. Furthermore, only Constant Quantization Parameter (CQP) and Constant Bit Rate (CBR) options are allowed. See -control_rate.

FFmpeg Latency Measurements

Latency measurements can be obtained by enabling -latency_logging. The following is a sample transcode, HEVC to AVC, pipeline:

ffmpeg -y -hwaccel ama -hwaccel_device /dev/ama_transcoder0 -c:v hevc_ama -out_fmt nv12 -i <INPUT> -filter_complex "scaler_ama=outputs=1:out_res=(1920x1080)" -c:v h264_ama -frames 50 -lookahead_depth 0 -latency_logging enable -f rawvideo -b:v 6M /tmp/hevc_to_scale_to_h264.h264

...
2023-05-25 12:27:01.028958 NOTICE [Channel] Latency @ output frame  24:  0 frames, 4.5 ms; 152.79 fps = (24-0)/(157.1 ms)
2023-05-25 12:27:01.194255 NOTICE [Channel] Latency @ output frame  49:  0 frames, 4.5 ms; 151.24 fps = (49-24)/(165.3 ms)
frame=   50 fps=0.0 q=-0.0 latency= 12ms Lsize=     189kB time=00:00:00.83 bitrate=1858.8kbits/s speed=2.27x
...

Note that latency reports are printed every 25th frame. In the case noted above, the end to end latency is 12 [ms].