Tuning Latency¶

Latency tuning allows for trade off among end to end delay, video bit rate, GOP composition, etc. Interactive applications that require low latency, can do so by setting relevant parameters in the video pipeline. It is understood that lowering the latency comes at the cost of increased bit rate, for a given visual quality.

Decoder Latency ¶

Decoding latency can be reduced by enabling the -low_latency option, in FFmpeg.

Encoder Latency ¶

AMD AMA Video SDK encoder performs multi-objective optimization with set constraints on bit rate, GOP topology, visual quality measures, etc. As such, it may be tuned to achieve a compromise between latency and quality or be specialized to ultra low latency.

Guidelines on Encoder Latency Configuration¶

Encoding latency can also be reduced by trading off compression rate or visual quality. The following table lists the encoder options which can used to reduce to that effect.

Encode Options	Notes
Look Ahead Depth	For best visual quality, it is recommend to let the buffer depth be determined automatically. If it is found that the selected depth adds unacceptable delay, then this option can be set explicitly. Supported range is 0 - 46+number of B frames.
Number of B frames	It is understood that for every inserted B frame there will be a frame period delay.

See Encoding Compatibility Matrix combination of allowable parameters.

Automatic Look Ahead Depth Calculation¶

The default VQ optimized look ahead buffer depths are:

800 ms for 8 bit

600 ms for 10 bit

Ultra Low Latency (ULL) Mode¶

Ultra Low Latency (ULL) encoding is enabled by setting -lookahead_depth flag to 0.

Notes

In ULL encoding mode, frames are always processed in display order. As such, this mode is not compatible with B frames. Furthermore, only Constant Quantization Parameter (CQP) and Constant Bit Rate (CBR) options are allowed. See -control_rate.

Latency Adjustment¶

The overall latency can be further tuned by adjusting the -bufsize parameter. This parameter allows for tuning strict and relaxed ULL modes. Both relaxed ULL and strict ULL modes have the lowest achievable encoding latency, in AMD AMA SDK. Strict ULL has lower transmission latency than relaxed ULL, by restricting frame size variations at the expense of lower VQ. Relaxed ULL has better VQ than strict ULL, by allowing larger frame size variations. Such variations may results in higher transmission latency, depending on network bandwidth.

FFmpeg Latency Measurements ¶

Latency measurements can be obtained by configuring Unified Logging. The following example demonstrates how to measure decoder, encoder and end to end latencies:

export LOG_AMA_CONFIG="destination=file, location=log.txt, max_size=1000MB | log_level=WARN | debug_file_line=1 | debug_time_stamp=1 | debug_pid=1 | debug_thread=1 | perf_log=1"
export LOG_AMA_FILTER_PERF="*.*=INFO"
ffmpeg -y -nostdin -hwaccel ama -hwaccel_device /dev/ama_transcoder0   -re -c:v h264_ama  -low_latency 1 -ih264_1080p60.mp4 -c:v av1_ama -lookahead_depth 0 -f null /dev/null
...
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'h264_1080p60.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.76.100
  Duration: 00:00:20.00, start: 0.000000, bitrate: 536 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080, 530 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
 Stream mapping:
 ...

The log file tracks all timing info for all components of the pipeline. To generate a human readable output, use the parse_logs.py utility. This script will generate an output the looks like the following:

python3 <DOWNLOAD PATH>/parse_logs.py ./log.txt
============================ INSTANCE INFO START ===============================

DEC ::
        0. h264_ama@0x55746a79cd80
                DECODER { PerfBeg, PerfEnd }
                DECSDK::DWLReserveCmdBuf { PerfBeg, PerfEnd }
                DECSDK::rsv_osalsubmit { PerfBeg, PerfEnd }
                DECSDK::rsv_osalwait { PerfBeg, PerfEnd }
                DECSDK::DWLDMA_RC2EP { PerfBeg, PerfEnd }
                DECSDK::DWLEnableCmdBuf { PerfBeg, PerfEnd }
                DECSDK::en_osalsubmit { PerfBeg, PerfEnd }
                DECSDK::en_osalwait { PerfBeg, PerfEnd }
                DECSDK::DWLReleaseCmdBuf { PerfBeg, PerfEnd }
                DECSDK::rel_osalsubmit { PerfBeg, PerfEnd }
                DECSDK::rel_osalwait { PerfBeg, PerfEnd }
------------------------------------------------------------


ENC ::
        0. av1_ama@0x55746a79d9c0
                ENCODER { PerfBeg, PerfEnd }
                ENCODER::PutFrame { PerfBeg, PerfEnd }
                Encoder::SDK { PerfBeg, PerfEnd }
                ENCODER::GetPkt { PerfBeg, PerfEnd }
------------------------------------------------------------



============================= INSTANCE INFO END =================================

DEC_0.DECODER latency (avg) = 66.926 ms [min = 9.301 ms, max = 81.653 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECODER --> PerfEnd@h264_ama@0x55746a79cd80-DECODER)

DEC_0.DECSDK::DWLReserveCmdBuf latency (avg) = 0.050 ms [min = 0.020 ms, max = 0.080 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::DWLReserveCmdBuf --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::DWLReserveCmdBuf)

DEC_0.DECSDK::rsv_osalsubmit latency (avg) = 0.012 ms [min = 0.006 ms, max = 0.031 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::rsv_osalsubmit --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::rsv_osalsubmit)

DEC_0.DECSDK::rsv_osalwait latency (avg) = 0.013 ms [min = 0.004 ms, max = 0.029 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::rsv_osalwait --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::rsv_osalwait)

DEC_0.DECSDK::DWLDMA_RC2EP latency (avg) = 0.046 ms [min = 0.032 ms, max = 0.065 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::DWLDMA_RC2EP --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::DWLDMA_RC2EP)

DEC_0.DECSDK::DWLEnableCmdBuf latency (avg) = 1.734 ms [min = 1.664 ms, max = 1.886 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::DWLEnableCmdBuf --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::DWLEnableCmdBuf)

DEC_0.DECSDK::en_osalsubmit latency (avg) = 0.012 ms [min = 0.005 ms, max = 0.042 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::en_osalsubmit --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::en_osalsubmit)

DEC_0.DECSDK::en_osalwait latency (avg) = 1.639 ms [min = 1.606 ms, max = 1.801 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::en_osalwait --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::en_osalwait)

DEC_0.DECSDK::DWLReleaseCmdBuf latency (avg) = 0.044 ms [min = 0.019 ms, max = 0.089 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::DWLReleaseCmdBuf --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::DWLReleaseCmdBuf)

DEC_0.DECSDK::rel_osalsubmit latency (avg) = 0.015 ms [min = 0.006 ms, max = 0.032 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::rel_osalsubmit --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::rel_osalsubmit)

DEC_0.DECSDK::rel_osalwait latency (avg) = 0.009 ms [min = 0.004 ms, max = 0.024 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECSDK::rel_osalwait --> PerfEnd@h264_ama@0x55746a79cd80-DECSDK::rel_osalwait)

ENC_0.ENCODER latency (avg) = 81.518 ms (APPLICATION LEVEL) [min = 5.777 ms, max = 106.021 ms]
         (PerfBeg@av1_ama@0x55746a79d9c0-ENCODER --> PerfEnd@av1_ama@0x55746a79d9c0-ENCODER)

ENC_0.ENCODER::PutFrame latency (avg) = 0.105 ms [min = 0.006 ms, max = 3.267 ms]
         (PerfBeg@av1_ama@0x55746a79d9c0-ENCODER::PutFrame --> PerfEnd@av1_ama@0x55746a79d9c0-ENCODER::PutFrame)

ENC_0.Encoder::SDK latency (avg) = 4.839 ms [min = 4.022 ms, max = 20.838 ms]
         (PerfBeg@av1_ama@0x55746a79d9c0-Encoder::SDK --> PerfEnd@av1_ama@0x55746a79d9c0-Encoder::SDK)

ENC_0.ENCODER::GetPkt latency (avg) = 0.008 ms [min = 0.003 ms, max = 0.017 ms]
         (PerfBeg@av1_ama@0x55746a79d9c0-ENCODER::GetPkt --> PerfEnd@av1_ama@0x55746a79d9c0-ENCODER::GetPkt)

PostDecode_CH1 latency (avg) = 81.573 ms [min = 6.884 ms, max = 106.047 ms]
         (PerfEnd@h264_ama@0x55746a79cd80-DECODER --> PerfEnd@av1_ama@0x55746a79d9c0-ENCODER)

End2End_CH1 latency (avg) = 148.499 ms [min = 16.821 ms, max = 174.677 ms]
         (PerfBeg@h264_ama@0x55746a79cd80-DECODER --> PerfEnd@av1_ama@0x55746a79d9c0-ENCODER)

The above delineates component based and end to end timing information. In this example, the end to end delay is 148.499 [ms] and the overall encoder delay, i.e. FFmpeg and SDK, is 81.518 [ms].