Attention

This version of the SDK has been superseded by the latest release of the SDK.

C API Programming Guide

Overview

The AMD AMA Video SDK provides a C-based application programming interface (API) which facilitates the integration of AMD transcoding capabilities in proprietary frameworks. This API is provided in the form of plugins leveraging the Xilinx Media Accelerator (XMA) library and the Xilinx Resource Manager (XRM) library.

The XMA Library

The XMA library (libxma) is meant to simplify the development of applications managing and controlling video accelerators such as decoders, scalers, filters, and encoders.

The XRM Library

The XRM library is used to manage the hardware accelerators available in the system. XRM keeps track of total system capacity for each of the compute units such as the decoder, scaler, and encoder. The XRM library makes it possible to perform actions such as reserving, allocating and releasing resources; calculating resource load and max capacity.

The AMD AMA Video SDK Plugins

The AMD AMA Video SDK provides 4 different plugins, each corresponding to a specific hardware accelerated feature of the card:

  • The decoder plugin

  • The encoder plugin

  • The filter plugin

  • The scaler plugin

Any combination of plugins can be used when integrating with a proprietary framework.


General Application Development Guide

Integration layers for applications using the AMD AMA Video SDK are organized around the following steps:

  1. Initialization

  2. Resource Reservation

  3. Session Creation

  4. Runtime Processing

  5. Cleanup

Resource Initialization and Allocation

Applications using the plugins must first create a XRM plugin using the xrm_PLUGIN_reserve(), where PLUGIN is one of dec, enc, or scale. This is done using the XRM APIs, as described in detail in the XRM API Reference Guide below.

Session Creation

Once the resources have been allocated, the application must create dedicated plugin sessions for each of the hardware accelerators that need to be used (decoder, scaler, encoder, filter).

To create a session, the application must first initialize all the required properties and parameters of the particular plugin. It must then call the corresponding session creation function. A complete reference for all the plugins is provided below.

Runtime Processing

The plugins provide functions to send data from the host and receive data from the device. The data is in the form of video frames (XmaFrame). It is also possible to do zero-copy operations where frames are passed from one hardware accelerator to the next without being copied back to the host. The send and receive functions are specific to each plugin and the return code should be used to determine the next suitable action. A complete reference for all the plugins is provided below.

Cleanup

When the application finishes, it should destroy each plugin session using the corresponding destroy function. Doing so will free the resources on the AMD devices for other jobs and ensure that everything is released and cleaned-up properly.

The application should also use the xrm_PLUGIN_release(), where PLUGIN is one of dec, enc, or scale to release allocated resources. This is done using the XRM APIs, as described in detail in the XRM API Reference Guide below.


Compiling and Linking with the AMD AMA Video SDK Plugins

The plugins can be dynamically linked to the application. The required package to build applications is XRM. This package is provided as part of the AMD AMA Video SDK.

To provide the necessary declarations in your application, include the following header in your source code:

#include <xrm.h>

To compile and link your application with the plugins, add the following lines to your CMakeList.txt file:

target_link_libraries (${PROJECT_NAME} PUBLIC xma PUBLIC xrm_interface)
target_include_directories (${PROJECT_NAME}
  PUBLIC "${PROJECT_SOURCE_DIR}/include"
  PUBLIC "$<TARGET_PROPERTY:xrm,INTERFACE_INCLUDE_DIRECTORIES>")

Common XMA Data Structures

struct XmaParameter

Type-Length-Value data structure used for passing custom arguments to a plugin. The declaration of XmaParameter can be found in the /opt/amd/ama/ma35/include/xma/xmaparam.h file.

struct XmaFrameProperties

Data structure describing the frame dimensions for XmaFrame. The declaration of XmaFrameProperties can be found in the /opt/amd/ama/ma35/include/xma/xmabuffers.h file.

struct XmaFrame

Data structure describing a raw video frame and its buffers. XmaFrame structures can be received from the decoder or sent to the encoder. They are also used as input and outputs for the scaler and the look-ahead. The declaration of XmaFrame can be found in the /opt/amd/ama/ma35/include/xma/xmabuffers.h file.

The AMD AMA Video SDK plugins supports 3 types of frames:

  • XMA_HOST_BUFFER_TYPE frames must be copied to the device using the upload filter before they can be used by the device..

  • XMA_DEVICE_ONLY_BUFFER_TYPE frames must be copied to the host using the download filter before they can be used by the host application.

  • NO_BUFFER frames do not contain buffer data and are used as place holders to be filled in by XMA plugins..

The upload filter plugin send function only supports XMA_HOST_BUFFER_TYPE frames, while all other send frame functions only support XMA_DEVICE_BUFFER_TYPE frames. All frames used in receive functions must contain "dummy" data. All parameters in those frames will be filled in by the respective receive functions. All frames must be allocated using the provided XMA functions.

struct XmaDataBuffer

Data structure describing a buffer containing encoded video data. XmaDataBuffer structures can be sent to the decoder or received from the encoder. The declaration of XmaDataBuffer can be found in the /opt/amd/ama/ma35/include/xma/xmabuffers.h file.

All XmaDataBuffer structures must be allocated using one of the provided functions. XmaDataBuffer structures used in the encoder receive function must be allocated with "dummy" data and will be filled in by the encoder.

Note

Xma[Scaler|Encoder|Filter]Properties.sw_format

sw_format, which is a member of XmaScalerProperties, XmaEncoderProperties, and XmaFilterProperties structures, represents the pixel format of a frame on the device. Therefore, its assigned value is determined by the direction of data flow. Specifically:

XMA_NV12_FMT_TYPE and XMA_P010LE_FMT_TYPE, with flags = XMA_FRAME_PROPERTY_FLAG_TILE_4x4, are internal formats that can only be generated by decoder/scaler and not by the upload filter.
XMA_PACKED10_FMT_TYPE is an internal format that can only be generated by decoder/scaler and not the upload filter.
XMA_RGB24_FMT_TYPE is reserved for future use.
XMA_YUV420P_FMT_TYPE and XMA_YUV420P10LE_FMT_TYPE are external formats that can be used by the upload filter

Common XMA Data Reference

This section describes the commonly used calls and APIs that are required in most applications.

XMA Log Interface

The external interface to the xma logging consists of the following XMA functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmalogger.h file.

int32_t xma_log_init(XmaLogLevelType log_level, XmaLogType log_type, XmaLogHandle *handle, ...)

This function creates a log session and must be called prior to logging. (See /opt/amd/ama/ma35/include/xma/xmalogger.h for details on variadic usage.)

void xma_logmsg(XmaLogHandle handle, XmaLogLevelType level, const char *name, const char *msg, ...)

This function logs a message.

void xma_log_release(XmaLogHandle handle)

This function releases a log session that was previously created with the xma_log_init() function.

XMA Session Interface

The external interface to the xma session consists of the following XMA functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xma.h file.

int32_t xma_initialize(XmaLogHandle log, XmaInitParameter *init_params, XmaHandle *handle)

This function creates a XMA session and must be called prior to creating any plugins.

void xma_release(XmaHandle handle)

This function releases a XMA session that was previously created with the xma_initialize() function.

XMA Data Buffer Interface

The external interface to the xma data buffer consists of the following XMA functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmabuffers.h file.

XmaDataBuffer *xma_data_buffer_alloc(XmaHandle handle, size_t size, bool dummy)

This function creates a XmaDataBuffer of the given size.

XmaDataBuffer *xma_data_from_buffer_clone(XmaHandle handle, uint8_t *data, size_t size, xma_data_buffer_clone_free_callback_function free_callback, void *opaque)

This function creates a XmaDataBuffer that points to the provided buffer. xma_data_buffer_clone_free_callback_function will be called, when the buffer is no longer needed.

void xma_data_buffer_free(XmaDataBuffer *data)

This function releases a previously created XmaDataBuffer.

XMA Frame Interface

The external interface to the xma Frame consists of the following XMA functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmabuffers.h file.

int32_t xma_frame_planes_get(XmaHandle handle, XmaFrameProperties *frame_props)

This returns the number of planes for a given pixel format.

int32_t xma_frame_get_plane_height(XmaHandle handle, XmaFrameProperties *frame_props, size_t plane)

This returns the height of the plane based on the frame height, pixel format, and plane.

int32_t xma_frame_get_plane_stride(XmaHandle handle, XmaFrameProperties *frame_props, size_t plane)

This returns the stride of the plane based on the frame width, pixel format, and plane.

int32_t xma_frame_get_plane_size(XmaHandle handle, XmaFrameProperties *frame_props, size_t plane)

This returns the size of the plane based on the frame width, height, pixel format, and plane.

XmaFrame *xma_frame_alloc(XmaHandle handle, XmaFrameProperties *frame_props, bool dummy)

This function creates a XmaFrame with the given properties.

XmaFrame *xma_frame_from_buffers_clone(XmaHandle handle, XmaFrameProperties *frame_props, XmaFrameData *frame_data, xma_frame_clone_free_callback_function free_callback, void *opaque)

This function creates a XmaFrame that points to the provided data and with the given properties. xma_frame_clone_free_callback_function will be called when the data is no longer needed.

XmaFrame *xma_frame_clone(XmaHandle handle, XmaFrame *xma_frame)

This function creates a XmaFrame that uses the same buffers as the provided XmaFrame, but can have side data added or removed without affecting the original XmaFrame.

int32_t xma_frame_inc_ref(XmaFrame *frame)

This function increases the reference count of the given XmaFrame.

int32_t xma_frame_dec_ref(XmaFrame *frame)

This function decreases the reference count of the given XmaFrame.

void xma_frame_free(XmaFrame *frame)

This function releases a XmaFrame that was created with the previous functions.

int32_t xma_frame_add_side_data(XmaFrame *frame, XmaFrameSideData *side_data)

This function adds the side data to the frame. If there is already side data of the same type associated with the frame, it is removed and the new side data is set. The reference count of the side_data buffer is incremented by 1 on successful execution.

XmaFrameSideData *xma_frame_get_first_side_data(XmaFrame *frame)

This function returns the handle to the first side data of any type. Added in v1.1.

XmaFrameSideData *xma_frame_get_side_data(XmaFrame *frame, enum XmaFrameSideDataType type)

This function returns the handle to the first side data of the given type.

XmaFrameSideData *xma_frame_get_next_side_data(XmaFrame *frame, XmaFrameSideData *side_data)

This function returns the handle to the next side data of any type. Added in v1.1.

XmaFrameSideData *xma_frame_get_next_side_data_of_type(XmaFrame *frame, XmaFrameSideData *side_data)

This function returns the handle to the next side data of the given type. Added in v1.1.

int32_t xma_frame_remove_side_data(XmaFrame *frame, XmaFrameSideData *side_data)

This function removes the side data from the frame. The side data buffer reference count is decremented by 1. If it results in a reference count of zero, then the side data is freed.

int32_t xma_frame_remove_side_data_type(XmaFrame *frame, enum XmaFrameSideDataType type)

This function removes all side data of the given type. Any side data buffer that is removed has it's reference count decremented by 1. If it results in a reference count of zero, then the side data buffer is freed.

void xma_frame_clear_all_side_data(XmaFrame *frame)

This function removes all side data from the frame. The reference count of each side data buffer associated with the frame is decremented by 1. If it results in a reference count of zero, then the side data is freed.

XMA Side Band Data

Passing side band data through xma consists of the following XMA functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmabuffers.h file.

XmaFrameSideData *xma_side_data_alloc(XmaHandle handle, enum XmaFrameSideDataType type, XmaBufferType buffer_type, uint64_t size)

This function allocates side data handle, with reference count 1.

void xma_side_data_free(XmaFrameSideData *side_data)

This function decrements the reference count of the side_data by 1.

int32_t xma_side_data_inc_ref(XmaFrameSideData *side_data)

This function increments the reference count of the side_data by 1

int32_t xma_side_data_dec_ref(XmaFrameSideData *side_data)

This function decrements the reference count of the side_data by 1.

int32_t xma_side_data_get_refcount(XmaFrameSideData *side_data)

This function returns the reference count of the side_data.

int32_t xma_side_data_read(XmaFrameSideData *side_data)

This function copies data from device to host. Deprecated in v1.1 and above.

int32_t xma_side_data_write(XmaFrameSideData *side_data)

This function copies data from host to device. Only use this function if you created the side data and it has not yet been sent to any sessions, otherwise risk of a race condition can occur.

int32_t xma_side_data_get_metadata(XmaFrameSideData *side_data, XmaParameter *metadata)

This function gets the value of specified metadata. Added in v1.1.

int32_t xma_side_data_set_metadata(XmaFrameSideData *side_data, XmaParameter *metadata)

This function replaces or creates the specified metadata, depending on whether it exists on not. Added in v1.1.

1.0 SDK API

The following subsections describe APIs applicable to 1.0 SDK. Note that these APIs serve as bases for future versions. In order to allow forwards and backwards ABI compatibility, the application can now set which version of the API to use. If no version is set, version 1.0 is assumed. When a version is set, only features available in that version are allowed. This means if a feature was deprecated in an earlier version, or was added in a later version, it cannot be used and will either be ignored or returned as an error.

To set the API version, use the following code snippet:

XmaParameter params[1];
uint32_t api_version = XMA_API_VERSION_M_ m;

params[0].name = (char*)XMA_API_VERSION;
params[0].type = XMA_UINT32;
params[0].length = sizeof(uint32_t);
params[0].value = &api_version;

xma_init_param.params        = params;
xma_init_param.param_cnt     = 1;
xma_initialize(logger, &xma_init_param, handle);

, where M and m are major and minor versions of the SDK, e.g., for SDK 1.0 the macro becomes XMA_API_VERSION_1_0.

Decoder Plugin Reference

Decoder Interface

The external interface to the decoder plugin consists of the following XMA functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmadecoder.h file.

XmaDecoderSession *xma_dec_session_create(XmaDecoderProperties *dec_props)

This function creates a decoder session and must be called prior to decoding data. The hardware resources required to run the session must be previously reserved using the XRM APIs and should not be released until after the session is destroyed. The number of sessions allowed depends on several factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

int32_t xma_dec_session_set_log(XmaDecoderSession *session, XmaLogHandle handle)

This function changes the logging from the default (set in xma_initialize()) to some other logging.

int32_t xma_dec_session_send_data(XmaDecoderSession *session, XmaDataBuffer *data, int32_t *data_used)

This function sends input frame data to the hardware decoder by way of the plugin. The application needs to parse the input encoded stream and send one frame of data at a time in a XmaDataBuffer data structure.

The data_used value indicates the amount of input data consumed by the decoder.

If the function returns XMA_SUCCESS, then the decoder was able to consume the entirety of the available data and data_used will be set accordingly. In this case, the application can proceed with fetching decoded data using the xma_dec_session_recv_frame() API.

If the function returns XMA_TRY_AGAIN, then the decoder was did not consume any of the input data and data_used will be reported as 0. In this case, the application can proceed with fetching previously decoded data with the xma_dec_session_recv_frame() function but must send the same input again using using xma_dec_session_send_data() until the function returns XMA_SUCCESS.

If the function returns XMA_SEND_MORE_DATA, then the data sent is being processed, but more is required before any frames will be available.

Once the application has sent all the input frames to the decoder, it must notify the decoder by sending a null buffer. The application should then continue sending null buffers in order to flush out all the output YUV frames.

int32_t xma_dec_session_get_properties(XmaDecoderSession *dec_session, XmaFrameProperties *fprops);

This function returns the decoder properties such as width, height, output format, and frame rate.

int32_t xma_dec_session_recv_frame(XmaDecoderSession *session, XmaFrame *frame)

This function tries to fetch a decoded YUV frame from the hardware accelerator.

If the function returns XMA_SUCCESS, a valid YUV frame pointer is available in the buffer pointer of the XmaFrame argument.

If the function returns XMA_TRY_AGAIN, then the decoder still needs some input data to produce a complete YUV output frame.

If the function returns XMA_EOS, then the decoder has flushed out all the frames.

int32_t xma_dec_session_destroy(XmaDecoderSession *session)

This function destroys a decoder session that was previously created with the xma_dec_session_create() function.

Decoder Properties

The AMD AMA Video SDK decoder is configured using a combination of standard XMA decoder properties and custom decoder parameters, both of which are specified using a XmaDecoderProperties data structure. The declaration of XmaDecoderProperties can be found in the /opt/amd/ama/ma35/include/xma/xmadecoder.h file.


struct XmaDecoderProperties

This data structure is used to configure the AMD AMA Video SDK decoder.


Standard XMA Decoder Properties

When using the decoder plugin, the following members of the XmaDecoderProperties data structure must be set by the application:

hwdecoder_type

Specifying type of decoder to reserve.

params

Array of custom initialization parameters. See the next section for the list of custom parameters supported by the decoder plugin.

param_cnt

Count of custom parameters.

width

Width in pixels of incoming video stream/data. Valid values are even integers between 128 and 3840. Portrait mode is supported.

height

Height in pixels of incoming video stream/data. Valid values are even integers between 128 and 2160.

bits_per_pixel

Bits per pixel for primary plane of output video. Must be set to 8 or 10 bits per pixel.

flags

Sets on-device frame property to XMA_FRAME_PROPERTY_FLAG_TILE_4x4 and/or XMA_FRAME_PROPERTY_FLAG_COMPRESS.

handle

handle to XMA device.

Custom Decoder Parameters

In addition to the standard properties, the following XmaParameter custom parameters are supported by the decoder plugin:

"out_fmt"

XMA pixel format. See -out_fmt for available options.

“low_latency”

Set to 1 to enable low latency mode.

"latency_logging"

Set to 1 to enable logging of latency information to the xma log.


Scaler Plugin Reference

Scaler Interface

The external interface to the scaler plugin consists of the following XMA application-level functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmascaler.h file.

XmaScalerSession *xma_scaler_session_create(XmaScalerProperties *props)

This function creates scaler session and must be called prior to sending input frames. The hardware resources required to run the session must be previously reserved using the XRM APIs and should not be released until after the session is destroyed. The number of sessions allowed depends on several factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

This function changes the logging from the default (set in xma_initialize()) to some other logging.

int32_t xma_scaler_session_send_frame(XmaScalerSession *session, XmaFrame *frame)

This function sends a YUV frame to the underlying XMA plugin( and eventually to hardware to scale the input frame to one or multiple resolutions.

The application can take further action depending upon the return value from this API.

If the function returns XMA_SUCCESS, then the application can proceed to fetch scaled output frames.

If the function returns XMA_SEND_MORE_DATA, then the application should proceed with sending next YUV frame.

If the function returns XMA_FLUSH_AGAIN it means that the application should keep flushing the scaler.

Once the application has sent all the input frames to the scaler, it must notify the scaler by sending a null frame to flush the scaler.

int32_t xma_scaler_session_recv_frame_list(XmaScalerSession *session, XmaFrame **frame_list)

This function is called after calling the xma_scaler_session_send_frame(). This function returns a list of output frames with every call until it reaches end of scaling. Return codes can only be XMA_SUCCESS and XMA_ERROR.

int32_t xma_scaler_session_destroy(XmaScalerSession *session)

This function destroys scaler session that was previously created with the xma_scaler_session_create() function.


Scaler Properties

The AMD AMA Video SDK scaler is configured using a combination of standard XMA scaler properties, standard XMA scaler input and output properties and custom scaler parameters, all of which are specified using XmaScalerProperties and XmaScalerInOutProperties data structures.


struct XmaScalerProperties

This data structure is used to configure the video scaler. The declaration of XmaScalerProperties can be found in the /opt/amd/ama/ma35/include/xma/xmascaler.h file.

struct XmaScalerInOutProperties

This data structure is used to configure the input and outputs of the video scaler. The XmaScalerFilterProperties data structure contains one XmaScalerInOutProperties for the scaler input and an array of 16 XmaScalerInOutProperties for the scaler outputs. The declaration of XmaScalerInOutProperties can be found in the /opt/amd/ama/ma35/include/xma/xmascaler.h file.


Standard XMA Scaler Properties

When using the scaler plugin, the following members of the XmaScalerProperties data structure must be set by the application:

hwscaler_type

Type of scaler. Must be set to XMA_ABR_SCALER_TYPE.

num_outputs

Number of scaler outputs.

params

Array of custom initialization parameters. See the next section for the list of custom parameters supported by the scaler plugin.

param_cnt

Count of custom parameters.

input

Input property of type XmaScalerInOutProperties

output

Output properties an array of type XmaScalerInOutProperties of size num_outputs

XMA Scaler Input and Output Properties

When configuring the scaler input and outputs, the following members of the XmaScalerInOutProperties data structure must be set by the application:

format

Host side input video format. Must be set to XMA_VPE_FMT_TYPE

sw_format

Device side video format. Valid values are XMA_NV12_FMT_TYPE, XMA_P010LE_FMT_TYPE, XMA_PACKED10_FMT_TYPE, XMA_RGB24_FMT_TYPE, XMA_YUV420P_FMT_TYPE, XMA_YUV420P10LE_FMT_TYPE.

width

Width in pixels of video stream/data. Valid values are integers between 128 and 3840, in multiples of 4. Portrait mode is supported.

height

Height in pixels of video stream/data. Valid values are even integers between 128 and 2160, in multiples of 4.

framerate

Framerate data structure specifying frame rate per second.

flags

Sets on-device frame property to XMA_FRAME_PROPERTY_FLAG_TILE_4x4 and/or XMA_FRAME_PROPERTY_FLAG_COMPRESS

Other members of XmaScalerInOutProperties are not applicable to the scaler plugin and should not be used.

Custom Scaler Parameters

In addition to the standard properties, the following XmaParameter custom parameters are supported by the scaler plugin:

"mix_rate"

This parameter is used to configure mix-rate sessions where some scaler outputs are configured at the input frame rate and some other outputs will be configured at half the rate.

“top”

Cropping the top of the video before scaling.

“width”

Cropping the width of the video before scaling.

“height”

Cropping the height of the video before scaling.

"latency_logging"

Set to 1 to enable logging of latency information to XMA logs. Set to 0 to disable logging.


Encoder Plugin Reference

Encoder Interface

The external interface to the encoder plugin consists of the following XMA application-level functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmaencoder.h file.

XmaEncoderSession *xma_enc_session_create(XmaEncoderProperties *enc_props)

This function creates an encoder session and must be called prior to encoding input YUV. The hardware resources required to run the session must be previously reserved using the XRM APIs and should not be released until after the session is destroyed. The number of sessions allowed depends on several factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

int32_t xma_enc_session_set_log(XmaEncoderSession *session, XmaLogHandle handle)

This function changes the logging from the default (set in xma_initialize()) to some other logging.

int32_t xma_enc_session_send_frame(XmaEncoderSession *session, XmaFrame *frame)

This function sends a YUV frame to the hardware encoder by way of the plugin.

Each time the application calls this function, it must provide a XmaFrame containing YUV data and information about this frame (XmaFrameProperties).

If the function returns XMA_SUCCESS, then the application can proceed to fetch the encoded data using the xma_enc_session_recv_data() API.

If the function returns XMA_SEND_MORE_DATA, then the application must send the next YUV frame before calling xma_enc_session_recv_data().

Once the application has sent all the input frames to the encoder, it should notify the hardware by sending a null frame. If the API returns XMA_SUCCESS after a null frame is sent, then the application can call xma_enc_session_recv_data() but must send a null frame again.

int32_t xma_enc_session_recv_data(XmaEncoderSession *session, XmaDataBuffer *data, int32_t *data_size)

This function is called after calling the function xma_enc_session_send_frame(). The application is responsible for allocating the "dummy" XmaDataBuffer using one of the provided functions. It is also responsible for releasing it when done.

If the function returns XMA_SUCCESS, then valid output data is available.

If the function returns XMA_RESEND_AND_RECV, a data buffer is not ready to be returned and the length of the data buffer is set to 0. The XmaDataBuffer is untouched and can either be reused or released.

If the function returns XMA_EOS, the encoder has flushed all the output frames. The XmaDataBuffer is untouched and must be released.

int32_t xma_enc_session_destroy(XmaEncoderSession *session)

This function destroys an encoder session that was previously created with the xma_enc_session_create() function.

Encoder Properties

The AMD AMA Video SDK encoder is configured using a combination of standard XMA encoder properties and custom encoder parameters, both of which are specified using a XmaEncoderProperties data structure.


struct XmaEncoderProperties

This data structure is used to configure the video encoder. The declaration of XmaEncoderProperties can be found in the /opt/amd/ama/ma35/include/xma/xmaencoder.h file.

Standard XMA Encoder Properties

When using the encoder plugin, the following members of the XmaEncoderProperties data structure must be set by the application:

hwencoder_type

Specify type of encoder to reserve.

format

Host side input video format. Must be set to XMA_VPE_FMT_TYPE

sw_format

Device side input video format. Valid values are XMA_NV12_FMT_TYPE, XMA_P010LE_FMT_TYPE, XMA_PACKED10_FMT_TYPE, XMA_RGB24_FMT_TYPE, XMA_YUV420P_FMT_TYPE, XMA_YUV420P10LE_FMT_TYPE.

width

Width in pixels of incoming video stream/data. Valid values are even integers between 128 and 3840. Portrait mode is supported.

height

Height in pixels of incoming video stream/data. Valid values are even integers between 128 and 2160.

framerate

Framerate data structure specifying frame rate per second

lookahead_depth

The lookahead module depth to give start giving lookahead data. Supported values are 0 to 40.

rc_mode

Rate control mode for custom rate control Supported values are 0 (constant QP), 1 (CBR), 2 (VBR), and 3 (CVBR).

bitrate

Bitrate of output data (in kbps).

qp

Fixed quantization value. Valid values are 0-51 for H.264 or HEVC, 0-255 for AV1, or -1 (not used).

gop_size

Maximum group of pictures size in frames.

temp_aq_gain

Temporal AQ Gain. Valid values are 0-255.

spat_aq_gain

Spatial AQ Gain. Valid values are 0-255.

minQP

Minimum QP. Valid values are 0-51 for H.264 or HEVC, 0-255 for AV1, or -1 (not used).

maxQP

Maximum QP. Valid values are 0-51 for H.264 or HEVC, 0-255 for AV1, or -1 (not used).

profile

Encoding profile. Valid values are 0 (Baseline), 1 (Main), 2 (High), 3 (High 10), or 4 (High 10 Intra) for H.264, 100 (Main), 101 (Main Intra), 102 (Main 10), or 103 (Main 10 Intra) for HEVC, or 200 (Main) for AV1, or -1 (Auto) for any codec.

level

Encoding level.

flags

Sets on-device frame property to XMA_FRAME_PROPERTY_FLAG_TILE_4x4 and/or XMA_FRAME_PROPERTY_FLAG_COMPRESS

params

Array of custom initialization parameters. See the next section for the list of custom parameters supported by the encoder plugin.

param_cnt

Count of custom parameters.

Custom Encoder Parameters

In addition to the standard properties, the following XmaParameter custom parameters are supported by the encoder plugin:

"slice"

Slice to perform the encode.

"latency_logging"

When enabled, it logs latency information to XMA log.

"spatial_aq"

Enables or disables adaptive spatial qp.

"temporal_aq"

Enables or disables adaptive temporal qp.

"qp_mode"

Sets qp mode to auto, relative load or uniform.

"tune_metrics"

Sets the tuning metric to none, VQ, PSNR, SSIM or VMAF.

"forced_idr"

Forces insertion of an IDR.

"crf"

Enables or disables CRF.

"expert_options"

sets encoder's expert option.

"device_type"

For AV1 encoder selects between Type 1 and Type 2.

Encoder Dynamic Parameters

Dynamic parameters allow for per frame updates to encoding parameters. These parameters are specified by either XmaDynamicEncParams or XmaDynamicEncParams_v2 structure. To insert dynamic data parameters to encoder, proceed as follows:

  1. Allocate side band data through xma_side_data_alloc()

  2. Set returned structure members, from Step 1, of XmaDynamicEncParams or XmaDynamicEncParams_v2

  3. Clone frame using xma_frame_clone()

  4. Attach XmaDynamicEncParams or XmaDynamicEncParams_v2 structure using xma_frame_add_side_data()

  5. Check return values for status checking

  6. Remove dynamic parameters from frame using xma_frame_dec_ref()

Note

Use XmaDynamicEncParams structure for XMA_API_VERSION_1_1 and below. For XMA_API_VERSION_1_1_2 and above, use XmaDynamicEncParams_v2 structure.

struct XmaDynamicEncParams

This data structure is used to configure the dynamic parameters of encoder. The declaration of XmaDynamicEncParams can be found in the /opt/amd/ama/ma35/include/xma/xmasidedata.h file. See Video Encoding for details.

Dynamic Encoder Parameters

is_spatial_aq_gain_changed

Change indicator flag for spatial_aq_gain

spatial_aq_gain

Spatial adaptive quantizer gain factor

is_temporal_aq_gain_change

Change indicator flag for temporal_aq_gain

temporal_aq_gain

Temporal adaptive quantizer gain factor

is_temporal_mode_changed

Change indicator flag for temporal_aq_mode

temporal_aq_mode

Enable/disable temporal quantizer

is_spatial_mode_changed

Change indicator flag for spatial_aq_mode

spatial_aq_mode

Enable/disable spatial quantizer

is_bit_rate_changed

Change indicator flag for bit_rate_kbps

bit_rate_kbps

Bit rate in kbps

is_b_frames_changed

Change indicator flag for num_b_frames

num_b_frames

Number of B frames

is_min_qp_changed

Change indicator flag for min_qp

min_qp

Minimum QP value

is_max_qp_changed

Change indicator flag for min_qp

max_qp

Maximum QP value

struct XmaDynamicEncParams_v2

This data structure is used to configure extra dynamic parameters for encoding. The declaration of XmaDynamicEncParams_v2 can be found in the /opt/amd/ama/ma35/include/xma/xmasidedata.h file. See Video Encoding for details.

Dynamic Encoder Parameters Version 2

v1

XmaDynamicEncParams memeber

is_qp_changed

Change indicator flag for qp

qp

QP value

is_qp_i_offset_changed

Change indicator flag for qp_i_offset

qp_i_offset

QP offset for I frames

is_qp_b_offset_changed

Change indicator flag for qp_b_offset

qp_b_offset

QP offset for B frames

is_min_bit_rate_changed

Change indicator flag for min_bit_rate_kbps

min_bit_rate_kbps

Minimum bit rate in kbps

is_max_bit_rate_changed

Change indicator flag for max_bit_rate_kbps

max_bit_rate_kbps

Maximum bit rate in kbps

is_bit_rate_changed

Change indicator flag for bit_rate_kbps

reserved

For future use


Filter Plugin Reference

Filter Interface

The filter plugin is based on the Filter XMA plugin type. The external interface to the filter plugin consists of the following XMA application-level functions:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xma/xmafilter.h file.

XmaFilterSession *xma_filter_session_create(XmaFilterProperties *props)

This function creates a filter session and must be called prior to sending YUV frames to the filter. The hardware resources required to run the session must be previously reserved using the XRM APIs and should not be released until after the session is destroyed. The number of sessions allowed depends on several factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

int32_t xma_filter_session_set_log(XmaFilterSession *session, XmaLogHandle handle)

This function changes the logging from the default (set in xma_initialize()) to some other logging.

int32_t xma_filter_session_send_frame(XmaFilterSession *session, XmaFrame *frame)

This function sends YUV frames to the filter module in hardware. After sending a frame, the application can take further action depending upon the return value from this API.

If this function returns XMA_SUCCESS, then the application can proceed to recieve a frame from this filter.

If this function returns XMA_TRY_AGAIN, it means the input frame has not been consumed and needs to re-send the same input frame after calling receive frame.

Once the application sends all input frames to the filter module, it should continue sending null frames until all the frames have been flushed out from the filter.

int32_t xma_filter_session_recv_frame(XmaFilterSession *session, XmaFrame *frame)

This function is called after calling the function xma_filter_session_send_frame().

If this function returns XMA_SUCCESS, then the "dummy" frame provided by the application has been filled in and can be sent to the next plugin.

If this function returns XMA_RESEND_AND_RECV, then no frames have been sent to the filter for processing. The XmaFrame is untouched and may be reused or released. If this function returns XMA_TRY_AGAIN, then frames have been sent and are currently being processed, but are not yet ready. The c:struct:XmaFrame is untouched and may be reused or released. If this function returns XMA_EOS, then all frames have been processed. The c:struct:XmaFrame is untouched and must be released.

int32_t xma_filter_session_destroy(XmaFilterSession *session)

This function destroys the filter session that was previously created with the xma_filter_session_create() function.

Filter Properties

The AMD AMA Video SDK filter is configured using a combination of standard XMA filter properties, standard XMA filter input and output properties and custom filter parameters, all of which are specified using XmaFilterProperties and XmaFilterPortProperties data structures.

struct XmaFilterProperties

This data structure is used to configure the filter function. The declaration of XmaFilterProperties can be found in the /opt/amd/ama/ma35/include/xma/xmafilter.h file.

struct XmaFilterPortProperties

This data structure is used to configure the input and output of the filter. The XmaFilterProperties data structure contains one XmaFilterPortProperties for the filter input and one XmaFilterPortProperties for the output output. The declaration of XmaFilterPortProperties can be found in the /opt/amd/ama/ma35/include/xma/xmafilter.h file.

Standard XMA Filter Properties

When using the filter plugin, the following members of the XmaFilterPortProperties data structure must be set by the application:

hwfilter_type

Sets the direction of stream entry to XMA_UPLOAD_FILTER_TYPE or XMA_DOWNLOAD_FILTER_TYPE.

params

Array of custom initialization parameters. See the next section for the list of custom parameters supported by the filter plugin.

param_cnt

Count of custom parameters.

input

Input property of type XmaFilterPortProperties

output

Output property of type XmaFilterPortProperties

Standard XMA Filter Port Properties

When configuring the filter input or output, the following members of the XmaFilterPortProperties data structure must be set by the application:

format

Input video format. Must be set to XMA_VPE_FMT_TYPE.

sw_format

Device side input video format. Valid values are XMA_NV12_FMT_TYPE, XMA_P010LE_FMT_TYPE, XMA_PACKED10_FMT_TYPE, XMA_RGB24_FMT_TYPE, XMA_YUV420P_FMT_TYPE, XMA_YUV420P10LE_FMT_TYPE.

width

Width in pixels of incoming video stream/data. Valid values are even integers between 128 and 3840 Portrait mode is supported.

height

Height in pixels of incoming video stream/data. Valid values are even integers between 128 and 2160.

framerate

Framerate data structure specifying frames per second.

flags

Sets on-device frame property to XMA_FRAME_PROPERTY_FLAG_TILE_4x4 or XMA_FRAME_PROPERTY_FLAG_COMPRESS

framerate

Framerate data structure specifying frame rate per second.

Custom Filter Parameters

In addition to the standard properties, the following XmaParameter custom parameters are supported by the filter plugin:

"latency_logging"

When enabled, it logs latency information to XMA log.


Ultra Low Latency Considerations

Decoder Low Latency Mode

To enable low latency in the decoder, add a XmaParameter with the name low_latency, of type XMA_INT32, and value of 1. After each frame is sent to the decoder, the decoder will return XMA_SUCCESS. The application can immediately call xma_dec_session_recv_frame(). It will return either XMA_SUCCESS (indicating a frame has been returned), XMA_EOS (indicating the end of stream has been reached), or XMA_RESEND_AND_RECV (indicating the frame is still being processed and xma_dec_session_recv_frame() must keep being called until it is ready).

Encoder Ultra Low Latency Mode

To enable Ultra Low Latency in the encoder, set XmaEncoderProperties.lookahead_depth = 0. After each frame is sent to the encoder, the encoder will return XMA_SUCCESS. The application can immediately call xma_enc_session_recv_data(). It will return either XMA_SUCCESS (indicating a frame has been returned), XMA_EOS (indicating the end of stream has been reached), or XMA_RESEND_AND_RECV (indicating the frame is still being processed and xma_enc_session_recv_data() must keep being called until it is ready).

1.1 SDK API

The following subsections delineate 1.1 API improvements over 1.0. To use 1.1 APIs, set XMA_API_VERSION parameter to XMA_API_VERSION_1_1. (See here for how to do this.)

Note

SDK 1.1.1 is not backward ABI compatible with SDK 1.0. This implies that 1.0 applications need to be migrated to 1.1.1 SDK, manually.

Sideband Data

The main difference between SDK 1.1 and 1.0 are:

  1. v1.0 only allowed 1 sideband data of each type, adding another would replace the first. v1.1 allows multiple of each type. New functions are added to access/modify the sideband data.

  2. HDR10 sideband data type are deprecated and replaced with several different SEI sideband data types

  3. Applications can define their own sideband data types and add the sideband data to frames. Filters will pass this sideband data downstream. Currently, only frames can have sideband data.

  4. All sideband data (both predefined and user defined) can have metadata. Metadata is XmaParameters attached to sideband data used for adding additional information about a sideband data buffer to a sideband data. Each sideband data can have only one sideband metadata with a given name. If an application tries to add new metadata to a sidedata and a metadata with the same name already exists, the original metadata will be deleted and replaced by the new metadata. Metadata exists solely on the host.

  5. Added sidedata functions are:

    - xma_side_data_get_metadata(): Gets the value of sideband metadata.
- xma_side_data_set_metadata(): Adds/replaces sideband metadata to sideband data.
- xma_frame_get_first_side_data(): Get the first sidedata of any type that is attached to a given frame.
- xma_frame_get_next_side_data(): Get the next sidedata of any type that is attached to a given frame.
- xma_frame_get_next_side_data_of_type(): Get the next sidedata of a given type that is attached to a given frame.

  6. Modified sidedata functions are:

    - xma_frame_get_side_data(): In v1.0 it will continue to get the sidedata of a given type from a given frame. In v1.1 it will get the first sidedata of a given type from a given frame.
- xma_frame_remove_side_data_type(): In v1.0 it will continue to remove the sidedata of a given type. In v1.1 it will remove all sidedata of a given type.

  7. Deprecated sidedata functions are:

    - xma_side_data_read(): In v1.0 it copies the contents of the sidedata buffer from the device to the host.

Multithreading

  1. In the decoder, 2 XmaParameter parameters are added:XMA_DEC_PARAM_THREADS and XMA_DEC_PARAM_WAIT. If XMA_DEC_PARAM_THREADS is set to 1, a background thread is deployed to improve performance. Note that XMA_DEC_PARAM_THREADS is 0 by default. If the application is using different threads to send and receive data from the decoder, the parameter XMA_DEC_PARAM_WAIT will improve CPU utilization on the host. XMA_DEC_PARAM_WAIT is only available if XMA_DEC_PARAM_THREADS is 1. XMA_DEC_PARAM_WAIT is 0 by default.

  2. Similarly, in the encoder, 2 XmaParameter parameters are added: XMA_ENC_PARAM_THREADS and XMA_ENC_PARAM_WAIT. These behave the same as in the decoder.

  3. The encoder calling order of send and receive has changed to be more in line with the decoder and scaler.

    • xma_enc_session_send_frame() return values and expected application behavior:

      • XMA_SUCCESS: A frame is successfully sent to the encoder, regardless of if any data can be pulled from the encoder. The application should try calling xma_enc_session_recv_data() and proceed depending on the return value from xma_enc_session_recv_data().

      • XMA_TRY_AGAIN: The encoder's internal buffers are currently full. The application should try calling xma_enc_session_recv_data() and proceed according to it's return code. The next time xma_enc_session_send_frame() is called, send the same frame as was last sent, as it has not yet been processed. XMA_TRY_AGAIN will only be returned if XMA_ENC_PARAM_WAIT is 0.

    • xma_enc_session_recv_data() return values and expected application behavior:

      • XMA_SUCCESS: Compressed data was returned and xma_enc_session_send_frame() can be called.

      • XMA_RESEND_AND_RECV: No compressed data is ready and the application should call xma_enc_session_send_frame(). Multiple loops of xma_enc_session_send_frame() returning XMA_TRY_AGAIN and xma_enc_session_recv_data() returning XMA_RESEND_AND_RECV may occur if internal buffers are full but no compressed data is ready. Continue looping until one of the functions returns a different value. XMA_RESEND_AND_RECV will only be returned if XMA_ENC_PARAM_WAIT is 0.

      • XMA_EOS: The end of the stream has been reached.

  4. XMA_SCALER_PARAM_THREADS is enabled by default in the scaler. To disable threading and pipelining, set this parameter to 0.

Logging

XMA logging has been extended to be able to optionally output AMA logging. To use it, in the call to xma_log_init(), set the log type to XMA_LOG_TYPE_AMA.

Upgrading

  1. If any frame returned by a session needs to have sideband data added/removed, e.g. adding XMA_FRAME_SIDE_DATA_DYN_ENC_PARAMS, then:

    • Original frame will need to be cloned, using xma_frame_clone()

    • Original frame itself, not its clone, has to have it's reference count decremented, using xma_frame_dec_ref()

    • Clone frame can have its sideband data added or removed using xma_frame_add_side_data(), xma_frame_remove_side_data(), or xma_frame_remove_side_data_type().

    Following the above, the clone can then be passed on downstream, same as the original frame.

  2. xma_frame_add_side_data() does not remove existing sideband data from a frame, prior to adding a new sideband data. Instead it will add the new sideband data in addition to the old sideband data. To remove existing sideband data, call xma_frame_remove_side_data_type() first.

XRM API Reference

The Xilinx® FPGA Resource Manager (XRM) library is used to manage the hardware accelerators available in the system. XRM keeps track of total system capacity for each of the compute units such as the decoder, scaler, and encoder.

The XRM library includes a daemon, a command line tool and a C application programming interface (API). Using the library API, external applications can communicate with the XRM daemon and perform actions such as reserving, allocating and releasing resources.

More details on the XRM command line tool (xrmadm) and the XRM daemon (xrmd) can be found in the XRM Reference Guide section of the documentation.

The XRM C APIs are defined in /opt/amd/ama/ma35/include/xrm/xrm.h.

Decoder Resource Reservation with XRM

The decoder plugin uses the following function to reserve and release resources:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xrm_interface/xrm_dec_interface.h file.

int xrm_dec_reserve(XrmDecodeContext *xrm_dec_ctx, int dev_index, XrmInterfaceProperties *xrm_props)

This function checks and allocates the necessary xrm resources given the relevant decoder parameters

void xrm_dec_release(XrmDecodeContext *xrm_dec_ctx)

This function releases the xrm resources which were allocated.

XrmDecodeContext is defined in /opt/amd/ama/ma35/include/xrm_interface/xrm_dec_interface.h file.

Encoder Resource Reservation with XRM

The encoder plugin uses the following function to reserve and release resources:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xrm_interface/xrm_enc_interface.h file.

int32_t xrm_enc_reserve(XrmEncodeContext *xrm_enc_ctx, int dev_index, int slice_id, bool is_xav1, bool is_ull, XrmInterfaceProperties *xrm_props)

This function checks and allocates the necessary xrm resources given the relevant encoder parameters

void xrm_enc_release(XrmEncodeContext *xrm_enc_ctx)

This function releases the xrm resources which were allocated.

XrmEncodeContext is defined in /opt/amd/ama/ma35/include/xrm_interface/xrm_enc_interface.h file.

Scaler Resource Reservation with XRM

The scaler plugin uses the following function to reserve and release resources:

The declaration of these functions can be found in the /opt/amd/ama/ma35/include/xrm_interface/xrm_scale_interface.h file.

int32_t xrm_scale_reserve(XrmScaleContext *scaler_xrm_ctx, int dev_index, XrmInterfaceProperties *input_props, XrmInterfaceProperties *output_props, int num_outputs)

This function checks and allocates the necessary xrm resources given the relevant encoder parameters

void xrm_scale_release(XrmScaleContext *scaler_xrm_ctx)

This function releases the xrm resources which were allocated.

XrmScaleContext is defined in /opt/amd/ama/ma35/include/xrm_interface/xrm_scale_interface.h file.

Building Sample Examples

To build the included sample applications, follow the steps below:

cd /opt/amd/ama/ma35/examples/xma/
chmod -R 777 .
mkdir build && cd build
cmake ..
make -j