GstVideo.VideoDecoder¶
- Subclasses:
None
Methods¶
- Inherited:
- Structs:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Virtual Methods¶
- Inherited:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Properties¶
- Inherited:
Name |
Type |
Flags |
Short Description |
|---|---|---|---|
r/w |
Flags to use when automatically requesting sync points |
||
r/w |
Automatically request sync points when it would be useful |
||
r/w |
Discard frames marked as corrupted instead of outputting them |
||
r/w |
Max consecutive decoder errors before returning flow error |
||
r/w |
Minimum interval between force-keyunit requests in nanoseconds |
||
r/w |
Handle Quality-of-Service events from downstream |
Signals¶
- Inherited:
Fields¶
- Inherited:
Name |
Type |
Access |
Description |
|---|---|---|---|
element |
r |
||
input_segment |
r |
||
output_segment |
r |
||
padding |
[ |
r |
|
sinkpad |
r |
||
srcpad |
r |
||
stream_lock |
r |
Class Details¶
- class GstVideo.VideoDecoder(**kwargs)¶
- Bases:
- Abstract:
Yes
- Structure:
This base class is for video decoders turning encoded data into raw video frames.
The
GstVideo.VideoDecoderbase class and derived subclasses should cooperate as follows:- Configuration
Initially,
GstVideo.VideoDecodercalls start when the decoder element is activated, which allows the subclass to perform any global setup.GstVideo.VideoDecodercalls set_format to inform the subclass of caps describing input video data that it is about to receive, including possibly configuration data. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.Incoming data buffers are processed as needed, described in Data Processing below.
GstVideo.VideoDecodercalls stop at end of all processing.
- Data processing
The base class gathers input data, and optionally allows subclass to parse this into subsequently manageable chunks, typically corresponding to and referred to as ‘frames’.
Each input frame is provided in turn to the subclass’ handle_frame callback.
When the subclass enables the subframe mode with
gst_video_decoder_set_subframe_mode, the base class will provide to the subclass the same input frame with different input buffers to the subclass handle_frame callback. During this call, the subclass needs to take ownership of the input_buffer as GstVideoCodecFrame.input_buffer will have been changed before the next subframe buffer is received. The subclass will callgst_video_decoder_have_last_subframewhen a new input frame can be created by the base class. Every subframe will share the same GstVideoCodecFrame.output_buffer to write the decoding result. The subclass is responsible to protect its access.If codec processing results in decoded data, the subclass should call gst_video_decoder_finish_frame to have decoded data pushed downstream. In subframe mode the subclass should call gst_video_decoder_finish_subframe until the last subframe where it should call gst_video_decoder_finish_frame. The subclass can detect the last subframe using
GstVideo.VideoBufferFlags.MARKERon buffers or using its own logic to collect the subframes. In case of decoding failure, the subclass must call gst_video_decoder_drop_frame or gst_video_decoder_drop_subframe, to allow the base class to do timestamp and offset tracking, and possibly to requeue the frame for a later attempt in the case of reverse playback.
- Shutdown phase
The
GstVideo.VideoDecoderclass calls stop to inform the subclass that data parsing will be stopped.
- Additional Notes
Seeking/Flushing
When the pipeline is seeked or otherwise flushed, the subclass is informed via a call to its reset callback, with the hard parameter set to true. This indicates the subclass should drop any internal data queues and timestamps and prepare for a fresh set of buffers to arrive for parsing and decoding.
End Of Stream
At end-of-stream, the subclass parse function may be called some final times with the at_eos parameter set to true, indicating that the element should not expect any more data to be arriving, and it should parse and remaining frames and call
GstVideo.VideoDecoder.have_frame() if possible.
The subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named “sink” and “src”. It also needs to provide information about the output caps, when they are known. This may be when the base class calls the subclass’ set_format function, though it might be during decoding, before calling gst_video_decoder_finish_frame. This is done via gst_video_decoder_set_output_state
The subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, the base class provides limited framerate based interpolation.
Similarly, the base class provides some limited (legacy) seeking support if specifically requested by the subclass, as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates. To enable it, a subclass should call gst_video_decoder_set_estimate_rate to enable handling of incoming byte-streams.
The base class provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, the subclass should then be prepared for the parsing and frame processing stage to occur separately (in normal forward processing, the latter immediately follows the former), The subclass also needs to ensure the parsing stage properly marks keyframes, unless it knows the upstream elements will do so properly for incoming data.
The bare minimum that a functional subclass needs to implement is:
Provide pad templates
Inform the base class of output caps via gst_video_decoder_set_output_state
Parse input data, if it is not considered packetized from upstream Data will be provided to parse which should invoke gst_video_decoder_add_to_frame and gst_video_decoder_have_frame to separate the data belonging to each video frame.
Accept data in handle_frame and provide decoded results to gst_video_decoder_finish_frame, or call gst_video_decoder_drop_frame.
- add_to_frame(n_bytes)¶
- Parameters:
n_bytes (
int) – the number of bytes to add
Removes next n_bytes of input data and adds it to currently parsed frame.
- allocate_output_buffer()¶
- Returns:
allocated buffer, or
Noneif no buffer could be allocated (e.g. when downstream is flushing or shutting down)- Return type:
Gst.BufferorNone
Helper function that allocates a buffer to hold a video frame for self's current
GstVideo.VideoCodecState.You should use
GstVideo.VideoDecoder.allocate_output_frame() instead of this function, if possible at all.
- allocate_output_frame(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – aGstVideo.VideoCodecFrame- Returns:
Gst.FlowReturn.OKif an output buffer could be allocated- Return type:
Helper function that allocates a buffer to hold a video frame for self's current
GstVideo.VideoCodecState. Subclass should already have configured video state and set src pad caps.The buffer allocated here is owned by the frame and you should only keep references to the frame, not the buffer.
- allocate_output_frame_with_params(frame, params)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – aGstVideo.VideoCodecFrameparams (
Gst.BufferPoolAcquireParams) – aGst.BufferPoolAcquireParams
- Returns:
Gst.FlowReturn.OKif an output buffer could be allocated- Return type:
Same as
GstVideo.VideoDecoder.allocate_output_frameexcept it allows passingGst.BufferPoolAcquireParamsto the sub callGst.BufferPool.acquire_buffer.New in version 1.12.
- drop_frame(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – theGstVideo.VideoCodecFrameto drop- Returns:
a
Gst.FlowReturn, usuallyGst.FlowReturn.OK.- Return type:
Similar to
GstVideo.VideoDecoder.finish_frame(), but drops frame in any case and posts a QoS message with the frame’s details on the bus. In any case, the frame is considered finished and released.
- drop_subframe(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – theGstVideo.VideoCodecFrame- Returns:
a
Gst.FlowReturn, usuallyGst.FlowReturn.OK.- Return type:
Drops input data. The frame is not considered finished until the whole frame is finished or dropped by the subclass.
New in version 1.20.
- finish_frame(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – a decodedGstVideo.VideoCodecFrame- Returns:
a
Gst.FlowReturnresulting from sending data downstream- Return type:
frame should have a valid decoded data buffer, whose metadata fields are then appropriately set according to frame data and pushed downstream. If no output data is provided, frame is considered skipped. In any case, the frame is considered finished and released.
After calling this function the output buffer of the frame is to be considered read-only. This function will also change the metadata of the buffer.
- finish_subframe(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – theGstVideo.VideoCodecFrame- Returns:
a
Gst.FlowReturn, usuallyGst.FlowReturn.OK.- Return type:
Indicate that a subframe has been finished to be decoded by the subclass. This method should be called for all subframes except the last subframe where gst_video_decoder_finish_frame should be called instead.
New in version 1.20.
- get_allocator()¶
- Returns:
- allocator:
the
Gst.Allocatorused- params:
the
Gst.AllocationParamsof allocator
- Return type:
(allocator:
Gst.AllocatororNone, params:Gst.AllocationParams)
Lets
GstVideo.VideoDecodersub-classes to know the memory allocator used by the base class and its params.Unref the allocator after use it.
- get_buffer_pool()¶
- Returns:
the instance of the
Gst.BufferPoolused by the decoder; free it after use it- Return type:
- get_frame(frame_number)¶
- Parameters:
frame_number (
int) – system_frame_number of a frame- Returns:
pending unfinished
GstVideo.VideoCodecFrameidentified by frame_number.- Return type:
Get a pending unfinished
GstVideo.VideoCodecFrame
- get_frames()¶
- Returns:
pending unfinished
GstVideo.VideoCodecFrame.- Return type:
Get all pending unfinished
GstVideo.VideoCodecFrame
- get_input_subframe_index(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – theGstVideo.VideoCodecFrameto update- Returns:
the current subframe index received in subframe mode, 1 otherwise.
- Return type:
Queries the number of the last subframe received by the decoder baseclass in the frame.
New in version 1.20.
- get_latency()¶
- Returns:
- Return type:
Query the configured decoder latency. Results will be returned via min_latency and max_latency.
- get_max_decode_time(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – aGstVideo.VideoCodecFrame- Returns:
max decoding time.
- Return type:
Determines maximum possible decoding time for frame that will allow it to decode and arrive in time (as determined by QoS events). In particular, a negative result means decoding in time is no longer possible and should therefore occur as soon/skippy as possible.
- get_needs_format()¶
-
Queries decoder required format handling.
New in version 1.4.
- get_needs_sync_point()¶
-
Queries if the decoder requires a sync point before it starts outputting data in the beginning.
New in version 1.20.
- get_oldest_frame()¶
- Returns:
oldest pending unfinished
GstVideo.VideoCodecFrame.- Return type:
Get the oldest pending unfinished
GstVideo.VideoCodecFrame
- get_output_state()¶
- Returns:
GstVideo.VideoCodecStatedescribing format of video data.- Return type:
Get the
GstVideo.VideoCodecStatecurrently describing the output stream.
- get_packetized()¶
-
Queries whether input data is considered packetized or not by the base class.
- get_pending_frame_size()¶
- Returns:
The number of bytes pending for the current frame
- Return type:
Returns the number of bytes previously added to the current frame by calling
GstVideo.VideoDecoder.add_to_frame().New in version 1.4.
- get_processed_subframe_index(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – theGstVideo.VideoCodecFrameto update- Returns:
the current subframe processed received in subframe mode.
- Return type:
Queries the number of subframes in the frame processed by the decoder baseclass.
New in version 1.20.
- get_subframe_mode()¶
-
Queries whether input data is considered as subframes or not by the base class. If
False, each input buffer will be considered as a full frame.New in version 1.20.
- have_frame()¶
- Returns:
- Return type:
Gathers all data collected for currently parsed frame, gathers corresponding metadata and passes it along for further processing, i.e. handle_frame.
- have_last_subframe(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – theGstVideo.VideoCodecFrameto update- Returns:
a
Gst.FlowReturn, usuallyGst.FlowReturn.OK.- Return type:
Indicates that the last subframe has been processed by the decoder in frame. This will release the current frame in video decoder allowing to receive new frames from upstream elements. This method must be called in the subclass handle_frame callback.
New in version 1.20.
- merge_tags(tags, mode)¶
- Parameters:
tags (
Gst.TagListorNone) – aGst.TagListto merge, orNoneto unset previously-set tagsmode (
Gst.TagMergeMode) – theGst.TagMergeModeto use, usuallyGst.TagMergeMode.REPLACE
Sets the audio decoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_audio_decoder_merge_tags().
Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.
MT safe.
- negotiate()¶
-
Negotiate with downstream elements to currently configured
GstVideo.VideoCodecState. UnmarkGst.PadFlags.NEED_RECONFIGUREin any case. But mark it again if negotiate fails.
- proxy_getcaps(caps, filter)¶
- Parameters:
- Returns:
a
Gst.Capsowned by caller- Return type:
Returns caps that express caps (or sink template caps if caps ==
None) restricted to resolution/format/… combinations supported by downstream elements.New in version 1.6.
- release_frame(frame)¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – theGstVideo.VideoCodecFrameto release
Similar to
GstVideo.VideoDecoder.drop_frame(), but simply releases frame without any processing other than removing it from list of pending frames, after which it is considered finished and released.New in version 1.2.2.
- request_sync_point(frame, flags)¶
- Parameters:
Allows the
GstVideo.VideoDecodersubclass to request from the base class that a new sync should be requested from upstream, and that frame was the frame when the subclass noticed that a new sync point is required. A reason for the subclass to do this could be missing reference frames, for example.The base class will then request a new sync point from upstream as long as the time that passed since the last one is exceeding
GstVideo.VideoDecoder:min-force-key-unit-interval.The subclass can signal via flags how the frames until the next sync point should be handled:
If
GstVideo.VideoDecoderRequestSyncPointFlags.DISCARD_INPUTis selected then all following input frames until the next sync point are discarded. This can be useful if the lack of a sync point will prevent all further decoding and the decoder implementation is not very robust in handling missing references frames.If
GstVideo.VideoDecoderRequestSyncPointFlags.CORRUPT_OUTPUTis selected then all output frames following frame are marked as corrupted viaGst.BufferFlags.CORRUPTED. Corrupted frames can be automatically dropped by the base class, seeGstVideo.VideoDecoder:discard-corrupted-frames. Subclasses can manually mark frames as corrupted viaGstVideo.VideoCodecFrameFlags.CORRUPTEDbefore callingGstVideo.VideoDecoder.finish_frame().
New in version 1.20.
- set_estimate_rate(enabled)¶
- Parameters:
enabled (
bool) – whether to enable byte to time conversion
Allows baseclass to perform byte to time estimated conversion.
- set_interlaced_output_state(fmt, interlace_mode, width, height, reference)¶
- Parameters:
fmt (
GstVideo.VideoFormat) – aGstVideo.VideoFormatinterlace_mode (
GstVideo.VideoInterlaceMode) – AGstVideo.VideoInterlaceModewidth (
int) – The width in pixelsheight (
int) – The height in pixelsreference (
GstVideo.VideoCodecStateorNone) – An optional referenceGstVideo.VideoCodecState
- Returns:
the newly configured output state.
- Return type:
Same as
GstVideo.VideoDecoder.set_output_state() but also allows you to also set the interlacing mode.New in version 1.16..
- set_latency(min_latency, max_latency)¶
-
Lets
GstVideo.VideoDecodersub-classes tell the baseclass what the decoder latency is. If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.
- set_max_errors(num)¶
- Parameters:
num (
int) – max tolerated errors
Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. You can set -1 for never returning fatal errors. Default is set to
GstVideo.VIDEO_DECODER_MAX_ERRORS.The ‘-1’ option was added in 1.4
- set_needs_format(enabled)¶
- Parameters:
enabled (
bool) – new state
Configures decoder format needs. If enabled, subclass needs to be negotiated with format caps before it can process any data. It will then never be handed any data before it has been configured. Otherwise, it might be handed data without having been configured and is then expected being able to do so either by default or based on the input data.
New in version 1.4.
- set_needs_sync_point(enabled)¶
- Parameters:
enabled (
bool) – new state
Configures whether the decoder requires a sync point before it starts outputting data in the beginning. If enabled, the base class will discard all non-sync point frames in the beginning and after a flush and does not pass it to the subclass.
If the first frame is not a sync point, the base class will request a sync point via the force-key-unit event.
New in version 1.20.
- set_output_state(fmt, width, height, reference)¶
- Parameters:
fmt (
GstVideo.VideoFormat) – aGstVideo.VideoFormatwidth (
int) – The width in pixelsheight (
int) – The height in pixelsreference (
GstVideo.VideoCodecStateorNone) – An optional referenceGstVideo.VideoCodecState
- Returns:
the newly configured output state.
- Return type:
Creates a new
GstVideo.VideoCodecStatewith the specified fmt, width and height as the output state for the decoder. Any previously set output state on self will be replaced by the newly created one.If the subclass wishes to copy over existing fields (like pixel aspec ratio, or framerate) from an existing
GstVideo.VideoCodecState, it can be provided as a reference.If the subclass wishes to override some fields from the output state (like pixel-aspect-ratio or framerate) it can do so on the returned
GstVideo.VideoCodecState.The new output state will only take effect (set on pads and buffers) starting from the next call to
GstVideo.VideoDecoder.finish_frame().
- set_packetized(packetized)¶
- Parameters:
packetized (
bool) – whether the input data should be considered as packetized.
Allows baseclass to consider input data as packetized or not. If the input is packetized, then the parse method will not be called.
- set_subframe_mode(subframe_mode)¶
- Parameters:
subframe_mode (
bool) – whether the input data should be considered as subframes.
If this is set to
True, it informs the base class that the subclass can receive the data at a granularity lower than one frame.Note that in this mode, the subclass has two options. It can either require the presence of a
GstVideo.VideoBufferFlags.MARKERto mark the end of a frame. Or it can operate in such a way that it will decode a single frame at a time. In this second case, every buffer that arrives to the element is considered part of the same frame untilGstVideo.VideoDecoder.finish_frame() is called.In either case, the same
GstVideo.VideoCodecFramewill be passed to the GstVideoDecoderClass:handle_frame vmethod repeatedly with a different GstVideoCodecFrame:input_buffer every time until the end of the frame has been signaled using either method. This method must be called during the decoder subclass set_format call.New in version 1.20.
- set_use_default_pad_acceptcaps(use)¶
- Parameters:
use (
bool) – if the default pad accept-caps query handling should be used
Lets
GstVideo.VideoDecodersub-classes decide if they want the sink pad to use the default pad query handler to reply to accept-caps queries.By setting this to true it is possible to further customize the default handler with %GST_PAD_SET_ACCEPT_INTERSECT and %GST_PAD_SET_ACCEPT_TEMPLATE
New in version 1.6.
- do_drain() virtual¶
- Return type:
- do_finish() virtual¶
- Return type:
- do_handle_frame(frame) virtual¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) – The frame to handle- Return type:
- do_handle_missing_data(timestamp, duration) virtual¶
- Parameters:
- Returns:
Trueif the decoder should be drained afterwards.- Return type:
New in version 1.20.
- do_negotiate() virtual¶
-
Negotiate with downstream elements to currently configured
GstVideo.VideoCodecState. UnmarkGst.PadFlags.NEED_RECONFIGUREin any case. But mark it again if negotiate fails.
- do_parse(frame, adapter, at_eos) virtual¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) –adapter (
GstBase.Adapter) –at_eos (
bool) –
- Return type:
- do_set_format(state) virtual¶
- Parameters:
state (
GstVideo.VideoCodecState) –- Return type:
- do_transform_meta(frame, meta) virtual¶
- Parameters:
frame (
GstVideo.VideoCodecFrame) –meta (
Gst.Meta) –
- Return type:
Property Details¶
- GstVideo.VideoDecoder.props.automatic_request_sync_point_flags¶
- Name:
automatic-request-sync-point-flags- Type:
- Default Value:
GstVideo.VideoDecoderRequestSyncPointFlags.DISCARD_INPUT|GstVideo.VideoDecoderRequestSyncPointFlags.CORRUPT_OUTPUT- Flags:
GstVideo.VideoDecoderRequestSyncPointFlagsto use for the automatically requested sync points ifautomatic-request-sync-pointsis enabled.New in version 1.20.
- GstVideo.VideoDecoder.props.automatic_request_sync_points¶
-
If set to
Truethe decoder will automatically request sync points when it seems like a good idea, e.g. if the first frames are not key frames or if packet loss was reported by upstream.New in version 1.20.
- GstVideo.VideoDecoder.props.discard_corrupted_frames¶
-
If set to
Truethe decoder will discard frames that are marked as corrupted instead of outputting them.New in version 1.20.
- GstVideo.VideoDecoder.props.max_errors¶
-
Maximum number of tolerated consecutive decode errors. See
GstVideo.VideoDecoder.set_max_errors() for more details.New in version 1.18.
- GstVideo.VideoDecoder.props.min_force_key_unit_interval¶
-
Minimum interval between force-key-unit events sent upstream by the decoder. Setting this to 0 will cause every event to be handled, setting this to
Gst.CLOCK_TIME_NONEwill cause every event to be ignored.See
GstVideo.video_event_new_upstream_force_key_unit() for more details about force-key-unit events.New in version 1.20.