From Buzztard

Latency describes the time a signal is delayed due to processing. In GStreamer we need to deal with audio and video latency.

[edit] basic approach

Audio gets delayed by every element in the pipeline by the play-time of the buffer to process.

Example:

• Samplingrate: 44100 Hz
• Buffersize: 22050 Hz
• Delay per element: 22050 / 44100 = 0.5 seconds

For video it is easier, as pipeline elements usualy process one field (frame or halr-frame) per invocation. Thus video gets delayed by the show-time of the field.

Example:

• Videorate: 25 fps (PAL interlaced)
• Delay per element: 1/50 seconds

Lower bound of latency is definitely the size of the audio-sinks ringbuffer. Not sure if upper-bound is twice that size (realtime filling of the ringbuffer).

[edit] more accurate approach

The latency as described above can be determined by iterating over the plugin graph and summing up element latencies. The problem here is elements that use ring-buffers internaly and therewith cause extra latency. One idea here would be to use a GstQuery to ask the element about the latency. Elements that do not cause extra latency don't need to implement this query.

[edit] latency compensation

We might need delay elements (based on ringbuffers) to equalize different latencies in different element-graph-paths.

Example: One uses several elements to process the audio part of a video stream. Then the audio would lag behind of the video. In this case video needs to be delayed slightly to match the audio again.