MediaSync Class

Definition

[Android.Runtime.Register("android/media/MediaSync", ApiSince=23, DoNotGenerateAcw=true)]
public sealed class MediaSync : Java.Lang.Object

[<Android.Runtime.Register("android/media/MediaSync", ApiSince=23, DoNotGenerateAcw=true)>]
type MediaSync = class
    inherit Object

Inheritance

Object

JavaObject

Object

MediaSync

Attributes

RegisterAttribute

Remarks

MediaSync class can be used to synchronously play audio and video streams. It can be used to play audio-only or video-only stream, too.

MediaSync is generally used like this:

MediaSync sync = new MediaSync();
            sync.setSurface(surface);
            Surface inputSurface = sync.createInputSurface();
            ...
            // MediaCodec videoDecoder = ...;
            videoDecoder.configure(format, inputSurface, ...);
            ...
            sync.setAudioTrack(audioTrack);
            sync.setCallback(new MediaSync.Callback() {
                {@literal @Override}
                public void onAudioBufferConsumed(MediaSync sync, ByteBuffer audioBuffer, int bufferId) {
                    ...
                }
            }, null);
            // This needs to be done since sync is paused on creation.
            sync.setPlaybackParams(new PlaybackParams().setSpeed(1.f));

            for (;;) {
              ...
              // send video frames to surface for rendering, e.g., call
              // videoDecoder.releaseOutputBuffer(videoOutputBufferIx, videoPresentationTimeNs);
              // More details are available as below.
              ...
              sync.queueAudio(audioByteBuffer, bufferId, audioPresentationTimeUs); // non-blocking.
              // The audioByteBuffer and bufferId will be returned via callback.
              // More details are available as below.
              ...
                ...
            }
            sync.setPlaybackParams(new PlaybackParams().setSpeed(0.f));
            sync.release();
            sync = null;

            // The following code snippet illustrates how video/audio raw frames are created by
            // MediaCodec's, how they are fed to MediaSync and how they are returned by MediaSync.
            // This is the callback from MediaCodec.
            onOutputBufferAvailable(MediaCodec codec, int bufferId, BufferInfo info) {
                // ...
                if (codec == videoDecoder) {
                    // surface timestamp must contain media presentation time in nanoseconds.
                    codec.releaseOutputBuffer(bufferId, 1000 * info.presentationTime);
                } else {
                    ByteBuffer audioByteBuffer = codec.getOutputBuffer(bufferId);
                    sync.queueAudio(audioByteBuffer, bufferId, info.presentationTime);
                }
                // ...
            }

            // This is the callback from MediaSync.
            onAudioBufferConsumed(MediaSync sync, ByteBuffer buffer, int bufferId) {
                // ...
                audioDecoder.releaseBuffer(bufferId, false);
                // ...
            }

The client needs to configure corresponding sink by setting the Surface and/or AudioTrack based on the stream type it will play.

For video, the client needs to call #createInputSurface to obtain a surface on which it will render video frames.

For audio, the client needs to set up audio track correctly, e.g., using AudioTrack#MODE_STREAM. The audio buffers are sent to MediaSync directly via #queueAudio, and are returned to the client via Callback#onAudioBufferConsumed asynchronously. The client should not modify an audio buffer till it's returned.

The client can optionally pre-fill audio/video buffers by setting playback rate to 0.0, and then feed audio/video buffers to corresponding components. This can reduce possible initial underrun.

Java documentation for android.media.MediaSync.

Portions of this page are modifications based on work created and shared by the Android Open Source Project and used according to terms described in the Creative Commons 2.5 Attribution License.