Playing simultaneous tracks with media 3

[zoomGitS]

Playing simultaneous tracks with media 3

[tonihei]

If you are referring to comment like this one, I think the approach should still work as the underlying code hasn't changed in any significant ways since.

There is also ongoing work to allow proper audio mixing for playback of multiple tracks, but this may not be available soon.

Could you clarify in more detail what exactly isn't working and maybe we can point you in the right direction?

[zoomGitS]

//initilize

   private val exoPlayer by lazy {
        ExoPlayer.Builder(this, renderersFactory).setTrackSelector(trackSelector).build()
    }

//create media sources

val audioThreeFile = File(File(filesDir, "audio_test"), "audio_one_trimed.mp3")
        val audioFourFile = File(File(filesDir, "audio_test"), "auudio_two_trimed.mp3")

        val defaultDataSourceFactory = DefaultDataSource.Factory(applicationContext)

        val sourceOne = ProgressiveMediaSource.Factory(defaultDataSourceFactory)
            .createMediaSource(MediaItem.fromUri(Uri.fromFile(audioThreeFile)))
        val sourceTwo = ProgressiveMediaSource.Factory(defaultDataSourceFactory)
            .createMediaSource(MediaItem.fromUri(Uri.fromFile(audioFourFile)))

        val mergingMediaSource = MergingMediaSource(true, sourceOne, sourceTwo)


        exoPlayer.setMediaSource(mergingMediaSource)
        exoPlayer.prepare()
        exoPlayer.playWhenReady = true

//create renderersFactory

private val renderersFactory = object : DefaultRenderersFactory(this) {
        override fun buildAudioRenderers(
            context: Context,
            extensionRendererMode: Int,
            mediaCodecSelector: MediaCodecSelector,
            enableDecoderFallback: Boolean,
            audioSink: AudioSink,
            eventHandler: Handler,
            eventListener: AudioRendererEventListener,
            out: ArrayList<Renderer>
        ) {
            super.buildAudioRenderers(
                context,
                extensionRendererMode,
                mediaCodecSelector,
                enableDecoderFallback,
                audioSink,
                eventHandler,
                eventListener,
                out
            )
            out.add(AudioRendererWithoutClock(context, mediaCodecSelector))
            out.add(AudioRendererWithoutClock(context, mediaCodecSelector))
            out.add(AudioRendererWithoutClock(context, mediaCodecSelector))
        }
    }

//create trackSelector

private val trackSelector = object : TrackSelector() {
        override fun selectTracks(
            rendererCapabilities: Array<out RendererCapabilities>,
            trackGroups: TrackGroupArray,
            periodId: MediaSource.MediaPeriodId,
            timeline: Timeline
        ): TrackSelectorResult {
            val audioRenderers: Queue<Int> = ArrayDeque()
            val configs = arrayOfNulls<RendererConfiguration>(rendererCapabilities.size)
            val selections = arrayOfNulls<ExoTrackSelection>(rendererCapabilities.size)
            for (i in rendererCapabilities.indices) {
                if (rendererCapabilities[i].trackType == C.TRACK_TYPE_AUDIO) {
                    audioRenderers.add(i)
                    configs[i] = RendererConfiguration.DEFAULT
                }
            }
            for (i in 0 until trackGroups.length) {
                if (MimeTypes.isAudio(trackGroups[i].getFormat(0).sampleMimeType)) {
                    val index = audioRenderers.poll()
                    if (index != null) {
                        selections[index] = FixedTrackSelection(trackGroups[i], 0)
                    }
                }
            }
            return TrackSelectorResult(configs, selections, Any())
        }

        override fun onSelectionActivated(info: Any?) {

        }
    }

//create AudioRenderer

final class AudioRendererWithoutClock extends MediaCodecAudioRenderer {
    public AudioRendererWithoutClock(Context context, MediaCodecSelector mediaCodecSelector) {
        super(context, mediaCodecSelector);
    }

    @Override
    public MediaClock getMediaClock() {
        return null;
    }
}

But getting this error-

[zoomGitS]

Maybe the issue lies in the RenderersFactory where I am adding three new AudioRendererWithoutClock instances. However, I actually only need one additional AudioRendererWithoutClock to render the second audio.

[tonihei]

Thanks for highlighting the problem! This is indeed related to the additional renderers. Having more renderers than needed is not usually an issue, but there is a bug in the TrackSelector code I provided in the GitHub issue. The configs[i] = RendererConfiguration.DEFAULT line needs to move further down to where we decide which renderer to use. I fixed my original post in google/ExoPlayer#6589 (comment) too to avoid further confusion. Note that this is still just an example though and it wouldn't handle other renderer types like video.