Issues with BT and FFMpeg extension

[Tolriq]

Issues with BT and FFMpeg extension

[tonihei]

Config(48000, 12, 1152000)

This looks like it might be failing because the audio track buffer size is too big (> 1MB). The suppression also indicates the DefaultAudioSink automatically retried with a 1MB buffer but still failed.

As far as I see, the error code "-12" from AudioTrack also means NO_MEMORY, pointing in the same direction.

Are there any other tracks playing at the same time or any other sort of memory pressure? Alternatively, you could retry with an even lower audio track buffer somehow, but it feels like if a 1MB buffer is failing then slightly smaller buffers might fail too.

[Tolriq]

@tonihei not using ffmpeg and not making any other change does fix it for the user.

(I have an option that just change the order of the factory)

This is the same for the other linked issue, not using ffmpeg fixes it.

Seems there's something different the ffmpeg does that can impact BT on some devices but I'm clueless.

If you have ideas of hidden stuff on the extension I can try that last users is more cooperative so I might get him to run tests.

[tonihei]

The only reason I can imagine at the moment is that using FFmpeg causes too much memory to be used, leaving not enough memory for the AudioTrack. I'm not sure if this is a reasonable hypothesis, but it could be related to the fact that both need to allocate native memory outside of the Java layer.

[Tolriq]

That would not explain the other issue that looks similar, I'm currently trying to read code and add some logs on my custom build to help move on those as it's a pain for my users.

I've triple checked and both paths are using the exact same sink:

val audioSink = DefaultAudioSink.Builder(context)
            .setAudioProcessorChain(
                DefaultAudioSink.DefaultAudioProcessorChain(
                    SilenceSkippingAudioProcessor(),
                    SonicAudioProcessor(),
                    replayGainProcessor,
                ),
            )
            .setEnableAudioTrackPlaybackParams(true)
            .setEnableFloatOutput(hires)
            .build()
        if (haveRApi) {
            audioSink.setOffloadMode(offloadMode)
        }

It's just a matter of the order of creation to pass from one case to the other

override fun createRenderers(
        eventHandler: Handler,
        videoRendererEventListener: VideoRendererEventListener,
        audioRendererEventListener: AudioRendererEventListener,
        textRendererOutput: TextOutput,
        metadataRendererOutput: MetadataOutput,
    ): Array<Renderer> {
        return if (preferInternalDecoder) {
            arrayOf(
                getFfmpegRenderer(eventHandler, audioRendererEventListener),
                getDefaultRenderer(eventHandler, audioRendererEventListener),
                MetadataRenderer(metadataRendererOutput, eventHandler.looper),
            )
        } else {
            arrayOf(
                getDefaultRenderer(eventHandler, audioRendererEventListener),
                getFfmpegRenderer(eventHandler, audioRendererEventListener),
                MetadataRenderer(metadataRendererOutput, eventHandler.looper),
            )
        }
    }

Reading internals I think the real cause may be hidden and those devices returns a wrong error code and it's not an issue with memory but just some parameters passed to the sink that will then generate different AudioTrack settings.

Specially since both DecoderAudioRenderer and MediaCodedcAudioRenderer do not create exactly the same the outputFormat passed down.

To be sure I'll add some debug logs at start of DefaultAudioSink.configure() to log the exact full params and see if there's differences with and without ffmpeg for that user.

Do you think about something else that could be useful?

And @tonihei since this is really low level, can you clarify if the AudioTrack itself is somehow related to BT and how? Or it's it's before and then the fact that playing outside BT is working gives another clue. Like a missing event somewhere that prevent releasing the AudioTrack.

[tonihei]

To be sure I'll add some debug logs at start of DefaultAudioSink.configure() to log the exact full params and see if there's differences with and without ffmpeg for that user.

That sounds useful to check.

can you clarify if the AudioTrack itself is somehow related to BT and how

AudioTrack is related to BT in that it handles the entire output path of the audio data. So it will behave differently depending the path and also the capabilities of the output device will be different.