rustaudio/coreaudio-rs

A friendly rust interface to Apple's Core Audio API.

coreaudio-rs's Issues

Device input channels incorrectly reported in StreamFormat.

Hi,

I've noticed a device's default number of input channels isn't correctly reported when querying StreamFormat. The number of output channels on the other hand is correctly reported.

You can reproduce this behaviour by running:

use coreaudio::audio_unit::macos_helpers::{audio_unit_from_device_id, get_device_id_from_name};

use std::env;

fn main() {
    let args: Vec<String> = env::args().collect();
    let device = &args[1];
    let au_in = audio_unit_from_device_id(get_device_id_from_name(device).unwrap(), true).unwrap();
    let au_out =
        audio_unit_from_device_id(get_device_id_from_name(device).unwrap(), false).unwrap();

    println!(
        "{} Input {:?}",
        device,
        au_in.input_stream_format().unwrap()
    );
    println!(
        "{} Output {:?}",
        device,
        au_out.output_stream_format().unwrap()
    );
}

Running with UMC1820 (10 in, 12 out) returns:

UMC1820 Input StreamFormat { sample_rate: 44100.0, sample_format: F32, flags: IS_FLOAT | IS_PACKED | IS_NON_INTERLEAVED, channels: 2 }
UMC1820 Output StreamFormat { sample_rate: 44100.0, sample_format: F32, flags: IS_FLOAT | IS_PACKED, channels: 12 }

I'm not sure whether this is an issue with the library or CoreAudio itself, so any help with this would be appreciated!

Rename crate from 'coreaudio-rs' to 'coreaudio'

Every crate on crates.io is for Rust 😄 no reason for the -rs suffix.

Remove unnecessary Vec allocation in `extern "C" fn input_proc`

There should definitely be no allocations occurring in the realtime audio callback.

It should be possible to change this array of channels from a Vec to a slice, using from_raw_parts_mut instead.

Function for returning an Iterator of device ID's along with details

Now that the `AudioUnit` type is not a builder type, perhaps use "getters" and "setters" for clarity.

Might make sense to rename the following:

• render_callback -> set_render_callback
• sample_rate -> get_sample_rate
• stream_format -> get_stream_format

Separate up the audio unit `SubType` variants into specific enums associated with each `Type` variant.

This could significantly improve the ergonomics and safety of the AudioUnit::new API.

Rather than taking both a Type and SubType, it should take any T: Into<Type>, where:

• Type's variants hold their own sub type data
• From<$subtype> (for every $subtype) is implemented for Type

This way a user can just pass any single type that can be automatically converted to a Type.

[QUESTIONs] can rust & more so these libraries create audio units for macOS

howdy people,

I cant seem to find a forum post on this & really the questions I have. id rather ask here first before signing up to another forum thing.

ive been interested in audio unit & development for years. but finding the right out look to use is another. I do have apple old audio unit workflow, but personally I dont like c++. though have had things working & getting things made to a point.
ive seen things for the D programming language with the awesome looking Dplug - but I feel rust as a language would suit me more.

I mainly do game dev stuff, but am actually a musician & developer of music stuffs. I use haxe/kha for game dev & do GLSL stuff

but since ive seen this repo plus the libraries for rust audio stuff. its interested me more to go with rust as a language to learn & take on board to develop AUs/VSTs

but the main question[s] are:
• can rust build on macOS a AU [audio unit] - if so, is there any examples or tutorials
• are there any examples of VSTs/AUs to look at. I cant seem to find much at all.
• is there an IRC - so I could ask more questions

please excuse that this isn't an issue per-se. but id like to ask these just before I sign up

thanks & all the best

set_input_callback does not resize the AudioBufferList buffer correctly

Hi!

I'm experiencing an issue that occurs when the number of samples provided by the callback set using "kAudioOutputUnitProperty_SetInputCallback" in AudioUnit.set_input_callback increases.

On my device, this happens whenever a bluetooth headset is selected as the current audio device.

When switching from the speaker or earpeice (which provide 1024 samples), to a bluetooth headset, the change in the number of samples is detected and the buffer is resized using "vec.reserve_exact(data_byte_size - len);"

However, the mDataByteSize property of the buffer is not updated to reflect this change. This results in the framework printing the error:
3072 frames, 2 bytes/frame, expected 6144-byte buffer; ioData.mBuffers[0].mDataByteSize=2048; kAudio_ParamError

I tried updating this value and API worked for a few seconds before crashing with memory corruption.
e.g. malloc: Incorrect checksum for freed object 0x10700b800: probably modified after being freed.

I'm unsure how the corruption is occurring, and for now I have worked around this in our project by allocating a new buffer when the number of samples changes.

Any thoughts?

Thanks

What is the aim of this library?

In my project I need mode direct access to the CoreAudio api that CPAL provides, so I'm considering using this crate.
But after looking closer I'm not so sure.
There are several open PRs that look useful, but haven't gotten any response despite being open for a very long time. The changes that do happen appear to be closely coupled to the needs of CPAL. There are also the common task of switching sample rate, where the low-level stuff is in CPAL rather than here (#80).

My question then is this, sorry if it's a bit blunt.
Is this library meant to cover the needs for general CoreAudio use, or is it just the CoreAudio backend for CPAL?

Move `audio_format` module out of `audio_unit` module and into the top-level.

ABSD and the associated format and flags seems to be a Core Audio wide concept. Audio units however are only compatible with the linear PCM format, so it doesn't matter to host the audio_format module as a child of audio_unit.

Support configuring AVSession

Seems like iOS 17+ requires configuring an AVSession with an AVAudioSessionCategoryPlayback

(Possibly) Related issues:

• RustAudio/cpal#862
• RustAudio/cpal#842
• NiklasEi/bevy_kira_audio#118

It's unclear if a fix should go in coreaudio-rs or in coreaudio-sys.

Input Device is not starting in Feedback Examples

In both feedback examples the input device callback is not getting called, although in the same terminal other audio applications get audio input (so it should have microphone access rights).

MacBook Pro 14-inch, 2021
MacOS Ventura 13.2.1

Can anyone reproduce this?

Don't use "pub extern crate"

As seen in this line from the recent nightly builds on travis, pub extern crate shouldn't be used, and we should use extern crate and pub use instead.

Builder method for setting the stream format

Fix or determine why the macOS-14 github runner fails to run sine example.

In #110, I tried to add the sine example as CI and then used the macOS-14 github runner and got:

    Finished test [unoptimized + debuginfo] target(s) in 15.54s
     Running `/Users/runner/work/coreaudio-rs/coreaudio-rs/target/debug/deps/coreaudio-9dc00e3150a190d0`

running 2 tests
test audio_unit::macos_helpers::test_get_audio_device_ids ... ok
     Running `/Users/runner/work/coreaudio-rs/coreaudio-rs/target/debug/deps/sine-cb5e40f7142557a6`
test audio_unit::macos_helpers::test_get_audio_device_ids_for_scope ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s


running 1 test
test sine has been running for over 60 seconds
libunwind: malformed __unwind_info at 0x1A1335964 bad second level page
libunwind: malformed __unwind_info at 0x19EB3AAAC bad second level page
libunwind: malformed __unwind_info at 0x19EB3AAAC bad second level page
libunwind: malformed __unwind_info at 0x19EB3AAAC bad second level page
libunwind: malformed __unwind_info at 0x19EB3AAAC bad second level page

But with many many more lines of unwinde_info at...

`AudioUnit::set_sample_rate` should be `&mut self` rather than `&self` for API clarity.

Although it's not technically required, a mutation is being performed on the underlying AudioUnit that our type represents, and this will make that clearer and improve safety.

Work out if it is possible to implement `Clone` for `AudioUnit` in a sensible manner.

I'm not sure if the CoreAudio API provides a safe way of cloning AudioUnits, but this might be nice to have from a usability point of view.

AudioUnits should only be clone if their RenderCallback is also Clone.

Add proper support for handling different sample types and stream formats at runtime.

My favourite solution to this is using something like CPAL's UnknownTypeBuffer has an intermediary.

Documentation failing to build for docs.rs

Despite specifying the macos target for documentation to build in our Cargo.toml, the build will still fail when building for docs.rs. This is because docs.rs attempts to build for macOS via cross-compilation from Linux. This should be first solved in coreaudio-sys before attempting to fix coreaudio-rs.

Related issues / PRs:

• RustAudio/coreaudio-sys#40
• RustAudio/coreaudio-sys#38
• rust-lang/docs.rs#853

Return a `Result<StreamFormat, Error>` instead of `panic!`ing in `StreamFormat::from_asbd`.

Remove warning from example on nightly

https://travis-ci.org/RustAudio/coreaudio-rs/jobs/85088606#L90

Please explicitly update coreaudio-sys

Hello!

This repo currently relies upon coreaudio-sys, and just lists version "0.2". My Cargo is defaulting that to "0.2.2", but "0.2.3" of coreaudio-sys has upgraded its bindgen version to the much more recent v0.5.0. 0.2.2 however, is still stuck on 0.32.3.

This is a problem as I have another package which uses a more recent bindgen, and cargo can't let me use both packages at once. For now, I essentially can't use this package, cpal, or rodio.

Here's the error message cargo gives me incase there's a simpler solution for me personally:

error: multiple packages link to native library `clang`, but a native library can be linked only once

package `clang-sys v0.21.2`
    ... which is depended on by `bindgen v0.32.3`
    ... which is depended on by `coreaudio-sys v0.2.2`
    ... which is depended on by `coreaudio-rs v0.9.1`
    ... which is depended on by `cpal v0.10.0`
    ... which is depended on by `rodio v0.10.0`
    ... which is depended on by `clockwork v0.1.0 (/Users/jjspira/Documents/Rust/clockwork)`
links to native library `clang`

package `clang-sys v0.26.4`
    ... which is depended on by `bindgen v0.47.3`
    ... which is depended on by `rustcord-sys v0.2.2`
    ... which is depended on by `rustcord v0.2.2`
    ... which is depended on by `clockwork v0.1.0 (/Users/jjspira/Documents/Rust/clockwork)`
also links to native library `clang`

Thanks guys!

Split separate AudioUnit types into their own modules/types

Is it possible to create audiounit cdylibs with this library?

Sorry if this issue is out of place, but I had no idea where else to ask this question. I've been working on rust-vst2 (vst bindings for rust) and ddcore (a rust vst gui lib) for some time now and would like to assess what kind of work is involved in doing a similar thing for audiounit. With vst, simply exporting the right extern functions and compiling as a cdylib, packaging into a bundle and placing it in your plugins directory is enough to start a working vst plugin. I assume that au is similar, but am unsure where to start.

I have noticed some audiounits and vsts out there actually share a binary, (eg same binary, just copied into different bundles, .component and .vst) and have extern entry points for both vst and audiounit in the one binary. This also gives me some hope it might be possible to unify the vst and au work.

Does anyone have any clarity on this?

Update coreaudio-sys version please

The old bindgen cause so many conflicts.

Clarify formats when working with input/output scopes on an AudioUnit's input/output elements.

Each audio unit actually has 4 formats:

• Input and output scope formats for the input element.
• Input and output scope formats for the output element.

#47 is related to this but perhaps not general enough in its approach to solving this.

Perhaps we should change stream_format to always require taking the Scope and Element as arguments? cc @plietar

Test more feature permutations in CI workflow

Currently, features are used to gate compilation of support for different frameworks.

So far, only the default feature set is tested in CI. We should ensure that compilation succeeds under any combination of features, and at least test both --no-default-features and --all-features in the CI workflow.

Expose the AudioUnitSet/GetProperty functions as nice methods on the AudioUnit type.

We will only be able to make this so nice, however I think it is necessary to really give users proper control and thorough access to their audio units.

kAudioDevicePropertyBufferFrameSize doesn't exist for iOS.

Need a way to to figure out the kAudioDevicePropertyBufferFrameSize for iOS. I suspect you need to include AVAudioSession which is iOS only until Catalina in which case it becomes cross platform.

Use from_raw and into_raw instead of transmuting the box for callback ownership

I saw this forum post today, and it's mentioned that using Box's from_raw and into_raw to go to/from a c-pointer is preferred to using transmute. I'm guessing if nothing else this is to make it more explicit what's going on.

Try and remove `Box`ing wherever possible within the render_callback builder method.

There's a lot of boxing going on - all of which should be theoretically unnecessary given that we're sharing the stack with the C code, not sending stuff to it.

It might be easier to address this once the Fn/FnMut/FnOnce trait implementations are stabilised.

Bug With Remote IO Unit or Misuse?

Core Audio noob here. I'm on a 2012 MacBook Pro with working audio, and I'm trying to use the Remote IO audio unit. Calling AudioUnit::new(IOType::RemoteIO) returns aNoMatchingDefaultDeviceFound error. Am I doing something obviously wrong here?

Thanks in advance.

Proper stream type bitflags

Ran into issues with these implementing/testing a fix for #12. They need to be proper bitflags so they can be combined and all valid values can be represented.

Stop using monolithic `Error` enum and use error types specific to each part of the API

Although convenient for us, the biggest problem with using the same Error type everywhere is that rather than only representing the possible errors that may occur for each particular part of the API, it enumerates all errors, making it difficult for the user to determine what errors might actually occur when using certain parts of the API. Each method/function should have it's own error type which only enumerates the actual possible errors. This will be tricky as the coreaudio documentation rarely enumerates the possible errors that might actually occur, and in most places just returns a generic error code.

Link to documentation broken

No input in `feedback` and `feedback_interleaved`

Summary

So the feedback examples from 0.12.0 are not working - there's no audio data captured (mic device is opened and there's an orange dot indicator in macOS status bar):

1. Create new project with cargo new --bin
2. Add coreaudio-rs = "0.12.0" to dependencies in Cargo.toml
3. Add https://github.com/RustAudio/coreaudio-rs/blob/b671130aca0e3eef25e194e2bfa66a8ad58a8829/examples/feedback.rs or https://github.com/RustAudio/coreaudio-rs/blob/b671130aca0e3eef25e194e2bfa66a8ad58a8829/examples/feedback_interleaved.rs to src/main.rs
4. You'll get all the output cb {} frames and no input cb {} frames logs

Info

• macOS: 14.5 (23F79)
• coreaudio-rs: 0.12.0

Support for switching sample rate

I'm considering using this library for implementing CoreAudio support in my project.
I need to be able to switch the sample rate, and looking at how to do that I found this: https://github.com/RustAudio/cpal/blob/master/src/host/coreaudio/macos/mod.rs#L536

That's a lot of code needed for a quite common operation. Shouldn't this be included in coreaudio-rs?