lvsti/coremediaio-dal-example

Apple's CoreMediaIO DAL plugin example - modernized

CoreMediaIO-DAL-Example

Modernized and extended version of Apple's CoreMediaIO sample code.

Before you jump right in

Apple announced the deprecation of CMIO DAL plugins at WWDC 2022, sunsetting them definitely in the next major release after macOS 13 (that is, in macOS 14). As a replacement, Camera Extensions have been introduced in macOS 12.3 which are now the preferred way to develop camera plugins. If you are about to start development, do yourself a favor: drop support for anything below macOS 12.3 and don't waste your time with DAL plugins.

If you are still here, read on.

Description

This repository started off as a fork of the official CoreMediaIO sample code by Apple. Unfortunately, the original code didn't age well since it was last updated in 2012 so here you can find the necessary changes to make it compile on modern systems.

Virtual camera support

The original sample was mainly focused around camera plugins backed by real hardware which may be of little interest to anyone who is not a camera manufacturer. So eventually the code was extended with virtual camera support, i.e. DAL plugins that appear and can act like video sources but have no accompanying hardware. These cameras may draw their frames from whatever source they please, including but not limited to baked-in still images, content generated on-the-fly by the plugin itself, and relayed frames originating from a user-space companion application. (Note that the present code only implements reading from a static framestore.)

Known issues

• the kext is not being maintained and is most probably dysfunctional
• IPC between the plugin and assistant is still built around the obsolete bootstrap server, this should be replaced by XPC
• and many more, see the Issues tab (and help reporting them if you come across one)

Requirements to build

Xcode 10.1+

Note: the sample YUV files in the KernelExtension folder have been removed from the repo because the increased traffic continuously exhausted my LFS quota. From now on, please refer to the official CoreMediaIO sample code bundle which also contains these files.

Contributing

The goal of this project is to maintain an up-to-date and relevant sample code for those who have just made first contact with the CoreMediaIO DAL world. Contributions along these lines are warmly welcome. Improvements, however, that are introducing a new feature or would steer the project away from its general/educational purpose would probably best be placed in a different fork.

License

MIT

coremediaio-dal-example's Issues

Example to send the buffers outside the plugin

Hello,
I have asked almost every repository about this. The ones that implement this do not compile due to various reasons.
Is there any way you can tell me an example that is able to send the buffer outside the sample?

Debugging workflow

First, thank you so much @lvsti for this repo—it has saved me a huge amount of time and is greatly appreciated! I have a hopefully simple question: how do I debug the Assistant?

I noticed a hook in CMIO_DPA_Sample_VCamServer.cpp to wait for the debugger. However, when I run the Assistant from Xcode, it exits with code 43... The call to bootstrap_check_in fails with error 1100 (BOOTSTRAP_NOT_PRIVILEGED).

From what I can understand, that's because the plugin actually has to initialize the port (?) with bootstrap_create_server. But I'm a bit stuck as to how to invoke the plugin from Xcode. Any hints would be so helpful.

Thanks again!

Replace deprecated bootstrap code

bootstrap_create_server() is deprecated

Can't get the plugin load in Catalina

Ws able to compile successfully but can't get any of the plugins (Sample or SampleVCam) to load. I am using your Cameo tool to browse the DALs (super nice tool, love those Swift wrappers). But I don't see them there. I wonder if it's a signing issue or I am missing some important steps in how to register them with the system. Any pointers?

[Question] Could not get video output after reboot

Hi,
I use SampleVCam.plugin, without kext installed.
Every time after reboot macOS, I could not get video using OBS. But if I use QuickTime movie recording function, the plugin works well. And the weird thing is after quitting QuickTime app, and opening OBS app, OBS can get video buffer from plugin.
I've put plist into /Library/LaunchDaemons and /Library/LaunchAgents folders.
Is there anything I missed ?

How can I use my iPhone as virtual camera input device to capture video?

I am creating an app that uses an iPhone as a virtual camera. Similar to Epoccam app (https://www.elgato.com/en/epoccam). How can I use this sample code to create a plugin for my iPhone camera as the input?

Wrong timestamps preventing playback in QuickTime

I'm getting some sort of incorrect timestamping issue that is preventing reliable playback in QuickTime player. Sometimes a few frames will play but then the rest will be dropped. I get errors in the Console like this.

default    12:12:03.725776-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1466    pts    48.865    dur    0.033    ht_ms    0    now_ms    43835002    flags    0x00000000    ***** DELTA 43835002 IS GREATER THAN FRAME DURATION OF 33 *****
default    12:12:03.833574-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1467    pts    48.898    dur    0.033    ht_ms    0    now_ms    43835110    flags    0x00000000    ***** DELTA 43835110 IS GREATER THAN FRAME DURATION OF 33 *****
default    12:12:03.940351-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1468    pts    48.932    dur    0.033    ht_ms    0    now_ms    43835217    flags    0x00000000    ***** DELTA 43835217 IS GREATER THAN FRAME DURATION OF 33 *****
default    12:12:04.045023-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1469    pts    48.965    dur    0.033    ht_ms    0    now_ms    43835322    flags    0x00000000    ***** DELTA 43835322 IS GREATER THAN FRAME DURATION OF 33 *****

I've also tried decoding this in OBS. I get similar messages (though OBS does not drop the frames)

default    12:12:03.725776-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1466    pts    48.865    dur    0.033    ht_ms    0    now_ms    43835002    flags    0x00000000    ***** DELTA 43835002 IS GREATER THAN FRAME DURATION OF 33 *****
default    12:12:03.833574-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1467    pts    48.898    dur    0.033    ht_ms    0    now_ms    43835110    flags    0x00000000    ***** DELTA 43835110 IS GREATER THAN FRAME DURATION OF 33 *****
default    12:12:03.940351-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1468    pts    48.932    dur    0.033    ht_ms    0    now_ms    43835217    flags    0x00000000    ***** DELTA 43835217 IS GREATER THAN FRAME DURATION OF 33 *****
default    12:12:04.045023-0700    obs    CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff699b0fc40    Asn=1469    pts    48.965    dur    0.033    ht_ms    0    now_ms    43835322    flags    0x00000000    ***** DELTA 43835322 IS GREATER THAN FRAME DURATION OF 33 *****

I'm digging in the code to try to understand these.

KEXT is not working

The kernel extension doesn't seem to load, probably due to code signing issues (OS imposes stricter security rules since the sample code was conceived in the Lion age).

Is there a Process to register the Object ID before publishing

@johnboiles @lvsti : If I need to publish the a DAL Plugin what is the process?
Also, how can I make sure no one can publish with same Object ID as mine?
How can I change the objectID?

recording multiple devices

can we use this approach for recording multiple devices at a time?

[Question] Possible to output a html5 page/canvas to a virtual webcam

Hi
Been trying to find something that can output an electron app to a virtual webcam
(osx-coreMediaIO / windows-directx).
Would it be possible with this example or similar, can see you did quite a few virtual webcam projects.

kext installation error on MacOS10.15.sdk

Switching from MacOS10.14.sdk to MacOS10.15.sdk gives me this error:
sudo kextutil -tn **./BuildResults/Debug/root/System/Library/Extensions/IOVideoSample.kext

kxld[com.apple.cmio.driver.Sample]: The super class vtable 'vtable for IOVideoDevice' for vtable 'vtable for IOVideoSampleDevice' is out of date. Make sure your kext has been built against the correct headers.
kxld[com.apple.cmio.driver.Sample]: The super class vtable 'vtable for IOVideoStream' for vtable 'vtable for IOVideoSampleStream' is out of date. Make sure your kext has been built against the correct headers.
Link failed (error code 5).

Any idea what could be wrong?

Unbind plugin from assistant

Is there anyway to untie with DPA? if the virualcam doesn't need access hardware, the kext will not be necessary, I also don't want DPA, according the document, looks like the DP plugin shall work with it self without DPA and kext (I use TCP/IP for the source of the video frames).
If you could provide pure working plugin which could find by app (WebRTC, Zoom etc), it will be great!!!!

I'm only getting empty output

Thanks for putting this and the Cameo tool together!

I think I have setup the virtual camera correctly. When I open your cameo app, I am able to see the virtual device, and "open it". However when I view the output it's all green. In PhotoBooth the output is always black. So I'm not sure if it's working properly or not. Is there anyway to send random noise for the frame in EmitFrame?

Also, whenever I switch to another device and back to the virtual device, whichever app i'm using crashes. Any suggestions?

Thanks!!

CMIO DAL Assistant freeze/hangs on macs with ( Iris, Iris Plus ) GPU.

Hello

Virtual Camera model is based on the CoreMediaIO-DAL-Example.

Our plugin is properly detected and loaded by the apps and functioning normally in general. CMIO DAL Assistant hangs/freezes on systems with Iris or Iris Plus GPU (we haven't see this issue on non-Iris systems). Freezing happens usually within 5-10 minutes after test execution on the mentioned platform.

Data collected before freeze:

• Memory Profiling: No memory leaks
• CPU loads for DAL Assistant: 0.6-1.2%
• GPU loads for DAL Assistant: 0%
  Data collected after freeze:
• "Console Crash Report" list is empty
• "Diagnostic Report" list is empty
• Sometimes the following event appears in the console: "CMIOHardware::CMIODeviceStopStream".

After plugin detection, from the console's "All messages" we always see a lot of warning messages like these:

• CMIO_Unit_Input_Device_DALPlugInProducer.cpp:584:PullInput 0x7ff418df06a0
• Asn=5229 pts 174.261 dur 0.033 ht_ms 0 now_ms 1428568 flags 0x00000000 ***** DELTA 1428568 IS GREATER THAN FRAME DURATION OF 33 *****
  We see these messages also on other (not Iris GPU) hardware configurations. We're not sure if this may be an indication of an issue that may cause the freeze problem on Iris GPU platforms.

STEPS TO REPRODUCE:

Test is performed with Zoom application (no other camera usage in the system during the test run).
Our plugin is properly detected and loaded by the app and functioning normally in general.
Freezing (we see black video frame) happens usually within 5-10 minutes inactivity after that. Activity Monitor shows 0% CPU load for our DAL assistant.
The only way around it to kill the DAL assistant process.

Combine with macam64 static library?

using legacy usb cams (e.g. the PS3 Eye Cam) with modern MacOs
raises a couple of interests due to so many remote use cases.

macam which provides a driver for many legacy USB cams has been ported to catalina as macam64 but this makes available a static library only. I'd love to use this with OBS Studio which can ingest CodeMedioIO DAL Plugins only.

How hard would it be to include support for macam64 creating a CoreMediaIO DAL Plugin?