one-click-studio/gabin

Automate camera switching with OBS, Bitfocus Companion and any audio api.

gabin's Issues

(pre-release 0.2.5) Crash au moment de lancer l'autocam

Après avoir configuré mon profil dans la pre-release 0.2.5, l'application a crashé.

La configuration était établie comme suit :

• Connexion ASIO à ReaRoute (Reaper)
• 4 canaux ASIO utilisés en entrée
• Dans OBS, 2 conteneurs avec chacun d'entre eux les 3 mêmes caméras
• Dans OBS, 2 scènes, la première avec 1 seul conteneur, la deuxième avec 2 conteneurs
• Le reste des réglages (durée des scènes, affectations des caméras…) dans Gabin sont les réglages par défaut

Voici la trace des logs :

01:37:32 [Gabin - main process 🤖] New gabin client connected 2023-05-26 01:37:32.0962139 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '620'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.0966568 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '623'. It is not used by any node and should be removed from the model.
2023-05-26 01:37:32.0968680 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '625'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.0973148 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '629'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.0975061 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '628'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.0978603 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '131'. It is not used by any node and should be removed from the model.
2023-05-26 01:37:32.0980087 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '134'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.0983212 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '136'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.0984340 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '140'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.0985649 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '139'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2137832 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '620'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2139599 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '623'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2142208 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '625'. It is not used by any node and should be removed from the model.
2023-05-26 01:37:32.2143900 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '629'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2147683 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '628'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2151971 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '131'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2153912 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '134'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2155519 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '136'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.2156992 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '140'. It is not used by any node and should be removed from the model.
2023-05-26 01:37:32.2158312 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '139'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3046578 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '620'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3048366 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '623'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3049589 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '625'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3051386 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '629'. It is not used by any node and should be removed from the model.
2023-05-26 01:37:32.3053454 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '628'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3058453 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '131'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3059883 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '134'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3061775 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '136'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3063369 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '140'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3064997 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '139'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3944424 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '620'. It is not used by any node and should be removed from the model.
2023-05-26 01:37:32.3949175 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '623'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3952927 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '625'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3954763 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '629'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] R[1] 2023-05-26 01[1] 2023-05-26 01:37:32.3956181 [W[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitiali[1] 2023-05-26 01:37:32.3956181[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitia[1] 2023-05-26 01:37:32.3956[1] 2023-05-26 01:37:32.3956[1] 2023-05-26[1] 2023-05-26[1] 2023-05-26 01:37:32.3956[1] 2023-05-26[1] 2023-05-26[1] 2023-05-26[1] 2023-05-26 01:37:32.3956181 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '628'. It is not used by any node and should be removed from the model.
2023-05-26 01:37:32.3959201 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '131'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3961775 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '134'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3962915 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '136'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3964072 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '140'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.3965608 [W:onnxruntime:, graph.cc:3490 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer '139'. It is not used by any node and should be removed from the model.
[1] 2023-05-26 01:37:32.4484837 [E:onnxruntime:, sequential_executor.cc:494 onnxruntime::ExecuteKernel] Non-zero status code returned while running Pad node. Name:'Pad_63' Status Message: Cannot use 'reflect' mode to pad dimension with a value of 0. Input shape:{1,1,1,0}
2023-05-26 01:37:32.4490770 [E:onnxruntime:, sequential_executor.cc:494 onnxruntime::ExecuteKernel] Non-zero status code returned while running If node. Name:'If_25' Status Message: Non-zero status code returned while running Pad node. Name:'Pad_63' Status Message: Cannot use 'reflect' mode to pad dimension with a value of 0. Input shape:{1,1,1,0}     [1]
[1] 01:37:32 [Gabin - main process 🤖] Error: Non-zero status code returned while running If node. Name:'If_25' Status Message: Non-zero status code returned while running Pad node. Name:'Pad_63' Status Message: Cannot use 'reflect' mode to pad dimension with a value of 0. Input shape:{1,1,1,0}
[1]     at C:\***\gabin\node_modules\.pnpm\[email protected]\node_modules\onnxruntime-node\lib\backend.ts:41:42
    at processTicksAndRejections (node:internal/process/task_queues:77:11)
[1] 01:37:32 [Gabin - main process 🤖] Cleaning...
01:37:32 [Gabin - main process 🤖] Client disconnected ***
01:37:32 [osc-server] osc-server server disconnected
01:37:32 [osc-server] cleaning osc-server server [nodemon] app crashed - waiting for file changes before starting...

prevent text selection anywere but in inputs

https://tailwindcss.com/docs/user-select

Add unit tests

playwright?

weird padding on number input

mics selection scrollbar

This view should have a scroll bar when needed :

We could use this : https://github.com/action-hong/unocss-preset-scrollbar as it's much less ugly.
(as in jeanne)

merge "profile config" and "obs config"

We should solve #3 at the same time.

"Edit settings" should also be merged with "Edit".

check OBS version

When connecting to OBS, check if version >= 28 and show error if not.

OSC implem

https://www.npmjs.com/package/osc-js

log crashes

Crashes are not logged

refresh OBS scenes

If you add / edit scenes too late, they don't appear here :

There should be a 🔄 button (I had to completely restart the app).

Test of the audio processing part

need test on the multi-channel processing

[Voice detection] Fine tune detection

It would be interesting to adjust some voice detection settings (like level threshold, speed of detection, etc).

missing "channel" in summary

Add it in a new col ?

check if obs connection is valid

During initial setup (when setting OBS ip & password), a check should be made that connection is valid.
If not, show error & prevent going forward.

changing cols titles make everything unaligned with table

Minor fixes

• x to close modal
• probabilities all at 100% in summary view
• OSC marked as not connected in running view
• disconnect view not triggered when in running view

filter audio by volume before checking if is speaking

default camera selection toggle

On "Auto cam settings" panel, default config should be :

• for each mic, one of the camera is selected, at 100% probability (or none if all are selected in other mics)

Not : every cam is enabled for every mic.

Make sure that "Reset all" button follows this too.

crash on M1 mac

Have this crash when opening latest build on mac :

cropped container step

If window is not wide enough, it crops this step horizontally :

Add overflow-y or prevent content being > 100%.

[API/Settings] Remote control to edit a profile

Inside a studio where hosts may move at different places, is there any possibility to easily change the shot associated to a mic source without having to edit a profile?

It may be interesting to control this kind of assignations, so it can be controlled by remote devices like Companion or custom Raspberry Pi devices.

buttons feedback on click

We should have a feedback when clicking on buttons. (a different level of opacity than hover for instance)

minor UI fixes

1. should be aligned

2. should have capital I for "I'm" -> no idea where this message is lowercased

3. there are small lags when starting / stopping Gabin, we should have an instant transition (with loading / skeletons if necessary)

4. this menu should not have rounded corners

5. arrow should be vertically aligned with +

6. some texts line-heights are very low, and make them hard to read :

clarify "Probability", "Min", "Max"

These are unclear :

We could add a small circled "i" (for information) after this 3 column titles. They would show a tooltip on hover like #4 with respective texts :

• Describes how often this camera will be shown (between 0 and 100%)
• Minimum duration a camera is shown. 0s will allow ugly glitches.
• Maximum duration a camera is shown before changing to other available camera angles. Not useful if no other cameras are enabled for this mic.

gabin message visual bugs

text is not horizontally centered in msg box in initial configuration

[UI/UX] Exit the profile editor

Currently, to exit the profile editor, we need to click "Cancel".

It may not be clear because of the meaning of "Cancel", even after having saved the profile changes.

Is it possible to add a close button to the top right corner of the editor, that will have the same role than the cancel button?

Impossible de sélectionner une source quand il y a plusieurs conteneurs ?

Bonjour,

Dans cette configuration, le conteneur 1 fonctionne aussi bien en autocam qu'en manuel. Mais il ne se passe absolument rien dans le conteneur 2 (ni en autocam, ni en manuel).

Ce bug semble présent dans la v0.1 et dans la v0.2.

Rec audio channels from Gabin

grab window from top & close/min/max buttons

Window should also be draggable from this region :

Crash on run

Documentation protocole OSC

Bonjour,

Je vois que la version 0.3 de Gabin sera en mesure de se connecter à un serveur OSC.
J'envisage de développer un genre de middleware entre OBS et Gabin, qui communiquerait en OSC, pour répondre à 2 problématiques :

• Pouvoir changer les assignations de l'activité audio aux différents plans de caméras ( → si quelqu'un change de place en cours de live)
• Pouvoir déclencher un changement de plan (qui serait demandé par Gabin) dans OBS, après que la caméra concernée (PTZ) a fini de se réorienter.

Ma question : est-ce que l'API pour que Gabin puisse se brancher en OSC est / sera documentée quelque part ?

Merci :)

1px tree bug

Import/Export config

• GUI
• OSC

Add profile picker in Companion

mic name add ellipsis

Text should not be on two lines :

Use text-ellipsis.

fix build

We must find a clean way to find this pnpm.cjs file and patch it in release.yml.
This currently prevents builds to work.

Ajouter une liste des choses à faire.

Bonjour, j'aimerais bien contribuer au projet, mais serait-il possible de mettre dans le readme une sorte de roadmap pour savoir sur quoi bosser?

log file / crash log

log everything, do not create infinite logs, logs crashes

clarify channel selection

It is unclear what these 2 options are :

Idea :
Replace "Mic 1"... with "Channel 1"...

Add a tooltip (with HeadlessUI popover for ex) saying on hover : "Add this channel as a Mic."
Replace input label "Mic 1 Custom Name" with "Channel 1 Name".

[BUG] serveur tcp

salut, je voulais voir ce que c'est http://127.0.0.1:6481/ et j'ai eu cette erreur ;)

vous devez regarder si c'est bien du json avant de le parse.
c'est dans ce fichier: https://github.com/one-click-studio/gabin/blob/main/src/main/servers/TcpServer.ts

socket.on('data', (data: Buffer) => {
  const request: TcpRequest = JSON.parse(data.toString()) //ici
  if (!request.type){
     this.logger.error('received request without type', request)
     return
 }
 this.logger.debug('received request', request)
 this.sendRequestToClients(request)
});

change edit menu

Remove timeline & useless steps. Allow quick edits on mappings.

add units in inputs

Add % units and "s" units inside inputs here :

You could mimic "Custom parsing" exemple here : https://www.naiveui.com/en-US/light/components/input-number#icon.vue

not all buttons on running

disabled button should not have hover effect

Camlock Logic

If next shot is the same as before, do not lock the container.

(+ minor unfocus bug)

Bug when stop and restart Gabin

OBS seems not connected

[UI/UX] Add a scenes and containers reminder to profile editor

Currently, it is not possible to display again the tutorial to create containers when editing a profile (or I did not find it :D )

Is it possible to add a kind of "Help" button somewhere to display again this tutorial?

[API] Remote control?

I see that Gabin may be connected to a "Companion" device. Is there any documentation anywhere to see what features may be controlled by a remote controller?

add "Empty scenes" / "Empty container" in select

When no scenes are found, selects should have a single "Empty" non-selectable item.

Compatibility with VMIX

https://www.vmix.com/