Comments (24)
Hello @oluwabajio
Thanks for your message and feature request. I have this on my road-map but unfortunately this can not be possible now due to lack of compatibility of Model (ckpt/*.pb) to Android Tensorflow Lite environment. This has been requested to original repository deezer/spleeter#354.
If you can port TensorFlow model somehow to Android App, this repository/library can be ported to Android App via Android NDK
Please keep the above mentioned issue in check to get update on when the Deezer Model will be upgraded/re-trained to support TF v2.x and TFLite. Once that is possible, I will extend this repository to support for TFLite as well which can later be portable to Android/iOS easily.
Thanks Jinay
Hi. Is this possible now? Can spleeter be implemented on Android apps now? What is the update to this issue?
from spleeter.
Ok, I would implement it tomorrow and share the link with you.
Thanks
from spleeter.
Hello, Thanks for this, i would keep tabs till when the models are compatible with tflite. I really do not want to go the NDK route.
from spleeter.
Hello @oluwabajio
Thanks for your message and feature request. I have this on my road-map but unfortunately this can not be possible now due to lack of compatibility of Model (ckpt/*.pb) to Android Tensorflow Lite environment. This has been requested to original repository deezer/spleeter#354.
If you can port TensorFlow model somehow to Android App, this repository/library can be ported to Android App via Android NDK
Please keep the above mentioned issue in check to get update on when the Deezer Model will be upgraded/re-trained to support TF v2.x and TFLite. Once that is possible, I will extend this repository to support for TFLite as well which can later be portable to Android/iOS easily.
Thanks
Jinay
from spleeter.
Hello,
I am still keeping tabs on this issue.
Spleeter is now compatible with TF2.
Announcement: https://gitter.im/spleeter/community
Do you have any idea how this new version can be ported to tflite.
Thanks.
from spleeter.
Thanks for the update, I have started working on this. Soon I will release with the update and functional TFLite Spleeter Codebase.
I already generated TFLite models with the help and update from gvne/spleeterpp#24 - I need to make some adjustment to my repository in order provie TFLite based inference as it is currently broken.
Already released model versions: https://github.com/jinay1991/spleeter/releases/tag/v2.3.
from spleeter.
Thanks for the update, I really appreciate.
Now implementing it with tflite on android, what are the pre-processing I need to do to the input mp3 audio data before feeding it to the tflite interpreter.
Also what post-processing needs to be done
from spleeter.
Hi @jinay1991 - Thanks for the update...But could you please confirm whether have you generated the TFLite model from the latest release of spleeter, which is compatible with TF2.0 or from the older version of spleeter models?
I would like to understand whether have you generated the TFLite from 'saved model' file or from 'check points' - as the tflite model converted from checkpoints are said to have performance issues (as per 'spleeter-tflite-convert' library). Could you let me know pls?
from spleeter.
Hi @jinay1991 - Thanks for the update...But could you please confirm whether have you generated the TFLite model from the latest release of spleeter, which is compatible with TF2.0 or from the older version of spleeter models?
I would like to understand whether have you generated the TFLite from 'saved model' file or from 'check points' - as the tflite model converted from checkpoints are said to have performance issues (as per 'spleeter-tflite-convert' library). Could you let me know pls?
Hi @VVasanth I have simply taken the original checkpoints from Spleeter Public Release - which is TF1.x, though I am still waiting for deezer to provide re-trained model with TF2.0 model definitions - (Issue: deezer/spleeter#354)
I haven't tried myself yet for the performance benchmark with native TFLite C++ APIs. (Though you are right, as per the author of "spleeter-tflite-converter" library, it has some performance issues.)
from spleeter.
Thanks for the update, I really appreciate.
Now implementing it with tflite on android, what are the pre-processing I need to do to the input mp3 audio data before feeding it to the tflite interpreter.
Would you be kind to share your work (Android App) (GitHub Link)?
Pre-processing: (Decode/Demux your audio file into raw data (i.e. plain array of floats) and pass it to waveform:0
tensor of the model)
Also what post-processing needs to be done
Post-processing: None (read output tensors depending on your chosen model, and you will receive raw data (i.e. plain array of floats) which can again Remux/Encode to your choice of container (MP3/MP4))
from spleeter.
@oluwabajio I have just published a release https://github.com/jinay1991/spleeter/releases/tag/2.3.0 which works fine with TFLite and provided Spleeter Model (derived from TF v1.15 checkpoints)
from spleeter.
Importing to Android for Bazel Project using NDK is explained here -> https://docs.bazel.build/versions/master/android-ndk.html
from spleeter.
Thanks for the update. i would work on it and share my feedbacks. Thanks.
from spleeter.
Please @jinay1991 can i communicate with you via mail, i have an issue with conversion of the input audio.
THanks
from spleeter.
I have a multimedia app in development, and have been looking into integrating Spleeter into my android app. I would be happy to help you move this forward, if you need additional help.
from spleeter.
Hello @plazar99 can we communicate via mail please. i really need to get spleeter working for my mobile app.
from spleeter.
Hello @plazar99 can we communicate via mail please. i really need to get spleeter working for my mobile app. my email is [email protected]
Hello @oluwabajio Is it possible to give updates on how you are getting spleeter to work on your app? I am also looking for the same. Thank you.
from spleeter.
Hello @Semester7 i integrated the tflite model in my android app, pass in the audio data, but i was not successful with it.
You can review my code here and see if i made errors.
from spleeter.
Hello, can you share the Android source code of the C++ spleeter integration? Because I am currently doing research in this area, thank you for your sharing and guidance.
@oluwabajio
@farihachaiti
@VVasanth
@jinay1991
@plazar99
from spleeter.
Hello, can you share the Android source code of the C++ spleeter integration? Because I am currently doing research in this area, thank you for your sharing and guidance.
@oluwabajio
@farihachaiti
@VVasanth
@jinay1991
@plazar99
@oluwabajio already provided his code:
"You can review my code here and see if i made errors. Also you can communicate with me at [email protected].
from spleeter.
Hello,
i would love to use spleeter in one of my android apps (java).
i have successfully implemented Deezer spleeter on my server (aivocalremover.com), but I would like to find out if its possible to bind your C++ spleter on my android java app.
My current questions are summarized as follows:
I want to use the spleeter to deploy on the Android side, and I hope that the interface of the spleeter c++ dynamic library can be called. At present, the dynamic library you gave still cannot be called.
What I want to know is the dynamic library you just gave, how can I use the C++ program to call it and realize the separation of the vocal accompaniment of music?
Because I want to deploy it on mobile android, although I have tried the method on android, but it doesn't seem to work.
from spleeter.
Hello @KangChou i tried to get the models working on android, but i was not successful with it.
Here is my code.
float floats[] = music_data_as_float;
for (int i = 0; i < 1280000-4; i+=2) {
floats[i/2] = sampleFloatArray[i];
floats[640000+(i/2)] = sampleFloatArray[i+1];
}
int[] probabilityShape = tflite.getInputTensor(0).shape();
DataType probabilityDataType = tflite.getInputTensor(0).dataType();
TensorBuffer tensorBuffer = TensorBuffer.createDynamic(imageDataType);
tensorBuffer.loadArray(floats, probabilityShape);
ByteBuffer inByteBuffer = tensorBuffer.getBuffer();
Object[] input = new Object[1];
input[0] = inByteBuffer;
TensorBuffer outputTensorBufferI =
TensorBuffer.createFixedSize(probabilityShape, probabilityDataType);
TensorBuffer outputTensorBufferV =
TensorBuffer.createFixedSize(probabilityShape, probabilityDataType);
Map outputs = new TreeMap<>();
outputs.put(0, outputTensorBufferI.getBuffer());
outputs.put(1, outputTensorBufferV.getBuffer());
from spleeter.
Hi, has this issue been resolved so far?
from spleeter.
Here is mine.
https://github.com/farihachaiti/DoKaraokeAI
from spleeter.
Related Issues (13)
- bazel build //application:spleeter --------->ERROR Build did NOT HOT 1
- How can I fix the error as below? HOT 3
- Windows support HOT 1
- Real-time use HOT 3
- Reg: Input shape of TFLite model is different from the Spleeter Model HOT 4
- Hello, how can I generate a callable dynamic library (so file) from spleeter C++? HOT 19
- How do I compile cpp programs? HOT 1
- How do I compile cpp programs? HOT 1
- Hello, it takes a long time to process a song with a size of about 40MB. I don't know if it is because of insufficient computing power or it needs to use GPU to solve it. HOT 2
- Spleeter_dynamic_library: Solutions and steps to implement the executable file of the algorithm HOT 5
- I changed how to use the following example to compile the executable problem, and how should the location of the project directory be stored in the specified path ? HOT 2
- bazel run //application:spleeter ---------> external/models/5stems/5stems.tflite HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spleeter.