How to use your own custom built TensorFlowLiteC.framework on iOS?

[am15h]

How to use your own custom built TensorFlowLiteC.framework on iOS?

[am15h]

Basically for now will have to clone this repo and include it using "path import" in your projects pubspec.yaml.

Then in your local version of tflite_flutter_plugin you can replace the ios/TensorFlowLiteC.framework with yours

[mgalgs]

Got it... I gave that a try but now I'm getting linker errors after dropping in my TensorFlowLiteC.framework... (Looks pretty similar to the errors I was getting when I was slamming my TensorFlowLiteC.framework into my pub-cache) 🤔

Build output

$ flutter build ios
Warning: You are using these overridden dependencies:
! tflite_flutter 0.5.0 from path external/tflite_flutter_plugin
Running "flutter pub get" in RoutespotterApp...                     0.7s
Building com.mgalgs.routespotter for device (ios-release)...
Automatically signing iOS for device deployment using specified development team in Xcode project: PH2D8HJC83
Running pod install...                                              2.3s
Running Xcode build...
Xcode build done.                                            4.3s
Failed to build iOS app
Error output from Xcode build:
↳
    ** BUILD FAILED **


Xcode's output:
↳
    Undefined symbols for architecture arm64:
      "tflite::tensor_utils::MatrixBatchVectorMultiplyAccumulate(signed char const*, int, int, signed char const*, float const*, int, int*, float*,
      tflite::CpuBackendContext*)", referenced from:
          l4840 in TensorFlowLiteC
      "tflite::ops::builtin::Register_FLOOR()", referenced from:
          l4485 in TensorFlowLiteC
      "tflite::ops::custom::Register_MFCC()", referenced from:
          l4485 in TensorFlowLiteC
      "_xnn_reallocate", referenced from:
          l4440 in TensorFlowLiteC
      "_xnn_aligned_allocate", referenced from:
          l4440 in TensorFlowLiteC
      "_xnn_aligned_deallocate", referenced from:
          l4440 in TensorFlowLiteC
      "flatbuffers::EnsureDirExists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
          l3532 in TensorFlowLiteC
      "flatbuffers::FileExists(char const*)", referenced from:
          l2367 in TensorFlowLiteC
      "flatbuffers::LoadFile(char const*, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*)", referenced from:
          l2367 in TensorFlowLiteC
      "flatbuffers::AbsolutePath(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
          l2254 in TensorFlowLiteC
      "tflite::tensor_utils::VectorBatchVectorCwiseProductAccumulate(short const*, int, short const*, int, int, int, short*)", referenced from:
          l1730 in TensorFlowLiteC
      "tflite::tensor_utils::VectorScalarMultiply(signed char const*, int, float, float*)", referenced from:
          l1729 in TensorFlowLiteC
      "tflite::tensor_utils::VectorBatchVectorAdd(float const*, int, int, float*)", referenced from:
          l1728 in TensorFlowLiteC
          l1729 in TensorFlowLiteC
      "tflite::tensor_utils::CwiseClipping(short*, int, short)", referenced from:
          l1727 in TensorFlowLiteC
          l1731 in TensorFlowLiteC
      "tflite::tensor_utils::MatrixBatchVectorMultiply(signed char const*, int, signed char const*, int, int, int, int, int, signed char*, signed char)", referenced from:
          l1727 in TensorFlowLiteC
      "tflite::tensor_utils::TwoGateSaturatingAdd(signed char const*, signed char, signed char const*, signed char, int, int, int, int, int, int, short*)", referenced from:
          l1727 in TensorFlowLiteC
      "tflite::tensor_utils::ApplySigmoidFloat(short const*, int, int, short*)", referenced from:
          l1727 in TensorFlowLiteC
      "tflite::tensor_utils::CwiseMul(short const*, short const*, int, int, int, short*)", referenced from:
          l1727 in TensorFlowLiteC
          l1731 in TensorFlowLiteC
      "tflite::tensor_utils::ApplyTanhFloat(short const*, int, int, int, short*)", referenced from:
          l1727 in TensorFlowLiteC
      "tflite::tensor_utils::ApplyTanh(int, short const*, int, int, short*)", referenced from:
          l1726 in TensorFlowLiteC
          l1730 in TensorFlowLiteC
      "tflite::tensor_utils::CwiseMul(short const*, short const*, int, int, int, int, int, signed char*)", referenced from:
          l1726 in TensorFlowLiteC
      "flatbuffers::StripExtension(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
          l814 in TensorFlowLiteC
          l1889 in TensorFlowLiteC
          l3564 in TensorFlowLiteC
      "tflite::tensor_utils::MatrixBatchVectorMultiplyAccumulate(signed char const*, int const*, signed char const*, int, int, int, int, int, int, int*, signed char*,
      tflite::CpuBackendContext*)", referenced from:
          l1726 in TensorFlowLiteC
      "tflite::tensor_utils::CwiseClipping(float*, int, float)", referenced from:
          l1722 in TensorFlowLiteC
          l1724 in TensorFlowLiteC
      "tflite::Subgraph::HasDelegates()", referenced from:
          l1472 in TensorFlowLiteC
      "tflite::Subgraph::IsCancelled()", referenced from:
          l1469 in TensorFlowLiteC
      "tflite::tensor_utils::AsymmetricQuantizeFloats(float const*, int, signed char*, float*, int*)", referenced from:
          l278 in TensorFlowLiteC
          l290 in TensorFlowLiteC
          l414 in TensorFlowLiteC
          l1254 in TensorFlowLiteC
          l1417 in TensorFlowLiteC
          l1724 in TensorFlowLiteC
          l4108 in TensorFlowLiteC
          ...
      "tflite::Subgraph::SetExecutionPlan(std::__1::vector<int, std::__1::allocator<int> > const&)", referenced from:
          l1465 in TensorFlowLiteC
      "tflite::tensor_utils::ApplyLayerNorm(short const*, short const*, int const*, int, int, int, int, int, short*)", referenced from:
          l1726 in TensorFlowLiteC
          l1730 in TensorFlowLiteC
      "_xnn_deallocate", referenced from:
          l4440 in TensorFlowLiteC
      "tflite::Subgraph::SetTensorParametersReadWrite(int, TfLiteType, char const*, unsigned long, int const*, TfLiteQuantization, bool, unsigned long, int const*)",
      referenced from:
          l1462 in TensorFlowLiteC
          l1464 in TensorFlowLiteC
          l3451 in TensorFlowLiteC
      "tflite::Subgraph::SetTensorParametersReadOnly(int, TfLiteType, char const*, unsigned long, int const*, TfLiteQuantization, char const*, unsigned long,
      tflite::Allocation const*, TfLiteSparsity*)", referenced from:
          l1461 in TensorFlowLiteC
          l1463 in TensorFlowLiteC
          l3451 in TensorFlowLiteC
      "tflite::Subgraph::AddTensors(int, int*)", referenced from:
          l1459 in TensorFlowLiteC
          l3454 in TensorFlowLiteC
      "flatbuffers::SaveFile(char const*, char const*, unsigned long, bool)", referenced from:
          l812 in TensorFlowLiteC
          l1892 in TensorFlowLiteC
          l3563 in TensorFlowLiteC
      "tflite::Subgraph::AddNodeWithParameters(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<int, std::__1::allocator<int> > const&,
      std::__1::vector<int, std::__1::allocator<int> > const&, char const*, unsigned long, void*, TfLiteRegistration const*, int*)", referenced from:
          l1454 in TensorFlowLiteC
          l3443 in TensorFlowLiteC
      "tflite::Subgraph::Invoke()", referenced from:
          l1458 in TensorFlowLiteC
          l3008 in TensorFlowLiteC
          l4742 in TensorFlowLiteC
          l5015 in TensorFlowLiteC
      "tflite::tensor_utils::Sub1Vector(short const*, int, short*)", referenced from:
          l1727 in TensorFlowLiteC
          l1731 in TensorFlowLiteC
      "tflite::Subgraph::ResetVariableTensors()", referenced from:
          l1460 in TensorFlowLiteC
      "tflite::tensor_utils::ApplySigmoid(short const*, int, int, short*)", referenced from:
          l1730 in TensorFlowLiteC
      "flatbuffers::StripPath(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
          l814 in TensorFlowLiteC
          l1889 in TensorFlowLiteC
          l3564 in TensorFlowLiteC
      "tflite::Subgraph::ReleaseNonPersistentMemory()", referenced from:
          l1457 in TensorFlowLiteC
          l5015 in TensorFlowLiteC
      "tflite::tensor_utils::SparseMatrixBatchVectorMultiplyAccumulate1x4(float const*, int const*, int const*, int, int, float const*, int, float*)", referenced from:
          l1282 in TensorFlowLiteC
          l1286 in TensorFlowLiteC
      "tflite::Subgraph::ResizeInputTensorStrict(int, std::__1::vector<int, std::__1::allocator<int> > const&)", referenced from:
          l1456 in TensorFlowLiteC
      "tflite::tensor_utils::SparseMatrixBatchVectorMultiplyAccumulate(signed char const*, unsigned char const*, int, int, signed char const*, float const*, int, float*)",
      referenced from:
          l1724 in TensorFlowLiteC
          l1729 in TensorFlowLiteC
      "tflite::Subgraph::ResizeInputTensor(int, std::__1::vector<int, std::__1::allocator<int> > const&)", referenced from:
          l1455 in TensorFlowLiteC
          l3007 in TensorFlowLiteC
          l3009 in TensorFlowLiteC
          l4741 in TensorFlowLiteC
      "tflite::tensor_utils::MatrixBatchVectorMultiply(short const*, signed char const*, int, int, int const*, int, int, int, int, signed char*)", referenced from:
          l1727 in TensorFlowLiteC
      "tflite::Subgraph::ReserveNodes(int)", referenced from:
          l1452 in TensorFlowLiteC
          l3443 in TensorFlowLiteC
      "tflite::Subgraph::SetExternalContext(TfLiteExternalContextType, TfLiteExternalContext*)", referenced from:
          l1445 in TensorFlowLiteC
      "tflite::Subgraph::SetCustomAllocationForTensor(int, TfLiteCustomAllocation const&)", referenced from:
          l1446 in TensorFlowLiteC
      "tflite::tensor_utils::MatrixBatchVectorMultiplyAccumulate(float const*, int, int, float const*, int, float*)", referenced from:
          l1252 in TensorFlowLiteC
          l1416 in TensorFlowLiteC
          l1722 in TensorFlowLiteC
          l1728 in TensorFlowLiteC
          l4106 in TensorFlowLiteC
      "tflite::Subgraph::RemoveAllDelegates()", referenced from:
          l1451 in TensorFlowLiteC
          l1470 in TensorFlowLiteC
          l1471 in TensorFlowLiteC
      "tflite::Subgraph::SetVariables(std::__1::vector<int, std::__1::allocator<int> >)", referenced from:
          l1449 in TensorFlowLiteC
          l3454 in TensorFlowLiteC
      "flatbuffers::ConCatPathFileName(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
          l2367 in TensorFlowLiteC
      "tflite::tensor_utils::ReductionSumVector(signed char const*, int*, int, int)", referenced from:
          l1724 in TensorFlowLiteC
          l4108 in TensorFlowLiteC
          l4109 in TensorFlowLiteC
          l4554 in TensorFlowLiteC
          l4864 in TensorFlowLiteC
      "tflite::tensor_utils::CwiseAdd(short const*, short const*, int, int, short*)", referenced from:
          l1727 in TensorFlowLiteC
          l1731 in TensorFlowLiteC
      "tflite::Subgraph::SetOutputs(std::__1::vector<int, std::__1::allocator<int> >)", referenced from:
          l1448 in TensorFlowLiteC
          l3454 in TensorFlowLiteC
      "tflite::Subgraph::Subgraph(tflite::ErrorReporter*, TfLiteExternalContext**, std::__1::vector<std::__1::unique_ptr<tflite::Subgraph,
      std::__1::default_delete<tflite::Subgraph> >, std::__1::allocator<std::__1::unique_ptr<tflite::Subgraph, std::__1::default_delete<tflite::Subgraph> > > >*,
      std::__1::unordered_map<int, std::__1::unique_ptr<tflite::resource::ResourceBase, std::__1::default_delete<tflite::resource::ResourceBase> >, std::__1::hash<int>,
      std::__1::equal_to<int>, std::__1::allocator<std::__1::pair<int const, std::__1::unique_ptr<tflite::resource::ResourceBase,
      std::__1::default_delete<tflite::resource::ResourceBase> > > > >*)", referenced from:
          l1441 in TensorFlowLiteC
      "flatbuffers::PosixPath(char const*)", referenced from:
          l2367 in TensorFlowLiteC
      "tflite::tensor_utils::CwiseClipping(signed char*, int, signed char)", referenced from:
          l1726 in TensorFlowLiteC
          l1727 in TensorFlowLiteC
      "tflite::tensor_utils::Sub1Vector(float const*, int, float*)", referenced from:
          l1722 in TensorFlowLiteC
          l1724 in TensorFlowLiteC
      "tflite::tensor_utils::BatchVectorBatchVectorDotProduct(short const*, short const*, int, int, int*)", referenced from:
          l1418 in TensorFlowLiteC
      "tflite::tensor_utils::MatrixBatchVectorMultiplyAccumulate(signed char const*, int, int, signed char const*, float const*, int, float*, float const*, int const*,
      int*, int*, bool*, tflite::CpuBackendContext*)", referenced from:
          l1254 in TensorFlowLiteC
          l1417 in TensorFlowLiteC
          l1724 in TensorFlowLiteC
          l1729 in TensorFlowLiteC
          l4108 in TensorFlowLiteC
      "tflite::Subgraph::SetCancellationFunction(void*, bool (*)(void*))", referenced from:
          l1468 in TensorFlowLiteC
      "tflite::tensor_utils::ReductionSumVector(int const*, int*, int, int)", referenced from:
          l1418 in TensorFlowLiteC
      "_xnn_allocate", referenced from:
          l4440 in TensorFlowLiteC
      "tflite::tensor_utils::MeanStddevNormalization(float const*, float*, int, int)", referenced from:
          l1728 in TensorFlowLiteC
          l1729 in TensorFlowLiteC
      "tflite::tensor_utils::VectorVectorDotProduct(float const*, float const*, int)", referenced from:
          l1416 in TensorFlowLiteC
          l1417 in TensorFlowLiteC
      "tflite::tensor_utils::IsZeroVector(float const*, int)", referenced from:
          l1254 in TensorFlowLiteC
          l1417 in TensorFlowLiteC
          l1722 in TensorFlowLiteC
          l1724 in TensorFlowLiteC
          l4108 in TensorFlowLiteC
      "tflite::tensor_utils::ApplyLayerNormFloat(short const*, short const*, int, int, int const*, int, int, short*)", referenced from:
          l1727 in TensorFlowLiteC
      "tflite::Subgraph::ModifyGraphWithDelegate(TfLiteDelegate*)", referenced from:
          l1451 in TensorFlowLiteC
          l1470 in TensorFlowLiteC
      "tflite::tensor_utils::MatrixScalarMultiplyAccumulate(signed char const*, int, int, int, int*)", referenced from:
          l764 in TensorFlowLiteC
          l1981 in TensorFlowLiteC
      "tflite::Subgraph::SetInputs(std::__1::vector<int, std::__1::allocator<int> >)", referenced from:
          l1447 in TensorFlowLiteC
          l3454 in TensorFlowLiteC
      "tflite::tensor_utils::ReductionSumVector(float const*, float*, int, int)", referenced from:
          l1416 in TensorFlowLiteC
          l1417 in TensorFlowLiteC
      "tflite::tensor_utils::MatrixBatchVectorMultiplyAccumulate(signed char const*, int const*, signed char const*, int, int, int, int, int, int, int*, short*,
      tflite::CpuBackendContext*)", referenced from:
          l1726 in TensorFlowLiteC
          l1730 in TensorFlowLiteC
      "tflite::tensor_utils::SymmetricQuantizeFloats(float const*, int, signed char*, float*, float*, float*)", referenced from:
          l1254 in TensorFlowLiteC
          l1417 in TensorFlowLiteC
          l1724 in TensorFlowLiteC
          l4108 in TensorFlowLiteC
          l4542 in TensorFlowLiteC
          l4551 in TensorFlowLiteC
          l4837 in TensorFlowLiteC
          ...
      "tflite::Subgraph::AllocateTensors()", referenced from:
          l1450 in TensorFlowLiteC
          l3006 in TensorFlowLiteC
          l3008 in TensorFlowLiteC
          l4741 in TensorFlowLiteC
          l5015 in TensorFlowLiteC
    ld: symbol(s) not found for architecture arm64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    note: Using new build system
    note: Building targets in parallel
    note: Planning build
    note: Constructing build description
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'url_launcher' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'video_player' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'shared_preferences' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'path_provider' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'package_info' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'image_picker' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'flutter_isolate' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'camera' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.3.99. (in target
    'Flutter' from project 'Pods')

Encountered error while building for device.

I'm using a dependency override so that the build system thinks I'm still using tflite_flutter: ^0.5.0 (otherwise I was getting dependency errors from tflite_flutter_helper):

dependency_overrides:
  tflite_flutter:
    path: "external/tflite_flutter_plugin/"

That's working fine. But as soon as I drop in my own TensorFlowLiteC.framework I'm getting the linker errors above.

My full steps for the build are:

cd ~/development
git clone [email protected]:tensorflow/tensorflow.git
cd tensorflow
./configure (answer yes when it asks about iOS support)
bazel build --config=ios_fat -c opt //tensorflow/lite/ios:TensorFlowLiteC_framework
cd <my_app>/external/tflite_flutter_plugin/ios
rm -rf TensorFlowLiteC.framework
unzip ~/development/tensorflow/bazel-bin/tensorflow/lite/ios/TensorFlowLiteC_framework.zip

I can successfully build the framework at tensorflow v2.3.1 (changing the bazel build path to //tensorflow/lite/experimantal/ios since it was still in the "experimental" directory as of v2.3.1). So something must have changed in tensorflow between v2.3.1 and master that's causing these linker errors... 🤔

Any suggestions on how to get this built against master? Do my build steps above look the same as yours?

Thanks!

[mgalgs]

Quick update... I bisected the tensorflow repo to determine where this build failure was introduced and I landed on this commit:

tensorflow/tensorflow@3e9fccb

I don't understand how that commit could have broken things so I'm still investigating... But I've done several manual builds at that commit and at the parent of that commit and have verified that the bazel version bump does indeed seem to be the breaker.

@am15h Can you confirm if you are able to build against the current TF master successfully?

[am15h]

@mgalgs I do not have access to a Mac, as of now. I can only confirm that TF 2.3.1 should work without any issues for iOS.

Opening an issue on tensorflow repository might help? They might be able to provide a resolution to this.

[mgalgs]

Looks like it was indeed an issue with Tensorflow itself, which has now been fixed.