hollance/inception-coreml

Running Inception-v3 on Core ML

Inception with CoreML

This is the Inception-v3 neural network running on the shiny new CoreML framework. It uses Inceptionv3.mlmodel from Apple's developer website.

It runs from a live video feed and performs a prediction as often as it can manage. If your device becomes too hot, change the setUpCamera() method in ViewController.swift to do videoCapture.fps = 5.

To use this app, open Inception.xcodeproj in Xcode 9 and run it on a device with iOS 11.

NOTE: The reported "elapsed" time is how long it takes the Inception-v3 neural net to process a single image. The FPS is the actual throughput achieved by the app.

inception-coreml's Issues

Elapsed time

Hi, thank you for sharing this demo app.
I have a question about the inference time.
On which device have you tested? (The screenshot iPad version)

Incorrect results, mostly predicting 'box turtle' and 'cd player' for every image.

I am using your project with xcode 10 beta on ios 11. Not sure if there is an issue during preprocessing, getting wrong results. Any idea what can be the issue.

Inception v3 is no longer available at the link provided

It looks like apple removed Inception v3 link from the URL you indicated.

https://developer.apple.com/machine-learning/models/

improve FPS

@hollance
thanx for the project it was helpful for deeplab real-time segmentation but the capture was very slow even with desiredFrameRate = 30, I tried to improve it to 60 or 240 but i got "libc++abi.dylib: terminating with uncaught exception of type NSException" error .
do you have any idea how can I solve it?
` private func configureSession() {

    guard permissionGranted else { return }

    captureSession.sessionPreset = quality

    guard let captureDevice = selectCaptureDevice() else { return }

           guard let captureDeviceInput = try? AVCaptureDeviceInput(device: captureDevice) else { 

return }

    guard captureSession.canAddInput(captureDeviceInput) else { return }  
  
   captureSession.addInput(captureDeviceInput)
    do {

        try captureDevice.lockForConfiguration()

        captureDevice.activeVideoMinFrameDuration = CMTimeMake(1, Int32(desiredFrameRate))

        captureDevice.activeVideoMaxFrameDuration = CMTimeMake(1, Int32(desiredFrameRate))

        captureDevice.unlockForConfiguration()
    } catch {print("dd")}

    let videoOutput = AVCaptureVideoDataOutput()

    videoOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "sample buffer"))

    guard captureSession.canAddOutput(videoOutput) else { return }

    captureSession.addOutput(videoOutput)

    guard let connection = videoOutput.connection(withMediaType: 

AVFoundation.AVMediaTypeVideo) else { return }

    guard connection.isVideoOrientationSupported else { return }

    guard connection.isVideoMirroringSupported else { return }

    connection.videoOrientation = .portrait

    connection.isVideoMirrored = position == .front
}`

ios:12.3.1
iphone:6+
xcode:10.1

Does the execution time includes image pre-processing?

Hi,

Does the execution time includes the image pre-processing time to input to first layer + inception-v3 model execution time + post processing time?

For example, tensorflow benchmark model shows the execution time of only the CNN model execution without and pre image processing or post processing. I was looking for such execution time benchmarking.