swift-ai/swift-ai

The Swift machine learning library.

Swift AI is a high-performance deep learning library written entirely in Swift. We currently offer support for all Apple platforms, with Linux support coming soon.

Tools

Swift AI includes a collection of common tools used for artificial intelligence and scientific applications:

• NeuralNet
  • A flexible, fully-connected neural network with support for deep learning
  • Optimized specifically for Apple hardware, using advanced parallel processing techniques
• Convolutional Neural Network
• Recurrent Neural Network
• Genetic Algorithm Library
• Fast Linear Algebra Library
• Signal Processing Library

Example Projects

We've created some example projects to demonstrate the usage of Swift AI. Each resides in their own repository and can be built with little or no configuration:

• NeuralNet-MNIST
  • A NeuralNet training example for the MNIST handwriting database
  • Trains a neural network to recognize handwritten digits
  • Built for macOS
• NeuralNet-Handwriting-iOS
  • A demo for handwriting recognition using NeuralNet
  • Pre-trained; just download and run
  • Built for iOS

Usage

Each module now contains its own documentation. We recommend that you read the docs carefully for detailed instructions on using the various components of Swift AI.

The example projects are another great resource for seeing real-world usage of these tools.

Compatibility

Swift AI currently depends on Apple's Accelerate framework for vector/matrix calculations and digital signal processing.

In order to provide support for more platforms, alternative BLAS solutions are being considered.

Contributing

Contributions to the project are welcome. We simply ask that you strive to maintain consistency with the structure and formatting of existing code.

Contact

Collin Hundley is the author and maintainer of Swift AI. Feel free contact him directly via email.

If you have a question about this library or are looking for guidance, we recommend opening an issue so a member of the community can help!

Consulting

If you're looking for for help with deep learning, computer vision, signal processing or other AI applications, you've come to the right place! Contact Collin for more information about consulting/contracting.

Donating

Your donation to Swift AI will help us continue building excellent open-source tools. All contributions are appreciated!

swift-ai's Issues

Re-think the storage, vDSP calls…

Someone pointed out this code base to me recently, and it's very cool stuff. While poking around I found things like this:

https://github.com/collinhundley/Swift-AI/blob/master/Source/Vector.swift#L14

While I also use a straight-up swift array to back my own publicly-available vectors at https://github.com/liscio/SMUGMath-Swift, I have recently found that it's not a great idea in practice—Arrays aren't necessarily guaranteed to be contiguous in memory. Instead I would recommend that you use ContiguousArray<Float> for the flat definition, and also surround your Accelerate accesses using withUnsafe{,Mutable}Pointer as I do in the code here:

https://github.com/liscio/SMUGMath-Swift/blob/master/SMUGMath/RealVectorOperations.swift

I don't yet have concrete plans to put my new code into the repo above because it is still in progress and tightly coupled with something else I'm working on.

Anyway—really cool repo, and subject matter!

Using Swift AI to detect emotion?

http://blogs.microsoft.com/next/2015/11/11/happy-sad-angry-this-microsoft-tool-recognizes-emotions-in-pictures/

how easy is it to use Swift AI to detect emotion?

number 7

Hey, maybe I can't write, but number 7 very often is not recognized.

Anyway, very good work.

Recurrent Neural Network

@KacheFlowe @joobei Hey guys, sorry for my apparent lack of communication over the last little while. New startup, crazy hours, no sleep... you know the routine! Definitely haven't left Swift AI behind though, and in fact it's finally made its way back up my priority list.

It's been some time since we've discussed Recurrent Neural Networks - any progress or thoughts from you guys? I've decided that it's the next component I want to tackle, partly because it would be particularly useful for some side projects that I have in mind. I definitely don't want to duplicate any effort that people have already made, if any, so I'd love to hear what you guys think about the idea.

What needs work on?

Hi. Since Github eliminated private messaging I find it difficult to comunicate with developers. I'm just wondering if there's a feature I can tackle or code I can take off yoru plate.

Add Travis CI?

That way failing pull requests could be seen automatically.

travis-ci.org does this for free and it's super popular.

Swift AI

Need to apply Mfcc and Spectrogram to the audio file

Hello,
I am creating the iOS Mobile application to detect snoring, I need to apply Mfcc and Spectrogram to the audio file and need to pass these two in the Tensorflow lite I need to apply Mfcc [Mel-frequency cepstral coefficients] and Spectrogram to the audio file. Can you suggest me any solution for this?

Thanks in Advance!

Where is actually the code?

"Swift AI is a high-performance deep learning library " --> but where is the code? Is it this only library https://github.com/Swift-AI/NeuralNet ?

Overhaul: Swift 3, SPM and Linux

To those following this repository:

We are in the process of performing a complete rewrite of Swift AI, which will include an update to Swift 3, full Linux support, many new features, and major improvements and optimizations across the board!

Here's an overview of what's happening:

Swift 3 / Package Manager

• Create a GitHub organization account
• Use SPM modules for each component
• Update full library to Swift 3
• Break out iOS and macOS examples into separate, dedicated packages

Features

• Brand-new feed-forward neural network
• Recurrent neural network
• Convolutional neural network
• Deep learning support
• Tools for building genetic algorithms
• Signal processing APIs (Fourier transforms, etc)

Linux

• Use a C BLAS library for optimized vector calculations
• OpenCL/CUDA acceleration

iOS / macOS

• Metal acceleration
• Experiment with Apple's new neural network tools for certain operations (iOS 10+)

API

• More intuitive API for creating, training and updating neural networks
• More intelligent training methods
• Potentially support Caffe models

Optimizations

• Better memory management. The neural network training routines are currently optimized for speed, at the expense of memory (large up-front allocations are made in order to avoid small heap allocations later). We can achieve a better balance.
• Make use of UnsafePointer when appropriate. Bypassing Swift's type safety in certain places can give us a big performance boost.
• Better random generation of weights. Some applications need weights generated along a normal distribution instead of a uniform distribution.
• Better support for alternative activation functions

Website

• A place for viewing documentation, sharing code and getting help
• This would be fun to have. But it also costs $$

Help and Contributions

Since I'll be making some dramatic changes to this library, I probably won't be accepting contributions until I've at least settled on the new APIs and standards. However, there are a few places where I could use some help:

• Feature/API suggestions: I'm especially interested in feedback from people who have used other machine learning frameworks like TensorFlow, Torch, etc.
• Deep learning experts: If this is you, let's be in touch.
• Logo and branding: The Swift bird is cool, but apparently some fruit company owns it.
• Sponsorship: Asking for money is lame. But it helps things move faster.

Thanks to everybody for using Swift AI. I'm excited to see what we build!

Collin

Performance of activation functions

I see in your code that you're computing activation in a non-vectorized way, i.e. element by element.
I expect the performance of this to be very slow, isn't it?

Thoughts on how to vectorize the activation computation?

Thanks in advance

feature - support caffe models

maybe possible to cherry pick some of this code
https://github.com/naru-hiyoko/simple-cnn-swift

https://github.com/noradaiko/caffe-ios-sample

http://deeplearningkit.org/wp-content/uploads/2016/01/DeepLearningKitPaper.pdf

Scalar multiplication on vectors/matrices

Why is the scalar value negated in the * operator overload?

Convert UIImage to [[Float]]

Hey folks,

Does anybody have a concise way to convert a UIImage to an array of Floats? I have tried to use a variant of the scanImage() function from the HandwritingViewController in the iOS examples but all of my images are coming out to be exactly the same. I have been stuck on this for a few days, not sure where I am going wrong. Any help would be greatly appreciated.

Take care y'all

GPU/Multi-thread CPU acceleration?

I was unsure from the docs, but will you be supporting these on OS X?

Ai

Cannot compile under new Swift 3 (need for latest iOS)

I accept conversion (no choice) but then 2 error messages:
1.

public func toString(decimalPlaces: Int) -> String {
    let power = pow(10.0, Double(decimalPlaces))
    return "\(round(power * self) / power)"
}

--> cannot use mutating member on immutable value "self" is immutable ...

and

let providerRef = CGDataProvider(data: Data(bytes: UnsafePointer(&data), count: data.count * sizeof(PixelData)))

--> ambiguous use of 'init'

errorSum > errorThreshold

Hi

I may have miss something in understanding the train function, which I though would keep on iterating till I get under the errorThreshold. but my errorSum = 105.794 while errorThreshold =0.2 , stop once I have the inputs have been backprogated once? is this the expected behaviour?
Also stupid question, but is there any way to asset the minimum amount of inputs to test for text (english ) recognition ?

Thanks for the support

Raph

Question regarding RNN

Hi Colin,

I'm working on building an RNN right now. Naturally an effective RNN needs a tokenizer. Would you prefer a tokenizer built from scratch or are we allowed to leverage other libraries? In particular, I was thinking of Ayaka Nonaka's Parsimmon's tokenizer feature. Thoughts?

B

There is no example of Song identification (e.g., Shazam).

Separate Storage

I believe there was a discussion about it should be possible to use the framework just by dragging in a single file (currently FFNN.swift). Right now, FFNN conforming to Storage is preventing this from happening. What's the issue with moving this snippet into Storage.swift?

Feature list

Hi, Collin

I wont to make attempt in bringing some new features to this great project! :)
Do you have road map or feature list ?

Thanks,
Roma

Where I can find a Multi-Layer Feed-Forward Neural Network example?

Met this framework tomorrow and I'm finding it awesome, by seeing documentation and reading what's it's possible, but I can't find any example to learn how to use it (feed/predict things). This isn't the complete repo? Where I can find the handwriting example?

Thanks.

AI Project

Hey Collin. I would like to ask if i can use this library to an app that will do this thing: You take a picture of a fruit with your camera and then the app give you results some of the recipes you can make with this fruit! Can i achieve this with this library? If not what should i use to achieve this result? Thanks in advance.

A bit puzzled about the results of using this FFNN

Hi,
I finally tried your FFNN (since until now I just lurked reading the code and the future developments), and I stumbled across something I don't fully understand:

I'm trying to train a FFNN with 784 inputs and 10 outputs, with 100 hidden nodes. Momentum is 0.5 and learning rate 0.7, error function Default and activation Sigmoid.

This is the output of the first iteration of the training (first input/output), I have 50000 items in my training set but all the items give the same result (so imagine this output pasted 50000 times).
The training ends immediately since the error is "0" and every test I make with a validation set "succeeds" with output [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] (that of course is wrong).

Hidden output cache [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
Output errors cache [-0.0, -0.0, -0.0, -0.0, -0.0, 0.0, -0.0, -0.0, -0.0, -0.0]
Hidden errors cache [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
Error 0.0
Iteration 0 got [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] with error 0.0 (expected output [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0]

Do you have an idea of why this happens?

APKit

I get an error saying:
Module file was created by an older version of the compiler; rebuild 'APKit' and try again

when I try to run the iOS example project.

Result of call to 'method' is unused

Xcode is giving me a warning. here is the code:

for (index, image) in self.trainImages.enumerated() {

                try self.network.update(inputs: image)
                let answer = trainAnswers[index]
                try self.network.backpropagate(answer: answer)
}

This is also resulting in the neural network getting stuck and not learning further.

Image classification

Hi Team,

Is it possible to classify the images using this library? like categories the images or detecting the object in image?

Need assistance on this.

Remove unnecessary use of `self`

Following the rejection of this proposal, and conforming to its corresponding rationale, we will be removing all usage of self within Swift AI. The obvious exceptions are instances where the compiler requires it (such as inside blocks), and in places where using self provides clarity which cannot otherwise be determined by the context. These situations, however, are rare, so the general rule of thumb will be "if the compiler can figure it out, so can you."

Recurrent Neural Network (RNN)

I am looking to implement an RNN in swift and I'm wondering if there's already some effort underway to add an RNN to Swift-AI. So that there is no duplication of effort. Is there some RNN effort by somebody?

Thanks in advance.

Testing?

There doesn't seem to be much testing going on. I've got a slave server dedicated to Jenkins. Also TDD would be cool. I heard it's popular with the kids.

Thoughts?

Support for deep FFNNs

Hi! I've developed a similar library, except focused on customization instead of performance. The only feature I miss from my implementation is the ability to specify an arbitrary number of hidden layers, and it would be nice to see it in Swift-AI.

The way I implemented it, the designated initializer would take an [Int] for the hidden parameter, each Int being the size of that hidden layer. The rest of the interface stays pretty much the same.

P.s. why are all member accesses qualified with self.? This is Swift man :)

Explanation

Can someone explain me these lines: let x = (-500 + (Float(index) * 1000) / Float(self.numPoints)) / 100 try! self.network.update(inputs: [x]) let answer = self.sineFunc(x) print("\(x):\(answer)") try! self.network.backpropagate(answer: [answer])
I don't get where the AI calculates the points here.

Is this abandoned due to CoreML?

Apple released CoreML which basically does all this very well and natively. There's no activity here after that.

Anyway, we don't have CoreML on linux, so Swift-AI would be really useful to run AI on the server.

Neural Networks for Regression are not possible

Tried to use a simple Neural Network without an activation layer at the output.
The output needs to be a continuous number (example price).
The API requires an activation function (linear), but the back-propagation is not implemented.

Swift 3 compatibility

Hey, I'd love to see this lib available in Swift 3 so I can implement it in my project.

Thanks 😄

Carthage / Cocoapods support

Hey guys,

I am thinking about making a PR for you guys to support Cocoapods and carthage. You mentioned in that you don't want to do this because your not sure how they will support other platforms in the future. I aggree. Thats why I think 3rd parties should support Cocoapods, Carthage and Swift package manager. If we supported Cocoapods and Carthage we would need to create a Xcode project and that would help a lot on the testing and CI front. What do you guys think. Would you accept this kind of PR?

Image tools

I would like to contribute to this project with image tools. Instead of having the image convertion inside a ViewController.

But for that i need the xcode 8 swift 3 branch merged.

iOS example app Link on home readme is broken.

Genetic Algorithm

This probably isn't an issue for this SDK, I just added this as a "discussion"...

I've been messing around with getting Neural Networks to play games, for example, but I want to take it a step further: Would it be possible to use something like "Mendel" (an iOS SDK for Genetic Algorithms) to evolve a Neural Network using this SDK?

• Tanmay Bakshi (TajyMany)

Basic neural network subroutines (BNNS) in Accelerate framework

Found this while looking at the whats new in macOS 10.12 Sierra and iOS 10
https://developer.apple.com/library/content/releasenotes/MacOSX/WhatsNewInOSX/Articles/OSXv10.html

I see that the Accelerate framework has gotten support for BNNS
https://developer.apple.com/reference/accelerate/1912851-bnns

Maybe BNNS yields better performance than vDSP.

How do I identify other characters?

won't compile under Xcode 8

getting multiple errors, e.g.

Interesting Parallels

Hey @collinhundley am I going out on a limb here by saying that the FFD and RNN are similar with the exception of the RNN possessing what appear to be recursive qualities?

Chinese character recognition?

Logo

Swift AI needs a logo. Any artists out there willing to give it a shot?

Cannot compile with Xcode 8.3

How to use the library as binary classifier for complex object

Hi,
your library looks pretty. I would like to know if I can use it as classifier for data.

Example:

Given a human object classified it as good or bad.

Consider this scenario:

Given a Human struct:

struct Human {
    var name: String
    var title: String
    var job: Job
    var residence: Location
    var isMarried: Bool
}

where:

struct Job {
    var salary: Double
}

struct Location {
    var address: String
}

and this extract of training data:

let human1 = Human(name: "Jonh", title: "Eng.", job: Job(salary: 2000), residence: Location(address: "Street 5"), isMarried: true)
let human2 = Human(name: "Albert", title: "Mr.", job: Job(salary: 0), residence: Location(address: "Street 2"), isMarried: false)
let human3 = Human(name: "Wayne", title: "Mrs.", job: Job(salary: 1000), residence: Location(address: "Street 4"), isMarried: true)
let human4 = Human(name: "Luke", title: "Doctor", job: Job(salary: 4000), residence: Location(address: "Street 5"), isMarried: false)

where human1 -> good
where human2 -> good
where human3-> bad
where human4 -> bad

How can I use your library as classifier for a new human object like this one?

let humanToClassified = Human(name: "Dart", title: "Developer", job: Job(salary: 2000), residence: Location(address: "Street 3"), isMarried: true)

Is humanToClassified good or bad ?

Getting -nan as a weight when training a model

My model is failing to train a NN. All the weights just get set to -nan so infer always fails (returning -nan). Do you have any advice on how to debug?

Sorry for such a vague issue. I'm only able to reproduce this on my training set (not on a super simple, contrived dataset). Hopefully, you've seen issues like this before and can point me in the right direction to give you more info.

Btw, this is a super awesome library. I am really disappointed that CoreML doesn't allow training a NN. The classification of this library is SO MUCH FASTER. Talking about much less than 1 ms compared to 100 ms.