Last Update: September 2, 2017.
Curated list of resources for iOS developers in following topics:
- Core ML
- Machine Learning Libraries
- Deep Learning Libraries
- Computer Vision
- Natural Language Processing
- Speech Recognition (TTS) and Generation (STT)
- Text Recognition (OCR)
- Other AI
- Machine Learning Web APIs
- Opensource ML Applications
- Game AI
- Other related staff
- iOS ML Blogs
- Mobile ML books
- GPU Computing Blogs
- Learn Machine Learning
- Other Lists
Most of the de-facto standard tools in domains listed above are written in iOS-unfriendly languages (Python/Java/R/Matlab) so finding something appropriate for your iOS application may be a challenging task.
Resources are sorted alphabetically or randomly. The order doesn’t reflect my personal preferences or anything else. Some of the resources are awesome, some are great, some are fun, and some can serve as an inspiration.
Pull-requests are welcome here.
- coremltools is a Python package. It contains converters from some popular machine learning libraries to the Apple format.
- Core ML is an Apple framework to run inference on device. It is highly optimized to Apple hardware.
Currently CoreML is compatible (partially) with the following machine learning packages:
Third-party converters available for:
Core ML currently doesn’t support training models, but still, you can replace model by downloading a new one from a server in runtime. Here is a demo of how to do it. It uses generator part of MNIST GAN as Core ML model.
||C++||GNU LGPL 2.1||GitHub||Cocoa Pods|
|lbimproved||k-nearest neighbors and Dynamic Time Warping||C++||Apache 2.0||GitHub|
||Swift||Apache 2.0||GitHub||Swift Package Manager|
||C++||3-clause BSD||GitHub||Cocoa Pods|
||C++||GNU LGPL||GitHub||Cocoa Pods|
||Objective-C||GNU GPL 3.0||GitHub|
Kalvar Lin's libraries
Multilayer perceptron implementations:
- Brain.js - JS
- SNNeuralNet - Objective-C port of brain.js
- MLPNeuralNet - Objective-C, Accelerate
- Swift-AI - Swift
- SwiftSimpleNeuralNetwork - Swift
- ios-BPN-NeuralNetwork - Objective-C
- ios-Multi-Perceptron-NeuralNetwork- Objective-C
- ios-KRDelta - Objective-C
- ios-KRPerceptron - Objective-C
On-Device training and inference
- Birdbrain - RNNs and FF NNs on top of Metal and Accelerate. Not ready for production.
- BrainCore - simple but fast neural network framework written in Swift. It uses Metal framework to be as fast as possible. ReLU, LSTM, L2 …
- Caffe - A deep learning framework developed with cleanliness, readability, and speed in mind. GitHub. [BSD]
- Caffe2 - a cross-platform framework made with expression, speed, and modularity in mind.
- ConvNetSwift - Swift port [work in progress].
- Deep Belief SDK - The SDK for Jetpac’s iOS Deep Belief image recognition framework
- TensorFlow - an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.
- tiny-dnn - header only, dependency-free deep learning framework in C++11.
- Torch is a scientific computing framework with wide support for machine learning algorithms.
Deep Learning: Running pre-trained models on device
These libraries doesn’t support training, so you need to pre-train models in some ML framework.
- Bender - Framework for building fast NNs. Supports TensorFlow models. It uses Metal under the hood.
- Core ML
- DeepLearningKit - Open Source Deep Learning Framework from Memkite for Apple’s tvOS, iOS and OS X.
- Espresso - A minimal high performance parallel neural network framework running on iOS.
- Forge - A neural network toolkit for Metal.
- Keras.js - run Keras models in a web view.
- KSJNeuralNetwork - A Neural Network Inference Library Built atop BNNS and MPS
- MXNet - MXNet is a deep learning framework designed for both efficiency and flexibility.
- Quantized-CNN - compressed convolutional neural networks for Mobile Devices
- WebDNN - You can run deep learning model in a web view if you want. Three modes: WebGPU acceleration, WebAssembly acceleration and pure JS (on CPU). No training, inference only.
Deep Learning: Low-level routines libraries
- BNNS - Apple Basic neural network subroutines (BNNS) is a collection of functions that you use to implement and run neural networks, using previously obtained training data.
- MetalPerformanceShaders - CNNs on GPU from Apple.
- NNPACK - Acceleration package for neural networks on multi-core CPUs. Prisma uses this library in the mobile app.
- STEM - Swift Tensor Engine for Machine-learning
- TensorFlow implementation of knowledge distilling method
- MobileNet-Caffe - Caffe Implementation of Google’s MobileNets
- keras-surgeon - Pruning for trained Keras models.
- ccv - C-based/Cached/Core Computer Vision Library, A Modern Computer Vision Library
- OpenCV – Open Source Computer Vision Library. [BSD]
- OpenFace – a state-of-the art open source tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
- trackingjs – Object tracking in JS
- Vision is an Apple framework for computer vision.
- CoreLinguistics - POS tagging (HMM), ngrams, Naive Bayes, IBM alignment models.
- GloVe Swift package. Vector words representations.
- Twitter text - An Objective-C implementation of Twitter’s text processing library. The library includes methods for extracting user names, mentions headers, hashtags, and more – all the tweet specific language syntax you could ever want.
- Verbal expressions for Swift, like regexps for humans.
- Word2Vec - Original C implementation of Word2Vec Deep Learning algorithm. Works on iPhone like a charm.
- Kaldi-iOS framework - on-device speech recognition using deep learning.
- OpenEars™: free speech recognition and speech synthesis for the iPhone - OpenEars™ makes it simple for you to add offline speech recognition and synthesized speech/TTS to your iPhone app quickly and easily. It lets everyone get the great results of using advanced speech UI concepts like statistical language models and finite state grammars in their app, but with no more effort than creating an NSArray or NSDictionary.
- TLSphinx, Tutorial
- ocrad.js - JS OCR
- Axiomatic - Swift unification framework for logic programming.
- Build Your Own Lisp In Swift
- Logician - Logic programming in Swift
- Swiftlog - A simple Prolog-like language implemented entirely in Swift.
- IBM Watson - Enable Cognitive Computing Features In Your App Using IBM Watson’s Language, Vision, Speech and Data APIs.
- AlchemyAPI - Semantic Text Analysis APIs Using Natural Language Processing. Now part of IBM Watson.
- Microsoft Project Oxford
- Google Prediction engine
- Google Translate API
- Google Cloud Vision API
- Amazon Machine Learning - Amazon ML is a cloud-based service for developers. It provides visualization tools to create machine learning models. Obtain predictions for application using APIs.
- PredictionIO - opensource machine learning server for developers and ML engineers. Built on Apache Spark, HBase and Spray.
- Wit.AI - NLP API
- Yandex SpeechKit Text-to-speech and speech-to-text for Russian language. iOS SDK available.
- Abbyy OCR SDK
- Clarifai - deep learning web api for image captioning. iOS starter project
- MetaMind - deep learning web api for image captioning.
- Api.AI - Build intelligent speech interfaces for apps, devices, and web
- CloudSight.ai - deep learning web API for fine grained object detection or whole screen description, including natural language object captions. Objective-C API client is available.
- DeepDreamer - Deep Dream application
- DeepDreamApp - Deep Dream Cordova app.
- Texture Networks, Lua implementation
- Feedforward style transfer, Lua implementation
- TensorFlow implementation of Neural Style
- Corrosion detection app
- ios_camera_object_detection - Realtime mobile visualize based Object Detection based on TensorFlow and YOLO model
- TensorFlow MNIST iOS demo - Getting Started with Deep MNIST and TensorFlow on iOS
- Drummer App with RNN and Swift
- enVision - Deep Learning Models for Vision Tasks on iOS\
- GoogLeNet on iOS demo
- Neural style in Android
- mnist-bnns - TensorFlow MNIST demo port to BNNS
- Benchmark of BNNS vs. MPS
- VGGNet on Metal
- A Sudoku Solver that leverages TensorFlow and iOS BNNS for deep learning.
- HED CoreML Implementation is a demo with tutorial on how to use Holistically-Nested Edge Detection on iOS with CoreML and Swift
Traditional Computer Vision
- Classical ELIZA chatbot in Swift
- InfiniteMonkeys - A Keras-trained RNN to emulate the works of a famous poet, powered by BrainCore
- Swift implementation of Joel Grus’s “Data Science from Scratch”
- Neural Network built in Apple Playground using Swift
- Introduction to AI Programming for Games
- dlib is a library which has many useful tools including machine learning.
- MicroPather is a path finder and A* solver (astar or a-star) written in platform independent C++ that can be easily integrated into existing code.
- Here is a list of some AI libraries suggested on OGRE3D website. Seems they are mostly written in C++.
- GameplayKit Programming Guide
Other related staff
- Accelerate-in-Swift - Swift example codes for the Accelerate.framework
- cuda-swift - Swift binding to CUDA. Not iOS, but still interesting.
- Dimensional - Swift matrices with friendly semantics and a familiar interface.
- Eigen - A high-level C++ library of template headers for linear algebra, matrix and vector operations, numerical solvers and related algorithms. [MPL2]
- Matrix - convenient matrix type with different types of subscripts, custom operators and predefined matrices. A fork of Surge.
- NDArray - Float library for Swift, accelerated with Accelerate Framework.
- Swift-MathEagle - A general math framework to make using math easy. Currently supports function solving and optimisation, matrix and vector algebra, complex numbers, big int, big frac, big rational, graphs and general handy extensions and functions.
- SwiftNum - linear algebra, fft, gradient descent, conjugate GD, plotting.
- Swix - Swift implementation of NumPy and OpenCV wrapper.
- Surge from Mattt
- Upsurge - generic tensors, matrices on top of Accelerate. A fork of Surge.
- YCMatrix - A flexible Matrix library for Objective-C and Swift (OS X / iOS)
- SigmaSwiftStatistics - A collection of functions for statistical calculation written in Swift.
- SORandom - Collection of functions for generating psuedorandom variables from various distributions
- RandKit - Swift framework for random numbers & distributions.
- fmincg-c - Conjugate gradient implementation in C
- libLBFGS - a C library of Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)
- SwiftOptimizer - QuantLib Swift port.
- IntuneFeatures framework contains code to generate features from audio files and feature labels from the respective MIDI files.
- matchbox - Mel-Frequency-Cepstral-Coefficients and Dynamic-Time-Warping for iOS/OSX. Warning: the library was updated last time when iOS 4 was still hot.
- LibXtract is a simple, portable, lightweight library of audio feature extraction functions.
- Charts - The Swift port of the MPAndroidChart.
- Core Plot
- Awesome iOS charts
- D3.js iOS binding
- BioJS - a set of tools for bioinformatics in the browser. BioJS builds a infrastructure, guidelines and tools to avoid the reinvention of the wheel in life sciences. Community builds modules than can be reused by anyone.
- BioCocoa - BioCocoa is an open source OpenStep (GNUstep/Cocoa) framework for bioinformatics written in Objective-C. [Dead project].
- iBio - A Bioinformatics App for iPhone.
- HDF5Kit - This is a Swift wrapper for the HDF5 file format. HDF5 is used in the scientific comunity for managing large volumes of data. The objective is to make it easy to read and write HDF5 files from Swift, including playgrounds.
- iSwift - Swift kernel for IPython notebook.
Regular mobile ML
- The “Machine, think!” blog by Matthijs Hollemans
- Pete Warden’s blog
Accidental mobile ML
- Google research blog
- Apple Machine Learning Journal
- Invasive Code blog
- Big Nerd Ranch - Use TensorFlow and BNNS to Add Machine Learning to your Mac or iOS App
- Intelligence in Mobile Applications
- An exclusive inside look at how artificial intelligence and machine learning work at Apple
- Presentation on squeezing DNNs for mobile
- Curated list of papers on deep learning models compression and acceleration
- OpenCL for iOS - just a test.
- Exploring GPGPU on iOS.
GPU-accelerated video processing for Mac and iOS. Article.
Concurrency and OpenGL ES - Apple programming guide.
- OpenCV on iOS GPU usage - SO discussion.
- Simon’s Gladman (aka flexmonkey) blog
- Memkite blog - startup intended to create deep learning library for iOS.
- Metal by Example blog
- objc-io article on Metal
Please note that in this section, I’m not trying to collect another list of ALL machine learning study resources, but only composing a list of things that I found useful.
- Academic Torrents. Sometimes awesome courses or datasets got deleted from their sites. But this doesn’t mean, that they are lost.
- Arxiv Sanity Preserver - a tool to keep pace with the ML research progress.
- Immersive Linear Algebra interactive book by J. Ström, K. Åström, and T. Akenine-Möller.
- “Natural Language Processing with Python” - free online book.
- Probabilistic Programming & Bayesian Methods for Hackers - An intro to Bayesian methods and probabilistic programming from a computation/understanding-first, mathematics-second point of view.
- “Deep learning” - the book by Ian Goodfellow and Yoshua Bengio and Aaron Courville
- Original Machine Learning Coursera course by Andrew Ng.
- Machine learning playlist on Youtube.
- Free online interactive book “Neural Networks and Deep Learning”.
- Heterogeneous Parallel Programming course.
- Deep Learning for Perception by Virginia Tech, Electrical and Computer Engineering, Fall 2015: ECE 6504
- CAP 5415 - Computer Vision by UCF
- CS224d: Deep Learning for Natural Language Processing by Stanford
- Machine Learning: 2014-2015 Course materials by Oxford
- Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition.
- Deep Learning for Natural Language Processing (without Magic)
- Videos from Deep Learning Summer School, Montreal 2015.
- Deep Learning Summer School, Montreal 2016