Last Update: January 12, 2018.

Curated list of resources for iOS developers in following topics:

Most of the de-facto standard tools in AI-related domains are written in iOS-unfriendly languages (Python/Java/R/Matlab) so finding something appropriate for your iOS application may be a challenging task.

This list consists mainly of libraries written in Objective-C, Swift, C, C++, JavaScript and some other languages that can be easily ported to iOS. Also, I included links to some relevant web APIs, blog posts, videos and learning materials.

Resources are sorted alphabetically or randomly. The order doesn’t reflect my personal preferences or anything else. Some of the resources are awesome, some are great, some are fun, and some can serve as an inspiration.

Have fun!

Pull-requests are welcome here.

Core ML

  • coremltools is a Python package. It contains converters from some popular machine learning libraries to the Apple format.
  • Core ML is an Apple framework to run inference on device. It is highly optimized to Apple hardware.

Currently CoreML is compatible (partially) with the following machine learning packages via coremltools python package:

Third-party converters to CoreML format are also available for some models from:

There are many curated lists of pre-trained neural networks in Core ML format: [1], [2], [3].

Core ML currently doesn’t support training models, but still, you can replace model by downloading a new one from a server in runtime. Here is a demo of how to do it. It uses generator part of MNIST GAN as Core ML model.

General-Purpose Machine Learning Libraries

Library Algorithms Language License Code Dependency manager
AIToolbox
  • Graphs/Trees
    • Depth-first search
    • Breadth-first search
    • Hill-climb search
    • Beam Search
    • Optimal Path search
  • Alpha-Beta (game tree)
  • Genetic Algorithms
  • Constraint Propogation
  • Linear Regression
  • Non-Linear Regression
    • parameter-delta
    • Gradient-Descent
    • Gauss-Newton
  • Logistic Regression
  • Neural Networks
    • multiple layers, several non-linearity models
    • on-line and batch training
    • feed-forward or simple recurrent layers can be mixed in one network
    • LSTM network layer implemented - needs more testing
    • gradient check routines
  • Support Vector Machine
  • K-Means
  • Principal Component Analysis
  • Markov Decision Process
    • Monte-Carlo (every-visit, and first-visit)
    • SARSA
  • Single and Multivariate Gaussians
  • Mixture Of Gaussians
  • Model validation
  • Deep Network
    • Convolution layers
    • Pooling layers
    • Fully-connected NN layers
 Swift Apache 2.0

GitHub

dlib
  • Deep Learning
  • Support Vector Machines
  • Reduced-rank methods for large-scale classification and regression
  • Relevance vector machines for classification and regression
  • A Multiclass SVM
  • Structural SVM
  • A large-scale SVM-Rank
  • An online kernel RLS regression
  • An online SVM classification algorithm
  • Semidefinite Metric Learning
  • An online kernelized centroid estimator/novelty detector and offline support vector one-class classification
  • Clustering algorithms: linear or kernel k-means, Chinese Whispers, and Newman clustering
  • Radial Basis Function Networks
  • Multi layer perceptrons
 C++ Boost GitHub
FANN
  • Multilayer Artificial Neural Network
  • Backpropagation (RPROP, Quickprop, Batch, Incremental)
  • Evolving topology training
 C++ GNU LGPL 2.1 GitHub Cocoa Pods
lbimproved k-nearest neighbors and Dynamic Time Warping C++ Apache 2.0 GitHub
MAChineLearning
  • Neural Networks
    • Activation functions: Linear, ReLU, Step, sigmoid, TanH
    • Cost functions: Squared error, Cross entropy
    • Backpropagation: Standard, Resilient (a.k.a. RPROP).
    • Training by sample or by batch.
  • Bag of Words
  • Word Vectors
 Objective-C BSD 3-clause GitHub

MLKit
  • Linear Regression: simple, ridge, polynomial
  • Multi-Layer Perceptron, & Adaline ANN Architectures
  • K-Means Clustering
  • Genetic Algorithms
 Swift MIT GitHub Cocoa Pods

Mendel		 Evolutionary/genetic algorithms Swift ? GitHub
multilinear-math
  • Linear algebra and tensors
  • Principal component analysis
  • Multilinear subspace learning algorithms for dimensionality reduction
  • Linear and logistic regression
  • Stochastic gradient descent
  • Feedforward neural networks
    • Sigmoid
    • ReLU
    • Softplus activation functions
 Swift Apache 2.0 GitHub Swift Package Manager
OpenCV
  • Multi-Layer Perceptrons
  • Boosted tree classifier
  • decision tree
  • Expectation Maximization
  • K-Nearest Neighbors
  • Logistic Regression
  • Bayes classifier
  • Random forest
  • Support Vector Machines
  • Stochastic Gradient Descent SVM classifier
  • Grid search
  • Hierarchical k-means
  • Deep neural networks
 C++ 3-clause BSD GitHub Cocoa Pods

Shark
  • Supervised:
    • Linear discriminant analysis (LDA)
    • Fisher–LDA
    • Linear regression
    • SVMs
    • FF NN
    • RNN
    • Radial basis function networks
    • Regularization networks
    • Gaussian processes for regression
    • Iterative nearest neighbor classification and regression
    • Decision trees
    • Random forest
  • Unsupervised:
    • PCA
    • Restricted Boltzmann machines
    • Hierarchical clustering
    • Data structures for efficient distance-based clustering
  • Optimization:
    • Evolutionary algorithms
    • Single-objective optimization (e.g., CMA–ES)
    • Multi-objective optimization
    • Basic linear algebra and optimization algorithms
 C++ GNU LGPL GitHub Cocoa Pods

YCML
  • Gradient Descent Backpropagation
  • Resilient Backpropagation (RProp)
  • Extreme Learning Machines (ELM)
  • Forward Selection using Orthogonal Least Squares (for RBF Net), also with the PRESS statistic
  • Binary Restricted Boltzmann Machines (CD & PCD)
  • Optimization algorithms:
    • Gradient Descent (Single-Objective, Unconstrained)
    • RProp Gradient Descent (Single-Objective, Unconstrained)
    • NSGA-II (Multi-Objective, Constrained)
 Objective-C GNU GPL 3.0 GitHub

Kalvar Lin's libraries		 Objective-C MIT GitHub

Multilayer perceptron implementations:

Deep Learning Libraries:

On-Device training and inference

  • Birdbrain - RNNs and FF NNs on top of Metal and Accelerate. Not ready for production.
  • BrainCore - simple but fast neural network framework written in Swift. It uses Metal framework to be as fast as possible. ReLU, LSTM, L2 …
  • Caffe - A deep learning framework developed with cleanliness, readability, and speed in mind. GitHub. [BSD]
  • Caffe2 - a cross-platform framework made with expression, speed, and modularity in mind.
  • Convnet.js - ConvNetJS is a Javascript library for training Deep Learning models by Andrej Karpathy. GitHub
  • Deep Belief SDK - The SDK for Jetpac’s iOS Deep Belief image recognition framework
  • TensorFlow - an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.
  • tiny-dnn - header only, dependency-free deep learning framework in C++11.
  • Torch is a scientific computing framework with wide support for machine learning algorithms.

Deep Learning: Running pre-trained models on device

These libraries doesn’t support training, so you need to pre-train models in some ML framework.

  • Bender - Framework for building fast NNs. Supports TensorFlow models. It uses Metal under the hood.
  • Core ML
  • DeepLearningKit - Open Source Deep Learning Framework from Memkite for Apple’s tvOS, iOS and OS X.
  • Espresso - A minimal high performance parallel neural network framework running on iOS.
  • Forge - A neural network toolkit for Metal.
  • Keras.js - run Keras models in a web view.
  • KSJNeuralNetwork - A Neural Network Inference Library Built atop BNNS and MPS
  • MXNet - MXNet is a deep learning framework designed for both efficiency and flexibility.
  • Quantized-CNN - compressed convolutional neural networks for Mobile Devices
  • WebDNN - You can run deep learning model in a web view if you want. Three modes: WebGPU acceleration, WebAssembly acceleration and pure JS (on CPU). No training, inference only.

Deep Learning: Low-level routines libraries

  • BNNS - Apple Basic neural network subroutines (BNNS) is a collection of functions that you use to implement and run neural networks, using previously obtained training data.
  • MetalPerformanceShaders - CNNs on GPU from Apple.
  • NNPACK - Acceleration package for neural networks on multi-core CPUs. Prisma uses this library in the mobile app.
  • STEM - Swift Tensor Engine for Machine-learning

Deep Learning: Model Compression

Computer Vision

Natural Language Processing

  • CoreLinguistics - POS tagging (HMM), ngrams, Naive Bayes, IBM alignment models.
  • GloVe Swift package. Vector words representations.
  • NSLinguisticTagger
  • Parsimmon
  • Twitter text - An Objective-C implementation of Twitter’s text processing library. The library includes methods for extracting user names, mentions headers, hashtags, and more – all the tweet specific language syntax you could ever want.
  • Verbal expressions for Swift, like regexps for humans.
  • Word2Vec - Original C implementation of Word2Vec Deep Learning algorithm. Works on iPhone like a charm.

Speech Recognition (TTS) and Generation (STT)

Text Recognition (OCR)

Other AI

Machine Learning Web APIs

Opensource ML Applications

Deep Learning

Traditional Computer Vision

NLP

Other

Game AI

Other related staff

Linear algebra

  • Accelerate-in-Swift - Swift example codes for the Accelerate.framework
  • cuda-swift - Swift binding to CUDA. Not iOS, but still interesting.
  • Dimensional - Swift matrices with friendly semantics and a familiar interface.
  • Eigen - A high-level C++ library of template headers for linear algebra, matrix and vector operations, numerical solvers and related algorithms. [MPL2]
  • Matrix - convenient matrix type with different types of subscripts, custom operators and predefined matrices. A fork of Surge.
  • NDArray - Float library for Swift, accelerated with Accelerate Framework.
  • Swift-MathEagle - A general math framework to make using math easy. Currently supports function solving and optimisation, matrix and vector algebra, complex numbers, big int, big frac, big rational, graphs and general handy extensions and functions.
  • SwiftNum - linear algebra, fft, gradient descent, conjugate GD, plotting.
  • Swix - Swift implementation of NumPy and OpenCV wrapper.
  • Surge from Mattt
  • Upsurge - generic tensors, matrices on top of Accelerate. A fork of Surge.
  • YCMatrix - A flexible Matrix library for Objective-C and Swift (OS X / iOS)

Statistics, random numbers

  • SigmaSwiftStatistics - A collection of functions for statistical calculation written in Swift.
  • SORandom - Collection of functions for generating psuedorandom variables from various distributions
  • RandKit - Swift framework for random numbers & distributions.

Mathematical optimization

  • fmincg-c - Conjugate gradient implementation in C
  • libLBFGS - a C library of Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)
  • SwiftOptimizer - QuantLib Swift port.

Feature extraction

  • IntuneFeatures framework contains code to generate features from audio files and feature labels from the respective MIDI files.
  • matchbox - Mel-Frequency-Cepstral-Coefficients and Dynamic-Time-Warping for iOS/OSX. Warning: the library was updated last time when iOS 4 was still hot.
  • LibXtract is a simple, portable, lightweight library of audio feature extraction functions.

Data Visualization

Bioinformatics (kinda)

  • BioJS - a set of tools for bioinformatics in the browser. BioJS builds a infrastructure, guidelines and tools to avoid the reinvention of the wheel in life sciences. Community builds modules than can be reused by anyone.
  • BioCocoa - BioCocoa is an open source OpenStep (GNUstep/Cocoa) framework for bioinformatics written in Objective-C. [Dead project].
  • iBio - A Bioinformatics App for iPhone.

Big Data (not really)

  • HDF5Kit - This is a Swift wrapper for the HDF5 file format. HDF5 is used in the scientific comunity for managing large volumes of data. The objective is to make it easy to read and write HDF5 files from Swift, including playgrounds.

IPython + Swift

  • iSwift - Swift kernel for IPython notebook.

iOS ML Blogs

Regular mobile ML

Accidental mobile ML

Other

GPU Computing Blogs

Metal

Mobile ML Books

Learn Machine Learning

Please note that in this section, I’m not trying to collect another list of ALL machine learning study resources, but only composing a list of things that I found useful.

  • Academic Torrents. Sometimes awesome courses or datasets got deleted from their sites. But this doesn’t mean, that they are lost.
  • Arxiv Sanity Preserver - a tool to keep pace with the ML research progress.

Free Books

Free Courses

Other Lists