If you are beginner or intermediate in Data Science or AI at some point you might I have crossed a point where you were unable to decide which framework to use for AI or Ml. We will walk through different types of frameworks in this blog and will have high level undestanding of different frameworks.
Table of Contents:
1. Pytorch & Torch
2. TensorFlow
3. Caffe
4. Caffe2 ( now part of Pytorch)
5. Theano ( almost dead)
6. CNTK
7. Chainer
8. DSSTNE
9. DyNet
10. Gensim
11. Keras
12. MxNet
13. Gluon
14. Paddle
15. BigDL
16. Other ML
Lets get into knowing the above frameworks!!
Pytorch & Torch
Torch is a computational framework with an API written in Lua that supports machine-learning algorithms. Some version of it is used by large tech companies such as Facebook and Twitter, which devote in-house teams to customizing their deep learning platforms. Lua is a multi-paradigm scripting language that was developed in Brazil in the early 1990s.
Pytorch is a python version of torch open-sourced by facebook. PyTorch offers dynamic computation graphs, which let you process variable-length inputs and outputs, which is useful when working with RNNs. Machine Learning researches prefer pytorch as it allows certain complex architectures to be built easily.
Pros and Cons:
(+) Lots of modular pieces that are easy to combine
(+) Easy to write your own layer types and run on GPU
(+) Lots of pretrained models
(-) You usually write your own training code (Less plug and play)
(-) No commercial support
(-) Spotty documentation
Tensorflow
TensorFlow is written with a Python API over a C/C++ engine that makes it run faster, developed and open-sourced by Google. TensorFlow is about more than deep learning. TensorFlow actually has tools to support reinforcement learning and other algos. TensorFlow runs dramatically slower than other frameworks such as CNTK and MxNet.
For the moment, TensorFlow does not support so-called “inline” matrix operations, but forces you to copy a matrix in order to perform an operation on it. Copying very large matrices is costly in every sense. TF takes 4x as long as the state of the art deep learning tools.
Pros and Cons:
(+) Python + Numpy
(+) Computational graph abstraction, like Theano
(+) Faster compile times than Theano
(+) TensorBoard for visualization
(+) Data and model parallelism
(-) Slower than other frameworks
(-) Much “fatter” than Torch; more magic
(-) Not many pretrained models
(-) Computational graph is pure Python, therefore slow
(-) No commercial support
(-) Drops out to Python to load each new training batch
(-) Not very toolable
(-) Dynamic typing is error-prone on large software projects
If you are interested in further digging, below are few links which explain nd comapare in detail:
Comparing Pytorch and Tensorflow
A critic on Tensorflow
Caffe
Caffe is widely used for machine-vision problems. It ported Matlab’s implementation of fast convolutional nets to C and C++.
Pros and Cons:
(+) Good for feedforward networks and image processing
(+) Good for finetuning existing networks
(+) Train models without writing any code
(+) Python interface is pretty useful
(-) Need to write C++ / CUDA for new GPU layers
(-) Not good for recurrent networks
(-) Cumbersome for big networks (GoogLeNet, ResNet)
(-) Not extensible, bit of a hairball
(-) No commercial support
(-) Probably dying; slow development
Caffe2
Caffe2 is the successor of Caffe, now its part of Pytorch.
Theano: RIP Theano and its Ecosystem
Theano wriiten in python, is a library that handles multidimensional arrays, like Numpy. Used with other libs, it is well suited to data exploration and intended for research. Its development has been ceased.
CNTK
CNTK stands for “Computational Network Toolkit”. CNTK is Microsoft’s open-source deep-learning framework, includes s feed-forward DNNs, convolutional nets and recurrent networks.
Chainer
Chainer is an open-source neural network framework with a Python API, whose core team of developers work at Preferred Networks. Until the advent of DyNet at CMU, and PyTorch at Facebook, Chainer was the leading neural network framework for dynamic computation graphs, or nets that allowed for input of varying length, a popular feature for NLP tasks
DSSTNE
DSSTNE stands for “Deep Scalable Sparse Tensor Network Engine”, developed by Amazon is a library for building models for machine and deep learning.
Although Amazon now preffered MxNet on aws.
DyNet
DyNet stands for “Dynamic Neural Network toolkit”, came out of Carnegie Mellon University and used to be called cnn. Its notable feature is the dynamic computation graph, which allows for inputs of varying length, which is great for NLP.
Gensim
Gensim is a fast implementation of word2vec implemented in Python. While Gensim is not a general purpose ML platform, for word2vec, it is at least an order of magnitude faster than TensorFlow. It is supported by the NLP consulting firm Rare Technologies.
Keras
Keras is a deep-learning library that sits atop TensorFlow and Theano, providing an intuitive API inspired by Torch.
Pros and Cons:
(+) Intuitive API inspired by Torch
(+) Works with Theano, TensorFlow and Deeplearning4j backends (CNTK backend to come)
(+) Fast growing framework
(+) Likely to become standard Python API for NNs
MxNet:
A fast and flexible library, MxNet involves Pedro Domingos and a team of researchers at the University of Washington. MxNet is a machine-learning framework with APIs is languages such as R, Python and Julia which has been adopted by Amazon Web Services.
Gluon
Gluon is an API over Amazon’s MxNet that was introduced by Amazon and Microsoft. It will also integrate with Microsoft’s CNTK. While it is similar to Keras in its intent and place in the stack, it is distinguished by its dynamic computation graph, similar to Pytorch and Chainer, and unlike TensorFlow or Caffe.
Paddle
Paddle is a deep-learning framework created and supported by Baidu. Its name stands for PArallel Distributed Deep LEarning. Paddle is the most recent major framework to be released, and like most others, it offers a Python API.
BigDL
BigDL, a new deep learning framework with a focus on Apache Spark, has a focus on Scala.
Other ML frameworks
The deep-learning frameworks listed above are more specialized than general machine-learning frameworks, of which there are many. Few popular Ml frameworks are:
scikit-learn
Apache Mahout
Apache SystemDs( formerly SystemMl)
Happy reading. For updates please follow me on medium and linkedIn .
Thanks & Regards.
