Machine Learning is one of the fastest-growing fields and deep learning represents its true bleeding edge. Deep Learning is a new area of research that is getting us closer in achieving one of the primary objectives of Machine Learning – Artificial Intelligence. Deep Learning is used widely in the fields of image recognition, Natural Language Processing (NLP), self-driving cars, and video classification.
Background
Since the 1940’s, scientists have tried to understand how the actual brain could learn by using cells (neurons) that are connected together. Eventually, artificial neurons were proposed, which were a combination of complex mathematical function referred to as neural network models. The community believed that if a machine can mathematically understand the data and then infer how data was related, then we could say that the machine is learning. Machine Learning is one of the fastest-growing fields and deep learning represents its true bleeding edge. Deep Learning is a new area of research that is getting us closer in achieving one of the primary objectives of Machine Learning – Artificial Intelligence. Deep Learning is used widely in the fields of image recognition, Natural Language Processing (NLP), self-driving cars, and video classification.
For various reasons, such as lack of computing power and the advent of other sophisticated machine learning algorithms, Deep Learning received less attention and research funding during its early period. However, the interest in neural nets was far from over. In mid 2000, some important developments made Deep Learning plausible:
- The possibility of training a very large and deep neural network became real.
- The arrival of powerful GPUs and programming tools made it possible to exploit these large networks.
- Vast amounts of data in the form of images, videos, etc. became available to train the neural network.
Image Reference Domino Data Lab
Applied Deep Learning
Complex Image Recognition
Previously the task of image recognition, which is trivial for the human brain, was much more difficult for machines. But today, deep learning enables machines to recognize images that are difficult even for humans. Deep learning is synonymous to deep neural networks which is a revelation since many consider deep learning as some black box. Most of the systems like security systems, motion sensor games, driverless cars, etc. are making use of the deep learning algorithms and have shown better results.
Classification of Search Queries
Deep learning can be used to classify search queries into different buckets, such asnavigational queries, transactional queries, informational queries, etc. The Word 2 Vec algorithm is used to convert search queries into vectors, which are then converted into images. These images are then arranged into different classes using Convolutional Neural Networks (CNN).
Conversational Assistants
The landscape for conversational assistants can be divided into different types ranging from personal assistants, such as Apple’s Siri, Amazon’s Alexa, Google Home, and Microsoft’s Cortana, to domain specific chat bots that try to solve problems in a specific area. The expectations of users are becoming more demanding and these systems need to be intuitive to keep the user engaged. Building a dialogue-based interface is challenging and problems arise from understanding user intent from a question, correct speech-to-text conversion, identifying context of the conversation, and providing relevant results. Deep learning or deep neural nets are being used to solve many of these problems with techniques such as Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM). Deep learning is a great tool to parse complex noisy unstructured text and build meaningful models to solve specific NLP problems.
Neural Network Architectures and Tools
Some noteworthy neural network architectures that solve NLP problems include:
- A Convolutional Neural Network: Works well for problems related to identifying context. A CNN learns about the morphological information and is able to give good results for correcting misspelled words or context identification. Expresso is the tool which is built on Caffe, the open-source framework, popularly used to develop Convolutional Neural Networks (CNN).
- Word embeddings or character embeddings: Useful where semantic similarity needs to be deduced. Thelgorithm works even for unseen words.
- Recursive Neural Networks (RNN) and LSTM: These can be used in cases where the association of words in a sequence needs to be identified. RNN’s are also good match for slot-filling or sequence labelling problems since the accuracy of the RNN models have been found to be better than Conditional Random Field (CRF) classifiers. The RNN technique is useful in businesses that rely ontask implementation like purchasing a movie ticket, reserving a table in restaurant, or booking a flight.
Of the Deep Learning tools and libraries, Python libraries (Keras & Caffe) are the choice for many developers to implement neural networks. Coupled with Theano or Tensor-flow, they provide abstraction for code execution over GPUs.
Latest Developments
Deep neural nets can be compressed to work with low compute power and memory
Researchers at the Indian Institute of Science (IISc) in Bangalore are working on compressing the neural networks so that they can be run on smaller devices with power similar to mobile devices. This development will help in embedding deep learning applications without using valuable computing and memory resources. Research is also being done on a GUI for designing, training and exploring deep learning frameworks.
Deep learning has become extremely popular in Machine Learning, Artificial Intelligence and data mining since it creates multiple new opportunities in data-related research areas, such as signal processing, pattern recognition as well as NLP.
Many tech giants are already using deep learning for various NLP projects and they have started sharing their APIs to the outside world.
Alexa, a voice enabled cloud offering by Amazon, allows people to connect an external product with a microphone. The system uses deep learning for speech recognition. Recently Amazon has opened up Alexa Skills APIs to add custom NLP. This is an interesting development as it allows user or developers to build custom products like home automation or conversational assistants using Alexa.
Google has also just released a new API for NLP on cloud platform – Cloud Natural Language API. According to Google, this API makes use of its best deep learning models (used in Google Search & Google Now) and offers powerful text analytics tools.
The NLP APIs being provided by these tech companies could prove to be a benefit many who are hoping to develop products around NLP but didn’t have breadth to start work from scratch.
Gartner has predicted five big data trends that will dominate 2016 and Deep Learning is one of them. Many well-known companies are already utilizing Deep Learning technologies, including Apple, Facebook, Google, IBM, Microsoft, PayPal, Pinterest, Twitter, Yahoo, YouTube and others. This technology trend has the potential to have a significant impact on an organization’s long-term plans, programs, and initiatives. As more companies invest in this technology, we will see a sudden acceleration in the industry applications of Deep Learning.
Synerzip Capabilities in Deep Learning
The team at Synerzip is applying deep learning in areas such as building domain specific conversational assistants, collecting insights from unstructured text to expand domain vocabulary, and analyzing customer reviews to provide customer sentiment feedback.
With expertise in Machine Learning and NLP, Synerzip is an ideal partner for companies who need help analyzing their big data efficiently and leveraging that big data to predict markets for expansion, and ultimately, achieve better business revenues.
