Last Updated: 2/6/2023

Deep Learning

What is Deep Learning?

Deep Learning, or DL for short, is a subfield of machine learning that uses multi-layered artificial neural networks to model and solve complex problems. It is inspired by the structure and function of the human brain, and its goal is to simulate the neural processes that enable human perception, thought, and decision-making.

“In short, deep learning is a subset of methods in the machine learning toolbox, primarily using artificial neural networks, which are a class of algorithm loosely inspired by the human brain.” — Deep Learning - Andrew W. Trask.

Although Machine Learning was a huge step in the right direction, something was still missing from its algorithms:

• First, the work of collecting, cleaning, and selecting the features still had to be done by hand by the developers, which involved whole days or weeks of dipping into the context of the problem to choose the best features.
• Secondly and most importantly, machine learning algorithms didn't have the computational power to train themselves. Neural networks have existed since 1958 and are, part of the machine learning algorithms toolbox, however, they were single layer structures (known as perceptrons). Training a deep neural network with the hardware available at that time was impossible. Thanks to developments in GPUs and other more powerful hardware, deep networks were able to deliver results and leave their single-layer predecessors in the past.

Deep Learning
Deep Learning is a sub-area of machine learning where algorithms with multiple successive layers (known as neural networks) can extract features from the input data and make predictions just like classic machine learning algorithms do.

AI, ML and DL

Some people may confuse the terms AI, ML and DL. And although they are related, they are not the same:

• Artificial Intelligence (AI) is the broader concept of machines being able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and language translation.
• Machine Learning (ML) is a subset of AI that involves teaching computers to learn from data, without being explicitly programmed. It involves building mathematical models that can make predictions or decisions based on input data.
• Deep Learning (DL) is a type of machine learning that uses artificial neural networks, which are modeled after the structure of the human brain. It is called "deep" because these neural networks have many layers, allowing the computer to learn and make complex decisions. Deep learning is particularly useful for tasks such as image and speech recognition.

How Deep Learning Works?

Deep Learning works by building a network of artificial neurons that process information sequentially. Each layer of the network is responsible for extracting higher-level features from the raw data, and the final output is generated by combining these features. This approach is often referred to as “deep” because of the many layers involved. We fed the network with a large amount of data to train a Deep Learning model. The model then makes predictions based on this data, and the accuracy of these predictions is used to adjust the weights of the artificial neurons. This process is repeated many times, allowing the model to improve its accuracy continuously. One of the key benefits of Deep Learning is its ability to automatically identify patterns and features in data without being explicitly told what to look for. This makes it well-suited for tasks such as image and speech recognition, natural language processing, and autonomous decision-making.

“In this context, learning means finding a set of values for the weights of all layers in a network, such that the network will correctly map example inputs to their associated targets.” — Deep Learning with Python, François Chollet.

Unlike other algorithms, most neural networks require a long training in which they try to find the parameters that allow them to extract the best features and obtain the most similar results to those expected. The number of parameters of a neural network can go from hundreds to billions (GPT-3's full version has a capacity of 175 billion machine learning parameters). Due to this, training time is way bigger than traditional algorithms. Unlike traditional Machine Learning algorithms, neural networks require much more time to train and can get stuck.

For example, assume the input data is an image (or matrix of pixels). The first layer typically abstracts the pixels and recognizes the edges of features in the picture. The next layer might build simple features from the edges, such as leaves and branches. The next layer could then recognize a tree, and so on. The data passing from one layer to the next could be seen as a transformation, turning the output of one layer into the input for the next one. Each layer has a different level of abstraction, and the machine can learn which features of the data to place in which layer/level on its own.

Deep Learning in real life

Deep Learning is being widely used today, and it is one of the most popular fields of Artificial Intelligence since it can solve a huge number of complex tasks such as:

• Computer Vision: images can train the machine to recognize features, and now the machines are demonstrating "superhuman" accuracy for image recognition. Some examples include image classification, object detection, and segmentation.
• Natural Language Processing: neural networks, especially recurrent ones, have allowed enormous advances in language-related issues. Some examples include text prediction, sentiment analysis, information extraction, translation, text-to-speech, and Question Answering.
• Time Series Analysis: networks with temporary data retention capacity allow predictions and forecasting on financial data, weather, or product prices.
• Content Generation: neural networks are the only ones capable of generating content with acceptable quality to the point that some GANs can even fool humans.
• Recommender Systems: Deep Learning can analyze large amounts of user data and make personalized recommendations, such as suggesting movies or products that a user may be interested in.

Extra Resources

Books

Videos