What is Deep Learning
What is deep learning refers to one of the most interesting questions asked today. The term Deep Learning is a new approach in machine learning that particularly benefits from big data. It can be considered as a new area of machine learning that moves machine learning actually closer to one of its original goals referring to artificial intelligence. This article provides an overview of the different methods and architectures under this umbrella term. More general details can be found in our article deep learning.
Convolutional Neural Network (CNN)
The connectivity architecture of a CNN is inspired by the organization of the animal visual cortex and works particularly well for image datasets. As a consequence the CNN architecture and its key elements like convolutions can be nicely used in problems where spatial properties are assumed to be in the dataset. This is related to the inherent architecture property based on shared weights and translation invariance characteristics. Therefore CNN are sometimes called ConvNets or shift invariant or space invariant artificial neural network (SIANN). In many areas the question of ‘what is deep learning’ is answered with CNNs representing one of the most often used techniques for it. More details can be found in our article on Convolutional Neural Network.
Siamese Neural Networks
The architecture of this neural network contains two or more identical sub networks that have the same parameters and weights. Sharing weights is thus similar to the CNN architecture above and helps to reduce the number of parameters and to reduce overfitting. Also the parameter updating is mirrored across both of these sub networks. As a consequence the Siamese Neural Networks are used for problems that involve finding similarity or a certain relationship between two comparable data input elements. One example is to use two sentences with each of the sub networks and the output is how similar both of these sentences are based on a specific similarity measure. This in turn is often used in signature verification between a new signature and the original in order to determine if both signatures have been created by the same person.
Generative Adversarial Network (GAN)
The architecture of a GAN is based on a system of two neural networks that compete against each other and is used in unsupervised machine learning problems. In contrast to Siamese Neural Networks above, the architecture is different in the sense that one network is generative and the other is discriminative. The approach is that the generative neural network trains with the objective to increase the error rate of the other discriminative network. The idea is that the generative neural network fools the discriminative network by using new synthesized instances that appear to have come from the real dataset. This architecture is typically implemented by using a deconvolutional neural network for the generative model and a convolutional neural network for the discriminative model. This architecture is often used in computer vision applications.
More Details on What is Deep Learning
We recommend to check out the following video about this subject:
