Deep learning · TechGene

Definition

A machine-learning approach that progressively extracts and combines features through multiple layers of trainable representations.

Deep learning uses multilayer models to learn representations. Research on deep belief networks in 2006 and large-scale convolutional classification in 2012 illustrate advances in training methods and engineering applications, respectively; multilayer networks have a longer history.

Not all modern deep learning depends on layer-wise pretraining. Data, architecture, optimization, and computing power jointly affect performance; growing scale alone is not sufficient to ensure reliability.

Principles and impact

Problems solved

Reduces complete reliance on manually designed features for complex tasks.

Principles

Learns successive representations through multilayer transformations and jointly adjusts parameters according to a learning objective.

Improvements

Not recorded

Significance

Supports multiple technical approaches, including vision, language, and generative models.

History and applications

People

Geoffrey E. Hinton、Yoshua Bengio、Yann LeCun

Organizations

Not recorded

Events

Not recorded

Applications

Visual recognition、Language modeling

Limitations

Training and inference require substantial resources, and models may suffer from overfitting, fragility, and difficulty of interpretation.

Upstream and downstream relations

Upstream

←Artificial neural networksusesDeep learning learns representations using multilayer neural architectures; this does not imply that all neural networks fall within deep learning.

Sources: A Fast Learning Algorithm for Deep Belief Nets ImageNet Classification with Deep Convolutional Neural Networks

←BackpropagationusesModern deep networks often use backpropagation to compute gradients; this relationship does not present it as the sole necessary condition for every method of training deep networks.

Sources: Learning Representations by Back-propagating Errors ImageNet Classification with Deep Convolutional Neural Networks

←Convolutional neural networksusesDeep convolutional networks are a specific architectural approach through which deep learning is applied to document and visual recognition.

Sources: Gradient-based Learning Applied to Document Recognition ImageNet Classification with Deep Convolutional Neural Networks

←Machine learningspecializesDeep learning uses multiple layers of trainable representations to learn, making it a class of machine-learning methods.

Sources: A Fast Learning Algorithm for Deep Belief Nets ImageNet Classification with Deep Convolutional Neural Networks

Downstream

→Deep reinforcement learningapplied inMethods such as DQN apply deep representations to action-value estimation, combining perceptual learning with reward-driven decision-making.

Sources: Human-level Control through Deep Reinforcement Learning

→Diffusion generative modelsapplied inDeep networks can be used to learn reverse diffusion or denoising processes; the 2015 and 2020 papers provide corresponding examples of generative modeling.

Sources: Deep Unsupervised Learning using Nonequilibrium Thermodynamics Denoising Diffusion Probabilistic Models

→Large language modelsapplied inThe large-scale pretrained language models covered here use deep networks to learn sequence distributions; deep learning provides the training and representation methods.

Sources: Language Models are Few-Shot Learners

Definition

Principles and impact

Problems solved

Principles

Improvements

Significance

History and applications

People

Organizations

Events

Applications

Limitations

Upstream and downstream relations

Upstream

Downstream

Explore this technology's connections