About: Attention (machine learning)

Facets (new session)
Description
Metadata
Settings
- Rule:
- Inverse Functional Properties:
- "Same As":

About: Attention (machine learning) Goto Sponge NotDistinct Permalink

An Entity of Type : owl:Thing, within Data Space : el.dbpedia.org associated with source document(s)

In neural networks, attention is a technique that mimics cognitive attention. The effect enhances some parts of the input data while diminishing other parts — the thought being that the network should devote more focus to that small but important part of the data. Learning which part of the data is more important than others depends on the context and is trained by gradient descent.

Attributes	Values
rdfs:label	Attention (machine learning) (en)
rdfs:comment	In neural networks, attention is a technique that mimics cognitive attention. The effect enhances some parts of the input data while diminishing other parts — the thought being that the network should devote more focus to that small but important part of the data. Learning which part of the data is more important than others depends on the context and is trained by gradient descent. (en)
dbp:wikiPageUsesTemplate	dbt:Reflist dbt:Short_description dbt:Slink dbt:Plain_image_with_caption dbt:Differentiable_computing
Subject	Machine learning
Link from a Wikipage to an external page	https://www.youtube.com/watch%3Fv=AIiwuClvH6k&vl=en-GB https://www.youtube.com/watch%3Fv=yGTUuEx3GkA
thumbnail	wiki-commons:Special:FilePath/Attention-1-sn.png?width=300
foaf:depiction
prov:wasDerivedFrom	http://en.wikipedia.org/wiki/Attention_(machine_learning)?oldid=1073579610&ns=0
Wikipage page ID	66001552 (xsd:integer)
page length (characters) of wiki page	15299 (xsd:nonNegativeInteger)
Wikipage revision ID	1073579610 (xsd:integer)
Link from a Wikipage to another Wikipage	Alex Graves (computer scientist) Attention Thought vector University College London GloVe (machine learning) Machine learning Word2vec Word embedding DeepMind One-hot Softmax function Gradient descent Explainable artificial intelligence Neural network Recurrent neural network
has abstract	In neural networks, attention is a technique that mimics cognitive attention. The effect enhances some parts of the input data while diminishing other parts — the thought being that the network should devote more focus to that small but important part of the data. Learning which part of the data is more important than others depends on the context and is trained by gradient descent. Attention-like mechanisms were introduced in the 1990s under names like multiplicative modules, sigma pi units, and hypernetworks. Its flexibility comes from its role as "soft weights" that can change during runtime, in contrast to standard weights that must remain fixed at runtime. Uses of attention include memory in neural Turing machines, reasoning tasks in differentiable neural computers, language processing in transformers, and multi-sensory data processing (sound, images, video, and text) in perceivers. (en)
foaf:isPrimaryTopicOf	http://en.wikipedia.org/wiki/Attention_(machine_learning)
is Wikipage redirect of	Attention mechanism Dot-product attention Multi-head attention
is Link from a Wikipage to another Wikipage of	AlphaFold Attention (disambiguation) Attention mechanism Attention network Convolutional neural network GPT-2 Dot-product attention Multi-head attention
is foaf:primaryTopic of	http://en.wikipedia.org/wiki/Attention_(machine_learning)

Faceted Search & Find service v1.17_git151 as of Feb 20 2025

Alternative Linked Data Documents: ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3240 as of Nov 11 2024, on Linux (x86_64-ubuntu_focal-linux-gnu), Single-Server Edition (72 GB total memory, 927 MB memory in use)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2025 OpenLink Software