Neural network

Edit · View history

Neural Network

A neural network is a computational model inspired by the biological neural networks that constitute animal brains. It consists of interconnected groups of artificial neurons that process information using a connectionist approach. Neural networks are a fundamental component of machine learning and deep learning, and are widely used for tasks such as classification, regression, and pattern recognition.

History

The concept of artificial neurons dates back to the 1940s with the work of Warren McCulloch and Walter Pitts. In 1958, Frank Rosenblatt introduced the perceptron, a simple single-layer neural network capable of linear classification. Interest waned in the 1970s after the publication of a book by Marvin Minsky and Seymour Papert highlighting the perceptron's limitations. A resurgence began in the 1980s with the development of backpropagation for training multi-layer networks, popularized by Geoffrey Hinton, David Rumelhart, and Ronald J. Williams. Since the 2010s, advances in computing power and large datasets have led to deep neural networks achieving state-of-the-art results in fields like computer vision and natural language processing.

Architecture

Neural networks are composed of layers of nodes (neurons), typically including:

• An input layer that receives raw data
• One or more hidden layers where computations are performed
• An output layer that produces the final result

Each connection between neurons has a weight that is adjusted during training. Most modern networks use non-linear activation functions such as the rectified linear unit (ReLU) or sigmoid.

Training

Training a neural network involves feeding it labeled examples and adjusting weights to minimize the error between predicted and actual outputs. This is achieved through backpropagation combined with an optimization algorithm like stochastic gradient descent. The process often requires large amounts of data and computational resources.

Types

Common types of neural networks include:

• Feedforward neural network – the simplest form where connections do not form cycles
• Convolutional neural network (CNN) – specialized for grid-like data such as images
• Recurrent neural network (RNN) – designed for sequential data with loops
• Generative adversarial network (GAN) – consisting of two networks competing to generate realistic data
• Transformer – a self-attention based architecture prominent in NLP

Applications

Neural networks are applied across many domains, including:

• Image recognition and object detection
• Speech recognition and synthesis
• Machine translation
• Medical diagnosis
• Autonomous vehicles
• Recommender systems