YouTip

Neural network is a computational model that mimics the way the human brain processes information. It consists of many interconnected nodes (neurons) arranged in layers. The power of neural networks lies in their ability to automatically learn complex patterns and features from large amounts of data without manually designing feature extractors. With the development of deep learning, neural networks have become a key technology for solving many complex problems. ### Neuron A neuron is the basic unit of a neural network. It receives input signals, performs weighted summation and adds a bias, then processes through an activation function to produce an output. The weights and biases of neurons are parameters that need to be adjusted during the network's learning process. **Input and Output:** * **Input**: The input is the starting point of the network, which can be feature data such as pixel values of images or word vectors of text. * **Output**: The output is the endpoint of the network, representing the model's prediction results, such as class labels in classification tasks. A neuron receives multiple inputs (e.g., x1, x2, ..., xn). If the weighted sum of inputs exceeds the activation threshold (activation potential), it produces a binary output. !(#) The output of a neuron can be viewed as the weighted sum of inputs plus a bias. The mathematical representation of a neuron: !(#) Here, wj is the weight, xj is the input, and Bias is the bias term. ### Layer The layers between the input layer and output layer are called hidden layers. The density and type of connections between layers constitute the network's configuration. A neural network consists of multiple layers, including: * **Input Layer**: Receives raw input data. * **Hidden Layer**: Processes input data. There can be multiple hidden layers. * **Output Layer**: Produces the final output result. Typical neural network architecture: !(#) Demonstration diagram: ② Input data Input image features (brightness, color, edges) into the network. The brighter the circle, the higher the activation value of that neuron, meaning it receives a stronger signal. Input layer Hidden layer Output layer Weight connections Activation signal ### Feedforward Neural Network (FNN) The Feedforward Neural Network (FNN) is a fundamental unit in the neural network family. The characteristic of feedforward neural networks is that data flows from the input layer through one or more hidden layers to the output layer, with no loops or feedback throughout the process. !(#) **Basic structure of feedforward neural network:** * **Input Layer**: The entry point for data into the network. Each node in the input layer represents an input feature. * **Hidden Layer**: One or more layers used to capture non-linear features of the data. Each hidden layer consists of multiple neurons, and each neuron adds non-linear capability through an activation function. * **Output Layer**: Outputs the network's prediction results. The number of nodes is related to the problem type. For example, in classification problems, the number of output nodes equals the number of classes. * **Connection Weights and Biases**: Each neuron's input is weighted by weights, summed, and added with a bias value, then passed through an activation function. ### Recurrent Neural Network (RNN) The Recurrent Neural Network (RNN) is a type of neural network specifically designed for processing sequential data. It can capture dependencies of temporal or sequential information in input data. What makes RNN special is its "memory capability", which allows it to preserve information from previous time steps in the network's hidden state. Recurrent neural networks are used to process data patterns that change over time. In RNN, the same layer is used to receive input parameters and display output parameters in the specified neural network. !(#) PyTorch provides powerful tools for building and training neural networks. Neural networks in PyTorch are implemented through the torch.nn module. The torch.nn module provides various network layers (such as fully connected layers, convolutional layers, etc.), loss functions, and optimizers, making it more convenient to build and train neural networks. !(#) In PyTorch, building a neural network typically requires inheriting from the nn.Module class. nn.Module is the base class for all neural network modules. You need to define the following two parts: * **`__init__()`**: Define network layers. * **`forward()`**: Define the forward propagation process of data. Simple Fully Connected Network (Fully Connected Network): ## Example import torch import torch.nn as nn # Define a simple neural network model class SimpleNN(nn.Module): def __init__ (self): super(SimpleNN,self). __init__ () # Define a fully connected layer from input layer to hidden layer self.fc1= nn.Linear(2,2)# Input 2 features, output 2 features # Define a fully connected layer from hidden layer to output layer self.fc2= nn.Linear(2,1)# Input 2 features, output 1 prediction def forward(self, x): # Forward propagation x = torch.relu(self.fc1(x))# Use ReLU activation function x =self.fc2(x)# Output layer return x # Create model instance model = SimpleNN() # Print model print(model) The output is as follows: SimpleNN( (fc1): Linear(in_features=2, out_features=2, bias=True) (fc2): Linear(in_features=2, out_features=1, bias=True)) PyTorch provides many common neural network layers. Here are a few common ones: * **`nn.Linear(in_features, out_features)`**: Fully connected layer, inputs `in_features` features, outputs `out_features` features. * **`nn.Conv2d(in_channels, out_channels, kernel_size)`**: 2D convolutional layer, used for image processing. * **`nn.MaxPool2d(kernel_size)`**: 2D max pooling layer, used for dimensionality reduction. * **`nn.ReLU()`**: ReLU activation function, commonly used in hidden layers. * **`nn.Softmax(dim)`**: Softmax activation function, usually used in output layer for multi-class classification problems. ### Activation Function Activation functions determine whether a neuron should be activated. They are non-linear functions that enable neural networks to learn and perform more complex tasks. Common activation functions include: * Sigmoid: Used for binary classification problems, output values between 0 and 1. * Tanh: Output values between -1 and 1, commonly used before the output layer. * ReLU (Rectified Linear Unit): One of the most popular activation functions currently, defined as `f(x) = max(0, x)`, helps solve the vanishing gradient problem. * Softmax: Commonly used in the output layer for multi-class classification problems, converts outputs to probability distributions. ## Example import torch.nn.functional as F # ReLU activation output = F.relu(input_tensor) # Sigmoid activation output = torch.sigmoid(input_tensor) # Tanh activation output = torch.tanh(input_tensor) ### Loss Function Loss functions measure the difference between the model's