Simple Implementation of Neural Network

Import Libraries

import numpy as np

Activation Function

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
    return x * (1 - x)

Initialize Parameters

def initialize_parameters(input_size, hidden_size, output_size):
    np.random.seed(1)
    W1 = np.random.randn(input_size, hidden_size)
    b1 = np.zeros((1, hidden_size))
    W2 = np.random.randn(hidden_size, output_size)
    b2 = np.zeros((1, output_size))
    return W1, b1, W2, b2

Forward Propagation

def forward_propagation(X, W1, b1, W2, b2):
    Z1 = np.dot(X, W1) + b1
    A1 = sigmoid(Z1)
    Z2 = np.dot(A1, W2) + b2
    A2 = sigmoid(Z2)
    return Z1, A1, Z2, A2

Backward Propagation

def backward_propagation(X, Y, Z1, A1, Z2, A2, W1, W2):
    m = X.shape[0]
    dZ2 = A2 - Y
    dW2 = np.dot(A1.T, dZ2) / m
    db2 = np.sum(dZ2, axis=0, keepdims=True) / m
    dZ1 = np.dot(dZ2, W2.T) * sigmoid_derivative(A1)
    dW1 = np.dot(X.T, dZ1) / m
    db1 = np.sum(dZ1, axis=0, keepdims=True) / m
    return dW1, db1, dW2, db2

Update Parameters

def update_parameters(W1, b1, W2, b2, dW1, db1, dW2, db2, learning_rate):
    W1 -= learning_rate * dW1
    b1 -= learning_rate * db1
    W2 -= learning_rate * dW2
    b2 -= learning_rate * db2
    return W1, b1, W2, b2

Training the Neural Network

def train_neural_network(X, Y, input_size, hidden_size, output_size, iterations, learning_rate):
    W1, b1, W2, b2 = initialize_parameters(input_size, hidden_size, output_size)
    for i in range(iterations):
        Z1, A1, Z2, A2 = forward_propagation(X, W1, b1, W2, b2)
        dW1, db1, dW2, db2 = backward_propagation(X, Y, Z1, A1, Z2, A2, W1, W2)
        W1, b1, W2, b2 = update_parameters(W1, b1, W2, b2, dW1, db1, dW2, db2, learning_rate)
        if i % 1000 == 0:
            loss = np.mean((Y - A2) ** 2)
            print(f"Iteration {i}: Loss {loss}")
    return W1, b1, W2, b2

Predict Function

def predict(X, W1, b1, W2, b2):
    _, _, _, A2 = forward_propagation(X, W1, b1, W2, b2)
    predictions = A2 > 0.5
    return predictions

Example Usage

# Example dataset (XOR problem)
X = np.array([[0, 0],
              [0, 1],
              [1, 0],
              [1, 1]])
Y = np.array([[0],
              [1],
              [1],
              [0]])

# Parameters
input_size = 2
hidden_size = 2
output_size = 1
iterations = 10000
learning_rate = 0.1

# Train the neural network
W1, b1, W2, b2 = train_neural_network(X, Y, input_size, hidden_size, output_size, iterations, learning_rate)

# Predict
predictions = predict(X, W1, b1, W2, b2)
print("Predictions:")
print(predictions)

This code provides a simple example of a neural network implementation from scratch using NumPy Library. The network is trained on the XOR problem, which is a classic example of a problem that is not linearly separable.

What is Perceptron and Neurons in the above program??????

Let’s understand the perceptron and neuron in the above program

1. Input Layer:
   • The input layer has 2 neurons, which correspond to the 2 features of the input data (X).
2. Hidden Layer:
   • The hidden layer has 2 neurons. This is specified by the hidden_size parameter.
3. Output Layer:
   • The output layer has 1 neuron, which corresponds to the single output for binary classification. This is specified by the output_size parameter.

Explanation:

• Each neuron in the input layer is connected to every neuron in the hidden layer through a set of weights (W1) and biases (b1).
• Each neuron in the hidden layer is connected to the neuron in the output layer through another set of weights (W2) and biases (b2).

Breakdown of Neurons:

• Input Layer: 2 neurons (features).
• Hidden Layer: 2 neurons.
• Output Layer: 1 neuron.

Perceptrons in the Program:

• Weights (W1, W2): These represent the connections between the neurons of different layers.
  • W1 connects the input layer to the hidden layer.
  • W2 connects the hidden layer to the output layer.
• Biases (b1, b2): These are additional parameters for each layer that help the model to fit the data better.
  • b1 corresponds to the biases for the hidden layer neurons.
  • b2 corresponds to the biases for the output layer neuron.

Perceptron Structure:

Each perceptron (or neuron) calculates its output by applying the activation function (sigmoid in this case) to the weighted sum of its inputs plus a bias. Here’s the structure:

Hidden Layer Neurons:

Z1 = np.dot(X, W1) + b1  # Weighted sum for hidden layer
A1 = sigmoid(Z1)         # Activation for hidden layer

Output Layer Neuron:

Z2 = np.dot(A1, W2) + b2  # Weighted sum for output layer
A2 = sigmoid(Z2)          # Activation for output layer

Visual Representation

Input Layer   ->  Hidden Layer  ->  Output Layer
  X (2)        ->      A1 (2)      ->     A2 (1)
[feature1]     ->   [neuron1]      ->  [output]
[feature2]     ->   [neuron2]  

Summary:

• Perceptrons: The program includes multiple perceptrons organized into layers.
  • Input Layer: 2 perceptrons.
  • Hidden Layer: 2 perceptrons.
  • Output Layer: 1 perceptron.
• Neurons Generated: A total of 5 neurons (2 input + 2 hidden + 1 output).