• Latest
  • Trending
Mechanics of Neural Network

Mechanics of Neural Network

June 30, 2022
ATC Ghana supports Girls-In-ICT Program

ATC Ghana supports Girls-In-ICT Program

April 25, 2023
Vice President Dr. Bawumia inaugurates  ICT Hub

Vice President Dr. Bawumia inaugurates ICT Hub

April 2, 2023
Co-Creation Hub’s edtech accelerator puts $15M towards African startups

Co-Creation Hub’s edtech accelerator puts $15M towards African startups

February 20, 2023
Data Leak Hits Thousands of NHS Workers

Data Leak Hits Thousands of NHS Workers

February 20, 2023
EU Cybersecurity Agency Warns Against Chinese APTs

EU Cybersecurity Agency Warns Against Chinese APTs

February 20, 2023
How Your Storage System Will Still Be Viable in 5 Years’ Time?

How Your Storage System Will Still Be Viable in 5 Years’ Time?

February 20, 2023
The Broken Promises From Cybersecurity Vendors

Cloud Infrastructure Used By WIP26 For Espionage Attacks on Telcos

February 20, 2023
Instagram and Facebook to get paid-for verification

Instagram and Facebook to get paid-for verification

February 20, 2023
YouTube CEO Susan Wojcicki steps down after nine years

YouTube CEO Susan Wojcicki steps down after nine years

February 20, 2023
Inaugural AfCFTA Conference on Women and Youth in Trade

Inaugural AfCFTA Conference on Women and Youth in Trade

September 6, 2022
Instagram fined €405m over children’s data privacy

Instagram fined €405m over children’s data privacy

September 6, 2022
8 Most Common Causes of a Data Breach

5.7bn data entries found exposed on Chinese VPN

August 18, 2022
  • Consumer Watch
  • Kids Page
  • Directory
  • Events
  • Reviews
Thursday, 19 June, 2025
  • Login
itechnewsonline.com
  • Home
  • Tech
  • Africa Tech
  • InfoSEC
  • Data Science
  • Data Storage
  • Business
  • Opinion
Subscription
Advertise
No Result
View All Result
itechnewsonline.com
No Result
View All Result

Mechanics of Neural Network

by ITECHNEWS
June 30, 2022
in Data Science, Leading Stories
0 0
0
Mechanics of Neural Network

Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks (SNNs), are a subset of machine learning and are at the heart of deep learning algorithms. Their name and structure are inspired by the human brain, mimicking the way that biological neurons signal to one another.

Neural networks comprise multiple layers and each layer contains a certain number of nodes, each node is connected to each node from previous layer node (except input layer).

YOU MAY ALSO LIKE

ATC Ghana supports Girls-In-ICT Program

Vice President Dr. Bawumia inaugurates ICT Hub

The first and the last layer are known as input layer and output layer respectively. Number of nodes in input layer are number of input features which can be represented as a matrix of nxm size where n is number of column and m in number of rows. Output as we can be represented in a matrix.

How ANNs works?

What a neural network do is try to find probability of each node in deep neural network (refer main image) and according to it finding is showing the output.
If you refer to the first image you see, each node is connected to the other node with lines, this line represent an equation.

 

N=Wia+bN=Wi​a+b

whole network can be written as

Z[n]=W[n+1]A[n]+b[n+1]Z[n]=W[n+1]A[n]+b[n+1]

Z → inactivated first layer
W → matrix of weights
A → Data
b → bias

This z function is passed through an activation function which help in understanding complex pattern in ANNs.
If you don’t use any activation function in your network, it will only understand linear relationships, which will lead to poor quality ANNs.

On each iteration, 2 values get tuned, i.e., W and b. each iteration comprises two type of prorogation forward and backward propagation

On the first iteration, some random weight and bias will be picked, and an activation function will be applied.
When the propagation finishes, cost function will be calculated which help in tuning of the Weights and biases.

Note:
code use here is from my custom neural network from digit datause here is from my custom neural network from digit data

self.W1 = np.random.rand(10, 784) - 0.5
self.b1 = np.random.rand(10, 1) - 0.5
self.W2 = np.random.rand(10, 10) - 0.5
self.b2 = np.random.rand(10, 1) - 0.5

def forward_propragation(self):
    Z1 = self.W1.dot(self.X) + self.b1
    A1 = self.ReLU(Z1)
    Z2 = self.W2.dot(A1) + self.b2
    A2 = self.softmax(Z2)

Cost function

MSE=costfunction=12mΣi=1m(yˉ−y)2MSE=costfunction=2m1​Σi=1m​(yˉ​−y)2
def backward_propragation(self, Z1, A1, Z2, A2):
    one_hot_Y = self.one_hot()

    dZ2 = A2 - one_hot_Y
    dW2 = 1 / m * dZ2.dot(A1.T)
    db2 = 1 / m * np.sum(dZ2)

    dZ1 = self.W2.T.dot(dZ2) * self.ReLU_deriv(Z1)
    dW1 = 1 / m * dZ1.dot(self.X.T)
    db1 = 1 / m * np.sum(dZ1)

    return dW1, db1, dW2, db2

This function to be low to have a high accuracy. When we have MSE do iterate back which is called a backward propagation.

By doing partial differentiation of the function, you get new weight and biases

Error at a particular node=rEtotalrWnError at a particular node=rWn​rEtotal​​

to find backward propagation manually

Image description

changing the weight and biases

Image description
small visualization how a neural net works

 

in mu custom neural net, I have 1 hidden layer and 10 nodes in it, if you use see the gradient descent function, I am just propagating forward and backward and updating the values

    def gradient_descent(self):
        for i in range(self.iterations):
            Z1, A1, Z2, A2 = self.forward_propragation()
            dW1, db1, dW2, db2 = self.backward_propragation(Z1, A1, Z2, A2)
            W1, b1, W2, b2 = self.update_params(self.W1, self.b1, self.W2, self.b2, dW1, db1, dW2, db2)
            if i % 10 == 0:
                print("ITERATION: ", i)
                print("PREDICTION: ", self.get_predictions(A2))

Full source code : https://github.com/ishubhamsingh2e/neural-network-scratch

Source: Shubham Singh
Tags: Mechanics of Neural Network
ShareTweetShare
Plugin Install : Subscribe Push Notification need OneSignal plugin to be installed.

Search

No Result
View All Result

Recent News

ATC Ghana supports Girls-In-ICT Program

ATC Ghana supports Girls-In-ICT Program

April 25, 2023
Vice President Dr. Bawumia inaugurates  ICT Hub

Vice President Dr. Bawumia inaugurates ICT Hub

April 2, 2023
Co-Creation Hub’s edtech accelerator puts $15M towards African startups

Co-Creation Hub’s edtech accelerator puts $15M towards African startups

February 20, 2023

About What We Do

itechnewsonline.com

We bring you the best Premium Tech News.

Recent News With Image

ATC Ghana supports Girls-In-ICT Program

ATC Ghana supports Girls-In-ICT Program

April 25, 2023
Vice President Dr. Bawumia inaugurates  ICT Hub

Vice President Dr. Bawumia inaugurates ICT Hub

April 2, 2023

Recent News

  • ATC Ghana supports Girls-In-ICT Program April 25, 2023
  • Vice President Dr. Bawumia inaugurates ICT Hub April 2, 2023
  • Co-Creation Hub’s edtech accelerator puts $15M towards African startups February 20, 2023
  • Data Leak Hits Thousands of NHS Workers February 20, 2023
  • Home
  • InfoSec
  • Opinion
  • Africa Tech
  • Data Storage

© 2021-2022 iTechNewsOnline.Com - Powered by BackUPDataSystems

No Result
View All Result
  • Home
  • Tech
  • Africa Tech
  • InfoSEC
  • Data Science
  • Data Storage
  • Business
  • Opinion

© 2021-2022 iTechNewsOnline.Com - Powered by BackUPDataSystems

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Go to mobile version