Building a Deep Neural Network in 6 Lines.

Power of (Python + Keras) modern deep Learning tools:

This guide walks through the process of building and training an Artificial Neural Network (ANN) using TensorFlow's Keras library. We'll cover everything from initialization to training, with explanations of key concepts along the way.

In just 3 lines of code, we will see how we can create a deep neural network.

Setup and Imports

import tensorflow as tf
from tensorflow import keras

What is Artificial Neural Network (ANN)?

An ANN is a model inspired by the human brain's structure and function. Just as our brain has approximately 80 billion neurons organized in different layers, an ANN consists of artificial neurons arranged in a sequence of layers.

Brain Analogy:

• Human brain: ~80 billion neurons arranged in layers
• Each layer might contain millions of neurons
• Information flows through these layers sequentially

ANN Structure:

• Multiple layers of artificial neurons
• Neurons in each layer connect to neurons in the next layer
• Information flows from input layer through hidden layers to output layer

Sequential model is one common way to build ANNs

ANN is like the concept of "a building". Sequential is like saying "I'm going to build it one floor at a time".

Initializing Artificial neural network as sequence of layers. We will instantiate like

variable = object of certain class

it's a sequential class, which allows exactly to build an ANN, but as a sequence of layers as opposed to computation graph.

import tensorflow as tf
from tensorflow import keras

ann = tf.keras.models.Sequential()

here, we are a basic creating our neural network to be a sequential class.

Next, we will look into adding layers such as input, hidden and output layers.

First layer in neural network:

The way to add a fully connected layer into an artificial neural network is to use dense class from tensorflow.Keras. We are going to use this class to create our first layer.

import tensorflow as tf
from tensorflow import keras
ann = tf.keras.models.Sequential()

ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))

in this dense function we can specify how many hidden neurons we want to have?

Famous question!!

How do we know how many neurons we want?

Is there a rule of thumb or should we just experiment?

It's a work of an artist. We need to pick 1 number here which wouldn't sound irrelevant or extravagant.

Adding first layer to our neural network, thus creating a shallow neural network .

Second layer (hidden) in neural network:

To transform from shallow network to deep network we simply add one more layer

import tensorflow as tf
from tensorflow import keras
ann = tf.keras.models.Sequential()
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))

ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))

just by copying the and pasting the same piece of code twice, we have a deep neural network.

What about the output layer?

import tensorflow as tf
from tensorflow import keras
ann = tf.keras.models.Sequential()
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))

ann.add(tf.keras.layers.Dense(units = _, activation=_))

How many neurons do we need at the output layer?

Depends on what we are doing? if we are predicting something like , a user will leave the app vs not. A user will leave the credit card company vs not. In binary classification case we have output of either 0 or 1. The output layer contains the dimension of the output you want to predict. So we want to pass 1 neurons.

What about the activation function?

we will be using sigmoid activation function instead of relu in the output. Why is that?Because sigmoid function allows to get not only he ultimate predictions but even better, it'll give you the probabilities that the binary outcome is 1 or 0.

1. All the fully connected layers will get Rectifier linear unit i.e Relu, whereas only the output will get the sigmoid function.
2. If we were predicting categorical data, our output activation function should not be sigmoid, rather should be softmax.

import tensorflow as tf
from tensorflow import keras
ann = tf.keras.models.Sequential()
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))

ann.add(tf.keras.layers.Dense(units = 1, activation='sigmoid'))

So far our architecture looks like:

Thus creating a deep neural network in 3 lines of code:

ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 1, activation='sigmoid'))

Our model is not ready to be trained on the data yet. First we need to compile our ANN with:

1. optimizer
2. loss function

then we will train our network .

Training Artificial Neural Network:

ann.compile(optimizer= _, loss= _, metrics =_)

parameter for the function:

1. Optimizer
2. By default Adam optimizer is a performance optimizer that can perform stochastic gradient descent.
3. Stochastic Gradient Descent: Mechanism that updates the weights in order to reduce the loss error between your predictions & the real result.
4. Loss function: what about the loss function?For loss function:
5. Binary outcome i.e 0 or 1
6. binary_crossentropy
7. Non binary : predicting three different category
8. categorical_crossentropy.
9. Metrics: what about the metrics to compile?
10. we can choose several metrics at the same time, which is why we will pass it as array.
11. We will choose the most essential one: accuracy.

import tensorflow as tf
from tensorflow import keras
ann = tf.keras.models.Sequential()
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 1, activation='sigmoid'))

ann.compile(optimizer= 'adam', loss= 'binary_crossentropy', metrics =['accuracy'])

Now to train our ANN, we call our ann object.A function to train machine learning model is normally fit() method.

ann.fit()

which almost always takes the same parameter.

1. X train: The matrix of features of training set
2. y train: dependent variable vector of the training set.

when training a artificial neural network, we need to provide two more parameters.

1. Batch size:Instead of comparing your prediction to the real result one by one & reducing the loss, this will tell to compare the several predictions to several real results into a batch.
2. Batch learning is more efficient & performant when training ANN.
3. Batch size tells exactly the number of prediction you want to have in the batch to be compared to that same number of real results.
4. The classic batch size is 32.
5. If you don't want to spend too much time tuning this hyperparameter. choose 32 as the

import tensorflow as tf
from tensorflow import keras
ann = tf.keras.models.Sequential()
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 6, activation='relu'))
ann.add(tf.keras.layers.Dense(units = 1, activation='sigmoid'))
ann.compile(optimizer= 'adam', loss= 'binary_crossentropy', metrics =['accuracy'])


ann.fit(X_train,y_train, batch_size = 32, epochs=100)

here epochs, is number of times we want to iterate to make our model learn & become more accurate.

This we created our first artificial neural network. This is how we can use our model.

ann.predict(sc.transform([[1, 0, 0, 600, 1, 40, 3, 60000, 2, 1, 1, 50000]]))

To conclude:

Python and TensorFlow offer an incredible toolkit in the world of deep learning. In just three lines of code, we can create a fully functional, three-layer neural network. With TensorFlow's Keras & Python enables anyone to construct deep learning models that can tackle complex problems, from image classification to language processing, with minimal code.

#python #artificialNeuralNetwork #deeplearning #machineLearning #datascience