All Codes and Steps of Assignment -11- Explained

 Assignment -11 -Building a Neural Network with TensorFlow and Bekerja Keras

STEP-1

Loading Required packages and Data

###1. Load Data and Splot Data

from tensorflow.keras.datasets import mnist

from tensorflow.keras.datasets import fashion_mnist

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense

from tensorflow.keras.utils import to_categorical

import matplotlib.pyplot as plt

 

from		tensorflow.		keras.		datasets			import		mnist
The mnist module provides the MNIST dataset, which is a collection of 60,000 28x28 grayscale images of handwritten digits, along with a test set of 10,000 images. After loading the dataset, you can preprocess the data, build neural networks, and train models for tasks like digit recognition or other image-related tasks.
From	tensorflow.		keras.		datasets			import		fashion_mnist
Import the Fashion MNIST dataset using TensorFlow and Keras. The Fashion MNIST dataset is a collection of grayscale images of 10 different types of clothing items, and it is often used as a benchmark for testing machine learning and deep learning algorithms. Each image is 28x28 pixels in size.
From	tensorflow.		keras.		models			import		Sequential
The Sequential module is used to create a linear stack of layers, which is the most common way to build a neural network in Keras. The Sequential class in Keras allows you to build a neural network model in a sequential manner, where you add one layer at a time. This is a common way to create simple feedforward neural network architectures.
From		tensorflow.		keras.		layers	import				Dense
The Dense module creates a fully-connected layer, which is a layer where each neuron is connected to every neuron in the previous layer.
From		tensorflow.		keras.		utils	import				to_categorical
The to_categorical module converts a label vector to a one-hot encoded vector, which is required by the Dense layer.

 

	import	Matplotlib.	pyplot	as	plt
as plt

 

 

•

 

 

STEP-2

 

Data Splitting

# splitting the data into test and train set

(X_train, Y_train), (X_test, Y_test) = fashion_mnist.load_data()

The fashion_mnist.load_data() function loads the Fashion MNIST dataset, which is a dataset of 60,000 28x28 grayscale images of 10 fashion categories, along with a test set of 10,000 images. The function returns two tuples, (X_train, Y_train) and (X_test, Y_test).

• X_train is the training data, which consists of 60,000 images. Each image is a 28x28 NumPy array of grayscale values, with values ranging from 0 to 255.
• Y_train is the training labels, which are a list of integers from 0 to 9. Each label corresponds to a fashion category.
• X_test is the test data, which consists of 10,000 images.
• Y_test is the test labels, which are a list of integers from 0 to 9.

 

 

STEP-3

Display the sample images from x_train

 

1.	n = 10	# Number of images to display
2.	plt.figure(figsize=(20, 4))	# Create a figure to display the images The code plt.figure(figsize=(20, 4)) is used to create a new figure in Matplotlib. The figsize argument specifies the width and height of the figure in inches. In this case, the figure will be 20 inches wide and 4 inches high.
		The figure() function is a versatile function that can be used to control many aspects of a figure, such as the title, the axes, and the grid. Here is a list of some of the most commonly used arguments of the figure() function: ·        figsize: The width and height of the figure in inches. ·        dpi: The resolution of the figure in dots per inch. ·        title: The title of the figure. ·        xlabel: The label for the x-axis. ·        ylabel: The label for the y-axis. ·        grid: Whether or not to show a grid on the figure.
3.	for i in range(n):	The code creates a loop that will iterate n times. The variable i will be used to index the subplots.
	plt.subplot(2, n, i + 1)	·       The code creates a subplot within the figure. The first two arguments, 2 and n, specify the number of rows and columns in the subplot grid. ·       The third argument, i + 1, specifies the index of the subplot. ·       The index starts at 1 in the upper left corner and increases to the right. ·       So, the code for i in range(n): will create n subplots, arranged in a 2xn grid.
	plt.imshow(X_test[i].reshape(28, 28))	The imshow() functison is used to display an image in Matplotlib. It takes two arguments: ü  The image data, which can be a NumPy array or a PIL Image object. ü  The colormap, which specifies the colors to use for the image. ·       The code is used to display the original image in the ith subplot. ·       The X_test array contains the test images, and the ith element of the array is the ith test image. ·       The reshape(28, 28) function reshapes the image to be 28x28 pixels, which is the standard size for MNIST images.
	plt.gray()	·        The code is used to set the colormap to grayscale. This means that the image will be displayed in shades of gray, from black to white. ·        The gray() function is a shortcut for the plt.set_cmap("gray") function. The set_cmap() function specifies the colormap to use for an image. The "gray" colormap is a grayscale colormap that maps each value in the image to a shade of gray.
	ax.get_xaxis().set_visible(False) and ax.get_yaxis().set_visible(False)	The code are used to hide the x-axis and y-axis labels and ticks in the ith subplot. ·        The get_xaxis() and get_yaxis() functions return the x-axis and y-axis objects for the subplot, respectively. ·        The set_visible() function sets the visibility of the axis object. In this case, the x-axis and y-axis will be hidden, so the labels and ticks will not be displayed. This can be useful if you want to focus on the image itself.
	plt.show()	# Show the figure with the images
	plt.close()	# Close the figure

 

 

STEP-4

Checking Shapes

print("X_train: ", X_train.shape) print("Y_train: ", Y_train.shape) X_train:  (60000, 28, 28) Y_train:  (60000,)

·        The shape of the X_train array tells us that it has 60000 training examples, each of which is a 28x28 matrix. ·        The shape of the Y_train array tells us that it has 60000 labels for the training examples.

STEP-5

Flattening the Images

 

X_train = X_train.reshape(60000, 784) and X_test = X_test.reshape(10000, 784)

The code reshape the training and test images to be 784-dimensional vectors. This is because the MNIST dataset is a 28x28 grayscale image dataset, and each pixel in the image can take on 256 values (0 to 255). So, each image can be represented as a 28x28x256 = 784-dimensional vector. The reshape() function takes two arguments: The array to be reshaped. The shape of the new array. In this case, the new shape is (60000, 784) for the training images and (10000, 784) for the test images. This means that each training image will be a 784-dimensional vector, and each test image will be a 784-dimensional vector.

 

 

STEP-6

Min-Max Scalling

X_train = X_train.astype('float32') and X_test = X_test.astype('float32'

The code) converts the data type of the training and test images to float32. This is because the MNIST dataset is a grayscale image dataset, and each pixel value in the image is an integer between 0 and 255. Converting the data type to float32 makes it easier for machine learning models to learn the patterns in the data. The astype() function takes one argument: The new data type. In this case, the new data type is float32. This means that each pixel value in the image will be represented as a floating point number between 0 and 1. Converting the data type to float32 is a common practice in machine learning, as it makes the data more compatible with most machine learning models.

train /= 255 and X_test /= 255

The code  normalizes the pixel values in the training and test images to a range between 0 and 1. This is done by dividing each pixel value by 255, which is the maximum value of a pixel in the MNIST dataset. Normalization is a common practice in machine learning, as it helps to improve the performance of machine learning models. It is important to normalize the data so that all of the features are on the same scale. This makes it easier for the machine learning model to learn the patterns in the data.

 

 

STEP-7

Processing the Target variable

classes = 10	# Number of classes in the dataset
# Convert the labels to one-hot encoded format Y_train = to_categorical(Y_train, classes) Y_test = to_categorical(Y_test, classes)	1.              The code converts the labels in the training and test sets to one-hot encoded format. This is done by creating a new vector for each label, where each element in the vector is either 0 or 1. The element that is 1 indicates the correct label for the image. For example, if the label for an image is 5, then the one-hot encoded vector for the image will be a 10-dimensional vector with a 1 in the fifth element and zeros in all of the other elements. 2.              The to_categorical() function takes two arguments: The labels. The number of classes. In this case, the number of classes is 10, which is the number of digits in the MNIST dataset. 3.              Converting the labels to one-hot encoded format is a common practice in machine learning, as it makes it easier for machine learning models to learn the patterns in the data. It is also required by some machine learning models, such as neural networks.
# Print the shapes of the preprocessed training data and labels print("New X_train shape: {} \nNew Y_train shape:{}".format(X_train.shape, Y_train.shape))       New X_train shape: (60000, 784) New Y_train shape:(60000, 10)	·  The code prints the shapes of the pre-processed training data and labels. The shape attribute of an array returns a tuple of the number of elements in each dimension. In this case, the X_train array has 60000 rows and 784 columns, so the output of the code will be: New X_train shape: (60000, 784) New Y_train shape: (60000, 10) The Y_train array has 60000 rows and 10 columns, so the output of the code will be: New Y_train shape: (60000, 10) · The shape of the X_train array tells us that it has 60000 training examples, each of which is a 784-dimensional vector. The shape of the Y_train array tells us that it has 60000 labels for the training examples, and each label is a 10-dimensional vector. The output of the code tells us that the pre-processed training data and labels have the correct shapes.

 

 

STEP-8

Setting up Hyper-parameters

input_size = 784	# Define the input size for each data sample (e.g., image pixels)
batch_size = 200	# Specify the number of data samples to process in each batch
hidden1 = 400	# Define the number of neurons in the first hidden layer
hidden2 = 20	# Define the number of neurons in the second hidden layer
classes = 10	# Define the total number of classes/categories in the dataset
epochs = 5	# Set the number of complete passes through the dataset during training

 

 

Step-9

Building the FCN Model

1	model = Sequential()	Build the model ### # Create a Sequential model, which allows us to build a neural network layer by layer
2	model.add(Dense(hidden1, input_dim=input_size, activation='relu'))	·       The code adds a Dense layer with hidden1 neurons to the model. The Dense layer is a fully connected layer, which means that each neuron in the layer is connected to all of the neurons in the previous layer. ·       The input_dim argument specifies the number of input features, which in this case is 784, since the MNIST dataset is a 28x28 grayscale image dataset. So, the input layer of the model will have 784 neurons. ·       The activation='relu' argument specifies the activation function for the hidden layer. The ReLU activation function is a non-linear function that helps to prevent the model from overfitting the training data.
3	model.add(Dense(hidden2, activation='relu'))	# Add the second hidden layer with 'hidden2' neurons, also using ReLU activation function ·       The code adds a second Dense layer with hidden2 neurons to the model. The hidden2 argument specifies the number of neurons in the second hidden layer. ·       The activation='relu' argument specifies the activation function for the second hidden layer. The ReLU activation function is a non-linear function that helps to prevent the model from overfitting the training data. ·       The code model.add(Dense(hidden2, activation='relu')) is similar to the code model.add(Dense(hidden1, input_dim=input_size, activation='relu')), except that it specifies a different number of neurons in the hidden layer. ·       The number of neurons in the hidden layers can be chosen by trial and error. A good starting point is to use the same number of neurons in each hidden layer. However, you may need to experiment with different numbers of neurons to find the best configuration for your model. ·       The activation function can also be chosen by trial and error. The ReLU activation function is a good choice for many problems, but you may need to experiment with different activation functions to find the best one for your model.
4	model.add(Dense(classes, activation='softmax'))	# Add the output layer with 'classes' neurons, using softmax activation function # Softmax activation ensures that the output values represent probabilities of each class ·       The code adds an output layer with classes neurons to the model. ·       The classes argument specifies the number of neurons in the output layer. In this case, the number of neurons is 10, since there are 10 digits in the MNIST dataset. ·       The activation='softmax' argument specifies the activation function for the output layer. The softmax activation function ensures that the output values represent probabilities of each class. ·       The softmax activation function is a common choice for the output layer of a classification model. It is a non-linear function that takes a vector of real numbers as input and outputs a vector of probabilities. The probabilities sum to 1, so they represent the probability of each class.
5	model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='sgd')	### Compilation ### # Compile the model by specifying the loss function, optimizer, and evaluation metrics ·       The code compiles the model by specifying the loss function, optimizer, and evaluation metrics. ·       The loss argument specifies the loss function.The categorical crossentropy loss function is a good choice for classification problems. It measures the distance between the predicted probabilities and the true labels. ·       The optimizer argument specifies the optimizer. The Adam optimizer is a good choice for many problems. It is a stochastic gradient descent optimizer that is adaptive and efficient. ·       The metrics argument specifies the evaluation metrics. The accuracy metric is a good choice for classification problems. It measures the fraction of predictions that are correct. Here are some additional details about the loss function, optimizer, and evaluation metrics: The loss function is used to measure the performance of the model on the training data. The optimizer is used to update the model's parameters to minimize the loss function. The evaluation metrics are used to measure the performance of the model on the test data. The loss function, optimizer, and evaluation metrics can be chosen by trial and error. A good starting point is to use the categorical crossentropy loss function, the Adam optimizer, and the accuracy metric. However, you may need to experiment with different options to find the best configuration for your model.
6	model.summary()	# Display a summary of the model architecture, showing the layers and parameter counts The code displays a summary of the model architecture, showing the layers and parameter counts. The summary will show the following information for each layer: The layer name. The number of neurons in the layer. The activation function for the layer. The number of parameters in the layer. The total number of parameters in the model is the sum of the number of parameters in each layer. Here are some additional details about the model summary: The model summary can be used to understand the complexity of the model. The model summary can be used to debug the model. The model summary can be used to compare different models.

 

 

Step-10

Training The Model

1		from time import time	# Import necessary libraries
2		tic = time()	# Record the current time to measure training time
	model.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs, verbose=1)		# Fit the model on the training data We are fitting a model using the fit function from TensorFlow/Keras. This is a crucial step in training a machine learning model. The fit function is used to train the model on the provided training data (X_train and Y_train) for a certain number of epochs. Here's a breakdown of the parameters used in this context: X_train: This should be your training input data, such as the train_images loaded from the Fashion MNIST dataset. Y_train: This should be your training labels, corresponding to the classes of the images in X_train. batch_size: This parameter determines the number of samples that will be used in each iteration to update the model's weights. It's common to use batch sizes to speed up training and reduce memory usage. epochs: The number of times the training data will be iterated over during training. verbose: This parameter controls how much information is displayed during training. A value of 1 usually means that progress bars and training updates will be shown.
		toc = time()	# Record the time after model training
		print("Model training took {} secs".format(toc - tic))	# Calculate and print the time taken for model training In this code, the tic variable records the start time before model training, and the toc variable records the end time after training. The difference between these two times gives you the duration of the training process. You can then print this duration in seconds.

 

 

 

Step-11

Testing The Model

# Import the necessary libraries from sklearn.metrics import accuracy_score import numpy as np import matplotlib.pyplot as plt	·       The code from sklearn. metrics import accuracy_score imports the accuracy_score function from the sklearn. metrics library. The accuracy_score function is used to calculate the accuracy of a classification model. ·       The code import numpy as np imports the numpy library. The numpy library provides functions for working with arrays and matrices. ·       The code import matplotlib.pyplot as plt imports the matplotlib.pyplot library. The matplotlib.pyplot library provides functions for plotting data. Here are some additional details about the accuracy_score function: The accuracy_score function takes two arguments: The predicted labels. The true labels. The accuracy_score function returns the accuracy of the model, which is the fraction of predictions that are correct.
y_pred_probs = model.predict(X_test, verbose=0) y_pred = np.where(y_pred_probs > 0.5, 1, 0)	# Predict probabilities for the test set using the trained model ·        y_pred_probs: This variable holds the predicted probabilities for each sample in the test set. These probabilities represent the model's confidence in the positive class (1) for each sample. ·        np.where(y_pred_probs > 0.5, 1, 0): This line applies a threshold of 0.5 to the predicted probabilities. If the predicted probability is greater than 0.5, it's classified as the positive class (1), otherwise, it's classified as the negative class (0). This thresholding operation converts the probability predictions into binary predictions
test_accuracy = accuracy_score(y_pred, Y_test) print("\nTest accuracy: {}".format(test_accuracy))	# Calculate and print the test accuracy using predicted and true labels ·        accuracy_score(y_pred, Y_test): This function takes two arguments: the predicted labels (y_pred) and the true labels (Y_test). It calculates the accuracy by comparing the predicted labels to the true labels and then returns the accuracy as a floating-point value between 0 and 1. ·        print("\nTest accuracy: {}".format(test_accuracy)): This line prints the calculated test accuracy. The format function is used to insert the value of test_accuracy into the string for display. The \n at the beginning of the string adds a newline for better formatting.
mask = range(20, 50)	# Define a mask for selecting a range of indices (20 to 49) ·        range(20, 50): This is a built-in Python function that generates a sequence of integers starting from 20 (inclusive) up to 50 (exclusive). So, the mask will contain the numbers from 20 to 49. ·        The mask you've defined can be used to index or select a subset of elements from a larger dataset, like an array or a list. For example, you can use it like this:
X_valid = X_test[0:20] actual_labels = Y_test[0:20]	# Select the first 20 samples from the test set for visualization The code X_valid = X_test[0:20] selects the first 20 samples from the test set for visualization. The slice() function takes two arguments: The start index. The end index. In this case, the start index is 0 and the end index is 19. This means that the X_valid array will contain the first 20 images from the test set. The code actual_labels = Y_test[0:20] selects the labels for the first 20 samples from the test set. Now you have the X_valid array containing the input images and the actual_labels array containing the true labels for those images. You can use these arrays for visualization and comparison with your model's predictions.
y_pred_probs_valid = model.predict(X_valid) y_pred_valid = np.where(y_pred_probs_valid > 0.5, 1, 0)	# Predict probabilities for the selected validation samples ·        model.predict(X_valid): This uses your trained model (model) to predict probabilities for the selected validation samples (X_valid). The resulting y_pred_probs_valid will contain the predicted probabilities for each sample. ·        np.where(y_pred_probs_valid > 0.5, 1, 0): This line applies a threshold of 0.5 to the predicted probabilities, just like you did earlier. If the predicted probability is greater than 0.5, it's classified as the positive class (1), otherwise, it's classified as the negative class (0). The resulting y_pred_valid array contains the binary predictions for the selected validation samples. Now you have both y_pred_valid and actual_labels for the selected validation samples, which you can use for comparison and visualization.
n = len(X_valid) plt.figure(figsize=(20, 4))	# Set up a figure to display images ·        n = len(X_valid): This line calculates the number of selected validation samples (images) in X_valid. ·       plt.figure(figsize=(20, 4)): This creates a new figure with a specified size. The figsize parameter takes a tuple of (width, height) in inches, and it controls the dimensions of the figure. In this case, the figure will have a width of 20 inches and a height of 4 inches. This code prepares the figure for displaying multiple images side by side, typically used for visualization purposes.
for i in range(n): # Display the original image     ax = plt.subplot(2, n, i + 1)     plt.imshow(X_valid[i].reshape(28, 28))     plt.gray()     ax.get_xaxis().set_visible(False)     ax.get_yaxis().set_visible(False)	·        for i in range(n):: This loop iterates over the range of selected validation samples (images) in X_valid. ·        predicted_digit = np.argmax(y_pred_probs_valid[i]): This line calculates the predicted digit by finding the index of the highest predicted probability in y_pred_probs_valid[i]. ·        ax = plt.subplot(2, n, i + 1 + n): This sets up a subplot in the second row of the figure to display the predicted digits above the images. The i + 1 + n index calculation ensures that the subplot is positioned correctly. ·        plt.text(0.5, 0.5, str(predicted_digit), fontsize=12, ha='center', va='center'): This places the predicted digit as text at the center of the subplot. The ha and va parameters ensure that the text is horizontally and vertically centered. ·        plt.axis('off'): This turns off the axes for a cleaner display of the predicted digits. This code adds the predicted digits above the images of the selected validation samples in the figure.
predicted_digit = np.argmax(y_pred_probs_valid[i])     ax = plt.subplot(2, n, i + 1 + n)     plt.text(0.5, 0.5, str(predicted_digit), fontsize=12, ha='center', va='center')     plt.axis('off')	# Display the predicted digit The code you have provided is used to plot the predicted digit for the i-th image in the validation set. The predicted_digit variable stores the index of the maximum probability in the y_pred_probs_valid[i] array, which is the array of predicted probabilities for the i-th image. The plt.text() function is then used to plot the predicted digit at the center of the image. The plt.axis('off') function is used to turn off the axis labels and ticks.

Here is an explanation of each line of code:

predicted_digit = np.argmax(y_pred_probs_valid[i])	This line of code uses the np.argmax() function to find the index of the maximum value in the y_pred_probs_valid[i] array. The y_pred_probs_valid[i] array is the array of predicted probabilities for the i-th image.
ax = plt.subplot(2, n, i + 1 + n)	This line of code creates a subplot for the i-th image. The plt.subplot() function takes three arguments: the number of rows, the number of columns, and the index of the subplot. In this case, there are 2 rows and n columns, so the index of the subplot for the i-th image is i + 1 + n.
plt.text(0.5, 0.5, str(predicted_digit), fontsize=12, ha='center', va='center')	This line of code plots the predicted digit at the center of the image. The plt.text() function takes four arguments: the x-coordinate, the y-coordinate, the text to be plotted, the font size, and the horizontal alignment and vertical alignment of the text. In this case, the text is the predicted digit, the font size is 12, and the horizontal alignment and vertical alignment are both set to 'center'.
plt.axis('off')	This line of code turns off the axis labels and ticks.