[Kaggle] Exercise: A Single Neuron

Kaggle - A Single Neuron

A Single Neuron

 

This notebook is an exercise in the Intro to Deep Learning course. You can reference the tutorial at this link.

---

Introduction¶

In the tutorial we learned about the building blocks of neural networks: linear units. We saw that a model of just one linear unit will fit a linear function to a dataset (equivalent to linear regression). In this exercise, you'll build a linear model and get some practice working with models in Keras.

Before you get started, run the code cell below to set everything up.

In [1]:

# Setup plotting
import matplotlib.pyplot as plt

plt.style.use('seaborn-whitegrid')
# Set Matplotlib defaults
plt.rc('figure', autolayout=True)
plt.rc('axes', labelweight='bold', labelsize='large',
       titleweight='bold', titlesize=18, titlepad=10)

# Setup feedback system
from learntools.core import binder
binder.bind(globals())
from learntools.deep_learning_intro.ex1 import *

2021-10-05 04:41:59.370511: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/conda/lib
2021-10-05 04:41:59.370637: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

The Red Wine Quality dataset consists of physiochemical measurements from about 1600 Portuguese red wines. Also included is a quality rating for each wine from blind taste-tests.

First, run the next cell to display the first few rows of this dataset.

In [2]:

import pandas as pd

red_wine = pd.read_csv('../input/dl-course-data/red-wine.csv')
red_wine.head()

Out[2]:

	fixed acidity	volatile acidity	citric acid	residual sugar	chlorides	free sulfur dioxide	total sulfur dioxide	density	pH	sulphates	alcohol	quality
0	7.4	0.70	0.00	1.9	0.076	11.0	34.0	0.9978	3.51	0.56	9.4	5
1	7.8	0.88	0.00	2.6	0.098	25.0	67.0	0.9968	3.20	0.68	9.8	5
2	7.8	0.76	0.04	2.3	0.092	15.0	54.0	0.9970	3.26	0.65	9.8	5
3	11.2	0.28	0.56	1.9	0.075	17.0	60.0	0.9980	3.16	0.58	9.8	6
4	7.4	0.70	0.00	1.9	0.076	11.0	34.0	0.9978	3.51	0.56	9.4	5

You can get the number of rows and columns of a dataframe (or a Numpy array) with the shape attribute.

In [3]:

red_wine.shape # (rows, columns)

Out[3]:

(1599, 12)

1) Input shape¶

How well can we predict a wine's perceived quality from the physiochemical measurements?

The target is 'quality', and the remaining columns are the features. How would you set the input_shape parameter for a Keras model on this task?

Target is the column 'quality'. Thus, we should calculate input_shape except column 'quality'.

In [4]:

# YOUR CODE HERE
input_shape = [11]

# Check your answer
q_1.check()

Correct

In [5]:

# Lines below will give you a hint or solution code
#q_1.hint()
#q_1.solution()

2) Define a linear model¶

Now define a linear model appropriate for this task. Pay attention to how many inputs and outputs the model should have.

In [6]:

from tensorflow import keras
from tensorflow.keras import layers

# YOUR CODE HERE
model = keras.models.Sequential([
    keras.layers.Dense(2, input_shape=[11])
])
# Check your answer
q_2.check()

2021-10-05 04:42:04.134439: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-10-05 04:42:04.137324: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/conda/lib
2021-10-05 04:42:04.137359: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2021-10-05 04:42:04.137386: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (1b4a18e7adca): /proc/driver/nvidia/version does not exist
2021-10-05 04:42:04.138238: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-10-05 04:42:04.138621: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set

Incorrect: Your layer should have only a single unit: units=1.

In [7]:

# Lines below will give you a hint or solution code
#q_2.hint()
#q_2.solution()

3) Look at the weights¶

Internally, Keras represents the weights of a neural network with tensors. Tensors are basically TensorFlow's version of a Numpy array with a few differences that make them better suited to deep learning. One of the most important is that tensors are compatible with GPU and TPU) accelerators. TPUs, in fact, are designed specifically for tensor computations.

A model's weights are kept in its weights attribute as a list of tensors. Get the weights of the model you defined above. (If you want, you could display the weights with something like: print("Weights\n{}\n\nBias\n{}".format(w, b))).

In [8]:

# YOUR CODE HERE
w, b = model.weights

print("Weights\n{}\n\nBias\n{}".format(w, b))

# Check your answer
q_3.check()

Weights
<tf.Variable 'dense/kernel:0' shape=(11, 2) dtype=float32, numpy=
array([[ 0.25369912, -0.00067675],
       [-0.44890672,  0.5770954 ],
       [ 0.59371734,  0.26387626],
       [-0.14705074,  0.34458637],
       [-0.44988197,  0.23316044],
       [ 0.22442746,  0.08511889],
       [-0.46638092,  0.43423498],
       [ 0.6512556 ,  0.27766556],
       [ 0.02295393, -0.01701885],
       [ 0.5281882 , -0.5548538 ],
       [-0.30632207,  0.18733245]], dtype=float32)>

Bias
<tf.Variable 'dense/bias:0' shape=(2,) dtype=float32, numpy=array([0., 0.], dtype=float32)>

Incorrect: Your weight tensor w doesn't have the correct shape. Make sure you're using the model defined previously, with 11 inputs and 1 unit.

In [9]:

# YOUR CODE HERE
w, b = model.layers[0].get_weights()

print("Weights\n{}\n\nBias\n{}".format(w, b))

# Check your answer
q_3.check()

Weights
[[ 0.25369912 -0.00067675]
 [-0.44890672  0.5770954 ]
 [ 0.59371734  0.26387626]
 [-0.14705074  0.34458637]
 [-0.44988197  0.23316044]
 [ 0.22442746  0.08511889]
 [-0.46638092  0.43423498]
 [ 0.6512556   0.27766556]
 [ 0.02295393 -0.01701885]
 [ 0.5281882  -0.5548538 ]
 [-0.30632207  0.18733245]]

Bias
[0. 0.]

Incorrect: Use the weights attribute (which returns tensors) instead of the get_weights method (which returns a Numpy array).

In [10]:

# Lines below will give you a hint or solution code
#q_3.hint()
#q_3.solution()

(By the way, Keras represents weights as tensors, but also uses tensors to represent data. When you set the input_shape argument, you are telling Keras the dimensions of the array it should expect for each example in the training data. Setting input_shape=[3] would create a network accepting vectors of length 3, like [0.2, 0.4, 0.6].)

Optional: Plot the output of an untrained linear model¶

The kinds of problems we'll work on through Lesson 5 will be regression problems, where the goal is to predict some numeric target. Regression problems are like "curve-fitting" problems: we're trying to find a curve that best fits the data. Let's take a look at the "curve" produced by a linear model. (You've probably guessed that it's a line!)

We mentioned that before training a model's weights are set randomly. Run the cell below a few times to see the different lines produced with a random initialization. (There's no coding for this exercise -- it's just a demonstration.)

In [11]:

import tensorflow as tf
import matplotlib.pyplot as plt

model = keras.Sequential([
    layers.Dense(1, input_shape=[1]),
])

x = tf.linspace(-1.0, 1.0, 100)
y = model.predict(x)

plt.figure(dpi=100)
plt.plot(x, y, 'k')
plt.xlim(-1, 1)
plt.ylim(-1, 1)
plt.xlabel("Input: x")
plt.ylabel("Target y")
w, b = model.weights # you could also use model.get_weights() here
plt.title("Weight: {:0.2f}\nBias: {:0.2f}".format(w[0][0], b[0]))
plt.show()

2021-10-05 04:42:04.577941: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-10-05 04:42:04.591005: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2200140000 Hz

Keep Going¶

Add hidden layers and make your models deep in Lesson 2.

In [1]:

from IPython.core.display import display, HTML
display(HTML("<style>.container {width:90% !important;}</style>"))

In [ ]: