Keras
Around a year back,Keras was integrated to TensorFlow 2.0, which succeeded TensorFlow 1.0. Now Keras is a part of TensorFlow.
The writer of Keras is Francois Chollette.
Data loading and preprocessing
Neural networks don’t process raw data, encoded JPEG image files, or CSV files. They handle vectorized and standardized representations.
Text files require to be read into string tensors and then split into individual words. Finally, the terms need to be indexed and turned in to integer typed tensors.
Image files are read and converted/decoded into integer arrays/tensors, and then transformed to floating-point and finally normalized to small values ( between 0,1).
CSV files need to be parsed, where numerical features are transformed into floating-point tensors with certain categorical features indexed and converted into integer tensors. Finally, each element needs to be normalized [0,1].
Data loading
Keras models allow three types of inputs:
NumPy arrays: This is a good option if the data fits in memory.
TensorFlow Dataset objects. : This is a high-performance option which is more suitable for datasets that do not fit in memory and that are either streamed from disk or distributed filesystems.
Before we start training a model, we will need to make your data available in one of these formats. If we have a large dataset and we are operating on GPU(s), we should consider using the Dataset object, since it will automatically take care of performance-critical details, such as:
Asynchronously preprocessing our data on CPU while our GPU is busy, and will buffer it to a queue,
in turn, prefetching data on GPU memory so that it’s quickly available when the GPU has finished processing the previous batch of data so that we can reach full GPU utilization on our machine.
Keras supports a wide of range of utilities to help us turn raw data on ours disk into a Dataset object:
tf.keras.preprocessing.image_dataset_from_directory : It turns image files sorted into class-specific folders into a well labelled dataset of image tensors which are of a definite shape.
tf.keras.preprocessing.text_dataset_from_directory is used for the same over text files.
Assuming the following directory structure :
We will try and load the directories as a Keras dataset.
import os
import glob
from PIL import Image
path=r"C:\Users\VAGISH\Documents\Lightshot"
new_path=r"C:\Users\VAGISH\LightShot"
for file in os.listdir(path):
img=Image.open(os.path.join(path,file)).resize((200,200))
final_path=os.path.join(new_path,file)
img.save(final_path)
import tensorflow as tf
path=r'C:\Users\VAGISH\Lightshot'
dataset = tf.keras.preprocessing.image_dataset_from_directory(path,batch_size=8,image_size=(200,200))
for data,labels in dataset:
print(data.dtype)
print(data.shape)
print(labels.dtype)
print(labels.shape)
print(labels)
Output: <dtype: 'float32'> (8, 200, 200, 3) <dtype: 'int32'> (8,) tf.Tensor([0 1 0 1 0 1 0 1], shape=(8,), dtype=int32)
Similarly, we can load text files organized in different folders to a dataset object using :
tf.keras.preprocessing.text_dataset_from_directory​
Data Preprocessing with Keras
Once we have data in the form of string/int/float Numpy arrays, or a dataset object that yields batches of string/int/float tensors, the next step is to pre process the data. Taking up keras courses will help you learn more about the concept.
This usually means:
1.Tokenization of string data, followed by indexing
2.Feature normalization
3.Rescaling data to small values (zero-mean and variance or in range [0,1])
4.Text Vectorization
Keras supports a text vectorization layer, which can be directly used in the models. It holds an index for mapping of words for string type data or tokens to integer indices
Normalization
It stores the mean and variance of all features.
The state of the preprocessing layer can be directly obtained by calling layer.adapt(data) on a sample of training data.
Example:
Converting strings to sequences of integer word indices
from tensorflow.keras.layers.experimental.preprocessing import TextVectorization
import numpy as np
training_data=np.array([["I am brave."],["I am a student"],["I study at school"]])
print(training_data)
vectorizer=TextVectorization(output_mode="tf-idf")
vectorizer.adapt(training_data)
data = vectorizer(training_data)
print(data)
OUTPUT:
[['I am vagish.']
['I am a student']
['I study at school']]
tf.Tensor(
[[0. 0.5596158 0.6931472 0.91629076 0. 0.
0. 0. 0. ]
[0. 0.5596158 0.6931472 0. 0. 0.91629076
0. 0. 0.91629076]
[0. 0.5596158 0. 0. 0.91629076 0.
0.91629076 0.91629076 0. ]], shape=(3, 9), dtype=float32)from tensorflow.keras.layers.experimental.preprocessing import TextVectorization
import numpy as np
training_data=np.array([["I am vagish."],["I am a student"],["I study at school"]])
print(training_data)
vectorizer=TextVectorization(output_mode="binary")
vectorizer.adapt(training_data)
data = vectorizer(training_data)
print(data)Output [['I am vagish.'] ['I am a student'] ['I study at school']] tf.Tensor( [[0. 1. 1. 1. 0. 0. 0. 0. 0.] [0. 1. 1. 0. 0. 1. 0. 0. 1.] [0. 1. 0. 0. 1. 0. 1. 1. 0.]], shape=(3, 9), dtype=float32)
Generating Bi-Gram type data
from tensorflow.keras.layers.experimental.preprocessing import TextVectorization
# Example training data, of dtype `string`.
training_data = np.array([["This is me"], ["And there they are"]])
vectorizer = TextVectorization(output_mode="binary", ngrams=2)
vectorizer.adapt(training_data)
int_data = vectorizer(training_data)
print(int_data)
Output
tf.Tensor(
[[0. 1. 1. 0. 1. 0. 1. 1. 1. 0. 0. 0. 1. 0. 0. 1. 1.]
[0. 1. 1. 0. 0. 1. 0. 0. 0. 1. 1. 1. 1. 1. 1. 0. 0.]], shape=(2, 17), dtype=float32)
Feature Normalization
from tensorflow.keras.layers.experimental.preprocessing import Normalization
training_data = np.random.randint(0, 256, size=(256, 200, 200, 3)).astype("float32")
normalizer = Normalization(axis=-1)
normalizer.adapt(training_data)
normalized_data = normalizer(training_data)
print(training_data)
print("var: %.4f" % np.var(normalized_data))
print("mean: %.4f" % np.mean(normalized_data))
Ouput:
[[[[177. 55. 50.]
[ 85. 57. 44.]
[219. 217. 10.]
…
[146. 36. 211.]
[212. 149. 89.]
[ 35. 247. 171.]]
[[ 79. 13. 43.]
[238. 12. 112.]
[236. 75. 190.]
…
[ 87. 37. 177.]
[ 50. 35. 137.]
[161. 232. 78.]]
[[ 28. 207. 133.]
[ 79. 76. 229.]
[ 29. 9. 215.]
…
[242. 135. 241.]
[ 12. 250. 58.]
[225. 83. 165.]]
…
[[193. 170. 77.]
[239. 14. 213.]
[ 86. 237. 197.]
…
[224. 117. 71.]
[209. 82. 100.]
[151. 60. 73.]]
[[ 25. 81. 58.]
[ 0. 89. 187.]
[ 67. 18. 184.]
…
[ 39. 75. 129.]
[ 32. 87. 50.]
[ 30. 140. 210.]]
[[ 38. 155. 70.]
[ 92. 131. 141.]
[145. 110. 166.]
…
[ 17. 181. 115.]
[255. 179. 177.]
[212. 231. 195.]]]
[[[ 48. 200. 236.]
[167. 245. 193.]
[ 9. 20. 242.]
…
[184. 103. 221.]
[182. 142. 114.]
[247. 100. 77.]]
[[ 62. 24. 81.]
[216. 63. 116.]
[ 75. 42. 156.]
…
[237. 21. 114.]
[146. 246. 227.]
[ 87. 94. 148.]]
[[140. 223. 121.]
[ 59. 14. 174.]
[ 14. 117. 165.]
…
[ 12. 17. 234.]
[238. 132. 56.]
[207. 145. 69.]]
…
[[ 20. 88. 4.]
[114. 42. 178.]
[236. 193. 2.]
…
[ 7. 192. 59.]
[233. 9. 249.]
[ 86. 43. 14.]]
[[166. 170. 128.]
[137. 74. 19.]
[ 8. 62. 33.]
…
[227. 17. 54.]
[ 17. 8. 113.]
[234. 66. 254.]]
[[148. 238. 223.]
[213. 197. 218.]
[ 6. 66. 68.]
…
[201. 164. 121.]
[ 28. 206. 122.]
[225. 158. 215.]]]
[[[122. 201. 85.]
[202. 15. 160.]
[120. 175. 103.]
…
[ 43. 102. 177.]
[182. 7. 7.]
[ 29. 47. 139.]]
[[143. 136. 123.]
[147. 61. 182.]
[110. 52. 149.]
…
[ 64. 73. 98.]
[ 17. 39. 80.]
[173. 105. 21.]]
[[ 48. 177. 124.]
[ 73. 108. 189.]
[192. 155. 27.]
…
[ 90. 81. 86.]
[101. 170. 161.]
[195. 218. 27.]]
…
[[201. 154. 27.]
[255. 116. 182.]
[ 44. 234. 154.]
…
[ 7. 130. 91.]
[ 13. 76. 124.]
[ 73. 223. 110.]]
[[117. 39. 143.]
[ 13. 50. 66.]
[ 52. 68. 137.]
…
[ 72. 122. 229.]
[ 61. 99. 126.]
[199. 73. 218.]]
[[ 45. 215. 255.]
[113. 178. 201.]
[255. 180. 247.]
…
[217. 86. 39.]
[219. 224. 181.]
[ 28. 14. 123.]]]
…
[[[208. 184. 12.]
[ 6. 110. 134.]
[ 26. 47. 47.]
…
[ 63. 58. 78.]
[182. 73. 225.]
[227. 28. 63.]]
[[226. 123. 213.]
[208. 175. 242.]
[ 69. 152. 59.]
…
[ 21. 122. 163.]
[118. 254. 119.]
[157. 146. 23.]]
[[ 67. 31. 252.]
[229. 106. 135.]
[151. 98. 56.]
…
[191. 76. 197.]
[ 82. 91. 119.]
[165. 119. 81.]]
…
[[116. 121. 114.]
[232. 203. 36.]
[ 46. 152. 17.]
…
[107. 68. 24.]
[199. 145. 182.]
[116. 35. 216.]]
[[230. 160. 172.]
[ 88. 174. 54.]
[139. 61. 229.]
…
[ 60. 137. 174.]
[ 46. 24. 11.]
[190. 120. 56.]]
[[183. 166. 218.]
[ 51. 65. 206.]
[236. 60. 202.]
…
[142. 199. 106.]
[163. 154. 71.]
[172. 172. 7.]]]
[[[150. 148. 113.]
[238. 221. 55.]
[ 47. 33. 118.]
…
[140. 234. 62.]
[ 75. 122. 59.]
[136. 35. 132.]]
[[112. 190. 42.]
[ 65. 117. 249.]
[ 3. 223. 41.]
…
[176. 245. 70.]
[129. 63. 91.]
[100. 25. 185.]]
[[ 94. 170. 6.]
[141. 191. 198.]
[149. 171. 254.]
…
[166. 209. 137.]
[ 95. 176. 155.]
[139. 103. 210.]]
…
[[ 49. 58. 76.]
[ 66. 237. 32.]
[ 69. 61. 218.]
…
[212. 77. 169.]
[103. 177. 251.]
[ 67. 28. 224.]]
[[ 54. 24. 250.]
[181. 61. 145.]
[235. 116. 49.]
…
[195. 169. 99.]
[ 42. 240. 9.]
[203. 177. 25.]]
[[ 8. 91. 78.]
[ 81. 25. 96.]
[ 92. 229. 69.]
…
[ 56. 8. 159.]
[244. 1. 94.]
[189. 58. 202.]]]
[[[ 2. 212. 25.]
[183. 164. 186.]
[106. 187. 178.]
…
[ 97. 20. 168.]
[144. 242. 246.]
[117. 15. 126.]]
[[130. 155. 79.]
[103. 105. 244.]
[163. 242. 237.]
…
[101. 168. 186.]
[ 22. 149. 226.]
[244. 170. 46.]]
[[140. 7. 163.]
[151. 89. 27.]
[244. 77. 210.]
…
[210. 173. 240.]
[176. 210. 44.]
[ 26. 145. 213.]]
…
[[249. 229. 66.]
[254. 176. 218.]
[ 31. 218. 183.]
…
[ 79. 140. 181.]
[148. 47. 242.]
[180. 142. 86.]]
[[ 59. 166. 236.]
[ 92. 207. 140.]
[225. 220. 153.]
…
[ 26. 253. 247.]
[148. 177. 60.]
[ 91. 136. 4.]]
[[178. 114. 26.]
[140. 38. 232.]
[207. 199. 243.]
…
[129. 180. 212.]
[ 47. 159. 97.]
[182. 179. 1.]]]]
var: 1.0000
mean: 0.0000
Rescaling and center cropping images
from tensorflow.keras.layers.experimental.preprocessing import CenterCrop
from tensorflow.keras.layers.experimental.preprocessing import Rescaling
training_data = np.random.randint(0, 256, size=(16, 200, 200, 3)).astype("float32")
cropper = CenterCrop(height=100, width=100)
scaler = Rescaling(scale=1.0 / 255)
output_data = scaler(cropper(training_data))
print(output_data)
print("shape:", output_data.shape)
print("min:", np.min(output_data))
print("max:", np.max(output_data))
Output :
tf.Tensor(
[[[[0.50980395 0.56078434 0.3647059 ]
[0.10196079 0.9215687 0.15686275]
[0.95294124 0.9490197 0.427451 ]
…
[0.4784314 0.77647066 0.77647066]
[0.37254903 0.78823537 0.9686275 ]
[0.15686275 0.72156864 0.8980393 ]]
[[0.5254902 0.6156863 0.07450981]
[0.0627451 0.8862746 0.24313727]
[0.9058824 0.23529413 0.5372549 ]
…
[0.64705884 0.16862746 0.5176471 ]
[0.31764707 0.7019608 0.9686275 ]
[0.8431373 0.8352942 0.7137255 ]]
[[0.6862745 0.05882353 0.35686275]
[0.909804 0.9607844 0.7294118 ]
[0.37254903 0.6156863 0.10196079]
…
[0.16078432 0.97647065 0.70980394]
[0.53333336 0.7803922 0.8117648 ]
[0.5568628 0.05882353 0.5411765 ]]
…
[[0.21176472 0.25490198 0.32156864]
[0.6666667 0.6509804 0.5568628 ]
[1. 0.3803922 0.3019608 ]
…
[0.9294118 0.75294125 0.3803922 ]
[0.3254902 0.34117648 0.21568629]
[0.77647066 0.43921572 0.5568628 ]]
[[0.45882356 0.20784315 0.28627452]
[0.23137257 0.16078432 0.45882356]
[0.7843138 0.0509804 0.5882353 ]
…
[0.96470594 0.20000002 0.0509804 ]
[0.49803925 0.07843138 0.14901961]
[0.7960785 0.3254902 0.7803922 ]]
[[0.00392157 0.77647066 0.14117648]
[0.30588236 0.5058824 0.01568628]
[0.01176471 0.41176474 0.09803922]
…
[0.13333334 0.25882354 0.13333334]
[1. 0.5529412 0.6 ]
[0.3921569 0.18431373 0.65882355]]]
[[[0.03529412 0.13725491 0.2392157 ]
[0.3372549 0.8588236 0.2627451 ]
[0.5647059 0.9568628 0.7176471 ]
…
[0.4784314 0.94117653 0.11764707]
[0.60784316 0.82745105 0.53333336]
[0.6117647 0.8588236 0.43137258]]
[[0.5019608 0.7843138 0.13333334]
[0.9686275 0.7568628 0.85098046]
[0.21568629 0.21960786 0.7058824 ]
…
[0.85098046 0.882353 0.5568628 ]
[0.44705886 0.4901961 0.17254902]
[0.94117653 0.16862746 0.31764707]]
[[0.79215693 0.13333334 0.9176471 ]
[0.20392159 0.44705886 0.3254902 ]
[0.909804 0.6745098 0.62352943]
…
[0.58431375 0.39607847 0.9490197 ]
[0.20000002 0.6666667 0.7686275 ]
[0.01568628 0.56078434 0.8196079 ]]
…
[[0.30588236 0.29803923 0.5176471 ]
[0.02745098 0.09411766 0.56078434]
[0.53333336 0.38823533 0.5568628 ]
…
[0.6666667 0.17254902 0.8235295 ]
[0.25490198 0.09019608 0.28627452]
[0.7176471 0.12941177 0.6 ]]
[[0.2901961 0.74509805 0.8588236 ]
[0.58431375 0.43921572 0.23137257]
[0.38431376 0.04313726 0.23529413]
…
[0.32156864 0.20000002 0.8862746 ]
[0.47450984 0.53333336 0.5137255 ]
[0.7019608 0.7725491 0.19215688]]
[[0.38823533 0.19607845 0.35686275]
[0.28235295 0.28235295 0.75294125]
[0.08627451 0.62352943 0.07450981]
…
[0.82745105 0.52156866 0.54509807]
[0.13333334 0.29411766 0.58431375]
[0.7137255 0.92549026 0.14117648]]]
[[[0.6039216 0.882353 0.227451 ]
[0.34901962 0.5058824 0.909804 ]
[0.95294124 0.78823537 0.5058824 ]
…
[0.9058824 0.8000001 0.58431375]
[0.2901961 0.5764706 0.5764706 ]
[0.3921569 0.6745098 0.02745098]]
[[0.60784316 0.8862746 0.08235294]
[0.39607847 0.97647065 0.46274513]
[0.5294118 0.38823533 0.5058824 ]
…
[0.74509805 0.4156863 0.52156866]
[0.06666667 0.77647066 0.49803925]
[0.9686275 0.3254902 0.14901961]]
[[0.0509804 0.65882355 0.8588236 ]
[0.6509804 0.2784314 0.8235295 ]
[0.60784316 0.19215688 0.4431373 ]
…
[0.627451 0.7372549 0.01176471]
[0.05490196 0.75294125 0.17254902]
[0.73333335 0.49803925 0.6745098 ]]
…
[[0.8313726 0.09411766 0.70980394]
[0.13333334 0.5529412 0.8941177 ]
[0.2901961 0.45098042 0.427451 ]
…
[0.47450984 0.35686275 0.85098046]
[0.8000001 0.05490196 0.9960785 ]
[0.6 0.57254905 0.10588236]]
[[0.5019608 0.16078432 0.79215693]
[0.04313726 0.07450981 0.3254902 ]
[0.7058824 0.2627451 0.22352943]
…
[0.49411768 0.26666668 0.07450981]
[0.05882353 0.1137255 0.73333335]
[0.4901961 0.4784314 0.65882355]]
[[0.44705886 0.27450982 0.9843138 ]
[0.45882356 0.11764707 0.48235297]
[0.57254905 0.10980393 0.81568635]
…
[0.454902 0.9450981 0.64705884]
[0.454902 0.6156863 0.04705883]
[0.59607846 0.64705884 0.01176471]]]
…
[[[0.23137257 0.7294118 0.7568628 ]
[0.2509804 0.8941177 0.14117648]
[0.8941177 0.96470594 0.18823531]
…
[0.8352942 0.36862746 0.74509805]
[0.50980395 0.87843144 0.13333334]
[0.62352943 0. 0.8352942 ]]
[[0.73333335 0.5176471 0.58431375]
[0.16862746 0.5058824 0.30588236]
[0.00392157 0.36862746 0.8078432 ]
…
[0.8196079 0.9803922 0.9333334 ]
[0.27450982 0.3529412 0.8000001 ]
[0.91372555 0.4901961 0.85098046]]
[[0.227451 0.04313726 0.7137255 ]
[0.28627452 0.6901961 0.97647065]
[0.8745099 0.9215687 0.77647066]
…
[0.4431373 0.94117653 0.96470594]
[0.05490196 0.8862746 0.09411766]
[0.7607844 0.64705884 0.3254902 ]]
…
[[0.9450981 0.74509805 0.7490196 ]
[0.34901962 0.6901961 0.41176474]
[0.65882355 0.01568628 0.43921572]
…
[0.36862746 0.5529412 0.4666667 ]
[0.81568635 0.68235296 0.909804 ]
[0.4901961 0.3921569 0.49411768]]
[[0.70980394 0.8078432 0.8352942 ]
[0.15686275 0.882353 0.3529412 ]
[0.3803922 0.24705884 0.3529412 ]
…
[0.5411765 0.9921569 0.454902 ]
[0.4156863 0.43137258 0.0627451 ]
[0.54901963 0.92549026 0.7568628 ]]
[[0.854902 0.18039216 0.61960787]
[0.68235296 0.10196079 0.07058824]
[0.23137257 0.5019608 0.48627454]
…
[0.627451 0.50980395 0.5568628 ]
[0.0509804 0.6509804 0.6 ]
[0.59607846 0.03529412 0.1137255 ]]]
[[[0.6509804 0.62352943 0.00392157]
[0.23137257 0.7803922 0.627451 ]
[0.94117653 0.05882353 0.10980393]
…
[0.24313727 0.5254902 0.02745098]
[0.28235295 0.25490198 0.64705884]
[0.7411765 0.05882353 0.16078432]]
[[0.7254902 0.20392159 0.5137255 ]
[0.69411767 0.29803923 0.2627451 ]
[0.2392157 0.80392164 0.69411767]
…
[0.46274513 0.7843138 0.40784317]
[0.19607845 0.16078432 0.24705884]
[0.14509805 0.3019608 0.04313726]]
[[0.13333334 0.69411767 0.43137258]
[0.8470589 0.6156863 0.49411768]
[0.4039216 0.37254903 0.59607846]
…
[0.45098042 0.9450981 0. ]
[0.3372549 0.49411768 0.9490197 ]
[0.4039216 0.27058825 0.227451 ]]
…
[[0.28235295 0.882353 0.6784314 ]
[0.75294125 0.46274513 0.0627451 ]
[0.9490197 0.6627451 0.25882354]
…
[0.8196079 0.17254902 0.3372549 ]
[0.97647065 0.8000001 0.5137255 ]
[0.9215687 0.227451 0.50980395]]
[[0.37647063 0.74509805 1. ]
[0.31764707 0.10980393 0.59607846]
[0.89019614 0.18823531 0.61960787]
…
[0.9490197 0.17254902 0.02745098]
[0.97647065 0.86274517 0.25882354]
[0.30588236 0.20784315 0.4039216 ]]
[[0.6392157 0.2627451 0.654902 ]
[0.96470594 0.27450982 0.7294118 ]
[0.16862746 0.40784317 0.6156863 ]
…
[0.9843138 0.03529412 0.40784317]
[0.86666673 0.9058824 0.9058824 ]
[0. 0.52156866 0.42352945]]]
[[[0.6 0.34509805 0.7137255 ]
[0.7686275 0.4156863 0.9803922 ]
[0.8862746 0.79215693 0.04705883]
…
[0.7254902 0.20000002 0.5411765 ]
[0.09019608 0.9843138 0.5176471 ]
[0.6 0.49411768 0.6117647 ]]
[[0.5294118 0.09411766 0.86666673]
[0.7960785 0.34117648 0.7607844 ]
[0.6039216 0.6117647 0.73333335]
…
[0.3803922 0.83921576 0.5254902 ]
[0.15686275 0.4784314 0.34901962]
[0.82745105 0.95294124 0.8941177 ]]
[[0.7568628 0.6666667 0.5764706 ]
[0.14117648 0.05490196 0.05490196]
[0.44705886 0.07450981 0.13333334]
…
[0.47450984 0.81568635 0.37254903]
[0.5686275 0.427451 0.06666667]
[0.9960785 0.9294118 0.29411766]]
…
[[0.7294118 0.7137255 0.21960786]
[0.41960788 0.81568635 0.7843138 ]
[0.61960787 0.6784314 0.9607844 ]
…
[0.6117647 0.427451 0.86274517]
[0.89019614 0.2509804 0.47450984]
[0.40000004 0.3803922 0.9607844 ]]
[[0.21568629 0.7254902 0.10588236]
[0.45098042 0.54509807 0.18823531]
[0.41176474 0.30980393 0.09411766]
…
[0.2627451 0.882353 0.8862746 ]
[0.4666667 0.89019614 0.69411767]
[0.38823533 0.64705884 0.42352945]]
[[0.00784314 0.33333334 0.7803922 ]
[0.5647059 0.7686275 0.03529412]
[0.44705886 0.19215688 0.7490196 ]
…
[0.6117647 0.7803922 0.60784316]
[0.7137255 0.6509804 0.18823531]
[0.87843144 0.9960785 0.9607844 ]]]], shape=(64, 100, 100, 3), dtype=float32)
shape: (64, 100, 100, 3)
min: 0.0
max: 1.0
Building models with keras Functional API
A simple input output transformation is known as a layer. Multiple layers can be stacked to get desired results.
A linear layer that maps its inputs to a 16 dimensional feature space can be written as :
D=keras.layers.Dense(units=)
A model is practically a directed acyclic graph of individual layers. The most powerful method to build keras models is by using the Functional API.
We start by specifying the input size and shape which are mandatory.If any dimension of the input is likely to vary we can specify it as None.
For example :
Lets consider we have input shape of (200,200,3) for a RGB image but for any dimensional input shape, we can specify it as (None,None,3)
For taking inputs of RGB images of any size we can do the following:
Inputs=keras.Input(shape=(None,None,3))
After the input has been defined, we can chain layers
from tensorflow import keras
from tensorflow.keras import layers
# Let's say we expect our inputs to be RGB images of arbitrary size
inputs = keras.Input(shape=(None, None, 3))
x = CenterCrop(height=100, width=100)(inputs)
x = Rescaling(scale=1.0 / 255)(x)
# Apply some convolution and pooling layers
x = layers.Conv2D(filters=32, kernel_size=(2, 2), activation="relu")(x)
x = layers.MaxPooling2D(pool_size=(1, 1))(x)
x = layers.Conv2D(filters=32, kernel_size=(2, 2), activation="relu")(x)
x = layers.MaxPooling2D(pool_size=(1, 1))(x)
x = layers.Conv2D(filters=32, kernel_size=(2, 2), activation="relu")(x)
x = layers.Conv2D(filters=32, kernel_size=(2, 2), activation="relu")(x)
x = layers.MaxPooling2D(pool_size=(1, 1))(x)
x = layers.Conv2D(filters=32, kernel_size=(2, 2), activation="relu")(x)
x = layers.MaxPooling2D(pool_size=(1, 1))(x)
x = layers.Conv2D(filters=32, kernel_size=(2, 2), activation="relu")(x)
x = layers.GlobalAveragePooling2D()(x)
num_classes = 10
outputs = layers.Dense(num_classes, activation="softmax")(x)
Once we have defined the directed acyclic graph of layers that turns our input(s) into our outputs, we instantiate a Model object:
model = keras.Model(inputs=inputs, outputs=outputs)
This model behaves basically like a bigger layer. You can call it on batches of data, like this:
data = np.random.randint(0, 256, size=(64, 200, 200, 3)).astype("float32")
processed_data = model(data)
print(processed_data.shape)
A model.summary can be printed.
model.summary()
Model: “functional_3”
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_4 (InputLayer) [(None, None, None, 3)] 0
_________________________________________________________________
center_crop_5 (CenterCrop) (None, 150, 150, 3) 0
_________________________________________________________________
rescaling_5 (Rescaling) (None, 150, 150, 3) 0
_________________________________________________________________
conv2d_11 (Conv2D) (None, 149, 149, 32) 416
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 149, 149, 32) 0
_________________________________________________________________
conv2d_12 (Conv2D) (None, 148, 148, 32) 4128
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 148, 148, 32) 0
_________________________________________________________________
conv2d_13 (Conv2D) (None, 147, 147, 32) 4128
_________________________________________________________________
conv2d_14 (Conv2D) (None, 146, 146, 32) 4128
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 146, 146, 32) 0
_________________________________________________________________
conv2d_15 (Conv2D) (None, 145, 145, 32) 4128
_________________________________________________________________
max_pooling2d_11 (MaxPooling (None, 145, 145, 32) 0
_________________________________________________________________
conv2d_16 (Conv2D) (None, 144, 144, 32) 4128
_________________________________________________________________
global_average_pooling2d_2 ( (None, 32) 0
_________________________________________________________________
dense_2 (Dense) (None, 10) 330
=================================================================
Total params: 21,386
Trainable params: 21,386
Non-trainable params: 0
The next step for us is to train the model on our data. The Model class has a built-in training loop, called the fit() method that accepts Dataset objects or Python generators that yield batches of data, or NumPy arrays.
Before we can call fit(), we need to specify an optimizer and a loss function.This is the compile() step:
model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=1e-4),
loss=keras.losses.CategoricalCrossentropy())
Loss and optimizer can be specified via their string identifiers (in this case their default constructor argument values are used):
model.compile(optimizer=’rmsprop’, loss=’categorical_crossentropy’)
Once our model is compiled, you can start “fitting” the model to the data. Here’s what fitting a model looks like with NumPy data:
model.fit(numpy_array_of_samples, numpy_array_of_labels,
batch_size=32, epochs=10)
Besides the data, you have to specify two key parameters: the batch_size and the number of epochs (iterations on the data). Here our data will get sliced on batches of 32 samples, and the model will iterate 10 times over the data during training.
Here’s what fitting a model looks like with a dataset:
model.fit(dataset_of_samples_and_labels, epochs=10)
Since the data yielded by a dataset is expect to be already batched, you don’t need to specify the batch size here.
Let’s look at it in practice with a toy example model that learns to classify MNIST digits:
# Get the data as Numpy arrays
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
# Build a simple model
inputs = keras.Input(shape=(28, 28))
x = layers.experimental.preprocessing.Rescaling(1.0 / 255)(inputs)
x = layers.Flatten()(x)
x = layers.Dense(128, activation="relu")(x)
x = layers.Dense(128, activation="relu")(x)
outputs = layers.Dense(10, activation="softmax")(x)
model = keras.Model(inputs, outputs)
model.summary()
# Compile the model
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy")
# Train the model for 1 epoch from Numpy data
batch_size = 64
print("Fit on NumPy data")
history = model.fit(x_train, y_train, batch_size=batch_size, epochs=1)
# Train the model for 1 epoch using a dataset
dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(batch_size)
print("Fit on Dataset")
history = model.fit(dataset, epochs=1)
{‘loss’: [0.11615095287561417]}
For a detailed overview of how to use fit(), see the guide to training & evaluation with the built-in Keras methods.
Keeping track of performance metrics
As you’re training a model, you want to keep track of metrics such as classification accuracy, precision, recall, AUC, etc. Besides, you want to monitor these metrics not only on the training data, but also on a validation set.
Monitoring metrics
You can pass a list of metric objects to compile(), like this:
model.compile(
optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics=[keras.metrics.SparseCategoricalAccuracy(name="acc")],
)
history = model.fit(dataset, epochs=1)
Passing validation data to fit()
You can pass validation data to fit() to monitor your validation loss & validation metrics. Validation metrics get reported at the end of each epoch.
val_dataset = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(batch_size)
history = model.fit(dataset, epochs=1, validation_data=val_dataset)938/938 [==============================] – 1s 1ms/step – loss: 0.0556 – acc: 0.9829 – val_loss: 0.1163 – val_acc: 0.9670
Using callbacks for checkpointing
If training goes on for more than a few minutes, it’s important to save your model at regular intervals during training. You can then use your saved models to restart training in case your training process crashes (this is important for multi-worker distributed training, since with many workers at least one of them is bound to fail at some point).
An important feature of Keras is callbacks, configured in fit(). Callbacks are objects that get called by the model at different point during training, in particular:
At the beginning and end of each batch
At the beginning and end of each epoch
Callbacks are a way to make model trainable entirely scriptable.
You can use callbacks to periodically save your model. Here’s a simple example: a ModelCheckpoint callback configured to save the model at the end of every epoch. The filename will include the current epoch.
callbacks = [
keras.callbacks.ModelCheckpoint(
filepath='path/to/my/model_{epoch}',
save_freq='epoch')
]
model.fit(dataset, epochs=2, callbacks=callbacks)
You can also use callbacks to do things like periodically changing the learning of your optimizer, streaming metrics to a Slack bot, sending yourself an email notification when training is complete, etc.
For detailed overview of what callbacks are available and how to write your own, see the callbacks API documentation and the guide to writing custom callbacks.
Monitoring training progress with TensorBoard
Staring at the Keras progress bar isn’t the most ergonomic way to monitor how your loss and metrics are evolving over time. There’s a better solution: TensorBoard, a web application that can display real-time graphs of your metrics (and more).
To use TensorBoard with fit(), simply pass a keras.callbacks.TensorBoard callback specifying the directory where to store TensorBoard logs:
callbacks = [
keras.callbacks.TensorBoard(log_dir='./logs')
]
model.fit(dataset, epochs=2, callbacks=callbacks)
You can then launch a TensorBoard instance that you can open in your browser to monitor the logs getting written to this location:
tensorboard –logdir=./logs
What’s more, you can launch an in-line TensorBoard tab when training models in Jupyter / Colab notebooks. Here’s more information.
After fit(): evaluating test performance & generating predictions on new data
Once you have a trained model, you can evaluate its loss and metrics on new data via evaluate():
loss, acc = model.evaluate(val_dataset) # returns loss and metrics
print("loss: %.2f" % loss)
print("acc: %.2f" % acc)
157/157 [==============================] – 0s 907us/step – loss: 0.1163 – acc: 0.9670
loss: 0.12
acc: 0.97
We can also generate NumPy arrays of predictions (the activations of the output layer(s) in the model) via predict():
predictions = model.predict(val_dataset)
print(predictions.shape)
(10000, 10)
Using fit() with a custom training step
By default, fit() is configured for supervised learning. If you need a different kind of training loop (for instance, a GAN training loop), you can provide your own implementation of the Model.train_step() method. This is the method that is repeatedly called during fit().
Metrics, callbacks, etc. will work as usual.
Here’s a simple example that reimplements what fit() normally does:
class CustomModel(keras.Model):
def train_step(self, data):
# Unpack the data. Its structure depends on your model and
# on what you pass to `fit()`.
x, y = data
with tf.GradientTape() as tape:
y_pred = self(x, training=True) # Forward pass
# Compute the loss value
# (the loss function is configured in `compile()`)
loss = self.compiled_loss(y, y_pred,
regularization_losses=self.losses)
# Compute gradients
trainable_vars = self.trainable_variables
gradients = tape.gradient(loss, trainable_vars)
# Update weights
self.optimizer.apply_gradients(zip(gradients, trainable_vars))
# Update metrics (includes the metric that tracks the loss)
self.compiled_metrics.update_state(y, y_pred)
# Return a dict mapping metric names to current value
return {m.name: m.result() for m in self.metrics}
# Construct and compile an instance of CustomModel
inputs = keras.Input(shape=(32,))
outputs = keras.layers.Dense(1)(inputs)
model = CustomModel(inputs, outputs)
model.compile(optimizer='adam', loss='mse', metrics=[...])
# Just use `fit` as usual
model.fit(dataset, epochs=3, callbacks=...)
Code can be written in keras using two methodologies:
1. Sequential API
2. Functional API
We will see Sequential API in this tutorial
Sequential API
A sequential model is appropriate where a plain stack of layers is required, given each layer has exactly one input tensor and one output tensor.
A sequential model is not appropriate when:
Designing a network with multiple inputs or multiple outputs
If any one of the layers has multiple inputs or multiple outputs
When sharing of layers is required
When we have a nonlinear network topology in place
Sequential model Creation
We can create a sequential model by first importing the sequential class from keras.models and then passing a list of layers (which give a stacking appearance to the layers) and then passing on this list as parameters to the sequential constructor.
Lets train a image detection model on the fashion mnist dataset:
Sample code in Tensorflow
# TensorFlow and tf.keras
import tensorflow as tf
from tensorflow import keras
# Helper libraries
import numpy as np
import matplotlib.pyplot as plt
fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
plt.figure()
plt.imshow(train_images[0])
plt.colorbar()
plt.grid(False)
plt.show()
train_images = train_images / 255.0
test_images = test_images / 255.0
plt.figure(figsize=(10,10))
for i in range(25):
plt.subplot(5,5,i+1)
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(train_images[i], cmap=plt.cm.binary)
plt.xlabel(class_names[train_labels[i]])
plt.show()
model = keras.Sequential([
keras.layers.Flatten(input_shape=(28, 28)),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dense(10)
])
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model.fit(train_images, train_labels, epochs=10)
test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)
print('\nTest accuracy:', test_acc)
probability_model = tf.keras.Sequential([model,
tf.keras.layers.Softmax()])
predictions = probability_model.predict(test_images)
np.argmax(predictions[0])
def plot_image(i, predictions_array, true_label, img):
predictions_array, true_label, img = predictions_array, true_label[i], img[i]
plt.grid(False)
plt.xticks([])
plt.yticks([])
plt.imshow(img, cmap=plt.cm.binary)
predicted_label = np.argmax(predictions_array)
if predicted_label == true_label:
color = 'blue'
else:
color = 'red'
plt.xlabel("{} {:2.0f}% ({})".format(class_names[predicted_label],
100*np.max(predictions_array),
class_names[true_label]),
color=color)
def plot_value_array(i, predictions_array, true_label):
predictions_array, true_label = predictions_array, true_label[i]
plt.grid(False)
plt.xticks(range(10))
plt.yticks([])
thisplot = plt.bar(range(10), predictions_array, color="#777777")
plt.ylim([0, 1])
predicted_label = np.argmax(predictions_array)
thisplot[predicted_label].set_color('red')
thisplot[true_label].set_color('blue')
# Plot the first X test images, their predicted labels, and the true labels.
# Color correct predictions in blue and incorrect predictions in red.
num_rows = 5
num_cols = 3
num_images = num_rows*num_cols
plt.figure(figsize=(2*2*num_cols, 2*num_rows))
for i in range(num_images):
plt.subplot(num_rows, 2*num_cols, 2*i+1)
plot_image(i, predictions[i], test_labels, test_images)
plt.subplot(num_rows, 2*num_cols, 2*i+2)
plot_value_array(i, predictions[i], test_labels)
plt.tight_layout()
plt.show()
plt.show()
Sample train images values
[[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
…
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]]
sample test images values
[[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
…
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]
[[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]]]
Model: “sequential_12”
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten_6 (Flatten) (None, 784) 0
_________________________________________________________________
dense_15 (Dense) (None, 128) 100480
_________________________________________________________________
dense_16 (Dense) (None, 10) 1290
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.4999 – accuracy: 0.8244
Epoch 2/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.3753 – accuracy: 0.8644
Epoch 3/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.3372 – accuracy: 0.8773
Epoch 4/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.3125 – accuracy: 0.8851
Epoch 5/20
1875/1875 [==============================] – 3s 1ms/step – loss: 0.2941 – accuracy: 0.8925
Epoch 6/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.2799 – accuracy: 0.8958
Epoch 7/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.2683 – accuracy: 0.9004
Epoch 8/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.2570 – accuracy: 0.9038
Epoch 9/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.2466 – accuracy: 0.9088
Epoch 10/20
1875/1875 [==============================] – 3s 1ms/step – loss: 0.2389 – accuracy: 0.9104
Epoch 11/20
1875/1875 [==============================] – 3s 1ms/step – loss: 0.2319 – accuracy: 0.9137
Epoch 12/20
1875/1875 [==============================] – 3s 1ms/step – loss: 0.2230 – accuracy: 0.9172
Epoch 13/20
1875/1875 [==============================] – 3s 1ms/step – loss: 0.2168 – accuracy: 0.9195
Epoch 14/20
1875/1875 [==============================] – 3s 1ms/step – loss: 0.2109 – accuracy: 0.9209
Epoch 15/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.2055 – accuracy: 0.9234
Epoch 16/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.1993 – accuracy: 0.9249
Epoch 17/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.1939 – accuracy: 0.9272
Epoch 18/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.1908 – accuracy: 0.9287
Epoch 19/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.1867 – accuracy: 0.9304
Epoch 20/20
1875/1875 [==============================] – 2s 1ms/step – loss: 0.1811 – accuracy: 0.9325
313/313 – 0s – loss: 0.3704 – accuracy: 0.8847
Test accuracy: 0.8847000002861023
[[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.00392157 0. 0. 0.05098039 0.28627451 0.
0. 0.00392157 0.01568627 0. 0. 0.
0. 0.00392157 0.00392157 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.01176471 0. 0.14117647 0.53333333 0.49803922 0.24313725
0.21176471 0. 0. 0. 0.00392157 0.01176471
0.01568627 0. 0. 0.01176471]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.02352941 0. 0.4 0.8 0.69019608 0.5254902
0.56470588 0.48235294 0.09019608 0. 0. 0.
0. 0.04705882 0.03921569 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0.60784314 0.9254902 0.81176471 0.69803922
0.41960784 0.61176471 0.63137255 0.42745098 0.25098039 0.09019608
0.30196078 0.50980392 0.28235294 0.05882353]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.00392157
0. 0.27058824 0.81176471 0.8745098 0.85490196 0.84705882
0.84705882 0.63921569 0.49803922 0.4745098 0.47843137 0.57254902
0.55294118 0.34509804 0.6745098 0.25882353]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0.00392157 0.00392157 0.00392157
0. 0.78431373 0.90980392 0.90980392 0.91372549 0.89803922
0.8745098 0.8745098 0.84313725 0.83529412 0.64313725 0.49803922
0.48235294 0.76862745 0.89803922 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.71764706 0.88235294 0.84705882 0.8745098 0.89411765
0.92156863 0.89019608 0.87843137 0.87058824 0.87843137 0.86666667
0.8745098 0.96078431 0.67843137 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.75686275 0.89411765 0.85490196 0.83529412 0.77647059
0.70588235 0.83137255 0.82352941 0.82745098 0.83529412 0.8745098
0.8627451 0.95294118 0.79215686 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0.00392157 0.01176471 0.
0.04705882 0.85882353 0.8627451 0.83137255 0.85490196 0.75294118
0.6627451 0.89019608 0.81568627 0.85490196 0.87843137 0.83137255
0.88627451 0.77254902 0.81960784 0.20392157]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0.02352941 0.
0.38823529 0.95686275 0.87058824 0.8627451 0.85490196 0.79607843
0.77647059 0.86666667 0.84313725 0.83529412 0.87058824 0.8627451
0.96078431 0.46666667 0.65490196 0.21960784]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0.01568627 0. 0.
0.21568627 0.9254902 0.89411765 0.90196078 0.89411765 0.94117647
0.90980392 0.83529412 0.85490196 0.8745098 0.91764706 0.85098039
0.85098039 0.81960784 0.36078431 0. ]
[0. 0. 0.00392157 0.01568627 0.02352941 0.02745098
0.00784314 0. 0. 0. 0. 0.
0.92941176 0.88627451 0.85098039 0.8745098 0.87058824 0.85882353
0.87058824 0.86666667 0.84705882 0.8745098 0.89803922 0.84313725
0.85490196 1. 0.30196078 0. ]
[0. 0.01176471 0. 0. 0. 0.
0. 0. 0. 0.24313725 0.56862745 0.8
0.89411765 0.81176471 0.83529412 0.86666667 0.85490196 0.81568627
0.82745098 0.85490196 0.87843137 0.8745098 0.85882353 0.84313725
0.87843137 0.95686275 0.62352941 0. ]
[0. 0. 0. 0. 0.07058824 0.17254902
0.32156863 0.41960784 0.74117647 0.89411765 0.8627451 0.87058824
0.85098039 0.88627451 0.78431373 0.80392157 0.82745098 0.90196078
0.87843137 0.91764706 0.69019608 0.7372549 0.98039216 0.97254902
0.91372549 0.93333333 0.84313725 0. ]
[0. 0.22352941 0.73333333 0.81568627 0.87843137 0.86666667
0.87843137 0.81568627 0.8 0.83921569 0.81568627 0.81960784
0.78431373 0.62352941 0.96078431 0.75686275 0.80784314 0.8745098
1. 1. 0.86666667 0.91764706 0.86666667 0.82745098
0.8627451 0.90980392 0.96470588 0. ]
[0.01176471 0.79215686 0.89411765 0.87843137 0.86666667 0.82745098
0.82745098 0.83921569 0.80392157 0.80392157 0.80392157 0.8627451
0.94117647 0.31372549 0.58823529 1. 0.89803922 0.86666667
0.7372549 0.60392157 0.74901961 0.82352941 0.8 0.81960784
0.87058824 0.89411765 0.88235294 0. ]
[0.38431373 0.91372549 0.77647059 0.82352941 0.87058824 0.89803922
0.89803922 0.91764706 0.97647059 0.8627451 0.76078431 0.84313725
0.85098039 0.94509804 0.25490196 0.28627451 0.41568627 0.45882353
0.65882353 0.85882353 0.86666667 0.84313725 0.85098039 0.8745098
0.8745098 0.87843137 0.89803922 0.11372549]
[0.29411765 0.8 0.83137255 0.8 0.75686275 0.80392157
0.82745098 0.88235294 0.84705882 0.7254902 0.77254902 0.80784314
0.77647059 0.83529412 0.94117647 0.76470588 0.89019608 0.96078431
0.9372549 0.8745098 0.85490196 0.83137255 0.81960784 0.87058824
0.8627451 0.86666667 0.90196078 0.2627451 ]
[0.18823529 0.79607843 0.71764706 0.76078431 0.83529412 0.77254902
0.7254902 0.74509804 0.76078431 0.75294118 0.79215686 0.83921569
0.85882353 0.86666667 0.8627451 0.9254902 0.88235294 0.84705882
0.78039216 0.80784314 0.72941176 0.70980392 0.69411765 0.6745098
0.70980392 0.80392157 0.80784314 0.45098039]
[0. 0.47843137 0.85882353 0.75686275 0.70196078 0.67058824
0.71764706 0.76862745 0.8 0.82352941 0.83529412 0.81176471
0.82745098 0.82352941 0.78431373 0.76862745 0.76078431 0.74901961
0.76470588 0.74901961 0.77647059 0.75294118 0.69019608 0.61176471
0.65490196 0.69411765 0.82352941 0.36078431]
[0. 0. 0.29019608 0.74117647 0.83137255 0.74901961
0.68627451 0.6745098 0.68627451 0.70980392 0.7254902 0.7372549
0.74117647 0.7372549 0.75686275 0.77647059 0.8 0.81960784
0.82352941 0.82352941 0.82745098 0.7372549 0.7372549 0.76078431
0.75294118 0.84705882 0.66666667 0. ]
[0.00784314 0. 0. 0. 0.25882353 0.78431373
0.87058824 0.92941176 0.9372549 0.94901961 0.96470588 0.95294118
0.95686275 0.86666667 0.8627451 0.75686275 0.74901961 0.70196078
0.71372549 0.71372549 0.70980392 0.69019608 0.65098039 0.65882353
0.38823529 0.22745098 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0.15686275 0.23921569 0.17254902 0.28235294 0.16078431
0.1372549 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]]
[[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.01176471 0.00392157 0. 0. 0.02745098
0. 0.14509804 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.00392157 0.00784314 0. 0.10588235 0.32941176
0.04313725 0. 0. 0. 0. 0.
0. 0.46666667 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.00392157 0. 0. 0.34509804 0.56078431
0.43137255 0. 0. 0. 0. 0.08627451
0.36470588 0.41568627 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.01568627 0. 0.20784314 0.50588235 0.47058824
0.57647059 0.68627451 0.61568627 0.65098039 0.52941176 0.60392157
0.65882353 0.54901961 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.00784314 0. 0.04313725 0.5372549 0.50980392 0.50196078
0.62745098 0.69019608 0.62352941 0.65490196 0.69803922 0.58431373
0.59215686 0.56470588 0. 0. ]
[0. 0. 0. 0. 0. 0.
0.00392157 0. 0.00784314 0.00392157 0. 0.01176471
0. 0. 0.45098039 0.44705882 0.41568627 0.5372549
0.65882353 0.6 0.61176471 0.64705882 0.65490196 0.56078431
0.61568627 0.61960784 0.04313725 0. ]
[0. 0. 0. 0. 0.00392157 0.
0. 0. 0. 0. 0.01176471 0.
0. 0.34901961 0.54509804 0.35294118 0.36862745 0.6
0.58431373 0.51372549 0.59215686 0.6627451 0.6745098 0.56078431
0.62352941 0.6627451 0.18823529 0. ]
[0. 0. 0. 0. 0. 0.
0.00784314 0.01568627 0.00392157 0. 0. 0.
0.38431373 0.53333333 0.43137255 0.42745098 0.43137255 0.63529412
0.52941176 0.56470588 0.58431373 0.62352941 0.65490196 0.56470588
0.61960784 0.6627451 0.46666667 0. ]
[0. 0. 0.00784314 0.00784314 0.00392157 0.00784314
0. 0. 0. 0. 0.10196078 0.42352941
0.45882353 0.38823529 0.43529412 0.45882353 0.53333333 0.61176471
0.5254902 0.60392157 0.60392157 0.61176471 0.62745098 0.55294118
0.57647059 0.61176471 0.69803922 0. ]
[0.01176471 0. 0. 0. 0. 0.
0. 0.08235294 0.20784314 0.36078431 0.45882353 0.43529412
0.40392157 0.45098039 0.50588235 0.5254902 0.56078431 0.60392157
0.64705882 0.66666667 0.60392157 0.59215686 0.60392157 0.56078431
0.54117647 0.58823529 0.64705882 0.16862745]
[0. 0. 0.09019608 0.21176471 0.25490196 0.29803922
0.33333333 0.4627451 0.50196078 0.48235294 0.43529412 0.44313725
0.4627451 0.49803922 0.49019608 0.54509804 0.52156863 0.53333333
0.62745098 0.54901961 0.60784314 0.63137255 0.56470588 0.60784314
0.6745098 0.63137255 0.74117647 0.24313725]
[0. 0.26666667 0.36862745 0.35294118 0.43529412 0.44705882
0.43529412 0.44705882 0.45098039 0.49803922 0.52941176 0.53333333
0.56078431 0.49411765 0.49803922 0.59215686 0.60392157 0.56078431
0.58039216 0.49019608 0.63529412 0.63529412 0.56470588 0.54117647
0.6 0.63529412 0.76862745 0.22745098]
[0.2745098 0.6627451 0.50588235 0.40784314 0.38431373 0.39215686
0.36862745 0.38039216 0.38431373 0.4 0.42352941 0.41568627
0.46666667 0.47058824 0.50588235 0.58431373 0.61176471 0.65490196
0.74509804 0.74509804 0.76862745 0.77647059 0.77647059 0.73333333
0.77254902 0.74117647 0.72156863 0.14117647]
[0.0627451 0.49411765 0.67058824 0.7372549 0.7372549 0.72156863
0.67058824 0.6 0.52941176 0.47058824 0.49411765 0.49803922
0.57254902 0.7254902 0.76470588 0.81960784 0.81568627 1.
0.81960784 0.69411765 0.96078431 0.98823529 0.98431373 0.98431373
0.96862745 0.8627451 0.80784314 0.19215686]
[0. 0. 0. 0.04705882 0.2627451 0.41568627
0.64313725 0.7254902 0.78039216 0.82352941 0.82745098 0.82352941
0.81568627 0.74509804 0.58823529 0.32156863 0.03137255 0.
0. 0. 0.69803922 0.81568627 0.7372549 0.68627451
0.63529412 0.61960784 0.59215686 0.04313725]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]]
