Tutorial: Binary Matcher with TensorFlow

--D. Thiebaut (talk) 15:26, 19 March 2017 (EDT)

---

This page illustrates how to design a simple multi-layer Tensorflow Neural Net to recognize integers coded in binary and output them as a 1-hot vector. For example, if we assume that we have 5 bits, then there are 32 possible combinations. We associate with each 5-bit sequence a 1-hot vector. For example, 0,0,0,1,1, which is 3 in decimal, is associated with 0,0,0,1,0,0,0,0...,0, which has 31 0s and one 1. The only 1 is at Index 3. Similarly, if we have 1,1,1,1,1, which is 31 in decimal, then its associated 1-hot vector is 0,0,0,0,...0,0,1, another group of 31 0s and one last 1. Our binary input is coded in 5 bits, and we make it more interesting by adding 5 additional random bits. So the input is a vector of 10 bits, 5 random, and 5 representing a binary pattern associated with a 1-hot vector. The 1- hot vector is the output to be predicted by the network.

...

Source

This tutorial is in the form of a Jupyter Notebook, and made available here in various forms:

• pdf
• tgz-zipped archive containing the markdown and notebook

Output

0 cost = 4.06461 Accuracy on training data = 7.78% Accuracy on test data = 0.00%
100 cost = 0.0568146 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
200 cost = 0.0249612 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
300 cost = 0.0156735 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
400 cost = 0.0113394 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
500 cost = 0.00885047 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
600 cost = 0.00724066 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
700 cost = 0.00611609 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
800 cost = 0.00528698 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
900 cost = 0.00465089 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1000 cost = 0.00414775 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1100 cost = 0.00374 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1200 cost = 0.00340309 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1300 cost = 0.00312006 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1400 cost = 0.00287907 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1500 cost = 0.00267149 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1600 cost = 0.00249079 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1700 cost = 0.00233219 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1800 cost = 0.00219193 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
1900 cost = 0.00206695 Accuracy on training data = 100.00% Accuracy on test data = 90.00%
2000 cost = 0.00195495 Accuracy on training data = 100.00% Accuracy on test data = 90.00%