TensorFlow.js has provided several examples – you can also see live demos here. I have modified the pacman example to build a Rock-Paper-Scissors game. Here’s a quick recording of the game.
Here’s whats happening under the covers:
- Loading a mobilenet model that’s pretrained – these models are based on convolutional neural network designed by researches at Google. They are “mobile-first” so are resource-friendly and execute fast. They are fast, small and pretty accurate.
- Now we build an image classifier that can detect if an image is rock, paper or scissors. Since we’re tackling a custom problem, we need to start with creating our dataset – this is where the dozen or so images of rock, paper and scissors were taken using the webcam and labeled appropriately.
- Final step is to retrain the mobilenet model – luckily for us TensorFlow comes packaged with great tools that you can use to retrain MobileNets without having to actually write any code.
We will be hosting this application on AppEngine shortly.
PS: As an aside, here’s a rock-paper-scissors app built by my daughter on scratch – https://scratch.mit.edu/projects/213853734/ – check it out 🙂