SmartMove

Smart Move

Smartmove is a student project imagined by four IT students of Polytech Grenoble. Smart Move is an IoT and AI approach for the sports. The project monitoring for teachers of Polytech Grenoble are available in this page https://air.imag.fr/index.php?title=RICM5_2017-2018_-_SmartMove.

Team project

Membre	Rôles
Anthony Geourjon	Chef de projet
Timothée Lemaire	Responsable hardware (black belt thinkpad)
Clément Rouquier	Responsable Dofus
Vincent Turrin	Batteur

• Team : Anthony Geourjon (leader), Timothée Lemaire, Vincent Turrin & Clément Rouquier
• Supervised by : Bernard Tourancheau, Didier Donsez

Abstract

In 2017, two highly innovative domains emerge  : the Internet of Things and the Artificial Intelligence and especially the machine learning. These technological advances open new possibilities to imagine new ways to help people. In this project, we present our proposal for the Open IoT challenge 4.0 organized by the Eclipse Foundation. This project is also part of the course of Wireless Sensor Networks given by Prof. Bernard Tourancheau at Polytech Grenoble (Engineering school)

Keywords

• IoT
• Machine learning,
• Sports

Documents

Rapport final - Presentation finale - Presentation de mi-parcours - Flyer - Poster - Vidéo 1

Introduction

Smart move is an ambitious project to connect sport equipments such as windsurf boards, skis, skateboards, etc.. The idea is to put smart cases to detect, record and transmit data (including movement, rotation, acceleration) to a master device (typically a smartphone or a smartwatch). With this data flow, the master device will recognize particular movements like a 360-rotation in skiing or a speed loop in windsurfing. To perform that, the master device will run a machine learning trained algorithm. The following figure shows an example of a connected windsurf board, monitored by smart cases as we would do in our project :

Architecture

We can generalize this architecture and split the project in three main parts : the smart cases, the master device and lastly the part which is situated in the cloud. The smart cases and the master device communicate through the use of Bluetooth Low Energy. The cloud will provide a REST API and communicate with the master device through HTTP requests. There are no direct communications between smart cases and the cloud.

The first part will be the set of the smart cases. A smart case is a very simple device with an accelerometer, a gyroscope, a Bluetooth transmitter and a battery. The goal of a smart case is to capture the movements of a part of the equipped device. Depending of the use case, the smart case could need to be waterproof, shock resistant or specifically need a long autonomy. We choice to use Arduino devices, because this technology is open-source and very adapted for prototyping. The Arduino 101 card integrates all the electronic features required and is also very cheap. According to the equipment instrumented we need more or less cards. As seen previously, we need 4 cards installed at the right places to instrument a windsurf board.

We need financial helps for this part. We need to buy some Arduino 101 cards and their batteries. We have already one card, have three others will be perfect to implement Smart move in the case of windsurfing. The cost of one Arduino 101 card is \$30 and a battery \$15. With the \$150 we can reach our goals of 4 cards/batteries and develop serenely our project.

The second part is called the master device. This device will be a smartphone or a smartwatch and will be the gateway of the smart cases. With the data and the machine learning algorithm, the device will provide all the information required for the user application. Even if the machine learning was not trained on the device, it must be powerful enough to run the trained model. We choose Tensor Flow Lite library, released by Google and running on Android. The smartphone doesn’t need a lot of calculation power. Our Honor 5C with Kirin 650 SoC (octa core at 1.7 GHz and GPU Mali T830 (40 GFlops)) and its 2 Go of RAM easily allows to run all of the Tensor Flow Lite example applications.

Lastly the third part will be the server on the cloud. The server will train the machine learning algorithm and take part as a classic server for the user application. As students we can’t pay a server on AMI AWS Deep Learning, so we will train our (simple) model on a desktop computer. But we will also use the free AWS student offer, to run the server application on EC2 instances.

Targeted industries

We want to target all the sport industries, but particularly the ones where it would me meaningful to monitor specific moves. For example, the windsurf case is very relevant. It could help a beginner learn the maneuvers, because it is really hard to start when you don't have any experience. The beginner could know what placements and body position to adopt depending on the wind behavior. For a more experienced user, it would be more a matter of training session than adopting the good moves. The user could follow a specific training session where he would need to do the required moves. It is relevant because a training session can quickly become boring, and our system could make this routine more lively and with a real goal. This advanced user could also learn from the data gathered. With a pre-programmed session a windsurf instructor could judge the level of a new student.

The majority of the sports still haven't been touched by the IT and electronic innovations. Only classic sports like cycling or trail running could enjoy application like tracking GPS, Strava, etc. The emergence of artificial intelligence, being able to run on small devices, like Tensor flow lite model on Android open a lot of new possibilities.

Prototype

At the beginning, we would like implement SmartMove for the windsurfing. But the windsurfing was not the best choice to prototype a project like smart move. To test our work, we should go to the lake and build sealing equipment. To bypass these constraints, we choose to work with windskate instead of windsurfing. In terms of functionality, the both are very close and we can work everywhere with non sealing equipment.

All the details for the implementation of Smart Move for windskate are available here (french, english version is in writing).

Media:CahierDesChargesWindskate.pdf

Machine learning model

To recognize the windsurf maneuvers, we will use a two-layer architecture. For the first layer : each smart case, will recognize its own move. For example, acceleration data will help us define the sail position compared to the board. Along with the angular information, we could find one case current position. The two board cases will allow us to keep a track of the current direction and the fallings.

The following is a little schema illustrating our model (in French).

Those data will be inputs for the second layer which will identify the global windsurf move. This layer will easily learn while being supervised.

Maneuver can be caracterized as the following : an almost static state, some active motion, back to an almost static state.

A move could be caracterized by an automate similar to this one.

This is the model as we see it now but it will evolve as we learn more about deep learning.

Here is an example move :

Planning

This is our planing in its current state (29/11/2017) :

Conclusion

With the recognition of moves, and the adaptability of the machine learning, which would make our system easily portable on a lot of activities. We can imagine to build a lot of new applications above Smart move. For example, we can develop an application which can directly detect your success or not and modify your training in real time to optimize your progression, by giving new exercices or by giving informations to improve the move. In addition, as described for the windsurf case, we could analyze a session afterward, and help to find the problems thanks to data converted to be readable by their user, or directly analyzed by the application. Innovative devices like smart glasses or pico projector like Cicret bracelet could also be integrated in user application to allow an easy interaction with the application during sport.

The power of machine learning is infinite, and it makes us think that our project has a big interest, because it could be applied in a lot of fields, compared to a standard solution where algorithms would be extremely specific for each use case.

Our first goal with Smart move is to do a proof of concept of the interests of connected sports equipments used with machine learning algorithm. We can see Smart move like a pattern of application. Our final goal is to use it to develop an application for windsurfing. The open-iot challenge 4.0 is a very good opportunity to get financial help and above all, feedbacks from highly skilled people on our work.

References

• https://developer.android.com/index.html
• https://www.tensorflow.org/
• https://www.arduino.cc/
• https://github.com/tensorflow/tensorflow
• http://www.mqtt.org