Sensors - Mission - Leka MLC & Movement Analysis Features

[ladislas]

Mission - Leka MLC & Movement Analysis Features

Introduction

Leka is an interactive educational toy. To react to interactions, Leka has an IMU, an Inertial Measurment Unit, composed of an accelerometer, a gyroscope and a magnetometer.

The component chosen for Leka is the LSM6DSOX.

The Mission

Use the MLC to detect different types of movements

Constraints

• Mbed OS based
• no ST BSP

Steps

1. TTHW (Time To Hello World)

This first step is to make sure the hardware is working using ST's provided software.

• - read the documentation
• - follow an ST examples and tutorials using all their tools (on Windows) and software (BSP if needed here)
• - make sure everything works, that you understand every concepts and that the process has no secret for you

EDIT 2020/06/06 (@ladislas ): ST's software doesn't work with our evaluations boards, moving to step 2.

2. TTHW + Mbed OS

This second step is to take the working example and start incorporating Mbed OS to the final working code.

• - analyse the code provided by ST and make it C++ friendly
• - start integrating Mbed OS eventually removing all mentions to ST's BSP and/or auto-generated code
• - make sure everything still works properly and that movements are recognized

3. Deep Dive + Mbed OS

Now that the sensors are working with Mbed OS, start fiddling with the options.

• - choose different interrupts and interrupts types
• - try, fail and try again
• - link interrupts with Mbed OS, create callbacks
• - try different things to get a feeling how those things work with each other deep down and how the final code should look like
• - create event queues
• - understand and play with memory slots

EDIT 2020/06/25 (@Aermanio): Apparently we can't access decision tree's memory slots individually

4. Adding our own movements

At this stage, we now have a working solution and we master every details of the collaboration between the hardware and Mbed OS. It's time to try with our own movements.

• - imagine the process to gather data
• - create a dedicated Mbed OS application to print data to the console
• - save data to a file with the correct format
• - analyze data with Weka/Unico
• - the final register map (.h) will be compiled in the program
• - movements should include : up/down, left/right and circle

5. Adding new movements while online

So now that our own process is setup, we want to be able to update the movements while the device is working. In practice, the data gathering and training will be done on other devices, and the update register data will be pushed to Leka through BLE.

• - add the ability to update a current model
• - improve the Mbed OS application with the ability to read data from the serial port
• - save those data in the device before updating the model (buffer)
• - imagine how the files generated by unico can be parsed and sent
• - add the ability to add a new model in a different slot
• - start with USB
• - make sure it works by adding a new movement (square for example)
• - if we have time, try doing the same through BLE

Resources

• ST Website - LSM6DSOX Datasheet
• ST Website - LSM6DSOX Resources
• ST Website - AN5259 - LSM6DSOX: Machine Learning Core
• ST Youtube Channel - Getting Started with LSM6DSOX
• ST Youtube Channel - LSM6DSOX Step-By-Step Tutorial (5 parts)
• ST Github - STMems Standard C Drivers - README
• ST Github - LSM6DSOX Drivers
• ST Github - STMems Machine Learning Core - README
• ST Github - LSM6DSOX MLC Examples
• Weka - Weka

[ladislas]

Added the following resources to the description:

• ST Github - STMems Standard C Drivers - README
• ST Github - LSM6DSOX Drivers
• ST Github - STMems Machine Learning Core - README
• ST Github - LSM6DSOX MLC Examples

[ladislas]

Also from https://github.com/leka/LekaOS/issues/2, stm32duino has a lot of examples for part 5.

• Example: https://github.com/stm32duino/X-NUCLEO-IKS01A3/tree/master/examples/X_NUCLEO_IKS01A3_LSM6DSOX_MLC
• MLC file: https://github.com/stm32duino/X-NUCLEO-IKS01A3/blob/master/examples/X_NUCLEO_IKS01A3_LSM6DSOX_MLC/lsm6dsox_activity_recognition_for_mobile.h#L42
• Loading process: https://github.com/stm32duino/X-NUCLEO-IKS01A3/blob/master/examples/X_NUCLEO_IKS01A3_LSM6DSOX_MLC/X_NUCLEO_IKS01A3_LSM6DSOX_MLC.ino#L99

[Aermanio]

Some unanswered questions

• It seems the configuration of the MLC is reset or lost on power loss
• Decision trees memory slots have to be set up in order during the configuration, we don't seem to be able to add a given decision tree in a specific memory slot, also selecting a particular tree to change does not seem possible, all the MLC has to be configured again
• The UCF files appear to use the PAGE_SEL (02h) register to access other advanced features register pages than 0 and 1 which are the only ones documented.
  We don't know if there is a documentation for pages 2-15 but nothing is said in the LSM6DSOX datasheet/Application Note. UCF file makes references to registers in page 3 of the memory for example.
• UCF code seems to set up a lot of configuration registers, is it possible to change these configurations after the tree configuration or should they stay the same.
• Is there any reason for us not to use Unico until the UCF génération? The documentation suggests that the features would be directly calculated by the MLC, yet we can't access directly to the results of thoses operations by ourselves.
• Need information concerning sensors data rate compared to MLC data rate