Demonstration video of atomic activity recognition using wearable devices (version 2)

This is a demonstration video of our atomic activity recognition system using wearable devices. Our system uses the following eight types of sensors: 1. Accelerometer, 2. Gravity, 3. Gyroscope, 4. Linear accelerometer, 5. Magnetometer in a smartphone (NEXUS 5X), 6. Accelerometer and 7. Gyroscope in a smartwatch (Microsoft band 2), and 8. Accelerometer in intelligent glasses (JINS MEME). In addition, our system recognises the following 29 atomic activities: 1. Lying, 2. Sitting, 3. Squatting, 4. Standing, 5. Walking, 6. Bending, 7. Bring, 8. CleanFloor, 9. CloseDoor, 10. CloseLid, 11. CloseTapWater, 12. Drink, 13. DryOffHand, 14. DryOffHandShake, 15. Gargle, 16. GettingUp, 17. LyingDown, 18. OpenDoor, 19. OpenLid, 20. OpenTapWater, 21. PuttingHandBack, 22. RubHand, 23. SittingDown, 24. SquattingDown, 25. StandUpFromSquatting, 26. StandingUp, 27. StretchingHand, 28. TakeFromFloor, 29. TalkTelephone. These activities are named as "atomic activities", since our future plan is to combine their recognition results to deduce higher-level activities (composite activities). In the video, the window at the top-left shows the actual scene where the user wearing the smartphone, smartwatch and intelligent glasses performs various activities. Recognition results are visualised in the browser-based UI (User Interface). In the UI, the small window at the top-left displayed the three atomic activities to which the system assigns the highest scores. "Please take into account a time delay that is caused by the slow data storing/retrieval in the UI." The three atomic activities displayed in the UI were actually recognised a few seconds before. This is only an implementation issue of the UI, and the actual activity recognition is done in about 0.01 seconds, as shown in the terminal. By taking the time delay into account, it can be seen that atomic activity recognition by our system is overall accurate, especially true-positive activities are mostly included in the three activities computed by the system. Please do not care the window at the bottom-left. From 0:00 to 2:35: The time-chart in the UI shows the temporal transition of recognition scores for each of 29 atomic activities (near real-time visualisation involving the above-mentioned time delay). From 3:20 to 3:52: The UI shows a "time-chart-type historical visualisation" of recognised atomic activities. This enables a user to know what kind of activities he/she performed at each of time points in the past. From 4:33 to end: The UI shows a "bar-graph-type historical visualisation" of recognised atomic activities. This visualises how long (how many hours) a user performed each of 29 atomic activities in the last day (week, month or year). One use-case is to show the bar-graph to a doctor. For example, "drinking" is an activity to feed liquid in a body and is very important for his/her live, but elderly people with dementia sometimes forget this activity. For this, the doctor can check the bar-graph and get an objective evaluation about whether a patient got enough liquid in the last day (or week). Also, based on the hours of "walking", the doctor can advise a patient to do more exercise. The demonstrated system has been developed in the project "Cognitive Village: Adaptively Learning Technical Support System for Elderly (Grant Number: 16SV7223K)" supported by German Federal Ministry of Education and Research (BMBF). For more detail, please visit the project web-site http://www.cognitive-village.de/ or contact us via [email protected] In addition, technical details are presented in the following papers: [1] Lukas Köping, Kimiaki Shirahama and Marcin Grzegorzek: "A General Framework for Sensor-based Human Activity Recognition", Computers in Biology and Medicine (to appear) [2] Kimiaki Shirahama and Marcin Grzegorzek: "On the Generality of Codebook Approach for Sensor-based Human Activity Recognition", Electronics (Special Issue "Data Processing and Wearable Systems for Effective Human Monitoring"), Vol. 6, No. 2, Article No. 44, 2017 Also, a source code for codebook-based feature extraction (off-line version) is available on http://www.pr.informatik.uni-siegen.d...