A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect

Shanfeng Hu, Hindol Bhattacharya, Matangini Chattopadhyay, Nauman Aslam and Hubert P. H. Shum
Proceedings of the 2018 International Conference on Software, Knowledge, Information Management and Applications (SKIMA), 2018

A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect

Abstract

Convenience internet access and ubiquitous computing have opened up new avenues for learning and teaching. They are now no longer confined to the classroom walls, but are available to anyone connected to the internet. E-learning has opened massive opportunities for learners who otherwise would have been constrained due to geographical distances, time and/or cost factors. It has revolutionized the learning methods and represents a paradigm shift from traditional learning methods. However, despite all its advantages, e-learning is not without its own shortcomings. Understanding the effectiveness of a teaching strategy through learner feedback has been a key performance measure and decision making criteria to fine tune the teaching strategy. However, traditional methods of collecting learner feedback are inadequate in a geographically distributed, virtual setup of the e-learning environment. Innovative and novel learner feedback collection mechanism is hence the need of the hour. In this work, we design and develop a deep learning based student feedback prediction system by recognizing the subtle facial motions during a student’s learning activity. This allows the system to infer the needs of the learners as if it is a realhuman teacher in order to provide the appropriate feedback. We propose a recurrent convolutional neural network structure to understand the color and depth streams of video taken by an RGB-D camera. Experimental results have shown that our system achieve high accuracy in estimating the feedback labels. While we demonstrate the proposed framework in an e-learning setup, it can be adapted to other applications such as in-house patient monitoring and rehabilitation training.

Downloads

YouTube

Citations

BibTeX

@inproceedings{hu18dualstream,
 author={Hu, Shanfeng and Bhattacharya, Hindol and Chattopadhyay, Matangini and Aslam, Nauman and Shum, Hubert P. H.},
 booktitle={Proceedings of the 2018 International Conference on Software, Knowledge, Information Management and Applications},
 series={SKIMA '18},
 title={A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect},
 year={2018},
 month={12},
 pages={1--8},
 numpages={8},
 doi={10.1109/SKIMA.2018.8631537},
 issn={2573-3214},
 publisher={IEEE},
 location={Phnom Penh, Cambodia},
}

RIS

TY  - CONF
AU  - Hu, Shanfeng
AU  - Bhattacharya, Hindol
AU  - Chattopadhyay, Matangini
AU  - Aslam, Nauman
AU  - Shum, Hubert P. H.
T2  - Proceedings of the 2018 International Conference on Software, Knowledge, Information Management and Applications
TI  - A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect
PY  - 2018
Y1  - 12 2018
SP  - 1
EP  - 8
DO  - 10.1109/SKIMA.2018.8631537
SN  - 2573-3214
PB  - IEEE
ER  - 

Plain Text

Shanfeng Hu, Hindol Bhattacharya, Matangini Chattopadhyay, Nauman Aslam and Hubert P. H. Shum, "A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect," in SKIMA '18: Proceedings of the 2018 International Conference on Software, Knowledge, Information Management and Applications, pp. 1-8, Phnom Penh, Cambodia, IEEE, Dec 2018.

Supporting Grants

The Royal Society
An Affective Smart Environment for Personalized Learning and Teaching
Royal Society International Exchanges (Ref: IE160609): £11,841, Principal Investigator ()
Received from The Royal Society, UK, 2017-2019
Project Page

Similar Research

Pierre Plantard, Hubert P. H. Shum and Franck Multon, "Motion Analysis of Work Conditions using Commercial Depth Cameras in Real Industrial Conditions", DHM and Posturography, 2019
Pierre Plantard, Hubert P. H. Shum and Franck Multon, "Ergonomics Measurements using Kinect with a Pose Correction Framework", Proceedings of the 2016 International Digital Human Modeling Symposium (DHM), 2016

 

 

Last updated on 25 March 2024
RSS Feed