Project Goal: Create a deep neural network for video classification of human actions in an industrial setting (based on the HRI30 dataset)
Application: allow collaborative robots to recognize human actions so they can perform collaborative tasks safely and efficiently in shared workspaces
Project Features:
Model trained and tested on the HRI30 video dataset with 30 classified human industrial actions such as walking, drilling, polishing, etc.
The model consists of a Mixed Convolutional 3D ResNet-18 (MC3_18) backbone using pretrained weights from KINETICS400_V1 with a custom feature extractor
Cross Entropy Loss Function
Stochastic Gradient Descent with momentum
ReLU activation function
Result:
implemented 8 training tests while adjusting the frame extraction method, parameter variation, learning rate, and video frames per clip
achieved 94% accuracy when tested on the validation data
further refinement needed to improve accuracy on test data and on live human-robot collaboration in industrial settings.
The HRI30 dataset contains subjects moving and/or holding a tool/object which determined the structure of our Mixed Convolutional 3D ResNet-18 neural network
Training results show a variety of responses depending on the parameters changed. The % accuracy is based on our validation data