Hector Motsepe
3 November 2021
The Integration of Artificial Intelligence and Saas has yet to reach its full potential in numerous industries. The contributing factor to this challenge is that we haven’t identified problems that AI can solve in these industries. The internet is full of solutions for machine learning problems that are relevant to tech giant companies only. For example, there are thousands of examples of how to build spam/ham detection, house price prediction, or sentiment analysis. But there are few solutions for problems in the energy sector on how AI can optimize energy consumption to reduce “load shedding”, or anomaly detection system for home appliances, such as geysers, to alert insurance companies about required maintenance.
These challenges have encouraged us to tap into different industries and shift our thinking from obsolete solutions to solving modern problems. I have attempted to build a SaaS called EduIntellect for the education industry, to eliminate challenges faced by universities and colleges as a result of the COVID-19 outbreak. The SaaS aims to personalize a student’s education and automate activities such as class attendance.
The Integration of Artificial Intelligence and SaaS
The features of the EduIntellect
Think of EduIntellect as a value-added product. The intention was not to build a new learning management system but rather to make the existing one intelligent.
Cheating Detection System
The cheating detection system monitors and ensures that students do their work and write their exams on their own. However, it is hard to monitor students who are writing exams at home. How do you ensure honesty and integrity? This system eliminates this challenge by detecting student facial movement, mouth behaviour, or background sound during an online examination.
Facial Recognition for participation monitoring and class attendance
This feature encourages class participation during remote learning by observing students’ facial movements during a lecture. Moreover, the feature can recognize students’ faces and record their attendance.
Sign language Detection
Disability is a major concern during remote learning. Some students cannot speak or hear. EduIntellect creates a conducive environment for disabled students during remote learning by detecting sign languages or any kind of nonverbal communication technique.
Intelligent Chatbot
The chatbot is more like a lecturer assistant. Students can ask anything related to admin information such as assignment and exam deadlines or even have a conversion around covered chapters to solidify their understanding.
The technical implementation of the systems
Facial Recognition Algorithm
I used OpenCV, TensorFlow, and other deep learning libraries to build the facial recognition algorithm. The algorithm is a series of several related problems. Firstly, I simplify the version of the image by using the HOG algorithm to encode the image. I then use the simplified image to locate the image that is similar to the generic HOC encoding of a face. I locate the main landmarks in the face by figuring out the pose of the face.
These landmarks help to warp the image so that the mouth and eyes are centred. The third step is to build a neural network that knows how to measure the features of the face. The image will pass through this neural network, which in turn will output 128 measurements. These measurements are used to match the closest measurements we’ve measured in the past, if the measurements are close then that is a match.
Since students should not speak or look away from their screen during an online exam, I have tuned the algorithm for cheating detection from detecting eye movement to predicting the mouth movement of the subjects.
Machine Learning Model on Cloud
I hosted the machine learning model on the cloud and leveraged it from our application. However, I faced a few issues with that, the model size and the time it takes to make a prediction were very bad. I decided to use an open-source deep learning framework for on-device inference called TensorFlow lite. This framework enabled me to convert the model into a more compressed flat buffer with a TensorFlow lite converter. The compressed .tflite file was then integrated into our application. This is also known as making predictions on the edge.
I am excited that I have managed to use machine learning to try and restore the value proposition of the universities. The possibilities of machine learning are endless. I think is time that we as industry experts consider reshaping and redesigning our solution to overcome modern challenges.
Integration at Integrove
Hector Motsepe is a junior data scientist at Integrove. We encourage all of our employees to be self-starters and enable them to start projects of their own. If you’d like to start your tech journey with us, check out our current vacancies!