Case StudyEducation

Artificial Intelligence Helps a US Online Education Company to Communicate Better with Students

A major US education management organization provides online curriculum to homeschooled children and other schools. Its adaptive literacy program is aimed at helping kindergarten to twelfth grade students improve foundational skills, literature, and language through games and short lessons.

Executive Summary

The AI assistant successfully reduced teachers’ workload by handling off-topic conversations and guiding students effectively. Continuous feedback and retraining kept the solution improving over time. This project also delivered a reusable framework for future Deep Learning initiatives in education.

At a glance

Boosting teacher productivity and guiding students with a custom AI assistant

Supporting online chats can be a daunting task for dedicated teachers. Students sometimes do not state clearly what they do not understand, and some engage in off-topic conversations, taking up significant time that could be focused on teaching and improving lessons.

Outcomes
  • Accurately dealt with off-topic behavior, easing teachers’ workload
  • Guided students through the learning process in a human-like way without giving answers away
  • Continuous improvement via human feedback loops and neural network retraining
  • Created a reusable framework for future Deep Learning projects within the education business group
The Challenge

What stood in the way

Supporting online chats can be a daunting task, even more if a dedicated teacher just happens to use that medium in order to communicate with and help students. As the reader might guess, students sometimes do not state clearly what they do not understand, and some of them engage in off-topic conversations. These issues take up a significant fraction of the teacher's time and efforts, time that could be focused on other tasks related to teaching and improving lessons.

The company realized its need to boost the productivity of its online teachers and recognized that an artificial intelligence-powered assistant would help them process conversations faster. Furthermore, artificial intelligence could assist students with the initial steps of working towards an answer without giving it away, deal with problematic cases, and reroute conversations when necessary.

The objective, then, was to create an AI assistant able to guide students through their learning process in a human-like way. This assistant had to be tailor-made to fit the client's e-learning environment and to comply with teacher behavior guidelines and educational regulations.

The Approach

How Orbis got to work

Not all AI solutions may be optimal for a given problem, so Orbis approached the project with an experimental mindset. Artificial Intelligence (AI) value creation was guided by rigorous validation of ideas and thorough testing of key hypotheses, followed by iterative cycles of improvement.

Performed an initial 12-week Proof Of Concept (POC) in which a standalone bot “brain” was developed and then deployed as a chat application, demonstrating its feasibility.
The POC included assessment of data capabilities and of required state-of-the-art technology.
A team consisting of data scientists, data architects and cloud computing specialists was specifically assembled to create and deploy the brain of the chatbot, its data infrastructure and the integration with existing platforms.
After the success of the initial POC, a production-ready version fully integrated with the client's platform was developed in approximately seven months and deployed in December 2017.

Currently, the solution is actively improved based on user feedback, with an improved version coming out approximately every two months. Additionally, a data platform was created to support the associated processes of chatbot training, performance feedback loops, and reporting.

This cloud-based big data platform stores transcripts of all ongoing conversations in the client's application, either between humans or involving the chatbot. This data grows at a rate of over 1 million lines per month and constitutes an ever-expanding corpus from which the AI learns.

The Solution

The result delivered

Orbis's solution was able to scale and differentiate something as unique as a teacher guiding a student to find answers to different problems. The performance of an AI solution depends on several factors: data availability is fundamental, but another key factor is the ability to transform the outputs of the AI algorithm into appropriate user experiences.

Working in close collaboration with the client, Orbis implemented a custom-built recurrent neural network architecture – the state-of-the-art solution for language processing – able to learn from historical data on millions of actual interactions between students and teachers.

The solution, named Orbis LearnGuide, was built with PyTorch, a Deep Learning framework for fast, flexible experimentation. The neural network was trained using specialized high-performance hardware. A cloud-based deployment then enabled scalability by launching new trained chatbot instances as required by user demand.

In six months of operation, the bot showed its ability to accurately deal with situations such as off-topic behavior, thus easing the teachers' workload.
Where performance deviated from the expected outcome, a feedback loop with human reviewers enabled retuning and retraining of the neural network.
Orbis created a framework and approach to enable the development of Deep Learning projects for future uses within the educational services provider business group.

This project demonstrated how a custom AI solution can be built around the client's specific needs. The experimental and data-centric approach also yielded valuable insights along the way, enriching and enhancing the final product.