Building Metacognition with an Interactive AI Math Tutor

TLDR

I built an open-source toy AI Math Tutor where the learner and the bot engage with an interactive activity.

The bot not only responds in text but also performs actions on the activity so that the interaction with the learner drives process-oriented learning. The video below shows a demo interaction of an activity I cloned from Brilliant.

I built this to show a new kind of interaction with AI tutors that has the potential to enable learners to exercise their curiosity and reflect on their thought processes and knowledge. The effectiveness of a system that uses a UX like this depends on the pedagogical aspects that utilize this UX effectively.

Why do we need a UX like this?

You might wonder about the need for a user experience like this. To explain that I would like to first show the benefits of human tutor-learner interactions and proceed to the limitations of current AI tutors, and hopefully you will understand the need.

Human Tutor-Learner interactions

I’ve seen that kids are often engaged in learning math when they’re doing something. However this isn’t really the case with unmotivated kids.

Another fact I’ve noticed with good human tutor-learner interactions is that they engage in a conversation by manipulating a shared artifact(lego blocks, game, etc).

This mode of interaction has a few key affordances:

1. A low-barrier medium for the learner to convey their thoughts because they can use their body and words.
2. Opportunity for the learner and the tutor to explore several states/configurations of the artifact by manipulating it.

These affordances in turn produce these benefits:

1. The tutor can better understand the learner.
2. The learner can better understand their thoughts, allowing them to reflect and learn.
3. The learner can explore their curiosity with ease(depending on the complexity of the manipulations).
4. The tutor can guide the learner through socratic processes-oriented learning rather than rote learning and memorization.

These benefits allow the learner to think about their thinking and actually learn by themselves without being told their mistakes and how to correct them.

Current state of AI Tutors

I’m going to choose Khanmigo, the AI-powered personal tutor built by Khan Academy because its pretty popular and is a good representative of AI tutor capabilities with regards to math.

Here is how the platform works when I answer a question about the slope of a line incorrectly.

Let’s break down what happened:

1. I submitted the wrong answer.
2. I got a popup from Khanmigo with a button that says Tutor me.
3. Clicking on this opened the chat panel with a message from the learner saying Help me check my answer.
4. The bot asks me how I found the slope using the points (0,2) and (2,0) .
5. I said I did 2/2 with the hope that it’ll pick up on my mistake.
6. The bot seems to have understood my mistake, gave me the formula and asked me how I would apply the formula and what do I get when I subtract the x and y coordinates.
7. I replied its 2-0/0-2 which is 2/2 to see if it gets to the bottom of my fundamental mistake.
8. It replies with the formula again and asks me what I get if I subtract the coordinates in the correct order.

Here’s what Khanmigo does well:

• Encourages the learner.
• Asks for what the learner has done so far to personalize the response.

Here’s what could be improved:

1. Instead of asking the learner to use the formula again and in the correct order, identify the more fundamental mistake with subtraction, help the learner identify their misunderstanding/mistake so that they solve it by themselves.
2. Use the graph to communicate by highlighting the points, the order, etc.
3. Ask the learner to communicate their work by talking aloud and sketching/highlighting on the graph.

As I mentioned in the beginning, the first improvement of identifying the fundamental issue depends on the pedagogical aspects of the system which is not the point of this post but is definitely something I will write about later.

Instead I want to bring attention to the second and third improvements because they are UX capabilities that implicitly enable the learner to think about their thinking.

This ability to be aware of one’s thought processes and have an understanding of the patterns behind them is called Metacognition.

Metacognition can not only help with learning but also with self-doubt and developing a “How can I” attitude. You can read more here for a brief overview on the topic.

So what is this new UX?

Now that we’ve seen the need and benefits of UX that enables process-oriented learning and builds metacognition, let’s see what I did.

Since the premise of this UX is shared interaction of an artifact, I chose a question on Linear Equations from Brilliant because they have so many interactive activities that allows process-oriented learning to take place but the feedback learners get is the same irrespective of their attempts because there is no chatbot in the platform yet.

So I cloned the activity from this question where learners gain an intuition of balancing an equation with two variables by trying different values for $x$ and $y$ until the “size” or “width” of both sides of the equation become equal.

One problem learners might run into is when they try to find a solution is when they choose a value for $x$ but can’t find an appropriate value for $y$ to satisfy the equation.

The question: Find a value for $x$ and

For example when