Home / For kids / A3

A3 How Machines Learn

How do we know if an AI is any good?

Stage
Creators, ages 9 to 11
Lessons
6
You make
A mini AI project and a one-page “how it works and where it fails”

Before you start. You can do all of these yourself. Ask a grown-up before you look anything up online.

You don’t need a computer tool for these lessons. You will build a paper “machine” from cards, which works the same way. The Try page has a digital machine too.

Safety rules · Teacher and parent version of this course

Your lessons

Open a lesson. Tick the boxes as you go. Do one lesson at a time.

Lesson 1: Rules Versus LearningAbout 40 minutes
  1. Get ready
  2. Do it
  3. Check yourself
  4. Show a grown-up

Get ready

You need:

  • Paper
  • Pencil
  • 8 safe things from around the kitchen or toy box (mugs, bowls, cups, plastic tubs)

Do it, step by step

Real AI test (with a grown-up): Spam test: rules or learning?

Optional, and a grown-up runs it. Under 13, the grown-up types and reads, and you direct and check. Read the AI lab guide first. About 25 minutes.

Why we do this. You compared rules with learning on paper. Now see which approach a real AI uses on a real job.

Prompt 1: Spam or not spam?

Here are 7 invented email subject lines. For each one, say SPAM or NOT SPAM and give one short reason. 1) WIN a FREE phone!!! Click now!!! 2) Your school trip form is due on Friday. 3) Congratulations!! You have been selected for a prize. Send your bank details. 4) Can we swap lunch plans to Thursday? 5) Last chance!!! 90% off all watches!!! 6) Your library book is ready to collect. 7) Free swim session this Saturday at the leisure centre.

Change only made-up parts in [brackets]. Never add a name, school, address, photo or anything private.

What to look for

  • First write 3 spam rules of your own (for example “has the word FREE”). Which of the 7 do your rules get wrong? Number 7 is a trap for the word FREE.
  • Did the AI get number 7 right? What reason did it give?
  • Are the AI’s reasons rules, or patterns from examples?

Write it down

Tool, prompt, what it said, what you checked, and whether it was right, wrong or not sure. Record sheet

Think about it. When would you pick rules? When would you pick learning?

Check yourself

What is the difference between a rule-based program and a learning model?

Open the answer

A rule-based program follows rules someone wrote. A learning model finds patterns in many examples.

Why do rules break?

Open the answer

Because the world has odd cases nobody thought to write a rule for.

Show a grown-up

Show a grown-up your rules and see if they can find an object that breaks them.

Next: Lesson 2

Lesson 2: Training and Testing DataAbout 45 minutes
  1. Get ready
  2. Do it
  3. Check yourself
  4. Show a grown-up

Get ready

You need:

  • 20 small pieces of paper
  • Pencil or pens

Do it, step by step

Real AI test (with a grown-up): Seen and unseen examples

Optional, and a grown-up runs it. Under 13, the grown-up types and reads, and you direct and check. Read the AI lab guide first. About 25 minutes.

Why we do this. A model must be tested on things it has not seen. See a real AI learn a pattern from examples.

Prompt 1: Teach the pattern

Here are examples of two made-up groups. cat = flim. hat = flim. bat = flim. dog = flam. log = flam. frog = flam. Now put these in flim or flam, one word each: mat, fog, rat, jog.

Prompt 2: Test on what it has seen

Now put these in flim or flam: cat, log.

Change only made-up parts in [brackets]. Never add a name, school, address, photo or anything private.

What to look for

  • It has to work out the pattern from the examples. What is it? (Hint: think of how the words sound.)
  • How many of the 4 unseen words did it get right?
  • Why is the second test (cat, log) a trick?

Write it down

Tool, prompt, what it said, what you checked, and whether it was right, wrong or not sure. Record sheet

Think about it. Which test tells you whether it really learned? Why?

Check yourself

Why must a model be tested on examples it has not seen?

Open the answer

If you test it on examples it has already learned, a high score does not tell you whether it works on new things.

Is a student who memorises answers the same as one who understands?

Open the answer

No. You can only tell the difference with new questions.

Show a grown-up

Explain to a grown-up why the 14 cards are a trick test.

Next: Lesson 3

Lesson 3: Build a ClassifierAbout 50 minutes
  1. Get ready
  2. Do it
  3. Check yourself
  4. Show a grown-up

Get ready

You need:

  • 24 small pieces of paper
  • Pencil or pens
  • Paper for a results table

Do it, step by step

Real AI test (with a grown-up): Build a classifier with an AI

Optional, and a grown-up runs it. Under 13, the grown-up types and reads, and you direct and check. Read the AI lab guide first. About 35 minutes.

Why we do this. You built a face classifier with cards. Now make your own categories and test a real AI on them.

Prompt 1: Your categories

Choose two groups of your own, such as “calm” and “exciting”. Write 12 short sentences, label each, and keep 4 as your hidden test. Give the AI 8 as examples.

Prompt 2: Teach and test

Here are labelled examples. [PASTE YOUR 8 LABELLED SENTENCES.] Now label each of these new sentences with one word: [PASTE YOUR 4 TEST SENTENCES WITHOUT LABELS.]

Change only made-up parts in [brackets]. Never add a name, school, address, photo or anything private.

What to look for

  • Count right out of 4 (your labels are the “truth”).
  • Where did the AI disagree? Is it wrong, or is the sentence hard to label?
  • Compare the score with your paper classifier.

Write it down

Tool, prompt, what it said, what you checked, and whether it was right, wrong or not sure. Record sheet

Think about it. Who decides the truth in a classifier? What if people disagree?

Check yourself

Why hold back 5 of each?

Open the answer

So there are cards the machine has never seen, to test it fairly.

What kind of faces confused it?

Open the answer

Probably faces that were halfway between happy and sad, or very different from the training faces.

Show a grown-up

Show a grown-up your results table and what you learned from the mistakes.

Next: Lesson 4

Lesson 4: Accuracy and MistakesAbout 40 minutes
  1. Get ready
  2. Do it
  3. Check yourself
  4. Show a grown-up

Get ready

You need:

  • Your results table from Lesson 3
  • Pencil and paper

Do it, step by step

Real AI test (with a grown-up): Which mistakes did it make?

Optional, and a grown-up runs it. Under 13, the grown-up types and reads, and you direct and check. Read the AI lab guide first. About 30 minutes.

Why we do this. You counted false alarms and misses. Now see if a real AI can find its own mistakes.

Prompt 1: Least sure

Look at your answers again. Which one are you least sure about, and why? Answer in two sentences.

Change only made-up parts in [brackets]. Never add a name, school, address, photo or anything private.

What to look for

  • Work out the accuracy from your last test.
  • Mark each mistake as a false alarm or a miss. Pick which group is “the thing you are looking for”.
  • Did the AI pick the answer it actually got wrong? Check.

Write it down

Tool, prompt, what it said, what you checked, and whether it was right, wrong or not sure. Record sheet

Think about it. Can you trust a machine to tell you when it is wrong?

Check yourself

Smoke alarm: how many false alarms, misses, and what accuracy?

Open the answer

2 false alarms, 1 miss. Right answers: 7 + 10 = 17 out of 20, so the accuracy is 85%.

Which mistake is worse for a smoke alarm?

Open the answer

A miss. A false alarm is annoying, but a miss could be dangerous.

Show a grown-up

Ask a grown-up which mistake would be worse for a machine that checks if a bridge is safe.

Next: Lesson 5

Lesson 5: Real, Fake or Made-up?About 45 minutes
  1. Get ready
  2. Do it
  3. Check yourself
  4. Show a grown-up

Get ready

You need:

  • Paper
  • Pencil

Do it, step by step

Try these

Case 1: The chain message. A message says: “BREAKING!!! School is closed forever tomorrow! Share this with 10 friends or you will have bad luck!” It does not say who sent it.

Open the answer

Do not trust or share it. It has no source, it pushes you to share quickly, and it uses scare words. Check the school’s own website or ask a teacher.

Case 2: The amazing picture. A picture shows a cat riding a skateboard past the Eiffel Tower. The fur looks perfect. The account that posted it has only this one picture and no other information.

Open the answer

It might be made or edited by a computer, but you cannot be sure from looking. The safe move is to say: “I don’t know if this is real.” Ask who posted it, look for other places it appears, and ask an adult. Not knowing is a fine answer.

Case 3: The museum website. A museum website says it opens at 10 o’clock and gives a phone number. The city’s tourism page says the same opening time.

Open the answer

Probably reliable. It comes from the museum itself, and a second source agrees. Check the web address looks right too.

Case 4: The voice message. A voice message sounds exactly like your favourite singer. It says: “Send me your password and I will give you free tickets!”

Open the answer

Never send a password. Computers can copy voices, and real people do not ask for passwords. Do not reply. Tell a grown-up.

Real AI test (with a grown-up): Invent a thing and see

Optional, and a grown-up runs it. Under 13, the grown-up types and reads, and you direct and check. Read the AI lab guide first. About 30 minutes.

Why we do this. Computers can make convincing things up. See what happens when you ask about something that does not exist.

Prompt 1: The invented book

Tell me about the children’s book called “[MAKE UP SILLY WORDS, E.g. THE WOBBLY PINEAPPLE OF ZINDOO]” by [MADE-UP AUTHOR]. What happens in it?

Prompt 2: Is it real?

Is that a real book? How sure are you?

Prompt 3: Check

Give me a website where I can check this.

Change only made-up parts in [brackets]. Never add a name, school, address, photo or anything private.

What to look for

  • Did it say it could not find the book, or did it describe what happens?
  • Ask a grown-up to check any website it gives. Does the page really exist and say that?
  • Both kinds of answer are useful. Write down which one you got.

Write it down

Tool, prompt, what it said, what you checked, and whether it was right, wrong or not sure. Record sheet

Think about it. What are your three rules for checking before you share?

Check yourself

What three questions do detectives ask?

Open the answer

Who made it? Why? Where else can I check?

Show a grown-up

Tell a grown-up about Case 2 and ask what they would do to check a picture.

Next: Lesson 6

Lesson 6: Using AI WiselyAbout 50 minutes
  1. Get ready
  2. Do it
  3. Check yourself
  4. Show a grown-up

Get ready

You need:

  • Everything from earlier in this course
  • Paper
  • A big sheet of paper for your one-pager

Do it, step by step

Real AI test (with a grown-up): Hints, not answers

Optional, and a grown-up runs it. Under 13, the grown-up types and reads, and you direct and check. Read the AI lab guide first. About 30 minutes.

Why we do this. Using AI wisely means it helps you learn, not just do the work. Test two ways of asking.

Prompt 1: Ask for hints

Here is a puzzle: I think of a number, double it, then add 6, and get 20. Give me hints to solve it, but do not give me the answer.

Prompt 2: Then ask for the answer

Now tell me the answer and explain how to get it.

Change only made-up parts in [brackets]. Never add a name, school, address, photo or anything private.

What to look for

  • Which way helped you learn more?
  • Did it respect “do not give me the answer”?
  • Write a one-line credit note: “I used an AI to help with…”

Write it down

Tool, prompt, what it said, what you checked, and whether it was right, wrong or not sure. Record sheet

Think about it. Add a rule about this to your “wise AI user” charter.

Check yourself

Where could your classifier fail?

Open the answer

With drawings unlike its training cards, or with ones that could be in either group.

Why say when AI helped you?

Open the answer

Being honest about how you made your work gives credit and builds trust.

Show a grown-up

Share your one-pager and charter with a grown-up. Ask: “One thing it does well, one way it could fail.”