← Back to Data Exploration with rakoon-ds
Practical Work 11

Understanding a model: importance, dependence, simulation, minimal change

A score says how often. It never says why, or on whom. Four tools that do, on a dataset whose answer you already know

Duration 75 min
Level Intermediate
Session Session 11
Mission dep-11

Objectives

By the end of this session you will be able to:

  • Tell permutation importance and native importance apart, and know when they disagree
  • Read a partial dependence curve without over-reading it
  • Use the what-if simulator to interrogate a single individual
  • Find the smallest change that flips a prediction, and judge whether it makes sense
  • Write an explanation a non-specialist can act on

Prerequisites

  • A rakoon-ds account on https://rakoon-ds.apps.way-up.io (free, browser only, nothing to install), and the group join code your instructor gives you
  • A recent Chrome, Edge or Firefox. Computation runs either in your own tab (browser engine) or on the server; the header pill tells you which
  • The mission for this session, assigned to your group. Open a project, then the Mission button in the workshop header: the panel opens next to Report. Click Check after each step
  • Practical work 1 finished: you know the Titanic and its deliberate leak
  • Transformation, chart and algorithm names come from the rakoon-ds registry and are served in French even when the interface is in English. Every step below gives you the French label you will click and, in code font, the registry key the mission checks against.

Data

The built-in titanic dataset, 1 309 passengers, target survivant. You use it precisely because you already know the answer: when the tool tells you something surprising, you can tell whether it is an insight or a bug.

Timing

The steps below add up to the announced duration. If you fall behind, Step 1 to Step 3 are the ones that must be finished.

#What you doTime
Step 1Two models, one of them dishonest12 min
Step 2Importance: two charts that disagree on purpose15 min
Step 3Partial dependence, and how not to over-read it12 min
Step 4The simulator: one individual, one slider13 min
Step 5The smallest change that flips the answer13 min
Step 6Write the explanation10 min
Total75 min

Instructions

Step 1: Two models, one of them dishonest12 min

  1. New project PW11 Explaining, add Titanic (passagers).
  2. Train a Forêt aléatoire (rf_clf) on survivant with every column. Experiment label explique.
  3. Train a second one on the seven honest columns only: classe, sexe, age, freres_conjoints, parents_enfants, prix_billet, port_embarquement.
ModelAccuracyF1AUC
all columns (with canot)0.96650.96630.9952
seven honest columns0.82010.81870.8756

Open the first model and find the list of automatically disabled columns. It should name nom and cabine: near-unique text columns are excluded and the studio says so. That list is the first thing to read on any result screen.

Step 2: Importance: two charts that disagree on purpose15 min

Open the results of the first model, scroll to the Explanations tab.

  1. Permutation importance: each variable is shuffled and the chart shows what the score loses. Reference figures, baseline accuracy 0.9665 on 328 test individuals:
VariableScore loss
canot0.4317
classe, sexe, freres_conjoints, parents_enfants, prix_billet, port_embarquement0.0000
age-0.0006

One variable carries everything and the other seven carry nothing, including sexe, which is the single best predictor of survival on this ship. That is not a paradox: once the model knows the lifeboat, sex adds no information. Permutation importance measures what a variable adds given the others, not what it is worth alone.

  1. Now open the same tab on the honest model:
VariableScore loss
sexe0.2165
classe0.0701
age0.0348
prix_billet0.0146
freres_conjoints0.0110
port_embarquement0.0091
parents_enfants-0.0134

Compare with Model's own importance, shown next to it. The native chart is computed inside the model on the prepared columns, so a categorical variable appears once per category (canot_None, sexe_homme, sexe_femme...). Two charts, two questions: "what did the model build itself on" against "what does the score lose without it".

Step 3: Partial dependence, and how not to over-read it12 min

  1. Still in Explanations, on the honest model: Partial dependence, variable age.
  2. The curve shows the average predicted probability as age varies, the other variables staying as they are, over a grid of 20 points from 0.17 to 80.
  3. Do the same for prix_billet and for classe.

Three questions to answer in writing:

  • Where does the age curve fall, and does it match the "women and children first" story?
  • The fare curve rises. Is that about money, or about the deck the cabin was on?
  • The curve is an average over everybody. Name one passenger for whom it would be wrong.

A partial dependence curve is not a causal claim. It says "when this number changes and nothing else does, the average prediction moves like this". On this ship, age and class are not independent, so "nothing else changes" describes passengers who never existed.

Step 4: The simulator: one individual, one slider13 min

  1. Click the honest model, then the Predict tab. It has two sub-tabs: Simulator and File.
  2. The simulator builds one input form per variable, bounded by what the training data contained: classe 1 to 3, sexe from femme / homme, age 0.17 to 80, freres_conjoints 0 to 8, parents_enfants 0 to 9, prix_billet 0 to 512.33, port_embarquement from Cherbourg / Queenstown / Southampton.
  3. Set up a third-class man, 30 years old, travelling alone, fare 8.05, boarding at Southampton. Read the prediction and the probabilities.
  4. Now drag the age slider from 30 down to 5. Watch the prediction and the curve beside it change live.
  5. Put the age back, then drag prix_billet up.

The curve next to the sliders is the partial dependence for this individual, not the average one. That is the difference between "what does the model do in general" and "what does it do to this person", and it is the difference that matters when the person is in front of you.

Step 5: The smallest change that flips the answer13 min

  1. With that same third-class man loaded, click Quel changement fait basculer ? (what change flips it?).
  2. The studio sweeps 40 values of each numeric variable and reports the smallest single change that switches the predicted class.

Reference answer from the instance, for that exact individual:

VariableFromToNew prediction
prix_billet8.0552.55oui
age3014.5oui
  1. Click Appliquer on the first one and watch the simulator move.
  2. Then judge it. Two questions, and they are not technical:
  • Is "pay 52 pounds instead of 8" a change this man could have made? What does the model actually mean by it?
  • If this were a loan model and the answer were "earn 44 000 more", would you send that sentence to the applicant?

The search moves one variable at a time, and only numeric ones. A real counterfactual usually needs several changes at once, and some changes are not available to the person (age, sex). A minimal change is a diagnostic tool, not advice.

Step 6: Write the explanation10 min

Pour into the Modélisation section: the permutation importance chart of both models, and the partial dependence curve.

Then write, in the Évaluation section, an explanation of one prediction for somebody who has never heard of a random forest. Constraints:

  • Under 150 words.
  • No jargon: no "feature", no "probability", no algorithm name.
  • Name the individual, the prediction, the two things that drove it, and one thing that would have changed it.
  • End with what you would not claim.

Then read the model card: results screen, Fiche tab. Its "limits" section is computed, not recited: test set size, rare classes, train / test gap, columns dropped, rows lost. Compare it with what you just wrote. If the card names a limit you did not, add it.

What you should have

  • Two models, 15 accuracy points apart, and the reason named
  • Two permutation importance charts, and an explanation of why they disagree
  • Three partial dependence curves read, and one over-reading avoided in writing
  • One individual simulated, one minimal change found and judged
  • A 150-word explanation with no jargon
  • The dep-11 mission at 5 / 5

Deliverables

  • Mission: dep-11 validated
  • Report: Modélisation and Évaluation sections, with the two importance charts
  • The 150-word explanation, read aloud to somebody from another course

Bonus

  • Train an Arbre de décision (tree_clf) instead and use Path through the tree: the tests your individual goes through, highlighted in the drawn tree. It is the only fully transparent explanation in the whole toolbox.
  • Train a k plus proches voisins (knn) and look at the neighbours table: the model is doing nothing but showing you the closest training rows and letting them vote. Compare that explanation with the forest's.
  • Train a Régression logistique (logreg) and open the equation view: coefficients and standardised effects. Which of the four explanation styles would you put in front of a regulator?

Resources