← Back to Data Exploration with rakoon-ds
Practical Work 7

Feature engineering: turn 0.49 into 0.89 without changing the algorithm

Calendar features, cyclical encodings with sine and cosine, binning, named categories, ratios by formula, and permutation importance to see which of them actually paid

Duration 75 min
Level Intermediate
Session Session 7
Mission dep-7

Objectives

By the end of this session you will be able to:

  • Establish a floor with the raw columns only, before engineering anything
  • Extract calendar features in one click and measure the gain
  • Encode a cyclical variable with sine and cosine, and say why an integer month lies
  • Turn a number into bins or into named categories, and choose between the two
  • Build a ratio or an interaction with a formula
  • Use permutation importance to keep the features that paid and drop the rest

Prerequisites

  • A rakoon-ds account on https://rakoon-ds.apps.way-up.io (free, browser only, nothing to install), and the group join code your instructor gives you
  • A recent Chrome, Edge or Firefox. Computation runs either in your own tab (browser engine) or on the server; the header pill tells you which
  • The mission for this session, assigned to your group. Open a project, then the Mission button in the workshop header: the panel opens next to Report. Click Check after each step
  • Practical works 1, 2 and 6 finished
  • Transformation, chart and algorithm names come from the rakoon-ds registry and are served in French even when the interface is in English. Every step below gives you the French label you will click and, in code font, the registry key the mission checks against.

Data

One built-in dataset, chosen because every feature you can build on it is a real idea about the world.

DatasetRows x colsTargetOne row is
bike_sharing (Vélos en libre-service, par jour)731 x 15total (bikes rented that day)one day, from 2011-01-01 to 2012-12-31

Columns: date, saison, annee, mois, jour_ferie, jour_semaine, jour_ouvre, meteo, temperature, temperature_ressentie, humidite, vent, occasionnels, abonnes, total.

occasionnels and abonnes sum to total. Drop them in Step 1 and never speak of them again. If you saw Practical work 6 you already know the studio will not warn you.

If your instructor gives you the air-passengers exam pack (train.csv, external_data.csv, holidays.xml, an IATA airport table), run the same six steps on it. Reference figures from the exam trial: R² 0.281 on the six raw columns, 0.619 after the date features, 0.718 after the geographic distance, 0.884 with LightGBM on the final set.

Timing

The steps below add up to the announced duration. If you fall behind, Step 1 to Step 3 are the ones that must be finished.

#What you doTime
Step 1The floor10 min
Step 2The calendar, in one click12 min
Step 3Cyclical encoding, and why an integer month lies15 min
Step 4Bins and named categories12 min
Step 5Ratios and interactions13 min
Step 6Which of them actually paid?13 min
Total75 min

Instructions

Step 1: The floor10 min

  1. New project PW7 Feature engineering, add Vélos en libre-service (par jour).
  2. Apply Supprimer des colonnes (drop_columns) on occasionnels and abonnes. Name it velos-honnete.
  3. Train Modèle de référence (médiane) (dummy_reg) on total. Experiment label features.
  4. Train Forêt aléatoire (rf_reg) on total with only four columns: temperature, humidite, vent, meteo.
ModelFeaturesR²RMSE
Baseline (median)all-0.02451 987.0
Random forest4 weather columns0.49331 397.4

That is your floor. Every number for the rest of the session is compared to 0.4933, not to zero.

Step 2: The calendar, in one click12 min

  1. On velos-honnete: family Dates, card Convertir en date (to_date) on date.
  2. Then Décomposer une date/heure (date_parts) on date. The available components are year, month, day, weekday, hour, minute, quarter, dayofyear, week, is_weekend. Take month, weekday, dayofyear, week, is_weekend.
  3. Name it velos-calendrier.
  4. Retrain the same forest, adding the new columns plus saison, annee, jour_ferie, jour_ouvre, temperature_ressentie.

Reference: R² 0.8889, RMSE 654.3. Same algorithm, same default settings, same 731 rows. Nearly forty points of R² came from columns that were already in the file and that nobody had extracted.

In Orange this step is a Python Script widget, and it is worth two points on the EPITA marking scheme precisely because it used to require code. Here it is a checkbox list. When a tool makes an exercise trivial, the exercise has to move: what is worth marking now is which components you took, and why.

Step 3: Cyclical encoding, and why an integer month lies15 min

December is month 12 and January is month 1. For any model that reads a number as a quantity, they are eleven apart. In reality they are neighbours.

  1. Family Colonnes, card Colonne par formule (expression).
  2. New column mois_sin, formula: sin(2 * 3.141592653589793 * date_month / 12).
  3. Again, new column mois_cos, formula with cos.
  4. Same treatment for the weekday: jour_sin and jour_cos over 7 instead of 12.
  5. Name the result velos-cyclique and retrain.

Nothing in the formula field advertises that sin and cos exist. They do (checked on the instance on 2026-09-09: sin(2 * 3.141592653589793 * mois / 12) returns 0.5 for January). Remember it, because the placeholder only shows a division.

On a tree-based model the gain is usually small, because a tree can already split "month > 10 or month < 3". Try it anyway, then try the same pair of columns on a Régression linéaire (linreg): that is where the encoding earns its keep. Write down both.

Step 4: Bins and named categories12 min

Two different tools for two different intentions.

CardKeyWhat it doesUse it when
Discrétiser (bins) (bin_numeric)bin_numericcuts into N buckets, equal width or quantileyou do not know where the boundaries should be
Catégoriser (seuils) (categorize)categorizecuts at thresholds you give and names the bucketsthe boundaries mean something (18 and 65 for an age)
  1. Bin temperature into 4 quantile buckets.
  2. Categorise humidite at thresholds 40, 70 with labels dry, normal, humid.
  3. Retrain and compare. Did discretising a number the forest already had help, hurt, or do nothing?

Expect "nothing much" on a tree, and say so in the report. Binning helps a linear model and a human reader far more than it helps a forest. A transformation that changes no score can still be the right one if it makes the result explainable.

Step 5: Ratios and interactions13 min

The two features that carry a real idea, rather than a reformatting.

  1. ressenti_ecart = temperature_ressentie - temperature, with Colonne calculée (simple) (derive_column) (left column, operator, right column or number) or with Colonne par formule (expression).
  2. confort = temperature * (1 - humidite / 100), with expression: an interaction, not a reformatting.
  3. Retrain with both added.

Backtick a column name in a formula when it contains a space: `ma colonne` + 1. The placeholder in the field shows the syntax.

The Assistant button next to the formula field asks a language model to write the formula from a description. It needs a Gemini key on the instance; without one, the field says how to enable it and you type the formula yourself, which takes ten seconds.

Step 6: Which of them actually paid?13 min

  1. Open the results of your best model, scroll to the Explanations tab.
  2. Read Permutation importance: each variable is shuffled at random and the chart shows what the score loses. The bar is the loss, the whisker is the spread over five shuffles.
  3. Compare it with Model's own importance just beside it: they disagree, and the disagreement is informative. The native one is computed on the prepared columns (so one bar per one-hot category), the permutation one on your original variables.
  4. Drop every feature whose permutation importance is at or below zero, with Supprimer des colonnes (drop_columns), and retrain.

A feature whose permutation importance is negative actively costs you: shuffling it improves the score. Removing it is not tidying, it is a result.

Finish the Préparation section of your report with one line per feature: what you built, why, and the R² before and after. That table is the deliverable.

What you should have

  • A floor at R² 0.4933 and a ceiling above 0.88, with the same algorithm throughout
  • Cyclical encodings built with sine and cosine, tested on a tree and on a linear model
  • One binning and one named categorisation, with an honest verdict on each
  • Two features that carry an idea: a difference and an interaction
  • A permutation importance chart used to remove features
  • The dep-7 mission at 6 / 6

Deliverables

  • Mission: dep-7 validated
  • Report: Préparation section with one line per feature (name, idea, R² before, R² after)
  • The importance chart poured into the report with its caption

Bonus

  • Build a lagged feature (yesterday's rentals) with the Python node (python). It is the one thing on this dataset that has no click-only equivalent, and it is the strongest feature of all. Then think about what it does to a random split.
  • Run the same feature set through Régression linéaire (linreg), Arbre de décision (tree_reg) and Gradient Boosting (gb_reg). The ranking of your features changes with the algorithm: explain why.
  • Use Partial dependence in the Explanations tab on temperature. Where does the curve turn around, and does that match what you know about cycling?

Resources