Back to Lab Notes
Lab Notes / Taxi-v3 reinforcement learning

A fair comparison needs a shared evaluation protocol

Saved outputs include 39 successes in 100 Monte Carlo episodes and 200 in 200 Q-learning episodes, under differing conditions.

Practical consequence

Compare methods with shared starts, limits, episode counts, and multiple seeds.

01

Question

Can the available Monte Carlo and Q-learning outputs support a reliable ranking of the methods?

02

Experiment

The saved Taxi-v3 outputs record successful episodes for Monte Carlo and Q-learning. Seeds and sometimes evaluation counts differ between methods.

03

Recorded results

Recorded runs with different seeds and episode counts; these ratios are not a controlled head-to-head evaluation.
MethodRecorded successes
Monte Carlo39 / 100 episodes
Q-learning200 / 200 episodes

Limits

The implementation labelled “first-visit Monte Carlo” requires correction before a new comparison. The saved counts alone do not establish general performance.

04

Interpretation

The outputs describe the available runs. Because the evaluation conditions differ, they do not rigorously rank the algorithms.

Practical consequence

Compare methods with shared starts, limits, episode counts, and multiple seeds.

05

Next experiment

Correct the first-visit Monte Carlo implementation, then evaluate both methods from the same initial states, with the same limits and episode counts, over multiple seeds.

06

Origin & contribution

Educational Taxi-v3 reinforcement-learning project. Counts come from an editorial synthesis of saved notebook outputs supplied for this update; the notebooks are not in this checkout.

The original learning code and precise personal contribution still need source review. The browser Taxi Lab is a separate reference reconstruction, not either trained policy.

ProjectTaxi-v3 reinforcement learningExplore the project