Back to selected work
01 / Reinforcement learningInteractive lab

Teaching a taxi to find its way.

A tiny city is a surprisingly useful place to think about sequential decisions. Explore the state, the route, and the reward — one action at a time.

ORIGINEducational reinforcement-learning project
THIS VERSIONProject exploration · browser reference
TOPICSPython · Gymnasium · Q-learning
TAXI LAB / BROWSER SIMULATION

A decision at every turn.

Pick a starting state, run a policy, or take the wheel yourself.

Ready to gos = 243
Taxi at row 3, column 3. Passenger at R. Destination B.RGYB0 · 1 · 2 · 3 · 4

Dashed outline: destination · Person: passenger · Solid lines: walls

Actions00
Total reward0
State243
Passenger waiting · R→ B
Or take the wheel

13 actions in the planned route

Reference implementation created for this portfolio. The BFS planner searches the known map; it is not a trained Q-learning or Monte Carlo policy.

01

The problem

A taxi must reach a waiting passenger, pick them up, and drop them at one of four destinations. Walls constrain the route. Every ordinary action costs −1, an invalid pickup or drop-off costs −10, and successful delivery earns +20. The goal is to complete the task with as few costly actions as possible.

02

How it works

The state combines 25 taxi positions, 5 passenger locations (including inside the taxi), and 4 destinations: 500 possible states. Six discrete actions update the environment deterministically. The browser runs this transition model locally and records every action, state, and reward in the episode.

01Initial state
02Policy → action
03Transition
04State + reward
03

Implementation & ownership

This portfolio includes a new TypeScript reconstruction of the environment and a breadth-first-search reference planner. Gymnasium defines the environment; no Gymnasium code, original coursework, trained policy, or original evaluation result is bundled. The design brief identifies Q-learning and Monte Carlo as the original project’s methods; their implementation and personal contribution remain to be verified from source.

04

Decisions & tradeoffs

A known-map planner gives the visitor a deterministic, understandable baseline. It finds a shortest successful route without training, which makes it useful for checking transitions. It cannot stand in for a learned policy or demonstrate performance under unknown dynamics. The distance overlay is Manhattan distance, not a learned value estimate.

05

Evaluation & next steps

The reference transition model is checked for state encoding, walls, rewards, and successful routes across the state space. Before comparing the original learning methods, correct the Monte Carlo implementation and progress counter, export the real policies, and evaluate every method under identical starting conditions and multiple random seeds. Report episode count, success rate, returns, and variance with the exports.

06

Sources & reproduction

The browser reference needs no Python server: open this page, choose a starting scenario, and run or step through the planner. Random actions are deterministic for a given scenario. Original notebooks and trained-policy JSON files have not been supplied. The links below document the environment and search algorithm; the GitHub profile is not presented as this project’s repository.

ABOUT THIS VERSION

The original notebooks, trained models, and evaluation artifacts were not included with the design brief. This page separates a working browser explainer from results that still need to be connected.

Next project / 02

Finding patterns in genomic data.

LET’S CONNECT

Have somethingin mind?

For software development, applied AI, or a conversation about the work — my inbox is open.