Teaching a taxi to find its way.
A tiny city is a surprisingly useful place to think about sequential decisions. Explore the state, the route, and the reward — one action at a time.
A decision at every turn.
Pick a starting state, run a policy, or take the wheel yourself.
Dashed outline: destination · Person: passenger · Solid lines: walls
13 actions in the planned route
Reference implementation created for this portfolio. The BFS planner searches the known map; it is not a trained Q-learning or Monte Carlo policy.
The problem
A taxi must reach a waiting passenger, pick them up, and drop them at one of four destinations. Walls constrain the route. Every ordinary action costs −1, an invalid pickup or drop-off costs −10, and successful delivery earns +20. The goal is to complete the task with as few costly actions as possible.
How it works
The state combines 25 taxi positions, 5 passenger locations (including inside the taxi), and 4 destinations: 500 possible states. Six discrete actions update the environment deterministically. The browser runs this transition model locally and records every action, state, and reward in the episode.
Implementation & ownership
This portfolio includes a new TypeScript reconstruction of the environment and a breadth-first-search reference planner. Gymnasium defines the environment; no Gymnasium code, original coursework, trained policy, or original evaluation result is bundled. The design brief identifies Q-learning and Monte Carlo as the original project’s methods; their implementation and personal contribution remain to be verified from source.
Decisions & tradeoffs
A known-map planner gives the visitor a deterministic, understandable baseline. It finds a shortest successful route without training, which makes it useful for checking transitions. It cannot stand in for a learned policy or demonstrate performance under unknown dynamics. The distance overlay is Manhattan distance, not a learned value estimate.
Evaluation & next steps
The reference transition model is checked for state encoding, walls, rewards, and successful routes across the state space. Before comparing the original learning methods, correct the Monte Carlo implementation and progress counter, export the real policies, and evaluate every method under identical starting conditions and multiple random seeds. Report episode count, success rate, returns, and variance with the exports.
Sources & reproduction
The browser reference needs no Python server: open this page, choose a starting scenario, and run or step through the planner. Random actions are deterministic for a given scenario. Original notebooks and trained-policy JSON files have not been supplied. The links below document the environment and search algorithm; the GitHub profile is not presented as this project’s repository.
The original notebooks, trained models, and evaluation artifacts were not included with the design brief. This page separates a working browser explainer from results that still need to be connected.
Finding patterns in genomic data.
Have somethingin mind?
For software development, applied AI, or a conversation about the work — my inbox is open.