{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Lesson 08: Sequence Models: RNN, GRU, and LSTM\n",
        "\n",
        "**CS230 inspiration:** Recurrent neural networks, sequence modeling, vanishing gradients, and gated units.\n",
        "\n",
        "**Demand forecasting focus:** Use a GRU to summarize demand history and produce a direct multi-step forecast.\n",
        "\n",
        "**Learning objectives**\n",
        "\n",
        "- Understand why recurrent state can represent ordered demand history.\n",
        "- Compare sequence models with convolutional sequence baselines.\n",
        "- Handle direct multi-horizon outputs.\n",
        "\n",
        "**Senior-readiness checkpoint:** You can articulate the tradeoffs between RNN-style recurrence and parallelizable alternatives.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 1 Explanation\n",
        "\n",
        "**What this cell does:** This setup cell imports the shared scientific Python stack and the curriculum helpers used by later cells. It also establishes the deterministic seed convention so repeated runs are comparable.\n",
        "\n",
        "**Why it matters:** A restart-and-run notebook should make its dependencies explicit before any modeling work begins. For forecasting, reproducibility is especially important because small data or seed changes can change validation conclusions.\n",
        "\n",
        "**Key operations to notice:** confirm that shared helper imports match the lesson's code path; check that `SEED` is defined before any generated data or model training\n",
        "\n",
        "**What to inspect:** Check that imports are grouped by purpose: general analysis packages, forecasting data helpers, metrics, plotting utilities, and any lesson-specific libraries.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "from demand_forecasting_curriculum.data import (\n",
        "    DEFAULT_FEATURE_COLUMNS,\n",
        "    generate_daily_demand,\n",
        "    make_sequence_dataset,\n",
        "    make_supervised_table,\n",
        "    time_series_split,\n",
        ")\n",
        "from demand_forecasting_curriculum.metrics import (\n",
        "    interval_coverage,\n",
        "    mase,\n",
        "    pinball_loss,\n",
        "    smape,\n",
        "    wmape,\n",
        ")\n",
        "from demand_forecasting_curriculum.plotting import plot_predictions, plot_series\n",
        "\n",
        "SEED = 42\n",
        "np.random.seed(SEED)\n",
        "\n",
        "import torch\n",
        "from torch import nn\n",
        "from torch.utils.data import DataLoader, TensorDataset\n",
        "from demand_forecasting_curriculum.modeling import set_seed\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 2 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Sequence Models: RNN, GRU, and LSTM** by working on synthetic demand generation, sequence-window construction. It creates or updates `df`, `X`, `y`, `meta`, `target_start`, `train_idx` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a GRU to summarize demand history and produce a direct multi-step forecast. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** creates a synthetic retail panel with known demand drivers; builds lookback windows for sequence models; batches examples for neural-network training; packages tensors into a PyTorch dataset\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "set_seed(SEED)\n",
        "df = generate_daily_demand(n_stores=4, n_items=6, periods=520, seed=SEED)\n",
        "X, y, meta = make_sequence_dataset(df, lookback=56, horizon=14)\n",
        "target_start = pd.to_datetime(meta[\"target_start\"])\n",
        "train_idx = target_start < target_start.quantile(0.70)\n",
        "valid_idx = (target_start >= target_start.quantile(0.70)) & (target_start < target_start.quantile(0.85))\n",
        "\n",
        "x_mean = X[train_idx].mean(axis=(0, 1), keepdims=True)\n",
        "x_std = np.where(X[train_idx].std(axis=(0, 1), keepdims=True) == 0, 1, X[train_idx].std(axis=(0, 1), keepdims=True))\n",
        "X_scaled = (X - x_mean) / x_std\n",
        "\n",
        "train_loader = DataLoader(\n",
        "    TensorDataset(torch.tensor(X_scaled[train_idx]).float(), torch.tensor(np.log1p(y[train_idx])).float()),\n",
        "    batch_size=256,\n",
        "    shuffle=True,\n",
        ")\n",
        "valid_loader = DataLoader(\n",
        "    TensorDataset(torch.tensor(X_scaled[valid_idx]).float(), torch.tensor(np.log1p(y[valid_idx])).float()),\n",
        "    batch_size=512,\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 3 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Sequence Models: RNN, GRU, and LSTM** by working on model definition. It creates or updates `model` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a GRU to summarize demand history and produce a direct multi-step forecast. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** summarizes ordered history with recurrent state\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "class GRUForecaster(nn.Module):\n",
        "    def __init__(self, n_features, hidden_size, horizon):\n",
        "        super().__init__()\n",
        "        self.gru = nn.GRU(input_size=n_features, hidden_size=hidden_size, batch_first=True)\n",
        "        self.head = nn.Sequential(nn.LayerNorm(hidden_size), nn.Linear(hidden_size, horizon))\n",
        "\n",
        "    def forward(self, x):\n",
        "        _, hidden = self.gru(x)\n",
        "        return self.head(hidden[-1])\n",
        "\n",
        "model = GRUForecaster(n_features=X.shape[-1], hidden_size=64, horizon=y.shape[-1])\n",
        "model(next(iter(train_loader))[0]).shape\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 4 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Sequence Models: RNN, GRU, and LSTM** by working on forecast metrics, model training. It creates or updates `optimizer`, `loss_fn`, `pred`, `actual` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a GRU to summarize demand history and produce a direct multi-step forecast. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** evaluates scale-weighted forecast error; updates model weights after backpropagation; computes gradients for model parameters; runs inference without tracking gradients\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-4)\n",
        "loss_fn = nn.L1Loss()\n",
        "\n",
        "for epoch in range(4):\n",
        "    model.train()\n",
        "    for xb, yb in train_loader:\n",
        "        optimizer.zero_grad(set_to_none=True)\n",
        "        loss = loss_fn(model(xb), yb)\n",
        "        loss.backward()\n",
        "        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n",
        "        optimizer.step()\n",
        "\n",
        "model.eval()\n",
        "with torch.no_grad():\n",
        "    preds, actuals = [], []\n",
        "    for xb, yb in valid_loader:\n",
        "        preds.append(model(xb).numpy())\n",
        "        actuals.append(yb.numpy())\n",
        "\n",
        "pred = np.expm1(np.vstack(preds))\n",
        "actual = np.expm1(np.vstack(actuals))\n",
        "{\"valid_wmape_all_horizons\": wmape(actual.ravel(), pred.ravel())}\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Lesson Recap\n",
        "\n",
        "In this lesson, we used 4 runnable code cells to work through **Sequence Models: RNN, GRU, and LSTM**. The practical focus was: Use a GRU to summarize demand history and produce a direct multi-step forecast.\n",
        "\n",
        "We did this because deep learning for demand forecasting is not only about choosing an architecture. The learner needs to understand the data contract, the target and horizon, the validation path, the metric, and the operational decision supported by the forecast.\n",
        "\n",
        "The core takeaways were:\n",
        "\n",
        "- Understand why recurrent state can represent ordered demand history.\n",
        "- Compare sequence models with convolutional sequence baselines.\n",
        "- Handle direct multi-horizon outputs.\n",
        "\n",
        "By the end, you should be able to explain not just what the code produced, but why each step was necessary and what could go wrong if the same step were skipped or performed with leakage.\n",
        "\n",
        "**Senior-readiness reminder:** You can articulate the tradeoffs between RNN-style recurrence and parallelizable alternatives.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Exercises\n",
        "\n",
        "- Replace GRU with LSTM and compare validation score and training speed.\n",
        "- Plot WMAPE by horizon to see whether longer horizons degrade smoothly.\n",
        "- Explain why gradient clipping is common in recurrent training loops.\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python (deep-learning-demand)",
      "language": "python",
      "name": "deep-learning-demand"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}