{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lesson 11: Global Models and Entity Embeddings\n", "\n", "**CS230 inspiration:** Deep networks, representation learning, and transfer across examples.\n", "\n", "**Demand forecasting focus:** Learn store and item embeddings so one global model can share signal across many series.\n", "\n", "**Learning objectives**\n", "\n", "- Distinguish local models from global multi-series models.\n", "- Use embeddings for categorical store and item identities.\n", "- Evaluate cold-ish and sparse-ish segment behavior.\n", "\n", "**Senior-readiness checkpoint:** You can decide when a global model helps and when it hides segment-specific failures.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Code Cell 1 Explanation\n", "\n", "**What this cell does:** This setup cell imports the shared scientific Python stack and the curriculum helpers used by later cells. It also establishes the deterministic seed convention so repeated runs are comparable.\n", "\n", "**Why it matters:** A restart-and-run notebook should make its dependencies explicit before any modeling work begins. For forecasting, reproducibility is especially important because small data or seed changes can change validation conclusions.\n", "\n", "**Key operations to notice:** confirm that shared helper imports match the lesson's code path; check that `SEED` is defined before any generated data or model training\n", "\n", "**What to inspect:** Check that imports are grouped by purpose: general analysis packages, forecasting data helpers, metrics, plotting utilities, and any lesson-specific libraries.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "\n", "from demand_forecasting_curriculum.data import (\n", " DEFAULT_FEATURE_COLUMNS,\n", " generate_daily_demand,\n", " make_sequence_dataset,\n", " make_supervised_table,\n", " time_series_split,\n", ")\n", "from demand_forecasting_curriculum.metrics import (\n", " interval_coverage,\n", " mase,\n", " pinball_loss,\n", " smape,\n", " wmape,\n", ")\n", "from demand_forecasting_curriculum.plotting import plot_predictions, plot_series\n", "\n", "SEED = 42\n", "np.random.seed(SEED)\n", "\n", "import torch\n", "from torch import nn\n", "from torch.utils.data import DataLoader, TensorDataset\n", "from demand_forecasting_curriculum.modeling import Standardizer, set_seed\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Code Cell 2 Explanation\n", "\n", "**What this cell does:** This cell advances **Global Models and Entity Embeddings** by working on synthetic demand generation, time-based splitting, supervised feature construction. It creates or updates `df`, `supervised`, `train`, `valid`, `_`, `numeric_cols` and prepares evidence that later cells use rather than hiding state in prose.\n", "\n", "**Why it matters:** The goal is tied to this lesson's forecasting focus: Learn store and item embeddings so one global model can share signal across many series. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n", "\n", "**Key operations to notice:** creates a synthetic retail panel with known demand drivers; turns time series rows into forecast-origin training examples; keeps train, validation, and test windows ordered by date; fits scaling parameters on training data only; batches examples for neural-network training; packages tensors into a PyTorch dataset\n", "\n", "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "set_seed(SEED)\n", "df = generate_daily_demand(n_stores=8, n_items=12, periods=500, seed=SEED)\n", "supervised = make_supervised_table(df, horizon=7)\n", "train, valid, _ = time_series_split(supervised, valid_days=60, test_days=60)\n", "\n", "numeric_cols = DEFAULT_FEATURE_COLUMNS\n", "scaler = Standardizer.fit(train[numeric_cols].to_numpy(dtype=np.float32))\n", "X_train_num = scaler.transform(train[numeric_cols].to_numpy(dtype=np.float32)).astype(np.float32)\n", "X_valid_num = scaler.transform(valid[numeric_cols].to_numpy(dtype=np.float32)).astype(np.float32)\n", "\n", "train_ds = TensorDataset(\n", " torch.tensor(train[\"store_id\"].to_numpy()).long(),\n", " torch.tensor(train[\"item_id\"].to_numpy()).long(),\n", " torch.tensor(X_train_num).float(),\n", " torch.tensor(np.log1p(train[\"y\"].to_numpy(dtype=np.float32))).float(),\n", ")\n", "valid_tensors = (\n", " torch.tensor(valid[\"store_id\"].to_numpy()).long(),\n", " torch.tensor(valid[\"item_id\"].to_numpy()).long(),\n", " torch.tensor(X_valid_num).float(),\n", ")\n", "train_loader = DataLoader(train_ds, batch_size=512, shuffle=True)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Code Cell 3 Explanation\n", "\n", "**What this cell does:** This cell advances **Global Models and Entity Embeddings** by working on model definition. It creates or updates `model` and prepares evidence that later cells use rather than hiding state in prose.\n", "\n", "**Why it matters:** The goal is tied to this lesson's forecasting focus: Learn store and item embeddings so one global model can share signal across many series. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n", "\n", "**Key operations to notice:** follow the assignments, final displayed object, and any checks in the cell because those are the evidence the next cell depends on\n", "\n", "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class EmbeddedDemandModel(nn.Module):\n", " def __init__(self, n_stores, n_items, n_numeric):\n", " super().__init__()\n", " self.store_embedding = nn.Embedding(n_stores, 4)\n", " self.item_embedding = nn.Embedding(n_items, 6)\n", " self.net = nn.Sequential(\n", " nn.Linear(n_numeric + 10, 96),\n", " nn.ReLU(),\n", " nn.Linear(96, 48),\n", " nn.ReLU(),\n", " nn.Linear(48, 1),\n", " )\n", "\n", " def forward(self, store_id, item_id, numeric):\n", " x = torch.cat(\n", " [self.store_embedding(store_id), self.item_embedding(item_id), numeric],\n", " dim=1,\n", " )\n", " return self.net(x).squeeze(-1)\n", "\n", "model = EmbeddedDemandModel(\n", " n_stores=int(df[\"store_id\"].max()) + 1,\n", " n_items=int(df[\"item_id\"].max()) + 1,\n", " n_numeric=len(numeric_cols),\n", ")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Code Cell 4 Explanation\n", "\n", "**What this cell does:** This cell advances **Global Models and Entity Embeddings** by working on forecast metrics, model training. It creates or updates `optimizer`, `loss_fn`, `scored` and prepares evidence that later cells use rather than hiding state in prose.\n", "\n", "**Why it matters:** The goal is tied to this lesson's forecasting focus: Learn store and item embeddings so one global model can share signal across many series. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n", "\n", "**Key operations to notice:** evaluates scale-weighted forecast error; updates model weights after backpropagation; computes gradients for model parameters; runs inference without tracking gradients\n", "\n", "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-4)\n", "loss_fn = nn.L1Loss()\n", "\n", "for epoch in range(6):\n", " model.train()\n", " for store_id, item_id, numeric, target in train_loader:\n", " optimizer.zero_grad(set_to_none=True)\n", " loss = loss_fn(model(store_id, item_id, numeric), target)\n", " loss.backward()\n", " optimizer.step()\n", "\n", "model.eval()\n", "with torch.no_grad():\n", " pred = np.expm1(model(*valid_tensors).numpy())\n", "\n", "scored = valid.assign(prediction=pred)\n", "wmape(scored[\"y\"], scored[\"prediction\"])\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Code Cell 5 Explanation\n", "\n", "**What this cell does:** This cell advances **Global Models and Entity Embeddings** by working on forecast metrics. It creates or updates `segment_scores` and prepares evidence that later cells use rather than hiding state in prose.\n", "\n", "**Why it matters:** The goal is tied to this lesson's forecasting focus: Learn store and item embeddings so one global model can share signal across many series. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n", "\n", "**Key operations to notice:** evaluates scale-weighted forecast error\n", "\n", "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "segment_scores = (\n", " scored.groupby(\"item_id\")\n", " .apply(lambda part: wmape(part[\"y\"], part[\"prediction\"]), include_groups=False)\n", " .reset_index(name=\"wmape\")\n", " .sort_values(\"wmape\", ascending=False)\n", ")\n", "segment_scores.head()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Lesson Recap\n", "\n", "In this lesson, we used 5 runnable code cells to work through **Global Models and Entity Embeddings**. The practical focus was: Learn store and item embeddings so one global model can share signal across many series.\n", "\n", "We did this because deep learning for demand forecasting is not only about choosing an architecture. The learner needs to understand the data contract, the target and horizon, the validation path, the metric, and the operational decision supported by the forecast.\n", "\n", "The core takeaways were:\n", "\n", "- Distinguish local models from global multi-series models.\n", "- Use embeddings for categorical store and item identities.\n", "- Evaluate cold-ish and sparse-ish segment behavior.\n", "\n", "By the end, you should be able to explain not just what the code produced, but why each step was necessary and what could go wrong if the same step were skipped or performed with leakage.\n", "\n", "**Senior-readiness reminder:** You can decide when a global model helps and when it hides segment-specific failures.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercises\n", "\n", "- Increase item embedding size and see whether validation WMAPE moves.\n", "- Hold out one item from training and discuss what the embedding model cannot know.\n", "- Plot learned item embeddings with PCA or t-SNE if you add a visualization dependency.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python (deep-learning-demand)", "language": "python", "name": "deep-learning-demand" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12" } }, "nbformat": 4, "nbformat_minor": 5 }