{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Lesson 13: Ralph Loops for Deep Learning Iteration\n",
        "\n",
        "**CS230 inspiration:** Hyperparameter tuning, ML strategy, error analysis, and project iteration under fast feedback.\n",
        "\n",
        "**Demand forecasting focus:** Use a Ralph-style loop to improve a demand forecaster one validated change at a time.\n",
        "\n",
        "**Learning objectives**\n",
        "\n",
        "- Translate Ralph loop principles into deep learning experiment workflows.\n",
        "- Create a deterministic context stack with specs, metrics, constraints, and a fix plan.\n",
        "- Run one model-improvement candidate per loop and apply fast validation backpressure.\n",
        "- Record learnings so future loops inherit decisions instead of rediscovering them.\n",
        "\n",
        "**Senior-readiness checkpoint:** You can use agentic iteration safely: fast loops, tight scope, reliable gates, and human judgment about when to stop.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 1 Explanation\n",
        "\n",
        "**What this cell does:** This setup cell imports the shared scientific Python stack and the curriculum helpers used by later cells. It also establishes the deterministic seed convention so repeated runs are comparable.\n",
        "\n",
        "**Why it matters:** A restart-and-run notebook should make its dependencies explicit before any modeling work begins. For forecasting, reproducibility is especially important because small data or seed changes can change validation conclusions.\n",
        "\n",
        "**Key operations to notice:** confirm that shared helper imports match the lesson's code path; check that `SEED` is defined before any generated data or model training\n",
        "\n",
        "**What to inspect:** Check that imports are grouped by purpose: general analysis packages, forecasting data helpers, metrics, plotting utilities, and any lesson-specific libraries.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "from demand_forecasting_curriculum.data import (\n",
        "    DEFAULT_FEATURE_COLUMNS,\n",
        "    generate_daily_demand,\n",
        "    make_sequence_dataset,\n",
        "    make_supervised_table,\n",
        "    time_series_split,\n",
        ")\n",
        "from demand_forecasting_curriculum.metrics import (\n",
        "    interval_coverage,\n",
        "    mase,\n",
        "    pinball_loss,\n",
        "    smape,\n",
        "    wmape,\n",
        ")\n",
        "from demand_forecasting_curriculum.plotting import plot_predictions, plot_series\n",
        "\n",
        "SEED = 42\n",
        "np.random.seed(SEED)\n",
        "\n",
        "import torch\n",
        "from demand_forecasting_curriculum.modeling import (\n",
        "    MLPRegressor,\n",
        "    Standardizer,\n",
        "    make_regression_loader,\n",
        "    predict_regression_model,\n",
        "    set_seed,\n",
        "    train_regression_model,\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Article Review\n",
        "\n",
        "Geoffrey Huntley's Ralph article describes a monolithic agent loop that repeatedly chooses one important task, works from a deterministic context stack, runs fast backpressure, records learnings, and gets retuned when it goes off track: https://ghuntley.com/ralph/\n",
        "\n",
        "For deep learning, the point is not to let an agent mutate experiments without supervision. The useful pattern is a disciplined improvement loop: one candidate change, one validation path, one updated experiment log, and one human-readable learning per turn.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 2 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Ralph Loops for Deep Learning Iteration** by working on Ralph-style iteration. It creates or updates `ralph_principles` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a Ralph-style loop to improve a demand forecaster one validated change at a time. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** creates an inspectable table for assumptions, metrics, or decisions\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "ralph_principles = pd.DataFrame(\n",
        "    [\n",
        "        (\"one item per loop\", \"try one modeling change before combining ideas\"),\n",
        "        (\"deterministic context\", \"load the same spec, metric, constraints, and current plan each loop\"),\n",
        "        (\"fast backpressure\", \"run cheap training, smoke tests, leakage checks, and validation metrics\"),\n",
        "        (\"do not assume missing work\", \"search existing experiment logs before proposing a new run\"),\n",
        "        (\"capture why tests matter\", \"record which gate protected the workflow and why\"),\n",
        "        (\"update the plan\", \"mark completed ideas and add new risks discovered during the loop\"),\n",
        "        (\"human judgment\", \"stop or redirect when the loop optimizes the wrong objective\"),\n",
        "    ],\n",
        "    columns=[\"Ralph principle\", \"DL experiment translation\"],\n",
        ")\n",
        "ralph_principles\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 3 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Ralph Loops for Deep Learning Iteration** by working on forecast metrics. It creates or updates `dl_spec` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a Ralph-style loop to improve a demand forecaster one validated change at a time. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** evaluates scale-weighted forecast error\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "dl_spec = {\n",
        "    \"task\": \"7-day-ahead demand forecasting\",\n",
        "    \"primary_metric\": \"validation WMAPE\",\n",
        "    \"decision_metric\": \"do not increase validation WMAPE while reducing training risk\",\n",
        "    \"constraints\": [\n",
        "        \"time-based splits only\",\n",
        "        \"no future-demand leakage\",\n",
        "        \"each loop tests one candidate change\",\n",
        "        \"training must stay fast enough for repeated iteration\",\n",
        "        \"record every attempt, including rejected changes\",\n",
        "    ],\n",
        "    \"quality_gates\": {\n",
        "        \"max_valid_wmape\": 0.36,\n",
        "        \"max_train_valid_gap\": 0.10,\n",
        "        \"required_columns\": [\"date\", \"store_id\", \"item_id\", \"y\", \"lag_7\"],\n",
        "    },\n",
        "}\n",
        "dl_spec\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 4 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Ralph Loops for Deep Learning Iteration** by working on synthetic demand generation, time-based splitting, supervised feature construction. It creates or updates `df`, `supervised`, `train`, `valid`, `_`, `feature_cols` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a Ralph-style loop to improve a demand forecaster one validated change at a time. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** creates a synthetic retail panel with known demand drivers; turns time series rows into forecast-origin training examples; keeps train, validation, and test windows ordered by date; fits scaling parameters on training data only; wraps NumPy arrays in PyTorch DataLoaders\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "set_seed(SEED)\n",
        "df = generate_daily_demand(n_stores=4, n_items=6, periods=460, seed=SEED)\n",
        "supervised = make_supervised_table(df, horizon=7)\n",
        "train, valid, _ = time_series_split(supervised, valid_days=60, test_days=45)\n",
        "\n",
        "feature_cols = [\"store_id\", \"item_id\", *DEFAULT_FEATURE_COLUMNS]\n",
        "missing = sorted(set(dl_spec[\"quality_gates\"][\"required_columns\"]) - set(supervised.columns))\n",
        "if missing:\n",
        "    raise ValueError(f\"Missing required columns: {missing}\")\n",
        "\n",
        "scaler = Standardizer.fit(train[feature_cols].to_numpy(dtype=np.float32))\n",
        "X_train = scaler.transform(train[feature_cols].to_numpy(dtype=np.float32)).astype(np.float32)\n",
        "X_valid = scaler.transform(valid[feature_cols].to_numpy(dtype=np.float32)).astype(np.float32)\n",
        "y_train = np.log1p(train[\"y\"].to_numpy(dtype=np.float32))\n",
        "y_valid = np.log1p(valid[\"y\"].to_numpy(dtype=np.float32))\n",
        "\n",
        "train_loader = make_regression_loader(X_train, y_train, batch_size=256, shuffle=True)\n",
        "valid_loader = make_regression_loader(X_valid, y_valid, batch_size=512, shuffle=False)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 5 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Ralph Loops for Deep Learning Iteration** by working on forecast metrics, model training, model definition, Ralph-style iteration. It creates or updates intermediate values and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a Ralph-style loop to improve a demand forecaster one validated change at a time. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** instantiates a feed-forward demand forecasting model; runs the shared PyTorch training loop; generates validation or test predictions from a trained model; evaluates scale-weighted forecast error\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def evaluate_candidate(candidate, run_id):\n",
        "    set_seed(SEED + run_id)\n",
        "    model = MLPRegressor(\n",
        "        n_features=X_train.shape[1],\n",
        "        hidden_sizes=candidate[\"hidden_sizes\"],\n",
        "        dropout=candidate[\"dropout\"],\n",
        "    )\n",
        "    history = train_regression_model(\n",
        "        model,\n",
        "        train_loader,\n",
        "        valid_loader,\n",
        "        epochs=candidate[\"epochs\"],\n",
        "        learning_rate=candidate[\"lr\"],\n",
        "        weight_decay=candidate[\"weight_decay\"],\n",
        "    )\n",
        "    pred = np.expm1(predict_regression_model(model, X_valid))\n",
        "    actual = np.expm1(y_valid)\n",
        "    valid_wmape = wmape(actual, pred)\n",
        "    train_mae = history[-1][\"train_mae\"]\n",
        "    valid_mae = history[-1][\"valid_mae\"]\n",
        "    train_valid_gap = valid_mae - train_mae\n",
        "    passed = (\n",
        "        valid_wmape <= dl_spec[\"quality_gates\"][\"max_valid_wmape\"]\n",
        "        and train_valid_gap <= dl_spec[\"quality_gates\"][\"max_train_valid_gap\"]\n",
        "    )\n",
        "    return {\n",
        "        \"run_id\": run_id,\n",
        "        \"candidate\": candidate[\"name\"],\n",
        "        \"valid_wmape\": valid_wmape,\n",
        "        \"train_mae_log\": train_mae,\n",
        "        \"valid_mae_log\": valid_mae,\n",
        "        \"train_valid_gap\": train_valid_gap,\n",
        "        \"passed_backpressure\": passed,\n",
        "    }\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 6 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Ralph Loops for Deep Learning Iteration** by working on model training, Ralph-style iteration. It creates or updates `fix_plan` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a Ralph-style loop to improve a demand forecaster one validated change at a time. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** records the next bounded Ralph-loop candidates; creates an inspectable table for assumptions, metrics, or decisions\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "fix_plan = pd.DataFrame(\n",
        "    [\n",
        "        {\n",
        "            \"candidate\": \"baseline_mlp\",\n",
        "            \"priority\": 1,\n",
        "            \"status\": \"ready\",\n",
        "            \"hypothesis\": \"A compact MLP is the control run for later loops.\",\n",
        "            \"hidden_sizes\": (64, 32),\n",
        "            \"dropout\": 0.00,\n",
        "            \"lr\": 1e-3,\n",
        "            \"weight_decay\": 0.0,\n",
        "            \"epochs\": 4,\n",
        "        },\n",
        "        {\n",
        "            \"candidate\": \"wider_regularized\",\n",
        "            \"priority\": 2,\n",
        "            \"status\": \"ready\",\n",
        "            \"hypothesis\": \"More capacity plus mild dropout may improve nonlinear effects.\",\n",
        "            \"hidden_sizes\": (128, 64),\n",
        "            \"dropout\": 0.08,\n",
        "            \"lr\": 1e-3,\n",
        "            \"weight_decay\": 1e-4,\n",
        "            \"epochs\": 4,\n",
        "        },\n",
        "        {\n",
        "            \"candidate\": \"slower_optimizer\",\n",
        "            \"priority\": 3,\n",
        "            \"status\": \"ready\",\n",
        "            \"hypothesis\": \"A lower learning rate may reduce noisy validation behavior.\",\n",
        "            \"hidden_sizes\": (96, 48),\n",
        "            \"dropout\": 0.05,\n",
        "            \"lr\": 5e-4,\n",
        "            \"weight_decay\": 1e-4,\n",
        "            \"epochs\": 4,\n",
        "        },\n",
        "    ]\n",
        ")\n",
        "fix_plan[[\"candidate\", \"priority\", \"status\", \"hypothesis\"]]\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 7 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Ralph Loops for Deep Learning Iteration** by working on forecast metrics, Ralph-style iteration. It creates or updates `experiment_log`, `learnings`, `current_best`, `experiment_log`, `learning_log` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a Ralph-style loop to improve a demand forecaster one validated change at a time. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** evaluates scale-weighted forecast error; records the next bounded Ralph-loop candidates; keeps a reproducible history of loop results; captures what future loops should remember; creates an inspectable table for assumptions, metrics, or decisions\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment_log = []\n",
        "learnings = []\n",
        "current_best = np.inf\n",
        "\n",
        "for run_id in range(1, 4):\n",
        "    next_idx = (\n",
        "        fix_plan.query(\"status == 'ready'\")\n",
        "        .sort_values(\"priority\")\n",
        "        .index[0]\n",
        "    )\n",
        "    candidate = fix_plan.loc[next_idx].to_dict()\n",
        "    candidate[\"name\"] = candidate[\"candidate\"]\n",
        "\n",
        "    result = evaluate_candidate(candidate, run_id=run_id)\n",
        "    improved = result[\"valid_wmape\"] < current_best\n",
        "    current_best = min(current_best, result[\"valid_wmape\"])\n",
        "    result[\"improved_best\"] = improved\n",
        "    experiment_log.append(result)\n",
        "\n",
        "    fix_plan.loc[next_idx, \"status\"] = \"accepted\" if result[\"passed_backpressure\"] else \"rejected\"\n",
        "    learnings.append(\n",
        "        {\n",
        "            \"loop\": run_id,\n",
        "            \"candidate\": result[\"candidate\"],\n",
        "            \"learning\": (\n",
        "                \"Keep as a reference point.\"\n",
        "                if improved\n",
        "                else \"Do not combine this with other changes until a stronger reason appears.\"\n",
        "            ),\n",
        "            \"backpressure_note\": (\n",
        "                \"Passed validation gates.\"\n",
        "                if result[\"passed_backpressure\"]\n",
        "                else \"Rejected by metric or train/validation gap gate.\"\n",
        "            ),\n",
        "        }\n",
        "    )\n",
        "\n",
        "experiment_log = pd.DataFrame(experiment_log)\n",
        "learning_log = pd.DataFrame(learnings)\n",
        "experiment_log.sort_values(\"valid_wmape\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Code Cell 8 Explanation\n",
        "\n",
        "**What this cell does:** This cell advances **Ralph Loops for Deep Learning Iteration** by working on forecast metrics, Ralph-style iteration. It creates or updates `loop_summary` and prepares evidence that later cells use rather than hiding state in prose.\n",
        "\n",
        "**Why it matters:** The goal is tied to this lesson's forecasting focus: Use a Ralph-style loop to improve a demand forecaster one validated change at a time. Each object is kept inspectable so the learner can connect code, assumptions, and modeling consequences.\n",
        "\n",
        "**Key operations to notice:** evaluates scale-weighted forecast error; records the next bounded Ralph-loop candidates; keeps a reproducible history of loop results; captures what future loops should remember\n",
        "\n",
        "**What to inspect:** Before moving on, inspect shapes, date ranges, metric values, segment behavior, or generated tables. If a value looks surprisingly good, surprisingly bad, or unavailable at forecast time, treat that as a modeling clue.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "loop_summary = {\n",
        "    \"best_run\": experiment_log.sort_values(\"valid_wmape\").iloc[0].to_dict(),\n",
        "    \"completed_plan\": fix_plan[[\"candidate\", \"priority\", \"status\", \"hypothesis\"]].to_dict(\"records\"),\n",
        "    \"learning_log\": learning_log.to_dict(\"records\"),\n",
        "}\n",
        "loop_summary\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Operating Rules for DL Ralph Loops\n",
        "\n",
        "Use this pattern for rapid iteration, but keep it bounded. A loop should not change the dataset, features, loss, architecture, and decision metric at the same time. If a run fails, update the plan with the reason. If the loop keeps improving a proxy metric while hurting the business decision, stop and rewrite the spec.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Lesson Recap\n",
        "\n",
        "In this lesson, we used 8 runnable code cells to work through **Ralph Loops for Deep Learning Iteration**. The practical focus was: Use a Ralph-style loop to improve a demand forecaster one validated change at a time.\n",
        "\n",
        "We did this because deep learning for demand forecasting is not only about choosing an architecture. The learner needs to understand the data contract, the target and horizon, the validation path, the metric, and the operational decision supported by the forecast.\n",
        "\n",
        "The core takeaways were:\n",
        "\n",
        "- Translate Ralph loop principles into deep learning experiment workflows.\n",
        "- Create a deterministic context stack with specs, metrics, constraints, and a fix plan.\n",
        "- Run one model-improvement candidate per loop and apply fast validation backpressure.\n",
        "- Record learnings so future loops inherit decisions instead of rediscovering them.\n",
        "\n",
        "By the end, you should be able to explain not just what the code produced, but why each step was necessary and what could go wrong if the same step were skipped or performed with leakage.\n",
        "\n",
        "**Senior-readiness reminder:** You can use agentic iteration safely: fast loops, tight scope, reliable gates, and human judgment about when to stop.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Exercises\n",
        "\n",
        "- Add one new candidate to the fix plan and explain why it is exactly one modeling change.\n",
        "- Create a new backpressure gate for leakage, runtime, calibration, or inventory cost.\n",
        "- Write a loop note that a future agent could use to avoid repeating a failed experiment.\n",
        "- Describe when you would stop the Ralph loop and ask for human review.\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python (deep-learning-demand)",
      "language": "python",
      "name": "deep-learning-demand"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}