{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# 实验 B：高分是否来自重复样品\n",
        "先猜：相同模型只改变划分方式，分数会不会改变？检查样品编号交集，再看误差。数据故意设计为可识别样品但不能预测新样品响应，帮助区分记忆与泛化。\n",
        "**数据声明：本实验使用固定随机种子生成的模拟教学数据，不是真实材料数据。**\n",
        "先安装依赖，再依次运行下方两格。图片和结果保存在当前目录的 outputs 文件夹。\n"
      ],
      "id": "lab-b-0"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "\"\"\"共同的模拟教学数据；不代表真实材料测量或物理规律。\"\"\"\n",
        "from pathlib import Path\n",
        "import csv\n",
        "import numpy as np\n",
        "\n",
        "OUT = Path.cwd() / 'outputs'\n",
        "OUT.mkdir(exist_ok=True)\n",
        "\n",
        "def materials():\n",
        "    rng = np.random.default_rng(42)\n",
        "    temperature = rng.uniform(300, 900, 240)\n",
        "    fraction = rng.uniform(0, 1, 240)\n",
        "    size = rng.uniform(10, 100, 240)\n",
        "    X = np.column_stack([temperature, fraction, size])\n",
        "    y = 2 + .004 * temperature + 3 * fraction**2 - .015 * size + rng.normal(0, .15, 240)\n",
        "    with (OUT / 'simulated_materials.csv').open('w', newline='', encoding='utf-8') as f:\n",
        "        w = csv.writer(f)\n",
        "        w.writerow(['temperature_K', 'fraction', 'size_nm', 'teaching_response'])\n",
        "        w.writerows(np.column_stack([X, y]))\n",
        "    return X, y\n"
      ],
      "id": "lab-b-1"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "\"\"\"实验 B：随机按行划分与按新样品划分，回答的是不同问题。\"\"\"\n",
        "import csv\n",
        "import json\n",
        "import numpy as np\n",
        "import matplotlib\n",
        "matplotlib.use('Agg')\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.model_selection import train_test_split, GroupShuffleSplit\n",
        "from sklearn.neighbors import KNeighborsRegressor\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from sklearn.pipeline import make_pipeline\n",
        "from sklearn.metrics import mean_absolute_error\n",
        "\n",
        "\n",
        "rng = np.random.default_rng(42)\n",
        "# 每个样品具有一个可重复测得的“指纹”，响应在本例中是随机的，不能外推。\n",
        "n_samples, repeats = 80, 8\n",
        "fingerprints = rng.normal(size=(n_samples, 6))\n",
        "responses = rng.normal(size=n_samples)\n",
        "groups = np.repeat(np.arange(n_samples), repeats)\n",
        "X = fingerprints[groups] + rng.normal(0, .005, (len(groups), 6))\n",
        "y = responses[groups] + rng.normal(0, .02, len(groups))\n",
        "with (OUT / 'simulated_repeats.csv').open('w', newline='', encoding='utf-8') as f:\n",
        "    w = csv.writer(f)\n",
        "    w.writerow(['sample_id'] + [f'fingerprint_{i}' for i in range(6)] + ['teaching_response'])\n",
        "    w.writerows(np.column_stack([groups, X, y]))\n",
        "row_train, row_test = train_test_split(np.arange(len(y)), test_size=.25, random_state=42)\n",
        "group_train, group_test = next(GroupShuffleSplit(n_splits=1, test_size=.25, random_state=42).split(X, y, groups))\n",
        "results = {}\n",
        "for name, train, test in [('random_rows', row_train, row_test), ('new_samples', group_train, group_test)]:\n",
        "    model = make_pipeline(StandardScaler(), KNeighborsRegressor(n_neighbors=1))\n",
        "    model.fit(X[train], y[train])\n",
        "    overlap = len(set(groups[train]) & set(groups[test]))\n",
        "    results[name] = {'mae': float(mean_absolute_error(y[test], model.predict(X[test]))), 'overlapping_samples': overlap}\n",
        "assert results['new_samples']['overlapping_samples'] == 0\n",
        "fig, ax = plt.subplots(figsize=(6, 4), constrained_layout=True)\n",
        "ax.bar(list(results), [v['mae'] for v in results.values()])\n",
        "ax.set(ylabel='Test MAE', title='Same model, different generalization questions')\n",
        "fig.savefig(OUT / 'b_split_comparison.png', dpi=150)\n",
        "plt.close(fig)\n",
        "(OUT / 'b_results.json').write_text(json.dumps(results, indent=2), encoding='utf-8')\n",
        "print(json.dumps(results, indent=2))\n"
      ],
      "id": "lab-b-2"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 运行后回答\n",
        "先猜：相同模型只改变划分方式，分数会不会改变？检查样品编号交集，再看误差。数据故意设计为可识别样品但不能预测新样品响应，帮助区分记忆与泛化。\n",
        "提交结果表、输出图、150 字解释和一个局限。改变设置属于新实验；不要看测试结果反复挑设置。\n"
      ],
      "id": "lab-b-3"
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.12"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}