{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f9aeb7ce",
   "metadata": {},
   "source": [
    "# Lab Minggu 17 — Final Lab: Template Pipeline End-to-End\n",
    "\n",
    "Notebook ini adalah **template proyek akhir**. Contoh kasusnya prediksi churn,\n",
    "tetapi strukturnya bisa kamu salin untuk data proyekmu sendiri.\n",
    "\n",
    "| Tahap KDD | Bagian notebook |\n",
    "|---|---|\n",
    "| Selection | 1. Masalah & data |\n",
    "| Preprocessing + Transformation | 2–3. EDA & Pipeline |\n",
    "| Data Mining | 4. Perbandingan model |\n",
    "| Interpretation | 5–7. Evaluasi, interpretasi, laporan |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb2c07fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.insert(0, \"..\")\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.inspection import permutation_importance\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import (ConfusionMatrixDisplay, RocCurveDisplay, classification_report,\n",
    "                             f1_score, precision_recall_curve)\n",
    "from sklearn.model_selection import StratifiedKFold, cross_validate, train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, StandardScaler\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "from common.datasets import churn_pelanggan\n",
    "\n",
    "SEED = 42  # satu seed untuk seluruh proyek → reproducible"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2cba897",
   "metadata": {},
   "source": [
    "## 1. Masalah bisnis & data\n",
    "\n",
    "**Masalah:** perusahaan layanan langganan kehilangan ±25% pelanggan. Tim retensi hanya\n",
    "sanggup menghubungi sebagian pelanggan per bulan → butuh model untuk memprioritaskan.\n",
    "\n",
    "**Metrik:** F1 (keseimbangan precision & recall) + recall sebagai perhatian utama\n",
    "(pelanggan churn yang terlewat = pendapatan hilang)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf244b50",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = churn_pelanggan()\n",
    "TARGET = \"churn\"\n",
    "print(df.shape)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62a552b3",
   "metadata": {},
   "source": [
    "## 2. EDA singkat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eceb91b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Proporsi churn:\", df[TARGET].mean().round(3))\n",
    "df.describe().T.round(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57668b86",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 3, figsize=(15, 3.5))\n",
    "df.groupby(\"kontrak\")[TARGET].mean().plot.bar(ax=axes[0], title=\"Churn rate per kontrak\", rot=0)\n",
    "df.boxplot(column=\"lama_langganan\", by=TARGET, ax=axes[1]); axes[1].set_title(\"Lama langganan vs churn\")\n",
    "df.groupby(\"jumlah_keluhan\")[TARGET].mean().plot(ax=axes[2], marker=\"o\", title=\"Churn rate vs jumlah keluhan\")\n",
    "plt.suptitle(\"\"); plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40cc6418",
   "metadata": {},
   "source": [
    "## 3. Split data & pipeline preprocessing\n",
    "\n",
    "Split **sebelum** apa pun di-fit. Semua transformasi masuk ke `Pipeline`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14422543",
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = df.drop(columns=TARGET), df[TARGET]\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, stratify=y, random_state=SEED)\n",
    "\n",
    "num_cols = [c for c in X.columns if pd.api.types.is_numeric_dtype(X[c])]\n",
    "cat_cols = [c for c in X.columns if c not in num_cols]\n",
    "print(\"Numerik    :\", num_cols)\n",
    "print(\"Kategorikal:\", cat_cols)\n",
    "\n",
    "def bangun_pipeline(model):\n",
    "    prep = ColumnTransformer([\n",
    "        (\"num\", Pipeline([(\"imputer\", SimpleImputer(strategy=\"median\")),\n",
    "                          (\"scaler\", StandardScaler())]), num_cols),\n",
    "        (\"cat\", Pipeline([(\"imputer\", SimpleImputer(strategy=\"most_frequent\")),\n",
    "                          (\"onehot\", OneHotEncoder(handle_unknown=\"ignore\"))]), cat_cols),\n",
    "    ])\n",
    "    return Pipeline([(\"prep\", prep), (\"model\", model)])\n",
    "\n",
    "bangun_pipeline(LogisticRegression())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "655ddea0",
   "metadata": {},
   "source": [
    "## 4. Perbandingan model dengan CV berstrata (fold sama untuk semua)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50f6a24a",
   "metadata": {},
   "outputs": [],
   "source": [
    "models = {\n",
    "    \"baseline (mayoritas)\": DummyClassifier(strategy=\"most_frequent\"),\n",
    "    \"decision tree d=5\": DecisionTreeClassifier(max_depth=5, random_state=SEED),\n",
    "    \"logreg balanced\": LogisticRegression(max_iter=1000, class_weight=\"balanced\"),\n",
    "    \"random forest\": RandomForestClassifier(n_estimators=200, class_weight=\"balanced\", random_state=SEED),\n",
    "}\n",
    "cv = StratifiedKFold(5, shuffle=True, random_state=SEED)\n",
    "baris = []\n",
    "for nama, mdl in models.items():\n",
    "    r = cross_validate(bangun_pipeline(mdl), X_train, y_train, cv=cv,\n",
    "                       scoring=[\"f1\", \"recall\", \"precision\", \"roc_auc\"])\n",
    "    baris.append({\"model\": nama,\n",
    "                  **{f\"{k[5:]}_mean\": v.mean() for k, v in r.items() if k.startswith(\"test_\")},\n",
    "                  \"f1_std\": r[\"test_f1\"].std()})\n",
    "hasil_cv = pd.DataFrame(baris).set_index(\"model\").sort_values(\"f1_mean\", ascending=False)\n",
    "hasil_cv.round(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8eda7536",
   "metadata": {},
   "outputs": [],
   "source": [
    "hasil_cv[\"f1_mean\"].plot.barh(xerr=hasil_cv[\"f1_std\"], figsize=(7, 3), title=\"F1 CV 5-fold (mean ± std)\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "461757cf",
   "metadata": {},
   "source": [
    "## 5. Evaluasi final pada data uji (sekali saja!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "372abe36",
   "metadata": {},
   "outputs": [],
   "source": [
    "terbaik = hasil_cv.index[0]\n",
    "pipe = bangun_pipeline(models[terbaik]).fit(X_train, y_train)\n",
    "pred = pipe.predict(X_test)\n",
    "print(\"Model terpilih:\", terbaik)\n",
    "print(classification_report(y_test, pred, digits=3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31699979",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(11, 4))\n",
    "ConfusionMatrixDisplay.from_predictions(y_test, pred, ax=axes[0], colorbar=False)\n",
    "RocCurveDisplay.from_estimator(pipe, X_test, y_test, ax=axes[1])\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "258f10f9",
   "metadata": {},
   "source": [
    "### Threshold sesuai kapasitas tim retensi\n",
    "Misal tim hanya sanggup menghubungi 20% pelanggan → ambil 20% probabilitas tertinggi."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7dc1db52",
   "metadata": {},
   "outputs": [],
   "source": [
    "proba = pipe.predict_proba(X_test)[:, 1]\n",
    "ambang = np.quantile(proba, 0.80)\n",
    "pred_kapasitas = (proba >= ambang).astype(int)\n",
    "print(f\"ambang={ambang:.3f}, dihubungi={pred_kapasitas.sum()}, \"\n",
    "      f\"churn tertangkap={((pred_kapasitas == 1) & (y_test == 1)).sum()} dari {y_test.sum()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85cf6524",
   "metadata": {},
   "source": [
    "## 6. Interpretasi: permutation importance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1491de59",
   "metadata": {},
   "outputs": [],
   "source": [
    "imp = permutation_importance(pipe, X_test, y_test, scoring=\"f1\", n_repeats=10, random_state=SEED)\n",
    "penting = pd.Series(imp.importances_mean, index=X_test.columns).sort_values()\n",
    "penting.plot.barh(title=\"Permutation importance (penurunan F1)\"); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a31155b",
   "metadata": {},
   "source": [
    "## 7. Laporan otomatis (Markdown)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7ef0431",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, precision_score, recall_score, roc_auc_score\n",
    "\n",
    "metrik = {\"accuracy\": accuracy_score(y_test, pred), \"precision\": precision_score(y_test, pred),\n",
    "          \"recall\": recall_score(y_test, pred), \"f1\": f1_score(y_test, pred),\n",
    "          \"roc_auc\": roc_auc_score(y_test, proba)}\n",
    "top = penting.sort_values(ascending=False)\n",
    "laporan = \"\\n\".join(\n",
    "    [f\"# Laporan Model Churn — {terbaik}\", \"\", \"## Metrik Evaluasi\", \"\", \"| Metrik | Nilai |\", \"|---|---|\"]\n",
    "    + [f\"| {k} | {v:.3f} |\" for k, v in metrik.items()]\n",
    "    + [\"\", \"## Fitur Terpenting\", \"\"]\n",
    "    + [f\"{i}. {n} ({v:.3f})\" for i, (n, v) in enumerate(top.head(3).items(), 1)]\n",
    ")\n",
    "print(laporan)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb3fa065",
   "metadata": {},
   "source": [
    "## ✍️ Coba sendiri (adaptasi ke proyekmu)\n",
    "\n",
    "1. Ganti dataset dengan data proyek kelompokmu. Bagian mana saja yang perlu diubah?\n",
    "2. Tambahkan satu model lagi (mis. `KNeighborsClassifier` atau `GaussianNB`) ke perbandingan.\n",
    "3. Lakukan tuning hyperparameter `C` pada logistic regression dengan `GridSearchCV` (tetap di dalam pipeline!).\n",
    "4. Tulis 3 rekomendasi bisnis berdasarkan permutation importance.\n",
    "5. Simpan pipeline terbaik dengan `joblib.dump` dan muat ulang untuk memprediksi 5 pelanggan baru."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbf55bba",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7893d57f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db858883",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 5."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6000f22",
   "metadata": {},
   "source": [
    "**Rekomendasi bisnis (no. 4):**\n",
    "\n",
    "1. ...\n",
    "2. ...\n",
    "3. ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d48cc0fd",
   "metadata": {},
   "source": [
    "## Selanjutnya\n",
    "Kerjakan kuis dan `exercise.py`, lalu jalankan `python check.py 17` dari folder utama."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}