Lompat ke konten
nalar

Probabilistic & Instance-Based Classifiers (Naïve Bayes & KNN)

Teorema Bayes, asumsi independensi Naïve Bayes (kategorikal, Gaussian, multinomial untuk teks), Laplace smoothing, serta klasifikasi berbasis jarak K-Nearest Neighbors.

Notebook lab (pratinjau)

Unduh lalu jalankan di Jupyter, VS Code, atau Google Colab.

Unduh lab.ipynb

Lab Minggu 7 — Naïve Bayes & K-Nearest Neighbors

Tujuan lab:

  1. Menghitung Naïve Bayes manual pada Play Tennis lalu mencocokkannya dengan sklearn.
  2. Memakai Gaussian NB & KNN untuk memprediksi risiko churn pelanggan.
  3. Membuat klasifikasi sentimen ulasan dengan Multinomial NB.
  4. Melihat pengaruh normalisasi dan nilai k pada KNN.

Baca lesson.md dulu. Jalankan sel satu per satu (Shift+Enter).

[1]
import sys
sys.path.insert(0, "..")

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import CategoricalNB, GaussianNB, MultinomialNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import OrdinalEncoder, StandardScaler
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.metrics import accuracy_score, classification_report

from common.datasets import play_tennis, churn_pelanggan, ulasan_produk

1. Naïve Bayes manual — Play Tennis

[2]
df = play_tennis()
df
[3]
prior = df["play"].value_counts(normalize=True)
prior
[4]
# Tabel likelihood P(nilai | kelas) per fitur (tanpa smoothing)
for fitur in ["outlook", "temperature", "humidity", "wind"]:
    print(pd.crosstab(df[fitur], df["play"], normalize="columns").round(3), "\n")
[5]
x_baru = {"outlook": "Sunny", "temperature": "Cool", "humidity": "High", "wind": "Strong"}

def skor_nb(df, x, alpha=0):
    skor = {}
    for kelas, grup in df.groupby("play"):
        s = len(grup) / len(df)
        for f, v in x.items():
            k = df[f].nunique()
            s *= ((grup[f] == v).sum() + alpha) / (len(grup) + alpha * k)
        skor[kelas] = s
    total = sum(skor.values())
    return {k: (round(v, 5), round(v / total, 3)) for k, v in skor.items()}

print("tanpa smoothing :", skor_nb(df, x_baru, 0))
print("Laplace alpha=1 :", skor_nb(df, x_baru, 1))

Bandingkan dengan sklearn CategoricalNB

alpha=1 di sklearn = Laplace smoothing.

[6]
enc = OrdinalEncoder()
X = enc.fit_transform(df.drop(columns="play"))
y = df["play"]
nb = CategoricalNB(alpha=1).fit(X, y)
x_enc = enc.transform(pd.DataFrame([x_baru]))
print(dict(zip(nb.classes_, nb.predict_proba(x_enc)[0].round(3))))

2. Gaussian NB vs KNN — prediksi churn

[7]
churn = churn_pelanggan()
fitur_num = ["lama_langganan", "biaya_bulanan", "jumlah_keluhan", "jam_pemakaian"]
X = churn[fitur_num].values
y = churn["churn"].values
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)
churn.groupby("churn")[fitur_num].mean().round(1)
[8]
gnb = GaussianNB().fit(X_tr, y_tr)
print("Gaussian NB akurasi:", round(accuracy_score(y_te, gnb.predict(X_te)), 3))
print("mean per kelas (theta_):\n", pd.DataFrame(gnb.theta_, columns=fitur_num).round(1))

KNN tanpa vs dengan normalisasi

[9]
knn_mentah = KNeighborsClassifier(n_neighbors=7).fit(X_tr, y_tr)
sc = StandardScaler().fit(X_tr)
knn_norm = KNeighborsClassifier(n_neighbors=7).fit(sc.transform(X_tr), y_tr)
print("KNN tanpa normalisasi :", round(accuracy_score(y_te, knn_mentah.predict(X_te)), 3))
print("KNN dengan z-score    :", round(accuracy_score(y_te, knn_norm.predict(sc.transform(X_te))), 3))

Tanpa normalisasi, biaya_bulanan (ratusan ribu) mendominasi jarak.

Pengaruh nilai k

[10]
ks = range(1, 40, 2)
acc_tr, acc_te = [], []
Xs_tr, Xs_te = sc.transform(X_tr), sc.transform(X_te)
for k in ks:
    model = KNeighborsClassifier(n_neighbors=k).fit(Xs_tr, y_tr)
    acc_tr.append(model.score(Xs_tr, y_tr))
    acc_te.append(model.score(Xs_te, y_te))

plt.plot(ks, acc_tr, "o-", label="latih")
plt.plot(ks, acc_te, "s-", label="uji")
plt.xlabel("k"); plt.ylabel("akurasi"); plt.legend(); plt.title("KNN: k vs akurasi")
plt.show()

k=1 → akurasi latih 100% (hafal data) tetapi akurasi uji lebih rendah = overfitting.

3. Klasifikasi sentimen dengan Multinomial NB

[11]
ul = ulasan_produk()
vec = CountVectorizer()
X_teks = vec.fit_transform(ul["teks"])
print("Ukuran kosakata:", len(vec.vocabulary_))
mnb = MultinomialNB(alpha=1).fit(X_teks, ul["sentimen"])

uji = ["barang bagus dan pengiriman cepat", "penjual lambat barang rusak", "harga murah tapi kualitas jelek"]
for t, p, pr in zip(uji, mnb.predict(vec.transform(uji)), mnb.predict_proba(vec.transform(uji))):
    print(f"{t:40s} -> {p:8s} {dict(zip(mnb.classes_, pr.round(2)))}")

Kata paling "khas" per kelas (log-likelihood terbesar):

[12]
kata = np.array(vec.get_feature_names_out())
for i, kelas in enumerate(mnb.classes_):
    top = np.argsort(mnb.feature_log_prob_[i])[::-1][:6]
    print(kelas, "->", ", ".join(kata[top]))

✍️ Coba sendiri

  1. Hitung manual posterior Play Tennis untuk X = (Rain, Mild, Normal, Weak) dengan α = 1. Cocokkan dengan CategoricalNB.
  2. Tambahkan fitur kategorikal kontrak (one-hot) ke model KNN churn. Apakah akurasi naik?
  3. Bandingkan KNN metric="manhattan" vs "euclidean" pada data churn yang sudah dinormalisasi.
  4. Tulis 3 ulasan buatanmu sendiri (termasuk yang ambigu) dan lihat prediksi Multinomial NB. Mengapa ulasan ambigu sulit?
[13]
# 1.
[14]
# 2.
[15]
# 3.

Jawaban no. 4:

  • ...

Selanjutnya

Kerjakan kuis dan exercise.py, lalu jalankan python check.py 7 dari folder utama.