{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "walkthrough-01",
   "metadata": {},
   "source": [
    "# MAMUT Walkthrough: Auditable Tabular Classification\n",
    "\n",
    "This example demonstrates the current MAMUT workflow on a deterministic public dataset. It uses a small search budget, reserves final holdout data, and inspects validation evidence without writing local artifacts."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "walkthrough-02",
   "metadata": {},
   "source": [
    "## Prepare a classification dataset\n",
    "\n",
    "The target is converted to readable class labels to demonstrate that public predictions preserve original labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "walkthrough-03",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_breast_cancer\n",
    "\n",
    "from mamut import Mamut\n",
    "\n",
    "X, y = load_breast_cancer(as_frame=True, return_X_y=True)\n",
    "y = y.map({0: \"malignant\", 1: \"benign\"})\n",
    "X.shape, y.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "walkthrough-04",
   "metadata": {},
   "source": [
    "## Fit a bounded candidate comparison\n",
    "\n",
    "Validation data selects the model. The holdout split remains separate for final diagnostics. The selected candidate is refit on all non-holdout modelling rows before holdout evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "walkthrough-05",
   "metadata": {},
   "outputs": [],
   "source": [
    "mamut = Mamut(\n",
    "    include_models=[\"LogisticRegression\", \"RandomForestClassifier\"],\n",
    "    score_metric=\"balanced_accuracy\",\n",
    "    optimization_method=\"random_search\",\n",
    "    n_iterations=1,\n",
    "    validation_size=0.2,\n",
    "    holdout_size=0.2,\n",
    "    refit_final_model=True,\n",
    "    evidence_cv_splits=2,\n",
    "    evidence_cv_repeats=1,\n",
    "    n_jobs=1,\n",
    "    random_state=42,\n",
    ")\n",
    "mamut.fit(X, y)\n",
    "mamut.validation_summary_.round(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "walkthrough-06",
   "metadata": {},
   "outputs": [],
   "source": [
    "mamut.holdout_summary_.round(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "walkthrough-07",
   "metadata": {},
   "source": [
    "## Inspect prediction and evidence contracts\n",
    "\n",
    "A locked holdout evidence view scores the selected model and simple baselines without evaluating alternate MAMUT candidates on the final holdout."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "walkthrough-08",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = mamut.predict(X.head())\n",
    "probabilities = mamut.predict_proba(X.head())\n",
    "predictions, probabilities.round(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "walkthrough-09",
   "metadata": {},
   "outputs": [],
   "source": [
    "evidence = mamut.generate_evidence(\n",
    "    dataset=\"holdout\",\n",
    "    include_candidate_comparison=False,\n",
    ")\n",
    "evidence[\"validation_integrity\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "walkthrough-10",
   "metadata": {},
   "outputs": [],
   "source": [
    "evidence[\"baseline_comparison\"].round(3), evidence[\"score_stability\"].round(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "walkthrough-11",
   "metadata": {},
   "source": [
    "## Generate a lightweight report\n",
    "\n",
    "Artifact writing and SHAP computation are configurable. This smoke-mode call retains evidence in memory without creating report files. For shareable output, call `evaluate()` with an `output_dir`, and use `save_models=True` only when serialized candidate models are needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "walkthrough-12",
   "metadata": {},
   "outputs": [],
   "source": [
    "report = mamut.evaluate(\n",
    "    include_shap=False,\n",
    "    write_html=False,\n",
    "    save_plots=False,\n",
    ")\n",
    "report[\"evaluation_dataset\"], report[\"evidence_available\"]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}