MAMUT Walkthrough: Auditable Tabular Classification

This example demonstrates the current MAMUT workflow on a deterministic public dataset. It uses a small search budget, reserves final holdout data, and inspects validation evidence without writing local artifacts.

Prepare a classification dataset

The target is converted to readable class labels to demonstrate that public predictions preserve original labels.

from sklearn.datasets import load_breast_cancer

from mamut import Mamut

X, y = load_breast_cancer(as_frame=True, return_X_y=True)
y = y.map({0: "malignant", 1: "benign"})
X.shape, y.value_counts()

Fit a bounded candidate comparison

Validation data selects the model. The holdout split remains separate for final diagnostics. The selected candidate is refit on all non-holdout modelling rows before holdout evaluation.

mamut = Mamut(
    include_models=["LogisticRegression", "RandomForestClassifier"],
    score_metric="balanced_accuracy",
    optimization_method="random_search",
    n_iterations=1,
    validation_size=0.2,
    holdout_size=0.2,
    refit_final_model=True,
    evidence_cv_splits=2,
    evidence_cv_repeats=1,
    n_jobs=1,
    random_state=42,
)
mamut.fit(X, y)
mamut.validation_summary_.round(3)

mamut.holdout_summary_.round(3)

Inspect prediction and evidence contracts

A locked holdout evidence view scores the selected model and simple baselines without evaluating alternate MAMUT candidates on the final holdout.

predictions = mamut.predict(X.head())
probabilities = mamut.predict_proba(X.head())
predictions, probabilities.round(3)

evidence = mamut.generate_evidence(
    dataset="holdout",
    include_candidate_comparison=False,
)
evidence["validation_integrity"]

evidence["baseline_comparison"].round(3), evidence["score_stability"].round(3)

Generate a lightweight report

Artifact writing and SHAP computation are configurable. This smoke-mode call retains evidence in memory without creating report files. For shareable output, call evaluate() with an output_dir, and use save_models=True only when serialized candidate models are needed.

report = mamut.evaluate(
    include_shap=False,
    write_html=False,
    save_plots=False,
)
report["evaluation_dataset"], report["evidence_available"]