Table of Contents

Getting Started

• Welcome to the Model Metrics Python Library Documentation!
  • Project Links
  • What is Model Evaluation?
  • Purpose of Model Metrics Library
  • Key Features
  • Prerequisites
  • Installation
  • Description

Model Training Overview

• Adult Income Dataset
  • Model Configuration & Hyperparameters (model_params.py)
  • Model Training (train.py)
    • Logistic Regression Output
    • Decision Tree Output
    • Random Forest Output
  • Loading (Retrieving) The Model Objects and Data Splits

Performance Assessment

• Model Performance Summaries
  • summarize_model_performance()
  • Binary Classification Models
    • Binary Classification Example 1
    • Binary Classification Example 2
  • Regression Models
    • Regression Example 1
    • Regression Example 2
    • Regression Example 3
• Lift Charts
  • show_lift_chart()
  • Lift Chart Example 1 (Grid Layout)
  • Lift Chart Example 2 (Overlay)
• Gain Charts
  • show_gain_chart()
  • Gain Chart Example 1 (Grid Layout)
  • Gain Chart Example 2 (Overlay)
• ROC AUC Curves
  • show_roc_curve()
  • ROC AUC Example 1 (Grid Layout)
  • ROC AUC Example 2 (Overlay)
  • ROC AUC Example 3 (by Category)
• Precision-Recall Curves
  • show_pr_curve()
  • Precision-Recall Example 1 (Grid Layout)
  • Precision-Recall Example 2 (Overlay)
  • Precision-Recall Example 3 (Categorical)
• Confusion Matrix Evaluation
  • show_confusion_matrix()
  • Confusion Matrix Example 1 (Threshold=0.5)
  • Confusion Matrix Example 2 (Classification Report)
  • Confusion Matrix Example 3 (Threshold = 0.37)
• Calibration Curves
  • show_calibration_curve()
  • Calibration Curve Example 1 (Grid-like)
  • Calibration Curve Example 2 (Overlay)
  • Calibration Curve Example 3 (by Category)
• Threshold Metric Curves
  • plot_threshold_metrics()
  • Threshold Curves Example 1 (Threshold=0.5)
  • Threshold Curves Example 2 (Targeted Metric Lookup)

Partial Dependence

• Partial Dependence Plots
  • 2D Partial Dependence Plots
    • plot_2d_pdp()
    • 2D Plots - CA Housing Example
  • 3D Partial Dependence Plots
    • plot_3d_pdp()
    • 3D Plots - CA Housing Example
    • Static Plot
    • Interactive Plot

Conceptual Notes

• Interpretive Context
  • Binary Classification Outputs
  • ROC Curve and AUC
  • Precision-Recall Curve and AUC-PR
  • Thresholding and Predictions
  • Summary Table: Theory Meets Interpretation
  • Interpretive Caveats
• Threshold Selection logic
• Calibration Trade-offs
  • Calibration Methodology
  • Alternative Calibration Methods
  • Dependence on Validation Data
  • Interpreting the Brier Score
• Partial Dependence Foundations

About Model Metrics

• Acknowledgements
• Contributors/Maintainers
• Citing Model Metrics
• Changelog
  • Version 0.0.2a
  • Version 0.0.1a
• References