Hire an Illini
Gaurush Hiranandani
- Advisor:
- Prof. Sanmi Koyejo
- Departments:
- Areas of Expertise:
- Classification, Ranking, Recommendation
- Metric Elicitation
- Statistical Machine Learning
- Thesis Title:
- Classification Performance Metric Elicitation
- Thesis abstract:
- Given a learning problem with real-world tradeoffs, which cost function should the model be trained to optimize? This is the metric selection problem in machine learning. Despite its practical interest, there is limited formal guidance on how to select metrics for machine learning applications. To address this issue, we propose and formalize metric elicitation as a principled framework for selecting the performance metric that best reflects implicit user preferences. We devise novel strategies for eliciting classification performance metrics using pairwise preference feedback over classifiers. Specifically, we elicit linear and linear-fractional metrics for binary and multiclass classification problems, which are then extended to a framework that elicits group-fair performance metrics in the presence of multiple sensitive groups. All the the elicitation strategies are robust to both finite sample and feedback noise, thus are useful in practice for real-world applications.
- Downloads:
Contact information:
gaurush2@illinois.edu