Подтвердите e-mail

Для публикаций, комментариев, реакций и сообщений подтвердите адрес.

←

Публикация

Jordan Nafa

I'm old enough to remember when people still thought SMOTE was a legitimate solution to class imbalance in predictive modeling 😮‍💨

Обсуждение

Ответы

Lauren Parlett, PhD

I only learned of SMOTE in the past few years. Has another method overtaken it or have methodological flaws been elucidated or emphasized?

Ответ для Lauren Parlett, PhD
Jordan Nafa

This is a good overview of methods and approaches for predicting rare events: dl.acm.org/doi/full/10.... In general, SMOTE doesn't perform better than doing nothing, and in some cases it can perform considerably worse. Class-weighting, or "cost-sensitive learning," is generally preferred nowadays

A Comprehensive Survey on Rare Event Prediction | ACM Computing SurveysRare event prediction involves identifying and forecasting events with a low probability using machine learning (ML) and data analysis. Due to the imbalanced data distributions, where the frequency of...dl.acm.org
Ответ для Lauren Parlett, PhD
Karandeep Singh

academic.oup.com/jamia/articl...

The harm of class imbalance corrections for risk prediction models: illustration and simulation using logistic regressionAbstractObjective. Methods to correct class imbalance (imbalance between the frequency of outcome events and nonevents) are receiving increasing interest facademic.oup.com
Ответ для Lauren Parlett, PhD
Mridul K. Thomas

There's a few useful threads over at CrossValidated about unbalanced classes. This is a good one that brings them together: stats.stackexchange.com/questions/35...

Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?TL;DR See title. Motivation I am hoping for a canonical answer along the lines of "(1) No, (2) Not applicable, because (1)", which we can use to close many wrong questions about unbalancedstats.stackexchange.com
Demetri

I remember lunch and learns on this topic, and I just wanted to tell them "You're literally making shit up"

Ответ для Demetri
Jordan Nafa

Fun Fact: The only time I've ever failed the technical screen portion of a job interview was back in October 2022 because my answer to a question about approaches to dealing with class imbalance was cost-sensitive learning, threshold tuning, or ensemble methods; definitely not SMOTE

Robin Blythe

My friend did his PhD on SMOTE and found it was quite a useful baseline for generating synthetic data, rather than as a class balancing thing, so it's only useless for the thing everyone thinks it's meant for!