Add polynomial regression learner

[imanechadli2003]

Summary

This PR adds a polynomial regression learner for mlr3extralearners.

• Learner ID: regr.polynomial
• Package used: stats (base R), using stats::lm() with stats::poly()
• Supported task type: regression
• Hyperparameters:
  • degree (integer, default = 2): degree of the polynomial
  • raw (logical, default = FALSE): whether to use raw polynomials

Files added

• R/learner_stats_regr_polynomial.R
• tests/testthat/test_stats_regr_polynomial.R
• tests/testthat/test_paramtest_stats_regr_polynomial.R

[imanechadli2003]

@tdhock Could you please review this polynomial regression learner? Thank you!

[imanechadli2003]

@tdhock Pouvez vous vérifier ça svp ?

[tdhock]

Hi @imanechadli2003 can you please add a comment here with the code and result figure for a benchmark that shows your learner with baselines? (featureless, cv_glmnet, nearest neighbors)

[imanechadli2003]

@tdhock Hi! Here is the benchmark code and result figure comparing regr.polynomial
with the baseline learners on two datasets (mtcars and airquality) with 5-fold cross-validation.
The full results are also in my TP9 report submitted on Moodle.

Code

library(mlr3)
library(mlr3learners)
library(mlr3tuning)
library(mlr3extralearners)
library(ggplot2)

tasks = list(
  tsk("mtcars"),
  as_task_regr(airquality[complete.cases(airquality),],
               target = "Ozone", id = "airquality")
)

lrn_poly = lrn("regr.polynomial", degree = 2L)
lrn_poly$id = "poly_regr"

lrn_knn = auto_tuner(
  tuner = tnr("grid_search", resolution = 10),
  learner = lrn("regr.kknn", k = to_tune(1L, 10L)),
  resampling = rsmp("cv", folds = 3),
  measure = msr("regr.mse")
)
lrn_knn$id = "kknn_tuned"

design = benchmark_grid(
  tasks = tasks,
  learners = list(lrn_poly, lrn("regr.featureless"),
                  lrn("regr.cv_glmnet"), lrn_knn),
  resamplings = rsmp("cv", folds = 5)
)

bmr = benchmark(design)
results = bmr$score(msr("regr.mse"))

ggplot(results, aes(x = regr.mse, y = learner_id)) +
  geom_point() +
  facet_grid(task_id ~ ., scales = "free_x") +
  labs(x = "MSE", y = "Learner")

Result

[tdhock]

poly_reg is worse than featureless in mtcars?
Can you try to fix that by tuning the polynomial degree? (auto_tuner)
And it is hard to see in that figure. (vertical panels with shared X axis between data sets)
Can you please use horizontal panels instead, facet_grid(. ~ data_set, scales="free") ?

[imanechadli2003]

@tdhock Hi , here is the updated benchmark with:

• polynomial degree tuned automatically using auto_tuner (degree from 1 to 5)
• horizontal panels as requested

poly_tuned is now better than featureless on both datasets.
The outlier on mtcars is likely due to the small dataset size (32 observations).

[tdhock]

can you please edit the first comment of the PR, add "Closes #420" to indicate that merging this PR provides the functionality asked for in that issue?
can you please edit your benchmark code to start with installation?

remotes::install_github("imanechadli2003/mlr3extralearners@polynomial-regression-learner")

can you please convert this PR from draft to regular PR to indicate that it is ready for review?