An introduction to pretraining, and to the R package:
More examples of pretraining and modeling:
Pretraining is a popular and powerful paradigm in machine learning. For example, suppose we have a modest-sized dataset of images of cats and dogs, and we want to fit a deep neural network to classify them. With pretraining, we start with a neural network trained on a large corpus of images, consisting of not just cats and dogs but hundreds of other image types. Then we fix all of the network weights except for the top layer(s) and train (or "fine tune") those weights on our dataset. This often results in dramatically better performance than the network trained solely on our smaller dataset.
Why should this work? By training our neural network first with a large, broad set of images, we learn general features like edges, textures, fur, eyes and so on. Then, during the fine tuning stage, we learn how to use these features to distinguish between cats and dogs.
Can pretraining help the lasso? Yes!
Here we present a framework for the lasso in which an overall model is fit to a large set of data, and then fine-tuned to a specific task. This latter dataset can be a subset of the original dataset, but does not need to be. Pretraining the lasso has a wide variety of applications, including stratified models, multinomial targets, multi-response models, conditional average treatment estimation and even gradient boosting.
This package fits pretrained generalized linear models for: (1) data with grouped observations, (2) data without grouped observations, but with multinomial responses, (3) data with multiple Gaussian responses and (4) time series data (data with repeated measurements over time).
Documentation and examples are available as vignettes within this package, and can be accessed through the "Articles" tab on this page. The vignettes also include examples of pretraining for settings not yet supported by this package, including conditional average treatment effect estimation and unsupervised pretraining.
Details of pretraining may be found in Craig et al. (2026).
All model fitting in this package is done with cv.glmnet, and our syntax closely follows that of the glmnet package (2010).
To install the development version of this package from GitHub:
# install.packages("remotes")
# remotes::install_github("erincr/ptLasso")
install.packages("ptLasso", repos = c('https://erincr.r-universe.dev', 'https://cloud.r-project.org'))
If you find a bug or have a feature request, please open a new issue.
Craig et al. "Pretraining and the lasso." Journal of the Royal Statistical Society Series B: Statistical Methodology 88.1 (2026): 261-281.
Friedman, Hastie, and Tibshirani. 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent." Journal of Statistical Software, Articles 33 (1): 1–22.