The package implements a collection of Generalized Elastic Net (GELnet) solvers, as outlined in the following publication: https://doi.org/10.1371/journal.pcbi.1004790 This vignette covers the basic usage of the gelnet package.
Models are assembled from small, independent pieces that are combined
with +, much like a ggplot2 plot is built up from
geoms, scales and themes added to a base ggplot() call. A
model definition always starts with gelnet(X), and is then
extended with a task (what is being predicted) and one or more
regularizers (how the model is penalized). The resulting definition is
handed to gelnet_train() to fit the model. Nearly all of
the regularization pieces have reasonable defaults, so the interface
stays simple when the extra flexibility isn’t needed.
Let \(X\) be a samples-by-features data matrix and \(y\) be a column vector of labels. Typically, \(X\) and \(y\) are determined by the prediction task at hand, but for the purposes of this tutorial, we are going to generate them randomly:
Let’s further assume that we are given a feature-feature relationship matrix \(A\). If working with genomic features, \(A\) might be the adjacency of a gene-gene interaction network. Again, we are going to generate a random matrix for the purposes of this tutorial:
We are now going to utilize the GELnet toolkit to learn a linear regression model, such that the model weights are more similar for the features that share an interaction on \(A\). As discussed in the manuscript, this can be achieved by formulating a feature-feature penalty matrix using either the graph Laplacian or \((I-D)\), where \(D\) is the graph diffusion matrix and \(I\) is the identity matrix. The gelnet package provides a function to compute the graph Laplacian from the adjacency. Here, we utilize the normalized Laplacian to keep the penalty term on the same scale as the traditional ridge regression:
A model definition is assembled by starting with the data matrix,
adding a task (model_lin() for linear regression) and the
L1- and L2-norm regularizers:
where we set the L1-norm and L2-norm penalties to 0.1 and 1, respectively. The terms can be added in any order, and the definition can be built up incrementally.
The model is trained by handing the definition to
gelnet_train():
## Training a linear regression model
## Initial objective value: 0.470201
## Final value is 0.36728 after iteration 6The response for new samples is computed via the dot product with the weights:
## [,1]
## [1,] -0.067236633
## [2,] -0.447742503
## [3,] -0.175409128
## [4,] -0.080966071
## [5,] 0.059396337
## [6,] -0.003023697
## [7,] 0.045376551
## [8,] -0.035755396
## [9,] 0.045334890
## [10,] 0.193699129Linear regression is one of the three types of prediction problems supported by the package. The other two are binary logistic regression and one-class logistic regression. The latter is outlined in the following paper: https://doi.org/10.1142/9789814749411_0037
The package recognizes the problem type based on which task is added
to the model definition: model_lin() for linear regression,
model_blr() for binary logistic regression, and
model_oclr() for one-class logistic regression. To train a
binary predictor, we have to provide \(y\) as a two-level factor, where the first
level is treated as the positive class.
y <- factor( y > 0, levels=c(TRUE,FALSE) )
mdef2 <- gelnet(X) + model_blr(y) + rglz_L1(0.1) + rglz_L2(1, P = L)
model2 <- gelnet_train( mdef2 )## Training a binary logistic regression model
## Initial objective value: 0.693147
## Final value is 0.668075 after iteration 3If we were to score the training data using this model, we can observe that the positive samples are receiving higher scores than the negative ones
## scores labels
## 1 0.154738786 TRUE
## 2 -0.245565853 TRUE
## 3 -0.368831848 FALSE
## 4 0.056206655 TRUE
## 5 0.187939317 TRUE
## 6 -0.409651293 FALSE
## 7 -0.398100244 FALSE
## 8 -0.001487814 TRUE
## 9 -0.265028201 FALSE
## 10 -0.010336384 TRUE
## 11 -0.659898556 FALSE
## 12 -0.101106056 FALSE
## 13 0.031961810 TRUE
## 14 -0.522069049 FALSE
## 15 -0.287473828 FALSE
## 16 -0.182250836 FALSE
## 17 -0.049072693 TRUE
## 18 -0.307122197 FALSE
## 19 -0.307199065 FALSE
## 20 -0.378781354 TRUEHowever, if there is class imbalance, the scores will tend to be
skewed towards the class with more samples. This can be addressed by
using the balanced argument of
model_blr():
mdef2bal <- gelnet(X) + model_blr(y, balanced=TRUE) + rglz_L1(0.1) + rglz_L2(1, P = L)
model2bal <- gelnet_train( mdef2bal )## Training a binary logistic regression model
## Initial objective value: 0.693147
## Final value is 0.672422 after iteration 2## scores labels
## 1 0.349542979 TRUE
## 2 -0.057392596 TRUE
## 3 -0.181550093 FALSE
## 4 0.245480462 TRUE
## 5 0.377568394 TRUE
## 6 -0.229275554 FALSE
## 7 -0.218530187 FALSE
## 8 0.186019193 TRUE
## 9 -0.076840125 FALSE
## 10 0.178615563 TRUE
## 11 -0.485874547 FALSE
## 12 0.083294274 FALSE
## 13 0.221138925 TRUE
## 14 -0.340590032 FALSE
## 15 -0.099195323 FALSE
## 16 0.006499425 FALSE
## 17 0.142318441 TRUE
## 18 -0.115978434 FALSE
## 19 -0.124052133 FALSE
## 20 -0.186996683 TRUETraditionally, the loss function for logistic regression is averaged over \(n\), the number of samples. This causes every sample to make the same contribution to the loss, which is what causes the skew towards the larger class. By using the balanced flag, the problem is reformulated slightly such that the loss is averaged over the positive and negative samples separately, and then the mean of both averages is used as the overall loss.
Finally, we can build a one-class logistic regression model using
just the positive samples. To train a one-class model we add
model_oclr(), which requires no labels:
j <- which( y == TRUE )
mdef1 <- gelnet(X[j,]) + model_oclr() + rglz_L1(0.1) + rglz_L2(1, P = L)
model1 <- gelnet_train( mdef1 )## Training a one-class model
## Initial objective value: 0.693147The model can now be used as a detector that recognizes the positive samples
## scores labels
## 1 0.52750494 TRUE
## 2 -0.01097148 TRUE
## 3 0.33395692 FALSE
## 4 0.66712728 TRUE
## 5 0.36844863 TRUE
## 6 -0.46454676 FALSE
## 7 0.26961477 FALSE
## 8 0.28740877 TRUE
## 9 0.53413741 FALSE
## 10 0.52913738 TRUE
## 11 -0.62328908 FALSE
## 12 -0.12038232 FALSE
## 13 0.39201546 TRUE
## 14 -0.24697369 FALSE
## 15 -0.08210189 FALSE
## 16 0.35184041 FALSE
## 17 0.74031258 TRUE
## 18 0.73936978 FALSE
## 19 0.07631607 FALSE
## 20 0.59820420 TRUE