Statistical Learning with a Nuisance Component (Extended Abstract)

1 Introduction

Predictive models based on modern machine learning methods are becoming increasingly widespread in policy making, with applications in health care, education, law enforcement and business decision-making. Most problems that arise in policy making are not pure prediction problems, but rather have a causal nature, such as attempting to predict counterfactual outcomes for different interventions or optimizing policies over such interventions. It is important to address the causal nature of these problems and build models that have a causal interpretation.

A common paradigm in the search of causality is that to estimate a model with a causal interpretation from observational data-e.g. data not collected via some randomized trial or via a known treatment policy-one typically needs to estimate many other quantities that are not of primary interest, but that can be used to de-bias a purely predictive ML model by formulating an appropriate loss. Examples of such nuisance parameters include the propensity for taking an action under the current policy, which can be used to form unbiased estimates for the reward of new policies, but is typically unknown in datasets that do not come from controlled experiments.

To make matters more concrete, let us walk through an example for which certain variants have been well-studied in machine learning (e.g., [Dud´ık et al., 2011; Swaminathan and Joachims, 2015; Nie and Wager, 2017; Kallus and Zhou, 2018]). Suppose a decision maker wants to estimate the causal effect of some treatment T ∈ {0, 1} on an outcome Y as a function of a set of observable features X; the causal effect will be denoted as θ(X). Typically, one has access to data consisting of tuples (Xi, Ti, Yi), where Xi is the observed feature for sample i, Ti is the treatment taken, and Yi is the observed outcome. Such settings are often referred to as having bandit feedback, since we only observe the outcome for the treatment that was chosen. Due to the bandit nature of the problem, one needs to create unbiased estimates of the unobserved outcome. A standard approach is to use the so-called doubly-robust formula, which is a combination of direct regression and inverse propensity scoring: if we let Y (t) i denote the potential outcome from treatment t in sample i, and let m(t) 0 (xi) ∶= E[Y (t) i ∣ xi] and p(t) 0 ∶= E[1{T = t} ∣ xi], then the following is an unbiased estimator for each potential outcome:

Given such an estimator, we can estimate the treatment effect by running a regression between the unbiased estimates and the features, i.e. solve minθ∈Θn ∑i (ˆ Y (1) - ˆ Y (0) - θ(Xi))2 over some model class Θn. In the population limit, with infinite samples, this corresponds to finding a model θ(x) that minimizes the population risk E[(ˆ Y (1) i - ˆ Y (0) i - θ(X))2 ]. Similarly, if one is interested in policy optimization rather than estimating treatment effects, they could use these unbiased estimates to solve minθ∈Θn ∑i( ˆ Y (0) i - ˆ Y (1) i )⋅θ(Xi) over a policy space Θn of functions mapping features to {0, 1}. When dealing with observational data, the functions m0 and p0 are not known, and must be estimated if we wish to evaluate the proxy labels ˆ Y (t) .The goal of the learner is to find a model θ that achieves good population risk when evaluated at the true nuisance functions as opposed to the estimated, since only then does the model have a causal interpretation.

This phenomenon is ubiquitous in causal inference and motivates us to formulate the abstract problem of statistical learning with a nuisance component: Given n i.i.d. examples from a distribution D, a learner is interested in finding a target parameter ̂ θn ∈ Θn so as to minimize a population risk function LD ∶ Θn × Gn → R. The population risk depends not just on the target parameter, but also on a nuisance parameter whose true value g0 ∈ Gn is unknown to the learner. The goal of the learner is to produce an estimate that has small excess risk when evaluated at the unknown true nuisance model:

Depending on the application, such an excess risk bound can take different interpretations. For many settings, such as treatment effect estimation, it is closely related to mean squared error, while in policy optimization problems it may correspond to regret. Following the tradition of statistical learning theory [Vapnik, 1995; Bousquet et al., 2004], we make excess risk the primary focus of our work, independent of the interpretation. We develop algorithms and analysis tools that generically address (2), then apply these tools to a number of applications of interest.

The problem of statistical learning with a nuisance component is strongly connected to the problem of semiparametric inference [Robinson, 1988; Kosorok, 2008], where a true parameter θ0 is the minimizer of a population risk that depends on unknown nuisance components. Our paper builds on a growing body of results on "double" or "debiased" machine learning in statistics and econometrics literature [Chernozhukov et al., 2017; Chernozhukov et al., 2018a; Chernozhukov et al., 2018c; Chernozhukov et al., 2018b] for addressing semiparametric inference problems. This line of research has focused on providing so-called "√n-consistent and asymptotically normal" estimates when the target parameter θ0 is low-dimensional and nuisance parameters belong to a nonparametric class. Unlike the semiparametric inference problem, statistical learning with a nuisance component does not require a well-specified model, nor a unique minimizer of

Algorithm 1 Two-Stage Estimation with Sample Splitting

Input: Sample set S = z1, . . . , zn

• 1: Split S into subsets S(1) = z1, . . . , z⌊n/2⌋, S(2) = S∖S(1) .
• 2: Let ̂gn be the output of Alg(Gn,S(1) ).
• 3: return ̂ θn, the output of Alg(Θn,S(2) ; ̂gn).

the population risk. Moreover, we do not ask for parameter recovery and asymptotic inference (i.e. asymptotically valid confidence intervals). Rather, we are content with an excess risk bound, regardless of whether there is an underlying true parameter to be identified. As a consequence, we provide guarantees even when the target parameter belongs to a large, potentially nonparametric class.

The case where the target parameter belongs to an arbitrary class has not been addressed at the level of generality we consider in the present work, but we mention some prior work that goes beyond the low-dimensional/parametric setup for special cases. [Athey and Wager, 2017] and [Zhou et al., 2018] give guarantees based on metric entropy of the target class for the specific problem of treatment policy learning. For estimation of treatment effects, various nonparametric classes have been used for the target class on a fairly cases by case basis, including kernels [Nie and Wager, 2017], random forests [Athey et al., 2019; Oprescu et al., 2019; Friedberg et al., 2018], and high-dimensional linear models [Chernozhukov et al., 2017; Chernozhukov et al., 2018b]. Our work unifies several of these papers into a single framework and our general results have implications and improve upon each of these directions.

Our approach is to reduce the problem of statistical learning with a nuisance component to the standard formulation of statistical learning. Rather than directly analyzing particular algorithms and models from machine learning, such as regularized regression, gradient boosting, or neural network estimation, we assume a black-box guarantee for the excess in the case where a nuisance value g ∈ Gn is known. In particular, our main theorem only asks for the existence of an algorithm Alg(Θn, S ; g) that for any given nuisance parameter g and data set S, achieves good excess risk with respect to the population risk LD(θ, g), i.e. with probability 1 - δ:

Likewise, we assume the existence of a black-box algorithm Alg(Gn, S) to estimate the nuisance component g0 from the data, with the required estimation guarantee varying from problem to problem.

Given access to the two black-box algorithms, we analyze a sample-splitting based simple meta-algorithm for statistical learning with a nuisance component presented as Algorithm 1. We can now state the main question addressed in this paper: When is the excess risk achieved by sample splitting robust to nuisance component estimation error?

In more technical terms, we seek to understand when the two-stage sample splitting estimation algorithm achieves an excess risk bound with respect to g0, in spite of error in the estimator ̂gn output by the first-stage algorithm. Robustness to nuisance estimation error allows the learner to use more complex models for nuisance estimation and-under certain conditions on the complexity of the target and nuisance model classes-to learn a target parameter whose error is, up to lower order terms, as good as if the learner had known the true nuisance model. Such a guarantee is typically referred to as achieving an oracle rate in semiparametric inference.

1.1 Overview of Results

We show that Neyman orthogonality, which has been used to prove oracle rates for inference in semiparametric models [Neyman, 1959; Neyman, 1979; Chernozhukov et al., 2018a; Chernozhukov et al., 2018b], is key to providing oracle rates for statistical learning with a nuisance component. We prove that if the population risk satisfies a functional analogue of Neyman orthogonality, then the estimation error of ̂gn has a second order impact on the overall excess risk (relative to g0) achieved by ̂ θn. To gain some intuition, Neyman orthogonality is weaker condition than double robustness, albeit similar in flavor, (see e.g. [Chernozhukov et al., 2016]) and is satisfied by both the treatment effect loss and the policy learning loss described in the introduction. In more detail, our extension of the Neyman orthogonality condition asks that a cross-functional derivative of the loss vanish to zero, when evaluated at the optimal target and nuisance parameter. Prior work on the classical notion of Neyman orthogonality provides a number of means through which to construct orthogonal losses whenever certain moment conditions are satisfied by the data generating process [Chernozhukov et al., 2018a; Chernozhukov et al., 2016; Chernozhukov et al., 2018b]. Indeed, orthogonal losses can be constructed in settings including treatment effect estimation, policy learning, missing data problems, estimation of structural econometric problems and game theoretic models.

We identify two regimes of excess risk behavior:

1. When the population risk is strongly convex with respect to the prediction of the target model (e.g. the treatment effect estimation loss), then typically so-called fast rates (e.g. rates of order of O(1/n) for parametric classes) are achievable had we known the true nuisance model. Letting RGn denote the estimation error of the nuisance component (root-mean-squared prediction error for most of our settings), then in the fast rate setting we show that orthogonality implies that the first stage error has an impact on the excess risk of the order of R4Gn (e.g. n-1/4 RMSE rates for the nuisance suffice when the target is parametric).
2. Absent any assumption on the convexity of the population risk (e.g. the treatment policy optimization loss), then typically slow rates (e.g. rates of order O(1/√n) for parametric classes) are achievable had we known the true nuisance model. In this case the impact is of nuisance estimation error is of the order R2Gn so, once again, n-1/4 RMSE rates for the nuisance suffice when the target is parametric.

To extend the sufficient conditions above to arbitrary classes, we give conditions on the relative complexity of the target and nuisance classes-quantified via metric entropy-under which the sample splitting meta-algorithm achieves oracle rates (assuming the two black-box estimation algorithms are appropriately instantiated). This allows us to extend several prior works beyond the parametric regime to complex nonparametric target classes. Our technical results extends the works of [Yang and Barron, 1999; Rakhlin et al., 2017], which provide minimax optimal rates without nuisance components and utilize the technique of aggregation in designing optimal algorithms.

The flexibility of our approach allows us to instantiate the framework with any machine learning model and algorithm of interest for both nuisance and target model estimation, and to utilize the vast literature on generalization bounds in machine learning to establish data-dependent and dimensionindependent rates for several classes of interests. For instance, our approach allows us to use recent work on size-independent generalization error of neural networks. We obtain sharp guarantees for these specific model classes and more as a consequence of a new analysis for empirical risk minimization with plug-in estimation of nuisance parameters in the presence of orthogonality. Our results on plugin empirical risk minimization extend the local Rademacher complexity analysis of generalization bounds [Koltchinskii and Panchenko, 2000; Bartlett et al., 2005], to account for the impact of the nuisance error. In the slow rate regime we also give a new analysis of variance-penalized empirical risk minimization, which allows us to recover and extend several prior results in the literature on policy learning. Our result improves upon the variancepenalized risk minimization approach of [Maurer and Pontil, 2009] by replacing the dependence on the metric entropy at a fixed approximation level with the critical radius, which is related to the entropy integral.

As a consequence of focusing on excess risk, we obtain oracle rates under weaker assumptions on the data generating process than in previous works. Notably, we obtain guarantees even when the target model is misspecified and the target parameters are not identifiable. For instance, for sparse highdimensional linear classes, we obtain optimal prediction rates with no restricted eigenvalue assumptions. We highlight the applicability of our results to four settings of primary importance in the literature: 1) estimation of heterogeneous treatment effects from observational data, 2) offline policy optimization, 3) domain adaptation, 4) learning with missing data. For each of these applications, our general theorems allow for the use of arbitrary estimators for the nuisance and target model classes and provide robustness to the nuisance estimation error.