A Comparison of Methods for Treatment Assignment with an Application to Playlist Generation
Authors: Fernández-Loría, Carlos; Provost, Foster; Anderton, Jesse; Carterette, Benjamin; Chandar, Praveen
Journal: Information Systems Research (2023)
<jats:p> This study presents a systematic comparison of methods for individual treatment assignment. We group the various methods proposed in the literature into three general classes of algorithms (or metalearners): learning models to predict outcomes (the O-learner), learning models to predict causal effects (the E-learner), and learning models to predict optimal treatment assignments (the A-learner). We discuss how the metalearners differ in their level of generality and their objective function, which has critical implications for modeling and decision making. Notably, we demonstrate that optimizing for the prediction of outcomes or causal effects is not the same as optimizing for treatment assignments, suggesting that, in general, the A-learner should lead to better treatment assignments than the other metalearners. We then compare the metalearners in the context of choosing, for each user, the best algorithm for playlist generation in order to optimize engagement. This is the first comparison of the three different metalearners on a real-world application at scale (based on more than half a billion treatment assignments). In addition to supporting our analytical findings, th…