Title: Markovian Interference and the Differences-in-Qs Estimator
Abstract: We consider experiments in dynamical systems where interventions on some experimental units impact other units through a limiting constraint (such as a limited inventory). Despite its importance, practical estimators for this `Markovian' interference problem are largely heuristic in nature, and their bias is not well understood. We formalize the problem of inference in such experiments as one of policy evaluation. Typical off-policy estimators apparently incur a large penalty in variance relative to state-of-the-art heuristics. We introduce the Differences-In-Q's (DQ) estimator. We show that the DQ estimator can in general have exponentially smaller variance than unbiased off-policy evaluation. At the same time, its bias is second order in the impact of the intervention. This yields a striking bias-variance tradeoff so that the DQ estimator effectively dominates state-of-the-art alternatives. Along the way, we develop a Laurent expansion for average cost dynamic programming that is of general interest. DQ has been implemented at scale on the world’s largest short-form video sharing platform; we discuss our experience with that application. Joint work with Andrew Li, Tianyi Peng and Andy Zheng.
Paper: https://arxiv.org/abs/2206.02371 (or the conference version: https://proceedings.neurips.cc/paper_files/paper/2022/file/03a9a9c1e15850439653bb971a4ad4b3-Paper-Conference.pdf) and a real-world application: https://dl.acm.org/doi/abs/10.1145/3604915.3608808.
Bio: Vivek is interested in the development of new methodologies and applications for large scale dynamic optimization. He received his Ph.D. in Electrical Engineering from Stanford University in 2007 and is the Patrick J. McGovern (1959) Professor at MIT. Vivek is a recipient of an INFORMS MSOM Student Paper Prize (2006), an INFORMS JFIG paper prize (2009, 2011), the NSF CAREER award (2011), MIT Sloan’s Outstanding Teacher award (2013), the INFORMS Simulation Society Best Publication Award (2014), the INFORMS Pricing and Revenue Management Best Publication Award (2015), the INFORMS MSOM Best Publication award in Management Science (2016), the MSOM Young Scholar Prize and the MIT-wide Jamieson Prize for Excellence in Teaching (2020). His practice based work has won the Wagner prize (2022) and has been judged a finalist for the Pierskalla Award (2011), the Gary L. Lilien ISMS-MSI Marketing Practice Prize (2016) and the Wagner Prize (2018). Vivek’s doctoral advisees have on various occasions won the Nicholson, MSOM, APS and RMP student paper prizes. Outside of academia, Vivek was co-founder/CTO at Celect (2014-19; acquired by Nike) and was a corresponding author of the Nature Communications paper underlying the technology at Seer (IPO in 2020). He serves on several technology startup advisory boards and has worked in various capacities in quantitive finance and private equity.
