Emerging infectious diseases are a cause of humanitarian and economic crises across the world. In developing regions, a serious epidemic can result in the collapse of healthcare infrastructure or even the failure of an affected state. The most recent 2013-2015 outbreak of Ebola virus disease in West Africa is an example of such an epidemic. The economic, infrastructure, and human costs of this outbreak provide strong motivation for the examination of adaptive treatment strategies that allocate resources in response to and anticipation of the evolution of an epidemic. We formalize adaptive management of an emerging infectious disease spreading across a set of locations as a treatment regime that maps up-to-date information on the epidemic to a subset of locations identified as high-priority for treatment. Because the disease dynamics are not known at outbreak, an optimal treatment regime must be estimated online. We propose a `safe' variant of Q-learning with Thompson Sampling for the management of an emerging infectious disease based on sampling from a truncated confidence distribution for parameters indexing the Q-function. Theoretical and empirical properties are examined.