Physical activity, such as walking and sitting, is commonly used in biomedical settings either as an outcome or covariate of interest. Researchers have traditionally relied on surveys to quantify activity levels of subjects in both research and clinical settings, but surveys are not objective in nature and have many known limitations, such as recall bias. Smartphones provide an opportunity for unobtrusive objective measurement of physical activity in naturalistic settings, but their data tends to be noisy and needs to be analyzed with care. We explored the potential of using a combination of various smartphone sensors to distinguish between different physical activities. I will describe the method and also demonstrate its application to a surgical cohort. Our results demonstrate the promise of smartphones for activity detection in naturalistic settings, but they also highlight common challenges in this field of research.