Online Program Home
My Program

Abstract Details

Activity Number: 499
Type: Contributed
Date/Time: Wednesday, August 3, 2016 : 8:30 AM to 10:20 AM
Sponsor: Section on Statistical Learning and Data Science
Abstract #320306
Title: Data Normalization by Fisher-Yates Transformation
Author(s): Yayan Zhang* and Javier Cabrera and Birol Emir
Companies: Merck and Rutgers University and Pfizer Inc & Columbia University
Keywords: data normalization ; statistical learning ; Fisher-Yates Transformation

Data normalization is a very popular technique to align the scales of measurements coming from different samples in genomic data, test scoring, and in patient reported outcomes, and questionnaire data in social sciences etc. All these data have in common that the instrument used to measure the outcome may modify the scale from sample to sample or test to test. This fact introduces high variability in the data that may observe the signal that would be observed when conducting standard analyses such as modeling or clustering. In order to address this issue the data is transform by a normalization method such as z-scores, linear or nonlinear quantile normalization or other. We show that the traditional transformation methods (linear transformations, z-scores, quantile normalization) are not always adequate to fulfill this task and we propose a new data normalization method applying the Fisher-Yates transformation. We performed a simulation to compare all these methods and to illustrate situation where Fisher-Yates would be more appropriate than the other methods. Finally we illustrate the Fisher-Yates normalization on data from clinical trials and DNA microarrays.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

Copyright © American Statistical Association