A Bootstrapping Approach to Adjust for Unequal Distribution of Compounds in the Chemical Space When Assessing the Performance of Three Toxicity Prediction Programs

• Activity Number: 90
• Type: Contributed
• Date/Time: Monday, August 9, 2004 : 9:00 AM to 10:50 AM
• Sponsor: Biopharmaceutical Section
• Author(s): Robert H. Gallavan, Jr*+ and Richard Mueller
• Companies: Pfizer Inc. and Pfizer Inc.
• Keywords: bootstrap ; in silico ; toxicology ; DEREK ; CASETOX ; TOPKAT

Abstract

This work was part of a study to evaluate the ability of three computer programs (TOPKAT, DEREK for Windows, and CASETOX) to predict mutagenicity for 520 drug candidates. Sensitivity, specificity, and accuracy were determined for each program based on the predicted and observed results of the Ames test. Initial results showed a marked difference in sensitivity (DEREK: 28 %; CASETOX: 50 %; TOPKAT 63 %). To determine if the results were skewed by an unequal distribution of compounds throughout the chemical space, the drug candidates were assigned to one of 188 classes based on chemical structure and 1,000 datasets of 188 compounds were generated by randomly selecting one compound from each class. The performance of each program was evaluated as before for each of these datasets and the median and upper and lower 95 % confidence intervals for sensitivity, specificity, and accuracy were determined using the empirical distribution functions. The median sensitivity of DEREK under these conditions (52 %) was within the 95 % confidence intervals for CASETOX and TOPKAT [(50 %, 69 %) for both], indicating that the three programs performed similarly, though poorly, across the chemical space.