Testing the association between SNP effects and a response is a common task. Such tests are often carried out through kernel machine methods such as SKAT. However, these testing procedures assume a normally distributed response, which is often violated. Other robust procedures such as QRKM restrict choice of loss function and only allow inference on conditional quantiles. We propose a general and robust kernel association test with flexible choice of loss function, no distributional assumptions, and has SKAT and QRKM as special cases. We evaluate our proposed robust test (RobKAT) across various data distributions through a simulation study. Across all distributions in our study, RobKAT controlled type I error and shows reasonable power. Our robust test has similar or greater power than SKAT in all distributional settings. Finally, we apply our robust kernel association test on data from the CATIE clinical trial to detect associations between genes in the MHC gene region and herpesvirus antibody levels in schizophrenia patients. Our robust test detected significant association with four genes (HST1H2BJ, MHC, POM12L2, and SLC17A1), three of which were undetected by SKAT.