In the clinical development of cancer immunotherapies and targeted therapies, the proportional hazard (PH) assumption used in the power calculation for time-to-event endpoints often does not hold. For example, due to the delayed and (for some patients) durable antitumor effect on cancer cells induced by immunotherapies, the survival curves of a randomized controlled study may take a while to separate and the curve for the immunotherapy agent may have a long and flat tail. Therefore, the log-rank test may lose power, and the interpretation of the hazard ratio (HR) from the standard Cox regression model is not straightforward. Kaplan-meier (KM) based methods such as the restricted mean survival time (RMST) and weighted KM-based tests are interesting alternative methods for statistical inference that do not rely on the PH assumption. The RMST is an appealing statistical measure to quantify treatment benefit in a clinically meaningful and interpretable manner. In this paper, we give an overview of these methods and present a simulation study to compare the performance of the KM-based methods with the HR/log-rank test under various non-PH patterns.