For equivalence and non-inferiority testing, defining the margin is often difficult. If the margin is too small, then the test's power will be substantially reduced. If the margin is too large, any claims of equivalence or non-inferiority will be meaningless. While it is generally expected that a margin is to be specified before any data is collected, it remains unclear how defining the margin afterwards will bias one's results. In this work, we discuss whether or not traditional frequentist guarantees about error rates will remain valid when a equivalence or non-inferiority margin is data-dependent. In circumstances for which traditional guarantees no longer hold, we present a number of alternatives.