Multivariate matching in observational studies tends to view covariate differences symmetrically. If matching is correcting an imbalance in age, such that treated subjects are typically older than controls, then the situation in need of correction is asymmetric. Correcting the bias may be easier if matching tries to avoid the typical case that creates the bias. We describe several easily used, asymmetric, directional penalties and illustrate how they can improve covariate balance in a matched sample. The investigator starts with a matched sample built in a conventional way, then diagnoses residual covariate imbalances in need of reduction, and achieves the needed reduction by slightly altering the distance matrix with directional penalties, creating a new matched sample. Unlike penalties commonly used in matching, a directional penalty can go too far, reversing the direction of the bias rather than reducing the bias, so the magnitude of the directional penalty matters and may need adjustment. We also explore the connection between directional penalties and a widely used technique in integer programming, namely Lagrangian relaxation of problematic linear side constraints.