Two-sample graph comparison arises in neuroscience, e.g., when comparing the brain networks of people from two populations. Standard practices include tests on aggregated univariate measures, which may not fully characterize a graph, or one test for each edge, which does not exploit graph structure. Assuming a latent space model (LSM) for random graphs, we propose an inferential framework that both compares entire graphs and utilizes underlying structure. Graphs in LSM are described by a set of latent positions and a function that maps distance to edge probability. Our method estimates two sets of common latent positions by multidimensional scaling, then constructs a test statistic by comparing interpoint distance matrices. Simulations show that our test has greater power than either the standard practices or a direct comparison of sample mean graphs. We apply our approach to compare the structural brain networks of (a) autistic vs healthy subjects, and (b) young vs old adults.