We are concerned with describing neural network model selection and training in the language of differential geometry and topology. Such a formulation is conceptually useful and provides a means to describe training a neural network as generating a sequence in the parameter space which converges to a point in the intersection of embedded submanifolds. Though we have only recently begun this work, we hope that such a formulation may ultimately yield a framework for describing a neural network's predictive ability for observations which differ substantially from the data with which the model was trained.