We consider one- and two-dimensional Ising models with varying interaction ranges. Using matrix product state techniques, we study the  dynamics of these systems and show a direct connection between the type of lowest-energy quasiparticles in the spectrum of the quench  Hamiltonian and the type of nonanalyticities occuring in the Loschmidt return rate, a dynamical analog of the free energy. Our results also  show a clear connection between the type of nonanalyticities and the phase of the long-time steady state in addition to how the order parameter decays at intermediate times. In particular, we discuss anomalous nonanalyticities that occur with no underlying local signature in the order parameter dynamics, unlike the traditional regular nonanalyticities that always correspond to zero crossings of the order parameter. Moreover, we demonstrate how dynamical quantum phase transitions can be used to extract the equilibrium physics of the model from short-time dynamics.