A unified approach to derive optimal finite differences is presented which combines three critical elements for numerical performance especially for multi-scale physical problems, namely, order of accuracy, spectral resolution and stability. The resulting mathematical framework reduces to a minimization problem subjected to equality and inequality constraints. We show that the framework can provide analytical results for optimal schemes and their numerical performance including, for example, the type of errors that appear for spectrally optimal schemes. By coupling the problem in this unified framework, one can effectively decouple the requirements for order of accuracy and spectral resolution, for example. Alternatively, we show how the framework exposes the tradeoffs between e.g. accuracy and stability and how this can be used to construct explicit schemes that remain stable with very large time steps. We also show how spectrally optimal schemes only bias odd-order derivatives to remain stable, at the expense of accuracy, while leaving even-order derivatives with symmetric coefficients. Schemes constructed within this framework are tested for diverse model problems with an emphasis on reproducing the physics accurately.