To adapt the large number of parameters in a speech recognition acoustic model with a small amount of data, some notion of parameter dependence is needed. We present a dependence model to relate parameters in a parsimonious framework using a Gaussian multiscale process defined by the evolution of a linear stochastic dynamical system on a tree. To adapt all classes from all adaptation data, we formulate adaptation as optimal smoothing of the tree process. This approach is used to adapt two types of models: Gaussians, and Gaussian processes (segment models) characterized by a polynomial mean trajectory. Recognition results presented on the Switchboard corpus show improvements in supervised and unsupervised modes.