We present an approach to linear prediction parameter estimation and model order selection that utilises Bayesian inference. The addition of a penalty term, or regulariser, to the conventional linear prediction data error term prior to minimising it facilitates the estimation of the maximum a posteriori parameters. A direct equivalence can be drawn between the type of regulariser used and the prior assumptions regarding the solution to a linear prediction problem. Mackay's Bayesian Evidence framework is used for the estimation of linear prediction parameters that reflect the role that prior assumptions play during the analysis of a speech segment. Quadratic regularisers are utilised to parametrise speech signals and the results are demonstrated with formant tracking and analysis-synthesis examples.