This paper will present the Pitch-Orthogonal LPC speech coding technique. Pitch-Orthogonal LPC codes the residual in a vector excitation speech coder (VXC ) using the projection of the target vector into each of a series of "codevector planes," where each plane is defined by a pair of vectors, the code and the pitch, which have been transformed to make their optimum gain functions independent. We will show how, using the "backward-transform" structure, POLPC can be used to take advantage of the low complexity offered by sparse, lattice, sparse-delta, and other efficient codebook structures in a way that standard simultaneous gain optimization VXC (SGOC) can't. Finally, we will show how this same backward-transform structure can be used with multi-stage coding to create low-complexity, jointly optimized multi-stage VXC speech coders, extending the dimension of coding accuracy by extending the codevector plane into a higher-dimensional hyperplane. Simulation results will be given which show that backward-transform POLPC provides quality identical to that of SGOC, at a computational complexity which is less than that of SGOC by a factor roughly proportional to the number of codevectors.