We consider interest rate models of Heath-Jarrow-Morton type, where the forward rates are driven by a multi-dimensional Wiener process, and where the volatility is allowed to be an arbitrary smooth functional of the present forward rate curve. Using ideas from differential geometry as well as from systems and control theory, we investigate when the forward rate process can be realized by a finite dimensional Markovian state space model, and we give general necessary and sufficient conditions, in terms of the volatility structure, for the existence of a finite dimensional realization. A number of concrete applications are given, and most previously known realization results for time homogenous Wiener driven models are included and extended. As a special case we give a general and easily applicable necessary and sufficient condition for when the induced short rate is a Markov process. In particular we show that the only forward rate models, with short rate dependent volatility structures, which generically give rise to a Markovian short rate are the affine ones. These models are thus the only generic short rate models from a forward rate point of view.