List of abstracts and full papers

P Sollich, V Heine, and M T Dove. The Ginzburg interval in soft-mode phase transitions: consequences of the Rigid Unit Mode picture. Journal of Physics: Condensed Matter, 6:3171-3196, 1994.

Abstract

In soft-mode structural phase transitions the Ginzburg temperature interval in which fluctuations and the interactions between them become important is often observed to be small on the scale of the transition temperature. We consider the size of the Ginzburg interval (GI) in framework and 'cogwheel' structures using the concept of 'rigid unit modes'. Such materials, as well as being very displacive, i.e. close to the soft-mode limit, have an extremely anisotropic phonon spectrum. Modelling these two properties with a suitable effective Hamiltonian for the degrees of freedom driving the transition we find that the GI can range from very small to large, depending on the balance between displaciveness and anisotropy. For the two perovskites SrTiO$_3$ and LaAlO$_3$ and the 'cogwheel' structure K$_2$SeO$_4$, we obtain values of the model parameters describing displaciveness and anisotropy from experimentally measured phonon dispersions and find, for the size of the GI, quantitative agreement with experiment. We also estimate typical values for the model parameters and the size of the GI for framework silicates, using quartz and cristobalite as examples. Finally, we use computer simulations to confirm the results of our theoretical analysis over a wider range of model parameters.

Full paper available as Adobe PDF
DOI: 10.1088/0953-8984/6/17/008

P Sollich. Query construction, entropy, and generalization in neural network models. Physical Review E, 49:4637-4651, 1994.

Abstract

We study query construction, algorithms, which aim at improving the generalization ability of systems that learn from examples by choosing optimal, nonredundant training sets. We set up a general probabilistic framework for deriving such algorithms from the requirement of optimizing a suitable objective function; specifically, we consider the objective functions entropy (or information gain) and generalization error. For two learning scenarios, the high-low game and the linear perceptron, we evaluate the generalization performance obtained by applying the corresponding query construction algorithms and compare it to training on random examples. We find qualitative differences between the two scenarios due to the different structure of the underlying rules (nonlinear and ``noninvertible'' versus linear); in particular, for the linear perceptron, random examples lead to the same generalization ability as a sequence of queries in the limit of an infinite number of examples. We also investigate learning algorithms which are ill matched to the learning environment and find that, in this case, minimum entropy queries can in fact yield a lower generalization ability than random examples. Finally, we study the efficiency of single queries and its dependence on the learning history, i.e., on whether the previous training examples were generated randomly or by querying, and the difference between globally and locally optimal query construction.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevE.49.4637

P Sollich. Finite-size effects in learning and generalization in linear perceptrons. Journal of Physics A, 27:7771-7784, 1994.

Abstract

Most properties of learning and generalization in linear perceptrons can be derived from the average response function $G$. We present a method for calculating $G$ using only simple matrix identities and partial differential equations. Using this method, we first rederive the known result for $G$ in the thermodynamic limit of perceptrons of infinite size $N$, which has previously been calculated using replica and diagrammatic methods. We also show explicitly that the response function is self-averaging in the thermodynamic limit. Extensions of our method to more general learning scenarios with anisotropic teacher-space priors, input distributions, and weight-decay terms are discussed. Finally, finite-size effects are considered by calculating the $O(1/N)$ correction to $G$. We verify the result by computer simulations and discuss the consequences for generalization and learning dynamics in linear perceptrons of finite size.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1088/0305-4470/27/23/020

D Barber, D Saad, and P Sollich. Test error fluctuations in finite linear perceptrons. Neural Computation, 7:809-821, 1995.

Abstract

We examine the fluctuations in the test error induced by random, finite, training and test sets for the linear perceptron of input dimension $N$ with a spherically constrained weight vector. This variance enables us to address such issues as the partitioning of a data set into a test and training set. We find that the optimal assignment of the test set size scales with $N^{2/3}$.

P Sollich. Learning unrealizable tasks from minimum entropy queries. Journal of Physics A, 28:6125-6142, 1995.

Abstract

In supervised learning, learning from queries rather than from random examples can improve generalization performance significantly. We study the performance of query learning for unrealizable tasks, where the student cannot learn from the perfectly. As a simple model scenario of this kind, we consider a linear perceptron student learning a general nonlinear perceptron teacher. Two kinds of queries for maximum information gain, i.e. minimum entropy, are investigated: minimum student space entropy (MSSE) queries, which are appropriate if the teacher space is unknown, and minimum teacher space entropy (MTSE) queries, which can be used if the teacher space is assumed to be known, but a student of a simpler form has deliberately been chosen. We find that for MSSE queries, the structure of the student space determines the efficacy of query learning. MTSE queries, on the other hand, which we investigate for the extreme case of a binary perceptron teacher, lead to a higher generalization error than random examples, due to a lack of feedback about the progress of the student in the way queries are selected.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1088/0305-4470/28/21/016

P Sollich. Learning in large linear perceptrons and why the thermodynamic limit is relevant to the real world. In G Tesauro, D S Touretzky, and T K Leen, editors, Advances in Neural Information Processing Systems 7, pages 207-214, Cambridge, MA, 1995. MIT Press.

Abstract

We present a new method for obtaining the response function G and its average $G$ from which most of the properties of learning and generalization in linear perceptrons can be derived. We first rederive the known results for the `thermodynamic limit' of infinite perceptron size N and show explicitly that G is self-averaging in this limit. We then discuss extensions of our method to more general learning scenarios with anisotropic teacher space priors, input distributions, and weight decay terms. Finally, we use our method to calculate the finite N corrections of order 1/N to $G$ and discuss the corresponding finite size effects on generalization and learning dynamics. An important spin-off is the observation that results obtained in the thermodynamic limit are often directly relevant to systems of fairly modest, `real-world' sizes.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich and D Saad. Learning from queries for maximum information gain in imperfectly learnable problems. In G Tesauro, D S Touretzky, and T K Leen, editors, Advances in Neural Information Processing Systems 7, pages 287-294, Cambridge, MA, 1995. MIT Press.

Abstract

In supervised learning, learning from queries rather than from random examples can improve generalization performance significantly. We study the performance of query learning for problems where the student cannot learn the teacher perfectly, which occur frequently in practice. As a prototypical scenario of this kind, we consider a linear perceptron student learning a binary perceptron teacher. Two kinds of queries for maximum information gain, i.e., minimum entropy, are investigated: Minimum student space entropy (MSSE) queries, which are appropriate if the teacher space is unknown, and minimum teacher space entropy (MTSE) queries, which can be used if the teacher space is assumed to be known, but a student of a simpler form has deliberately been chosen. We find that for MSSE queries, the structure of the student space determines the efficacy of query learning, whereas MTSE queries lead to a higher generalization error than random examples, due to a lack of feedback about the progress of the student in the way queries are selected.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Minimum entropy queries for linear students learning nonlinear rules. In Verleysen M, editor, Third European Symposium on Artificial Neural Networks (ESANN'95), Proceedings, pages 217-222, Brussels, 1995. D facto.

Abstract

We study the fundamental question of how query learning performs in imperfectly learnable problems, where the student can only learn to approximate the teacher. Considering as a prototypical scenario a linear perceptron student learning a general nonlinear perceptron teacher, we find that queries for minimum entropy in student space (i.e., maximum information gain) lead to the same improvement in generalization performance as for a noisy linear teacher. Qualitatively, the efficacy of query learning is thus determined by the structure of the student space alone; we speculate that this result holds more generally for minimum student space entropy queries in imperfectly learnable problems.

Full paper available as gzip'ped postscript or Adobe PDF

D Barber, D Saad, and P Sollich. Finite-size effects and optimal test set size in linear perceptrons. Journal of Physics A, 28:1325-1334, 1995.

Abstract

Fluctuations in the test error are important in the learning theory of finite-dimensional systems as they represent how well the test error matches the average test error. By explicitly finding the variance of the test error due to randomness present in both the data set and algorithm for a linear perceptron of dimension $n$, we are able to address such questions as the optimal test set size. Where exact results were not tractable, a good approximation is given to the variance. We find that the optimal test set size possesses a phase transition between linear and 2/3 power-law scaling in the system size $n$.

Full paper available as Adobe PDF
DOI: 10.1088/0305-4470/28/5/018

P Sollich. Learning from minimum entropy queries in a large committee machine. Physical Review E, 53:R2060-R2063, 1996.

Abstract

In supervised learning, the redundancy contained in random examples can be avoided by learning from queries. Using statistical mechanics, we study learning from minimum entropy queries in a large tree-committee machine. The generalization error decreases exponentially with the number of training examples, providing a significant improvement over the algebraic decay for random examples. The connection between entropy and generalization error in multilayer networks is discussed, and a computationally cheap algorithm for constructing queries is suggested and analyzed.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevE.55.811

D Barber, D Saad, and P Sollich. Finite size effects in online learning of multilayer neural networks. Europhysics Letters, 34:151-156, 1996.

Abstract

We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multilayer networks by calculating fluctuations possessed by finite-dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.

DOI: 10.1209/epl/i1996-00431-5

P Sollich and A Krogh. Learning with ensembles: When over-fitting can be useful. In D S Touretzky, M C Mozer, and M E Hasselmo, editors, Advances in Neural Information Processing Systems 8, pages 190-196, Cambridge, MA, 1996. MIT Press.

Abstract

We study the characteristics of learning with ensembles. Solving exactly the simple model of an ensemble of linear students, we find surprisingly rich behaviour. For learning in large ensembles, it is advantageous to use under-regularized students, which actually over-fit the training data. Globally optimal performance can be obtained by choosing the training set sizes of the students appropriately. For smaller ensembles, optimization of the ensemble weights can yield significant improvements in ensemble generalization performance, in particular if the individual students are subject to noise in the training process. Choosing students with a wide range of regularization parameters makes this improvement robust against changes in the unknown level of noise in the training data.

Full paper available as gzip'ped postscript or Adobe PDF

A Krogh and P Sollich. Statistical mechanics of ensemble learning. Physical Review E, 55:811-825, 1997.

Abstract

Within the context of learning a rule from examples, we study the general characteristics of learning, with ensembles. The generalization performance achieved by a simple model ensemble of linear students is calculated exactly in the thermodynamic limit of a large number of input components and shows a surprisingly rich behavior. Our main findings are the following. For learning in large ensembles, it is advantageous to use underregularized students, which actually overfit the training data. Globally optimal generalization performance can be obtained by choosing the training set sizes of the students optimally. For smaller ensembles, optimization of the ensemble weights can yield significant improvements in ensemble generalization performance, in particular if the individual students are subject to noise in the training process. Choosing students with a wide range of regularization parameters makes this improvement robust against changes in the unknown level of corruption of the training data.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevE.55.811

P Sollich and D Barber. Online learning from finite training sets: An analytical case study. In M C Mozer and M I Jordan and T Petsche, editors, Advances in Neural Information Processing Systems 9, pages 274-280, Cambridge, MA, 1997. MIT Press.

Abstract

We analyse online learning from finite training sets at non-infinitesimal learning rates $\eta$. By an extension of statistical mechanics methods, we obtain exact results for the time-dependent generalization error of a linear network with a large number of weights $N$. We find, for example, that for small training sets of size $p$ $\approx$ $N$, larger learning rates can be used without compromising asymptotic generalization performance or convergence speed. Encouragingly, for optimal settings of $\eta$ (and, less importantly, weight decay $\lambda$) at given final learning time, the generalization performance of online learning is essentially as good as that of offline learning.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich, F Lequeux, P Hébraud, and M E Cates. Rheology of soft glassy materials. Physical Review Letters, 78:2020-2023, 1997.

Abstract

We attribute similarities in the rheology of many soft materials (foams, emulsions, slurries, etc.) to the shared features of structural disorder and metastability. A generic model for the mesoscopic dynamics of ``soft glassy matter'' is introduced, with interactions represented by a mean-field noise temperature $x$. We find power law fluid behavior either with ($x<1$) or without ($1<x<2$) a yield stress. For $1<x<2$, both storage and loss modulus vary with frequency as $\omega^{x-1}$, becoming flat near a glass transition ($x=1$). Values of $x$ $\approx$ 1 may result from marginal dynamics as seen in some spin glass models.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevLett.78.2020

P Sollich. Query learning for maximum information gain in a multi-layer neural network. In S W Ellacott, J C Mason, and I J Anderson, editors, Mathematics of Neural Networks: Models, Algorithms and Applications, pages 339-343, Boston, MA, 1997. Kluwer Academic.

Abstract

In supervised learning, the redundancy contained in random examples can be avoided by learning from queries, where training examples are chosen to be maximally informative. Using the tools of statistical mechanics, we analyse query learning in a simple multi-layer network, namely, a large tree-committee machine. The generalization error is found to decrease exponentially with the number of training examples, providing a significant improvement over the slow algebraic decay for random examples. Implications for the connection between information gain and generalization error in multi-layer networks are discussed, and a computationally cheap algorithm for constructing approximate maximum information gain queries is suggested and analysed.

Full paper available as gzip'ped postscript or Adobe PDF

D Barber, P Sollich, and D Saad. Finite size effects in on-line learning in multilayer neural networks. In S W Ellacott, J C Mason, and I J Anderson, editors, Mathematics of Neural Networks: Models, Algorithms and Applications, pages 84-88, Boston, MA, 1997. Kluwer Academic.

Abstract

We extend the recent progress in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating the fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, and increase with the degree of symmetry of the initial conditions. Including a term to stimulate asymmetry in the learning process typically significantly decreases finite size effects and learning time.

P Sollich and D Barber. Online learning from finite training sets. Europhysics Letters, 38:477-482, 1997.

Abstract

We analyse online (gradient descent) learning of a rule from a finite set of training examples at non-infinitesimal learning rates $\eta$, calculating exactly the time-dependent generalization error for a simple model scenario. In the thermodynamic limit, we close the dynamical equation for the generating function of an infinite hierarchy of order parameters using `within-sample self-averaging'. The resulting dynamics is non-perturbative in $\eta$, with a slow mode appearing only above a finite threshold $\eta_{\rm min}$. Optimal settings of $\eta$ for given final learning time are determined and the results are compared with offline gradient descent.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1209/epl/i1997-00271-3

P Sollich and M E Cates. Projected free energies for polydisperse phase equilibria. Physical Review Letters, 80:1365-1368, 1998.

Abstract

A `polydisperse' system has an infinite number of conserved densities. We give a rational procedure for projecting its infinite-dimensional free energy surface onto a subspace comprising a finite number of linear combinations of densities (`moments'), in which the phase behavior is then found as usual. If the excess free energy of the system depends only on the moments used, exact cloud, shadow and spinodal curves result; two- and multi-phase regions are approximate, but refinable indefinitely by adding extra moments. The approach is computationally robust and gives new geometrical insights into the thermodynamics of polydispersity.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevLett.80.1365

P Sollich. Rheological constitutive equation for a model of soft glassy materials. Physical Review E, 58:738-759, 1998.

Abstract

We solve exactly and describe in detail a simplified scalar model for the low frequency shear rheology of foams, emulsions, slurries, etc. [P. Sollich, F. Lequeux, P. Hébraud, M.E. Cates, Phys. Rev. Lett. 78, 2020 (1997)]. The model attributes similarities in the rheology of such ``soft glassy materials'' to the shared features of structural disorder and metastability. By focusing on the dynamics of mesoscopic elements, it retains a generic character. Interactions are represented by a mean-field noise temperature $x$, with a glass transition occurring at $x=1$ (in appropriate units). The exact solution of the model takes the form of a constitutive equation relating stress to strain history, from which all rheological properties can be derived. For the linear response, we find that both the storage modulus $G'$ and the loss modulus $G''$ vary with frequency as $\omega^{x-1}$ for $1<x<2$, becoming flat near the glass transition. In the glass phase, aging of the moduli is predicted. The steady shear flow curves show power law fluid behavior for $x<2$, with a nonzero yield stress in the glass phase; the Cox-Merz rule does not hold in this non-Newtonian regime. Single and double step strains further probe the nonlinear behavior of the model, which is not well represented by the BKZ relation. Finally, we consider measurements of $G'$ and $G''$ at finite strain amplitude $\gamma$. Near the glass transition, $G''$ exhibits a maximum as $\gamma$ is increased in a strain sweep. Its value can be strongly overestimated due to nonlinear effects, which can be present even when the stress response is very nearly harmonic. The largest strain $\gamma_c$ at which measurements still probe the linear response is predicted to be roughly frequency-independent.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevE.58.738

P Sollich and D Barber. Online learning from finite training sets in non-linear networks. In M I Jordan, M J Kearns, and S A Solla, editors, Advances in Neural Information Processing Systems 10, pages 357-363, Cambridge, MA, 1998. MIT Press.

Abstract

Online learning is one of the most common forms of neural network training. We present an analysis of online learning from finite training sets for non-linear networks (namely, soft-committee machines), advancing the theory to more realistic learning scenarios. Dynamical equations are derived for an appropriate set of order parameters; these are exact in the limiting case of either linear networks or infinite training sets. Preliminary comparisons with simulations suggest that the theory captures some effects of finite training sets, but may not yet account correctly for the presence of local minima.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich and D Barber. Online learning from finite training sets and robustness to input bias. Neural Computation, 10:2201-2217,1998.

Abstract

We analyse online gradient descent learning from finite training sets at non-infinitesimal learning rates $\eta$. Exact results are obtained for the time-dependent generalization error of a simple model system: a linear network with a large number of weights $N$, trained on $p=\alpha N$ examples. This allows us to study in detail the effects of finite training set size $\alpha$ on, for example, the optimal choice of learning rate $\eta$. We also compare online and offline learning, for respective optimal settings of $\eta$ at given final learning time. Online learning turns out to be much more robust to input bias and actually outperforms offline learning when such bias is present; for unbiased inputs, online and offline learning perform almost equally well.

Full paper available as gzip'ped postscript or Adobe PDF

D Barber and P Sollich. On-line learning from finite training sets. In D Saad, editor, On-line learning in neural networks, pages 279-302, Cambridge, 1998. Cambridge University Press.

Abstract

We analyse online gradient descent learning from finite training sets at non-infinitesimal learning rates $\eta$ for both linear and n on-linear networks. In the linear case, exact results are obtained for the time-dependent generalization error of networks with a large number of weights $N$, trained on $p=\alpha N$ examples. This allows us to study in detail the effects of finite training set size $\alpha$ on, for example, the optimal choice of learning rate $\eta$. We also compare online and offline learning, for respective optimal settings of $\eta$ at given final learning time. Online learning turns out to be much more robust to input bias and actually outperforms offline learning when such bias is present; for unbiased inputs, online and offline learning perform almost equally well. Our analysis of online learning for non-linear networks (namely, soft-committee machines), advances the theory to more realistic learning scenarios. Dynamical equations are derived for an appropriate set of order parameters; these are exact in the limiting case of either linear networks or infinite training sets. Preliminary comparisons with simulations suggest that the theory captures some effects of finite training sets, but may not yet account correctly for the presence of local minima.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Learning curves for Gaussian processes. In M S Kearns, S A Solla, and D A Cohn, editors, Advances in Neural Information Processing Systems 11, pages 344-350, Cambridge, MA, 1999. MIT Press.

Abstract

I consider the problem of calculating learning curves (i.e., average generalization performance) of Gaussian processes used for regression. A simple expression for the generalization error in terms of the eigenvalue decomposition of the covariance function is derived, and used as the starting point for several approximation schemes. I identify where these become exact, and compare with existing bounds on learning curves; the new approximations, which can be used for any input space dimension, generally get substantially closer to the truth.

Full paper available as gzip'ped postscript or Adobe PDF

H C Rae, P Sollich, and A C C Coolen. On-Line Learning with Restricted Training Sets: Exact Solution as Benchmark for General Theories. In M S Kearns, S A Solla, and D A Cohn, editors, Advances in Neural Information Processing Systems 11, pages 316-322, Cambridge, MA, 1999. MIT Press.

Abstract

We solve the dynamics of on-line Hebbian learning in perceptrons exactly, for the regime where the size of the training set scales linearly with the number of inputs. We consider both noiseless and noisy teachers. Ouc calculation cannot be extended to non-Hebbian rules, but the solution provides a nice benchmark to test more general and advanced theories for solving the dynamics of learning with restricted training sets.

Full paper available as gzip'ped postscript or Adobe PDF

R M L Evans, M E Cates and P Sollich. Diffusion and rheology in a model of glassy materials. European Physical Journal B, 10:705-718, 1999.

Abstract

We study self-diffusion within a simple hopping model for glassy materials. (The model is Bouchaud's model of glasses [J.-P. Bouchaud, J. Physique I, 2:1705 (1992)], as extended to describe rheological properties [P. Sollich, F. Lequeux, P. Hébraud and M. E. Cates, Phys. Rev. Lett., 78:2020 (1997)].) We investigate quantitatively the breakdown, near the glass transition, of the (generalized) Stokes-Einstein relation between self-diffusion of a tracer particle and the (frequency dependent) viscosity of the system as a whole. This stems from the presence of a broad distribution of relaxation times of which different moments control diffusion and rheology. We also investigate the effect of flow (oscillatory shear) on self-diffusion and show that this causes a finite diffusivity in the temperature regime below the glass transition (where this was previously zero). At higher tempe ratures the diffusivity is enhanced by a power law frequency dependence that also characterises the rheological response. The relevance of these findings to soft glassy materials (foams, emulsions etc.) as well as to conventional glass-forming liquids is discussed.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1007/s100510050902

H C Rae, P Sollich, and A C C Coolen. On-Line Learning with Restricted Training Sets: An Exactly Solvable Case. Journal of Physics A, 32:3321-3339, 1999.

Abstract

We solve the dynamics of on-line Hebbian learning in large perceptrons exactly, for the regime where the size of the training set scales linearly with the number of inputs. We consider both noiseless and noisy teachers. Our calculation cannot be extended to non-Hebbian rules, but the solution provides a convenient and welcome benchmark with which to test more general and advanced theories for solving the dynamics of learning with restricted training sets.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1088/0305-4470/32/18/308

M E Cates and P Sollich. Rheology and glassy dynamics of foams. In J F Sadoc and N Rivier, editors, Foams and Emulsions, pages 207-236, Dordrecht, 1999. Kluwer Academic.

Abstract

After a brief `warm-up' discussion of osmotic pressure of foams, the basic phenomena of foam rheology are reviewed, focusing on linear viscoelastic spectra (elastic and loss moduli) with brief mention of nonlinear effects. Theoretical models for some of these properties are then described, starting with Princen's model for the elastic modulus $G_0$ of an ordered foam in two dimensions. There is a basic conflict between this model, which predicts a step-function onset of the modulus when droplets first contact at volume fraction $\phi$=$\phi_0$, and the experimental data (which show $G_0\sim \phi-\phi_0$). The three dimensional ordered case is reviewed next, focusing on anharmonic deformation theory which predicts a logarithmic softening of the modulus near $\phi_0$; this is still not soft enough to explain the observations. The 3D disordered case is then addressed; a combination of disorder and the anharmonic effect finally seems able to explain the data. We then consider the problem of the frequency-dependent loss modulus $G''(\omega)$ which describes dissipation in a foam. Somewhat alarmingly, the data suggest behaviour incompatible with linear response theory; reconciliation is possible if one invokes some very slow relaxation processes at timescales longer than experiment. We briefly describe the search for foam-specific slow relaxation mechanisms of surfactant and water transport, which so far has yielded no viable candidates. Since similar anomalies in $G''(\omega)$ are observed in several other systems, they are instead tentatively ascribed to a generic phenomenon: glassy dynamics. A recent model for the rheology of ``soft glassy matter" is then reviewed; though phenomenological, this suggests that glassy dynamics may be a useful concept in foam rheology.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Approximate learning curves for Gaussian processes. In ICANN99 - Ninth International Conference on Artificial Neural Networks, pages 437-442, London, 1999. The Institution of Electrical Engineers.

Abstract

I consider the problem of calculating learning curves (i.e., average generalization performance) of Gaussian processes used for regression. A simple expression for the generalization error in terms of the eigenvalue decomposition of the covariance function is derived, and used as the starting point for several approximation schemes. I identify where these become exact, and compare with existing bounds on learning curves; the new approximations, which can be used for any input space dimension, generally get substantially closer to the truth.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Probabilistic interpretation and Bayesian methods for Support Vector Machines. In ICANN99 - Ninth International Conference on Artificial Neural Networks, pages 91-96, London, 1999. The Institution of Electrical Engineers.

Abstract

Support Vector Machines (SVMs) can be interpreted as maximum a posteriori solutions to inference problems with Gaussian Process (GP) priors and appropriate likelihood functions. Focussing on the case of classification, I show first that such an interpretation gives a clear intuitive meaning to SVM kernels, as covariance functions of GP priors; this can be used to guide the choice of kernel. Second, a probabilitistic interpretation allows Bayesian methods to be used for SVMs: Using a local approximation of the posterior around its maximum (the standard SVM solution), I discuss how the evidence for a given kernel and noise parameter can be estimated, and how approximate error bars for the classification of test points can be calculated.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich and M R Evans. Glassy timescale divergence and anomalous coarsening in a kinetically constrained spin chain. Physical Review Letters, 83:3238-3241, 1999.

Abstract

We analyse the out of equilibrium behavior of an Ising spin chain with an asymmetric kinetic constraint after a quench to a low temperature $T$. In the limit $T\to 0$, we provide an exact solution of the resulting coarsening process. The equilibration time exhibits a `glassy' divergence $t_{\rm eq}=\exp(\mbox{const}/T^2)$ (popular as an alternative to the Vogel-Fulcher law), while the average domain length grows with a temperature dependent exponent, $\bar{d} \sim t^{T\ln 2}$. We show that the equilibration time $t_{\rm eq}$ also sets the timescale for the linear response of the system at low temperatures.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevLett.83.3238

P Sollich. Probabilistic methods for Support Vector Machines. In S A Solla, T K Leen and K-R Müller, editors, Advances in Neural Information Processing Systems 12, pages 349-355, Cambridge, MA, 2000. MIT Press.

Abstract

I describe a framework for interpreting Support Vector Machines (SVMs) as maximum a posteriori (MAP) solutions to inference problems with Gaussian Process priors. This can provide intuitive guidelines for choosing a `good' SVM kernel. It can also assign (by evidence maximization) optimal values to parameters such as the noise level $C$ which cannot be determined unambiguously from properties of the MAP solution alone (such as cross-validation error). I illustrate this using a simple approximate expression for the SVM evidence. Once $C$ has been determined, error bars on SVM predictions can also be obtained.

Full paper available as gzip'ped postscript or Adobe PDF

D Barber and P Sollich. Gaussian fields for approximate inference in layered sigmoid belief networks. In S A Solla, T K Leen and K-R Müller, editors, Advances in Neural Information Processing Systems 12, pages 393-399, Cambridge, MA, 2000. MIT Press.

Abstract

Layered Sigmoid Belief Networks are directed graphical models in which the local conditional probabilities are parameterised by weighted sums of parental states. Learning and inference in such networks are generally intractable, and approximations need to be considered. Progress in learning these networks has been made by using variational procedures. We demonstrate, however, that variational procedures can be inappropriate for the equally important issue of inference - that is, calculating marginals of the network. We introduce an alternative procedure, based on assuming that the weighted input to a node is approximately Gaussian distributed. Our approach goes beyond previous Gaussian field assumptions in that we take into account correlations between parents of nodes. This procedure is specialized for calculating marginals and is significantly faster and simpler than the variational procedure.

Full paper available as gzip'ped postscript or Adobe PDF

S M Fielding, P Sollich and M E Cates. Ageing and rheology in soft materials. Journal of Rheology, 44:323-369, 2000.

Abstract

We study theoretically the role of ageing in the rheology of soft materials. We define several generalized rheological response functions suited to ageing samples (in which time translation invariance is lost). These are then used to study ageing effects within a simple scalar model (the ``soft glassy rheology" or SGR model) whose constitutive equations relate shear stress to shear strain among a set of elastic elements, with distributed yield thresholds, undergoing activated dynamics governed by a "noise temperature", $x$. (Between yields, each element follows affinely the applied shear.) For $1<x<2$ there is a power-law fluid regime in which transients occur, but no ageing. For $x<1$, the model has a macroscopic yield stress. So long as this yield stress is not exceeded, ageing occurs, with a sample's apparent relaxation time being of order its own age. The (age-dependent) linear viscoelastic loss modulus $G''(\omega,t)$ rises as frequency is lowered, but falls with age $t$, so as to always remains less than $G'(\omega,t)$ (which is nearly constant). Significant ageing is also predicted for the stress overshoot in nonlinear shear startup and for the creep compliance. Though obviously oversimplified, the SGR model may provide a valuable paradigm for the experimental and theoretical study of rheological ageing phenomena in soft solids.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1122/1.551088

H Nishimori and P Sollich. Spin states and entropy of Ising spin glasses. Journal of the Physical Society of Japan, 69:A160-A164, 2000.

Abstract

The relative volume of spin states in the phase space is introduced for the $\pm J$ model of spin glasses. Analysis of its temperature dependence shows that the Nishimori line separates the pure ferromagnetic-like region in the high-temperature side of the ferromagnetic phase from the randomness-dominated region in the low-temperature side. The peak value of the relative volume of the perfect ferromagnetic state is shown to be equivalent to the entropy. Upper and lower bounds on the entropy are derived, which determine the value of the entropy quite precisely.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich, P B Warren and M E Cates. Moment free energies for polydisperse systems. Advances in Chemical Physics (I Prigogine and S A Rice, editors), 116:265-336, 2001.

Abstract

A polydisperse system contains particles with at least one attribute $\sigma$ (such as particle size in colloids or chain length in polymers) which takes values in a continuous range. It therefore has an infinite number of conserved densities, described by a density distribution $\rho(\sigma)$. The free energy depends on all details of $\rho(\sigma)$, making the analysis of phase equilibria in such systems intractable. However, in many (especially mean-field) models the excess free energy only depends on a finite number of (generalized) moments of $\rho(\sigma)$; we call these models truncatable. We show, for these models, how to derive approximate expressions for the total free energy which only depend on such moment densities. Our treatment unifies and explores in detail two recent separate proposals by the authors for the construction of such moment free energies. We show that even though the moment free energy only depends on a finite number of density variables, it gives the same spinodals and critical points as the original free energy and also correctly locates the onset of phase coexistence. Results from the moment free energy for the coexistence of two or more phases occupying comparable volumes are only approximate, but can be refined arbitrarily by retaining additional moment densities. Applications to Flory-Huggins theory for length-polydisperse homopolymers, and for chemically polydisperse copolymers, show that the moment free energy approach is computationally robust and gives new geometrical insights into the thermodynamics of polydispersity.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Bayesian methods for Support Vector Machines: Evidence and predictive class probabilities. Machine Learning, 46:21-52, 2002.

Abstract

I describe a framework for interpreting Support Vector Machines (SVMs) as maximum a posteriori (MAP) solutions to inference problems with Gaussian Process priors. This probabilistic interpretation can provide intuitive guidelines for choosing a `good' SVM kernel. Beyond this, it allows Bayesian methods to be used for tackling two of the outstanding challenges in SVM classification: how to tune hyperparameters--the misclassification penalty $C$, and any parameters specifying the kernel--and how to obtain predictive class probabilities rather than the conventional deterministic class label predictions. Hyperparameters can be set by maximizing the evidence; I explain how the latter can be defined and properly normalized. Both analytical approximations and numerical methods (Monte Carlo chaining) for estimating the evidence are discussed. I also compare different methods of estimating class probabilities, ranging from simple evaluation at the MAP or posterior average to full averaging over the posterior. A simple toy application illustrates the various concepts and techniques.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1023/A:1012489924661

N Clarke, J A Cuesta, R Sear, P Sollich and A Speranza. Phase equilibria in the polydisperse Zwanzig model of hard rods. Journal of Chemical Physics, 113:5817-5829, 2000.

Abstract

We study the phase behaviour of the Zwanzig model of suspensions of hard rods, allowing for polydispersity in the lengths of the rods. In spite of the simplified nature of the model (rods are restricted to lie along one of three orthogonal axes), the results agree qualitatively with experimental observations: the coexistence region broadens significantly as the polydispersity increases, and strong fractionation occurs, with long rods found preferentially in the nematic phase. These conclusions are obtained from an analysis of the exact phase equilibrium equations. In the second part of the paper, we consider the application of the recently developed ``moment free energy method'' to the polydisperse Zwanzig model. Even though the model contains non-conserved densities due to the orientational degrees of freedom, most of the exactness statements (regarding the onset of phase coexistence, spinodals, and critical points) derived previously for systems with conserved densities remain valid. The accuracy of the results from the moment free energy increases as more and more additional moments are retained in the description. We show how this increase in accuracy can be monitored without relying on knowledge of the exact results, and discuss an adaptive technique for choosing the extra moments optimally.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1063/1.1290473

P Sollich, H Nishimori, A C C Coolen and A J van der Sijs. Nontrivial phase behaviour in the infinite-range quantum Mattis model. Journal of the Physical Society of Japan, 69:3200-3213, 2000.

Abstract

We have solved the quantum version of the Mattis model with infinite-range interactions. A variational approach gives the exact solution for the infinite-range system, in spite of the non-commutative nature of the quantum spin components; this implies that quantum effects are not predominant in determining the macroscopic properties of the system. Nevertheless, the model has a surprisingly rich phase behaviour, exhibiting phase diagrams with tricritical, three-phase and critical end points.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1143/JPSJ.69.3200

P Sollich, A C C Coolen, L P Hughston, R F Streater, editors. Complex and disordered systems. American Institute of Physics Publishing, Melville, New York, 2001.

Abstract

From 10 to 14 July 2000, an international conference on Disordered and Complex Systems was held at King's College London; this was a satellite meeting of the XIIIth International Congress on Mathematical Physics, held the following week at Imperial College, London. The aim of the meeting was to take a deliberately broad view of the meaning of the terms ``disorder and complexity'', thus bringing together researchers working in a number of different areas and creating opportunities for possible synergies. The programme was structured around five areas: Glassy systems and neural networks, information geometry, quantum dynamics and quantum chaos, reaction-diffusion equations, and mathematical finance. While these topics may sound disparate, a substantial number of common threads appeared: Information geometry, for example, is being applied to subjects such as neural network learning, quantum mechanics and interest rate dynamics; the statistical mechanics of glassy systems is helping to understand the behaviour of large systems of economic agents; and Lévy statistics occur in glasses, in anomalous diffusion, and in asset price returns. In keeping with the aim of the conference, it is hoped that the reader will find in this book many other surprising connections between seemingly unrelated areas.

All speakers at the conference, as well as a selection of poster presenters, were invited to contribute a paper to the proceedings. The articles are grouped into five parts under the headings listed above; within each part, papers are arranged in alphabetical order by presenting author. Where there are several co-authors, the name of the presenting author is marked with an asterisk in the table of contents.

Part one of the proceedings is concerned with Glassy systems and neural networks. Here, the methods of the statistical mechanics of disordered systems are to the fore, and the papers in this section demonstrate the impressive sweep of topics to which these methods can be applied. There are, first of all, the structural glasses, of which window glass is the archetype; Barkai provides evidence for the occurrence of Lévy statistics in such systems. Cavagna studies a simple model for structural glasses and extracts information about its ``energy landscape''. Ritort focuses on fluctuations around a non-equilibrium state, and in particular on whether they can be understood by extending the concept of temperature to non-equilibrium situations. The kinetic spin model studied by Garrahan is more abstract--the ``caging in'' of atoms by their neighbours in a glass is represented by simple kinetic constraints--but nonetheless captures important features of glassy dynamics. Nicodemi applies similar ideas to granular materials, where ``jamming'' effects have recently been recognized as closely analogous to a glass transition. Goldbart studies yet another related class of materials, the vulcanized rubbers. These are in fact easier to deal with in an equilibrium framework than glasses, since the chemical crosslinks between different strands of a vulcanized polymer are permanent, rather than slowly evolving like the ``cages'' in a glass. Van Mourik, Nishimori and Parisi explore the theory of spin glasses, where glassiness arises from a random distribution of magnetic impurities (spins) in a host material. Many of the methods of the statistical mechanics of disordered systems--such as replica theory--were originally developed for these systems. Van Mourik advances a new interpretation of the Parisi scheme for the analysis of replica symmetry breaking, while Nishimori's work puts rigorous constraints on the regions of the phase diagram where such symmetry breaking can actually appear. Parisi, finally, discusses exciting links between equilibrium and non-equilibrium behaviour via fluctuation-dissipation relations and effective temperatures.

In neural network models, glassy effects often arise from the storage of conflicting patterns; Bollé and Skantzos address the resulting dynamical features, revealing links with information theory and random-field systems. Buhot and Kozowski focus on capacity calculations, where one asks how many patterns can be stored before interference destroys any possible recall. Saad broadens the picture out to error-correcting codes; the decoding problem has similarities to the task of finding the ground state of a spin glass. Giardina analyses a model of population dynamics in a glass-like landscape, where energy barriers can be surmounted by growth of the population. Sherrington, finally, demonstrates that statistical physics also has a part to play in the study of large systems of interacting economic agents.

Part two of this book deals with Information geometry. In the classical version of this theory, the set of probability measures on a finite space has a natural structure as a Riemannian manifold (with the Fisher metric), and also has two dual flat affine structures, those of Amari. Pistone, Combe and Nencka explain some progress that has been made in extending these ideas to infinite sample spaces, using ideas from the theory of Orlicz spaces. Fukumizu shows how these extensions are used in the theory of neural networks to cope with singularities. The quantum version of information geometry has many different good choices for the metric. Jencová analyses the family of metrics associated with Hasegawa's quantum $\alpha$-entropy, and one metric is singled out by Streater & Grasselli as the only one for which the entropy and free energy are dual. Gibilisco & Isola construct a whole family of affine structures, on the tangent bundle with its dual cotangent bundle. Grasselli shows that the free energy is an analytic function on a manifold of relatively bounded perturbations of a given state; these ideas can be used to describe dissipative dynamics in quantum theory. Finally, a relation between the quantum and classical theories via the semi-classical limit is described by Ghikas.

In part three, on Quantum dynamics and quantum chaos, Hasegawa describes how the statistical mechanics of random matrices leads to an understanding of the physics of the metal-insulator transition, while Fyodorov extends the subject to the spectrum of non-Hermitian random matrices; such matrices occur naturally in quantum chaotic systems. De Cock introduces a new way of analysing the properties of a sequence of vectors, using the matrix of scalar products. Aneva discusses a geometric definition of the operator $pq$ proposed by Berry as having a spectrum related to the Riemann zeros. Shukla suggests a universal method, based on Calogero-Moser dynamics, which seems to unify the treatment of many apparently different problems in chaos. Tomiya describes a numerical study of the distribution of modes in a quartic oscillator system, and finds that the transition to a Gaussian distribution is not at all sharp. The quantum $L^p$ spaces of Majewski arise in the dynamics of infinite dissipative systems, while Lorinczi's deep analysis tackles the infrared problem in a non-trivial quantum field theory. Cartas illustrates how topological bifurcations can occur as parameters controlling the dynamics of a model are varied.

Part four of the book contains papers on Reaction-diffusion equations. These are parabolic partial differential equations with a dissipative Lyapunov function that is identified with the entropy or free energy, and have been generalized to situations where the temperature is not assumed to be constant everywhere. Biler, Karch and Nadzieja take such a system and add the mutual energy, either Newtonian gravity or the Coulomb energy, to obtain an integro-differential system. They study existence, uniqueness and blow-up of the solutions; the methods involve versions of Leray-Schauder theory, and also make good use of scaling properties of the equations. Liskevich provides Gaussian estimates for general parabolic operators, which are useful also for nonlinear problems, while Archincheev shows that Lévy processes are behind some nonlinearities in model dynamics.

Part five, entitled New Directions in Mathematical Finance, includes eleven papers on interrelated topics concerning the behaviour of financial markets. Concepts arising in statistical physics, such as information manifolds and the scaling laws of critical phenomena, are put to use in various ways along side the established body of finance theory based on random price dynamics. For example, one of the central problems in modern finance is to understand the origins of the fat-tailed distributions of asset returns. This can be approached either by modelling the relevant processes directly, or by looking at the market as a whole and studying the interactions of agents. The resolution of this problem may have profound consequences for the risk management of derivatives. Several papers here address aspects of this problem (Balland; Bingham & Kiesel; Bolan, Hurd, Pivato & Seco; Iori; Shaw; and Woo). There are interesting overlaps with the mathematics of natural catastrophes and self-organised critical phenomena, with deep connections to the theory of Lévy processes. The great bond market crisis of 1998 provides a nice example of a ``risk avalanche''. This was triggered by a default on a sovereign debt, an event that can be modelled by a point process (as described in the article by Beumee). Two further papers here, by Brody & Hughston and by Webber, develop geometric ideas in the theory of interest rates with links to statistical physics. Finally, another key problem, closely related to modelling the optimal behaviour of market agents, is the more general task of managerial decision making, which is the topic of the theory of ``real options'', discussed in articles by Leppard and by Hughston & Zervos.

P Sollich and A Halees. Learning curves for Gaussian process regression: Approximations and bounds. Neural Computation, 14:1393-1428, 2002.

Abstract

We consider the problem of calculating learning curves (i.e., average generalization performance) of Gaussian processes used for regression. On the basis of a simple expression for the generalization error, in terms of the eigenvalue decomposition of the covariance function, we derive a number of approximation schemes. We identify where these become exact, and compare with existing bounds on learning curves; the new approximations, which can be used for any input space dimension, generally get substantially closer to the truth. We also study possible improvements to our approximations. Finally, we use a simple exactly solvable learning scenario to show that there are limits of principle on the quality of approximations and bounds expressible solely in terms of the eigenvalue spectrum of the covariance function.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Generalization of Plaskota's bound for Gaussian process learning curves. Technical report, unpublished.

Abstract

This paper is an extended version of the manuscript Learning curves for Gaussian process regression: Approximations and bounds by Sollich and Halees. The only difference is the addition of Appendix B, which gives the derivation of the generalized version of Plaskota's bound. The remainder of the paper has been left in place to provide the proper context.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Gaussian process regression with mismatched models. In T G Dietterich, S Becker and Z Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 519-526, Cambridge, MA, 2002. MIT Press.

Abstract

Learning curves for Gaussian process regression are well understood when the `student' model happens to match the `teacher' (true data generation process). I derive approximations to the learning curves for the more generic case of mismatched models, and find very rich behaviour: For large input space dimensionality, where the results become exact, there are universal (student-independent) plateaux in the learning curve, with transitions in between that can exhibit arbitrarily many over-fitting maxima. In lower dimensions, plateaux also appear, and the asymptotic decay of the learning curve becomes strongly student-dependent. All predictions are confirmed by simulations.

Full paper available as gzip'ped postscript or Adobe PDF

S M Fielding and P Sollich. Observable-dependence of fluctuation-dissipation relations and effective temperatures. Physical Review Letters, 88:050603, 2002.

Abstract

We study the non-equilibrium version of the fluctuation-dissipation theorem (FDT) within the glass phase of Bouchaud's trap model. We incorporate into the model an arbitrary observable $m$ and obtain its correlation and response functions in closed form. A limiting non-equilibrium FDT plot (of correlator vs response) is approached at long times for most choices of $m$, with energy-temperature FDT a notable exception. In contrast to standard mean field models, however, the shape of the plot depends nontrivially on the observable, and its slope varies continuously even though there is a single scaling of relaxation times with age. Non-equilibrium FDT plots can therefore not be used to define a meaningful effective temperature $T_{\rm eff}$ in this model. Consequences for the wider applicability of an FDT-derived $T_{\rm eff}$ are discussed.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1103/PhysRevLett.88.050603

P Sollich. Predicting phase equilibria in polydisperse systems (invited topical review). Journal of Physics: Condensed Matter, 14:R79-R117, 2002.

Abstract

Many materials containing colloids or polymers are polydisperse: They comprise particles with properties (such as particle diameter, charge, or polymer chain length) that depend continuously on one or several parameters. This review focusses on the theoretical prediction of phase equilibria in polydisperse systems; the presence of an effectively infinite number of distinguishable particle species makes this a highly nontrivial task. I first describe qualitatively some of the novel features of polydisperse phase behaviour, and outline a theoretical framework within which they can be explored. Current techniques for predicting polydisperse phase equilibria are then reviewed. I also discuss applications to some simple model systems including homopolymers and random copolymers, spherical colloids and colloid-polymer mixtures, and liquid crystals formed from rod- and plate-like colloidal particles; the results surveyed give an idea of the rich phenomenology of polydisperse phase behaviour. Extensions to the study of polydispersity effects on interfacial behaviour and phase separation kinetics are outlined briefly.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1088/0953-8984/14/3/201

P Sollich, S M Fielding and P Mayer. Fluctuation-dissipation relations and effective temperatures in simple non-mean field systems. Journal of Physics: Condensed Matter, 14:1683-1696, 2002.

Abstract

We give a brief review of violations of the fluctuation-dissipation theorem (FDT) in out-of-equilibrium systems; in mean field scenarios the corresponding fluctuation-dissipation (FD) plots can, in the limit of long times, be used to define a effective temperature $T_{\rm eff}$ that shares many properties of the true thermodynamic temperature $T$. We discuss carefully how correlation and response functions need to be represented to obtain meaningful limiting FD plots in non-mean field systems. A minimum requirement on the resulting effective temperatures is that they should be independent of the observable whose correlator and response are being considered; we show for two simple models with glassy dynamics (Bouchaud's trap model and the Glauber-Ising chain at zero temperature) that this is generically not the case. Consequences for the wider applicability of effective temperatures derived from FD relations are discussed; one intriguing possibility is that at least the limit of the FDT violation factor for well separated times may generically be observable-independent and so could yield a meaningful $T_{\rm eff}$.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1088/0953-8984/14/7/324

N B Wilding and P Sollich. Grand canonical ensemble simulation studies of polydisperse fluids. Journal of Chemical Physics, 16:7116-7126, 2002.

Abstract

We describe a Monte Carlo scheme for simulating polydisperse fluids within the grand canonical ensemble. Given some polydisperse attribute $\sigma$, the state of the system is described by a density distribution $\rho(\sigma)$ whose form is controlled by the imposed chemical potential distribution $\mu(\sigma)$. We detail how histogram extrapolation techniques can be employed to tune $\mu(\sigma)$ such as to traverse some particular desired path in the space of $\rho(\sigma)$. The method is applied in simulations of size-disperse hard spheres with densities distributed according to Schulz and log-normal forms. In each case, the equation of state is obtained along the dilution line, i.e. the path along which the scale of $\rho(\sigma)$ changes but not its shape. The results are compared with the moment-based expressions of Boublik et al (J. Chem. Phys. 54, 1523 (1971)) and Salacuse and Stell (J. Chem. Phys. 77, 3714 (1982)). It is found that for high degrees of polydispersity, both expressions fail to give a quantitatively accurate description of the equation of state when the overall volume fraction is large.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1063/1.1464829

A Speranza and P Sollich. Simplified Onsager theory for isotropic-nematic phase equilibria of length polydisperse hard rods. Journal of Chemical Physics, 117:5421-5436, 2002.

Abstract

Polydispersity is believed to have important effects on the formation of liquid crystal phases in suspensions of rod-like particles. To understand such effects, we analyse the phase behaviour of thin hard rods with length polydispersity. Our treatment is based on a simplified Onsager theory, obtained by truncating the series expansion of the angular dependence of the excluded volume. We describe the model and give the full phase equilibrium equations; these are then solved numerically using the moment free energy method which reduces the problem from one with an infinite number of conserved densities to one with a finite number of effective densities that are moments of the full density distribution. The method yields exactly the onset of nematic ordering. Beyond this, results are approximate but we show that they can be made essentially arbitrarily precise by adding adaptively chosen extra moments, while still avoiding the numerical complications of a direct solution of the full phase equilibrium conditions.
We investigate in detail the phase behaviour of systems with three different length distributions: a (unimodal) Schulz distribution, a bidisperse distribution and a bimodal mixture of two Schulz distributions which interpolates between these two cases. A three-phase isotropic-nematic-nematic coexistence region is shown to exist for the bimodal and bidisperse length distributions if the ratio of long and short rod lengths is sufficiently large, but not for the unimodal one. We systematically explore the topology of the phase diagram as a function of the width of the length distribution and of the rod length ratio in the bidisperse and bimodal cases.

Full paper available as cond-mat/0203325
DOI: 10.1063/1.1499718

P Sollich and F Ritort, editors. Proceedings of workshop on ``Glassy dynamics in kinetically constrained models''. Journal of Physics: Condensed Matter, 14(7), 2002.

Abstract

C Gold and P Sollich. Model selection for Support Vector Machine classification. Neurocomputing, 55:221-249, 2003.

Abstract

We address the problem of model selection for Support Vector Machine (SVM) classification. For fixed functional form of the kernel, model selection amounts to tuning kernel parameters and the slack penalty coefficient $C$. We begin by reviewing a recently developed probabilistic framework for SVM classification. An extension to the case of SVMs with quadratic slack penalties is given and a simple approximation for the evidence is derived, which can be used as a criterion for model selection. We also derive the exact gradients of the evidence in terms of posterior averages and describe how they can be estimated numerically using Hybrid Monte Carlo techniques. Though computationally demanding, the resulting gradient ascent algorithm is a useful baseline tool for probabilistic SVM model selection, since it can locate maxima of the exact (unapproximated) evidence. We then perform extensive experiments on several benchmark data sets. The aim of these experiments is to compare the performance of probabilistic model selection criteria with alternatives based on estimates of the test error, namely the so-called ``span estimate'' and Wahba's Generalized Approximate Cross-Validation (GACV) error. We find that all the ``simple'' model criteria (Laplace evidence approximations, and the Span and GACV error estimates) exhibit multiple local optima with respect to the hyperparameters. While some of these give performance that is competitive with results from other approaches in the literature, a significant fraction lead to rather higher test errors. The results for the evidence gradient ascent method show that also the exact evidence exhibits local optima, but these give test errors which are much less variable and also consistently lower than for the simpler model selection criteria.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1016/S0925-2312(03)00375-8

A Speranza and P Sollich. Isotropic-nematic phase equilibria of polydisperse hard rods: The effect of fat tails in the length distribution. Journal of Chemical Physics, 118:5213-5223, 2003.

Abstract

We study the phase behaviour of hard rods with length polydispersity, treated within a simplified version of the Onsager model. We give a detailed description of the unusual phase behaviour of the system when the rod length distribution has a "fat" (e.g. log-normal) tail up to some finite cutoff. The relatively large number of long rods in the system strongly influences the phase behaviour: the isotropic cloud curve, which defines the where a nematic phase first occurs as density is increased, exhibits a kink; at this point the properties of the coexisting nematic shadow phase change discontinuously. A narrow three-phase isotropic-nematic-nematic coexistence region exists near the kink in the cloud curve, even though the length distribution is unimodal. A theoretical derivation of the isotropic cloud curve and nematic shadow curve, in the limit of large cutoff, is also given. The two curves are shown to collapse onto each other in the limit. The coexisting isotropic and nematic phases are essentially identical, the only difference being that the nematic contains a larger number of the longest rods; the longer rods are also the only ones that show any significant nematic ordering. Numerical results for finite but large cutoff support the theoretical predictions for the asymptotic scaling of all quantities with the cutoff length.

Full paper available as cond-mat/0208193
DOI: 10.1063/1.1545444

S M Fielding and P Sollich. Equivalence of driven and ageing fluctuation-dissipation relations in the trap model. Physical Review E, 67:011101, 2003.

Abstract

We study the non-equilibrium version of the fluctuation dissipation (FD) relation in the glass phase of a trap model that is driven into a non-equilibrium steady state by external ``shear''. This extends our recent study of ageing FD relations in the same model, where we found limiting, observable independent FD relations for ``neutral'' observables that are uncorrelated with the system's average energy. In this work, for such neutral observables, we find the FD relation for a stationary weakly driven system to be the same, to within small corrections, as for an infinitely aged system. We analyse the robustness of this correspondence with respect to non-neutrality of the observable, and with respect to changes in the driving mechanism.

Full paper available as Adobe PDF or from cond-mat/0209645
DOI: 10.1103/PhysRevE.67.011101

F Ritort and P Sollich. Glassy dynamics of kinetically constrained models. Advances in Physics, 52:219-342, 2003.

Abstract

We review the use of kinetically constrained models (KCMs) for the study of dynamics in glassy systems. The characteristic feature of KCMs is that they have trivial, often non-interacting, equilibrium behaviour but interesting slow dynamics due to restrictions on the allowed transitions between configurations. The basic question which KCMs ask is therefore how much glassy physics can be understood without an underlying ``equilibrium glass transition''. After a brief review of glassy phenomenology, we describe the main model classes, which include spin-facilitated (Ising) models, constrained lattice gases, models inspired by cellular structures such as soap froths, models obtained via mappings from interacting systems without constraints, and finally related models such as urn, oscillator, tiling and needle models. We then describe the broad range of techniques that have been applied to KCMs, including exact solutions, adiabatic approximations, projection and mode-coupling techniques, diagrammatic approaches and mappings to quantum systems or effective models. Finally, we give a survey of the known results for the dynamics of KCMs both in and out of equilibrium, including topics such as relaxation time divergences and dynamical transitions, nonlinear relaxation, aging and effective temperatures, cooperativity and dynamical heterogeneities, and finally non-equilibrium stationary states generated by external driving. We conclude with a discussion of open questions and possibilities for future work.

Full paper available as Adobe PDF or from cond-mat/0210382
DOI: 10.1080/0001873031000093582

A Speranza and P Sollich. Isotropic-nematic phase equilibria in the Onsager theory of hard rods with length polydispersity. Physical Review E, 67:061702, 2003.

Abstract

We analyse the effect of a continuous spread of particle lengths on the phase behavior of rodlike particles, using the Onsager theory of hard rods. Our aim is to establish whether ``unusual'' effects such as isotropic-nematic-nematic (I-N-N) phase separation can occur even for length distributions with a single peak. We focus on the onset of I-N coexistence. For a log-normal distribution we find that a finite upper cutoff on rod lengths is required to make this problem well-posed. The cloud curve, which tracks the density at the onset of I-N coexistence as a function of the width of the length distribution, exhibits a kink; this demonstrates that the phase diagram must contain a three-phase I-N-N region. Theoretical analysis shows that in the limit of large cutoff the cloud point density actually converges to zero, so that phase separation results at any nonzero density; this conclusion applies to all length distributions with fatter-than-exponentail tails. Finally we consider the case of a Schulz distribution, with its exponential tail. Surprisingly, even here the long rods (and hence the cutoff) can dominate the phase behaviour, and a kink in the cloud curve and I-N-N coexistence again result. Theory establishes that there is a nonzero threshold for the width of the length distribution above which these long rod effects occur, and shows that the cloud and shadow curves approach nonzero limits for large cutoff, both in good agreement with the numerical results.

Full paper available as cond-mat/0211103
DOI: 10.1103/PhysRevE.67.061702

P Mayer, L Berthier, J P Garrahan, P Sollich. Fluctuation-dissipation relations in the non-equilibrium critical dynamics of Ising models. Physical Review E, 68:016116, 2003.

Abstract

We investigate the relation between two-time, multi-spin, correlation and response functions in the non-equilibrium critical dynamics of Ising models in d=1 and d=2 spatial dimensions. In these non-equilibrium situations, the fluctuation-dissipation theorem (FDT) is not satisfied. We find FDT `violations' qualitatively similar to those reported in various glassy materials, but quantitatively dependent on the chosen observable, in contrast to the results obtained in infinite-range glass models. Nevertheless, all FDT violations can be understood by considering separately the contributions from large wavevectors, which are at quasi-equilibrium and obey FDT, and from small wavevectors where a generalized FDT holds with a non-trivial limit fluctuation-dissipation ratio $X$. In d=1, we get $X = 1/2$ for spin observables, which measure the orientation of domains, while $X = 0$ for observables that are sensitive to the domain-wall motion. Numerical simulations in d=2 reveal a unique $X = 0.34$ for all observables. Measurement protocols for $X$ are discussed in detail. Our results suggest that the definition of an effective temperature $T_{\rm eff} = T / X$ for large length scales is generically possible in non-equilibrium critical dynamics.

Full paper available as cond-mat/0301493
DOI: 10.1103/PhysRevE.68.016116

P Sollich and M R Evans. Glassy dynamics in the asymmetrically constrained kinetic Ising chain. Physical Review E, 68:031504, 2003.

Abstract

We study the dynamics of the East model, comprising a chain of uncoupled spins in a downward-pointing field. Glassy effects arise at low temperatures $T$ from the kinetic constraint that spins can only flip if their left neighbour is up. We give details of our previous solution of the non-equilibrium coarsening dynamics after a quench to low $T$ (Phys. Rev. Lett. 83:3238, 1999), including the anomalous coarsening of down-spin domains with typical size $\bar{d} \sim t^{T\ln 2}$, and the pronounced `fragile glass'-divergence of equilibration times as $t_*=\exp(1/T^2\ln 2)$. We also link the model to the paste-all coarsening model, defining a family of interpolating models that all have the same scaling distribution of domain sizes. We then proceed to the problem of equilibrium dynamics at low $T$. Based on a scaling hypothesis for the relation between timescales and lengthscales, we propose a model for the dynamics of `superdomains' which are bounded by up-spins that are frozen on long timescales. From this we deduce that the equilibrium spin correlation and persistence functions should exhibit identical scaling behaviour for low $T$, decaying as $g(\tilde{t})$. The scaling variable is $\tilde{t}=(t/t_*)^{T\ln 2}$, giving strongly stretched behaviour for low $T$. The scaling function $g(\cdot)$ decays faster than exponential, however, and in the limit $T\to 0$ at fixed $\tilde{t}$ reaches zero at a finite value of $\tilde{t}$.

Full paper available as Adobe PDF or from cond-mat/0303318
DOI: 10.1103/PhysRevE.68.031504

P Sollich. Fluctuation-dissipation relations in trap models. Journal of Physics A, 36:10807-10818, 2003.

Abstract

Trap models are intuitively appealing and often solvable models of glassy dynamics. In particular, they have been used to study aging and the resulting out-of-equilibrium fluctuation-dissipation relations between correlations and response functions. In this note I show briefly that one such relation, first given by Bouchaud and Dean, is valid for a general class of mean-field trap models: it relies only on the way a perturbation affects the transition rates, but is independent of the distribution of trap depths and the form of the unperturbed transition rates, and holds for all observables that are uncorrelated with the energy. The model with Glauber dynamics and an exponential distribution of trap depths, as considered by Barrat and Mezard, does not fall into this class if the perturbation is introduced in the standard way by shifting all trap energies. I show that, surprisingly, a very similar relation between response and correlation nevertheless holds for the out-of-equilibrium dynamics at low temperatures. This points to intriguing parallels between trap models with energetic and entropic barriers.

Full paper available as cond-mat/0303637
DOI: 10.1088/0305-4470/36/43/009

M Fasolo and P Sollich. Equilibrium phase behavior of polydisperse hard spheres. Physical Review Letters, 91:068301, 2003.

Abstract

We calculate the phase behavior of hard spheres with size polydispersity, using accurate free energy expressions for the fluid and solid phases. Cloud and shadow curves, which determine the onset of phase coexistence, are found exactly by the moment free energy method, but we also compute the complete phase diagram, taking full account of fractionation effects. In contrast to earlier, simplified treatments we find no point of equal concentration between fluid and solid or re-entrant melting at higher densities. Rather, the fluid cloud curve continues to the largest polydispersity that we study (14%); from the equilibrium phase behavior a terminal polydispersity can thus only be defined for the solid, where we find it to be around 7%. At sufficiently large polydispersity, fractionation into several solid phases can occur, consistent with previous approximate calculations; we find in addition that coexistence of several solids with a fluid phase is also possible.

Full paper available as Adobe PDF or from cond-mat/0305211
DOI: 10.1103/PhysRevLett.91.068301

M E Cates and P Sollich. Tensorial constitutive models for disordered foams, dense emulsions, and other soft nonergodic materials. Journal of Rheology, 48:193-207, 2004.

Abstract

In recent years, the paradigm of `soft glassy matter' has been used to describe diverse nonergodic materials exhibiting strong local disorder and slow mesoscopic rearrangement. As so far formulated, however, the resulting `soft glassy rheology' (SGR) model treats the shear stress in isolation, effectively `scalarizing' the stress and strain rate tensors. Here we offer generalizations of the SGR model that combine its nontrivial aging and yield properties with a tensorial structure that can be specifically adapted, for example, to the description of fluid film assemblies or disordered foams.

Full paper available as cond-mat/0307098
DOI: 10.1122/1.1634985

P Mayer and P Sollich. General solutions for multispin two-time correlation and response functions in the Glauber-Ising chain. Journal of Physics A, 37:9-49, 2004.

Abstract

The kinetic Glauber-Ising spin chain is one of the very few exactly solvable models of non-equilibrium statistical mechanics. Nevertheless, existing solutions do not yield tractable expressions for two-time correlation and response functions of observables involving products of more than one or two spins. We use a new approach to solve explicitly the full hierarchy of differential equations for the correlation and response functions. From this general solution follow closed expressions for arbitrary multispin two-time correlation and response functions, for the case where the system is quenched from equilibrium at $T_i > 0$ to some arbitrary $T\geq0$. By way of application, we give the results for two and four-spin two-time correlation and response functions. From the standard mapping, these also imply new exact results for two-time particle correlation and response functions in one-dimensional diffusion limited annihilation.

Full paper available as cond-mat/0307214
DOI: 10.1088/0305-4470/37/1/002

M Fasolo, P Sollich and A Speranza. Phase equilibria in polydisperse colloidal systems. React. Funct. Polym., 58:187-196, 2004.

Abstract

Many materials containing colloids or polymers are polydisperse: they comprise particles with properties (such as diameter, charge, or polymer chain length) that depend continuously on one or several parameters. This effectively infinite number of distinguishable particle species makes the theoretical prediction of phase equilibria a highly non-trivial task. We give an overview of recent progress, focussing on the "moment free energy" method which reduces the problem to one involving only the densities of a finite number of quasi- species. Applications to isotropic-nematic phase equilibria of rod-like particles and to the phase behaviour of polydisperse hard spheres are described.

Full paper available as gzip'ped postscript or Adobe PDF

P Mayer, L Berthier, J P Garrahan, P Sollich. Reply to Comment on "Fluctuation-dissipation relations in the non-equilibrium critical dynamics of Ising models". Physical Review E, 70:018102, 2004.

Abstract

We have recently shown that in non-equilibrium spin systems at criticality the limit $X^\infty$ of the fluctuation-dissipation ratio $X(t,t_w)$ for $t\gg t_w \gg 1$ can be measured using observables such as magnetization or energy [Phys. Rev. E 68, 016116 (2003)]. Pleimling argues in a Comment [cond-mat/0309652] on our paper that for such observables correlation and response functions are dominated by one-time quantities dependent only on t, and are therefore not suitable for a determination of $X^\infty$. Using standard scaling forms of correlation and response functions, as used by Pleimling, we show that our data do have a genuine two-time dependence and allow $X(t,t_w)$ and $X^\infty$ to be measured, so that Pleimling's criticisms are easily refuted. We also compare with predictions from renormalization-group calculations, which are consistent with our numerical observation of a fluctuation-dissipation plot for the magnetization that is very close to a straight line. A key point remains that coherent observables make measurements of $X^\infty$ easier than the traditionally used incoherent ones, producing fluctuation-dissipation plots whose slope is close to $X^\infty$ over a much larger range.

Full paper available as cond-mat/0310545
DOI: 10.1103/PhysRevE.70.018102

N B Wilding and P Sollich. Phase equilibria and fractionation in a polydisperse fluid. Europhysics Letters, 67:219-225, 2004.

Abstract

We describe how Monte Carlo simulation within the grand canonical ensemble can be applied to the study of phase behaviour in polydisperse fluids. Attention is focused on the case of fixed polydispersity in which the form of the `parent' density distribution $\rho^0(\sigma)$ of the polydisperse attribute $\sigma$ is prescribed. Recently proposed computational methods facilitate determination of the chemical potential distribution conjugate to $\rho^0(\sigma)$. By additionally incorporating extended sampling techniques within this approach, the compositions of coexisting (`daughter') phases can be obtained and fractionation effects quantified. As a case study, we investigate the liquid-vapor phase equilibria of a size-disperse Lennard-Jones fluid exhibiting a large ($\delta=40\%$) degree of polydispersity. Cloud and shadow curves are obtained, the latter of which exhibit a high degree of fractionation with respect to the parent. Additionally, we observe considerable broadening of the coexistence region relative to the monodisperse limit.

Full paper available as cond-mat/0403391
DOI: 10.1209/epl/i2004-10064-2

H Nishimori and P Sollich. Error counting in a quantum error-correcting code and the ground-state energy of a spin glass. Journal of the Physical Society of Japan, 73(10):2701-2707, 2004.

Abstract

Upper and lower bounds are given for the number of equivalence classes of error patterns in the toric code for quantum memory. The results are used to derive a lower bound on the ground-state energy of the $\pm J$ Ising spin glass model on the square lattice with symmetric and asymmetric bond distributions. This is a highly non-trivial example in which insights from quantum information lead directly to an explicit result on a physical quantity in the statistical mechanics of disordered systems.

Full paper available as cond-mat/0405313
DOI: 10.1143/JPSJ.73.2701

P Mayer, H Bissig, L Berthier, L Cipelletti, J P Garrahan, P Sollich and V Trappe. Heterogeneous Dynamics of Coarsening Systems. Physical Review Letters, 93:115701, 2004.

Abstract

We show by means of experiments, theory and simulations, that the slow dynamics of coarsening systems displays dynamic heterogeneity similar to that observed in glass-forming systems. We measure dynamic heterogeneity via novel multi-point functions which quantify the emergence of dynamic, as opposed to static, correlations of fluctuations. Experiments are performed on a coarsening foam using Time Resolved Correlation, a recently introduced light scattering method. Theoretically we study the Ising model, and present exact results in one dimension, and numerical results in two dimensions. For all systems the same dynamic scaling of fluctuations with domain size is observed.

Full paper available as cond-mat/0405402
DOI: 10.1103/PhysRevLett.93.115701

M Fasolo and P Sollich. Fractionation effects in phase equilibria of polydisperse hard sphere colloids. Physical Review E, 70:041410, 2004.

Abstract

The equilibrium phase behaviour of hard spheres with size polydispersity is studied theoretically. We solve numerically the exact phase equilibrium equations that result from accurate free energy expressions for the fluid and solid phases, while accounting fully for size fractionation between coexisting phases. Fluids up to the largest polydispersities that we can study (around 14%) can phase separate by splitting off a solid with a much narrower size distribution. This shows that experimentally observed terminal polydispersities above which phase separation no longer occurs must be due to non-equilibrium effects. We find no evidence of re-entrant melting; instead, sufficiently compressed solids phase separate into two or more solid phases. Under appropriate conditions, coexistence of multiple solids with a fluid phase is also predicted. The solids have smaller polydispersities than the parent phase as expected, while the reverse is true for the fluid phase, which contains predominantly smaller particles but also residual amounts of the larger ones. The properties of the coexisting phases are studied in detail; mean diameter, polydispersity and volume fraction of the phases all reveal marked fractionation. We also propose a method for constructing quantities that optimally distinguish between the coexisting phases, using Principal Component Analysis in the space of density distributions. We conclude by comparing our predictions to perturbative theories for near-monodisperse systems and to Monte Carlo simulations at imposed chemical potential distribution, and find excellent agreement.

Full paper available as cond-mat/0405621
DOI: 10.1103/PhysRevE.70.041410

P Mayer, P Sollich. Observable dependent quasi-equilibrium in slow dynamics. Physical Review E, 71:046113, 2005.

Abstract

We present an example demonstrating that quasi-equilibrium fluctuation-dissipation behavior at short time differences is not a generic feature of systems with slow non-equilibrium dynamics. We analyze in detail the non-equilibrium fluctuation-dissipation ratio $X(t,t_{\rm w})$ associated with a defect-pair observable in the Glauber-Ising spin chain. It turns out that $X \neq 1$ throughout the short-time regime and in particular $X(t_{\rm w},t_{\rm w}) = 3/4$ for $t_{\rm w}\to\infty$. We discuss our results in the context of metastable states, which suggests that a violation of short-time quasi-equilibrium behavior could occur in general glassy systems for appropriately chosen observables.

Full paper available as cond-mat/0405711
DOI: 10.1103/PhysRevE.71.046113

P Sollich and C K I Williams. Using the equivalent kernel to understand Gaussian process regression. In L K Saul, Y Weiss and L Bottou, editors, Advances in Neural Information Processing Systems 17, pages 1313-1320, Cambridge, MA, 2005. MIT Press.

Abstract

The equivalent kernel is a way of understanding how Gaussian process regression works for large sample sizes based on a continuum limit. In this paper we show (1) how to approximate the equivalent kernel of the widely-used squared exponential (or Gaussian) kernel and related kernels, and (2) how analysis using the equivalent kernel helps to understand the learning curves for Gaussian processes.

Full paper available as gzip'ped postscript or Adobe PDF

N B Wilding, M Fasolo and P Sollich. Liquid-gas coexistence and critical point shifts in size-disperse fluids. Journal of Chemical Physics, 121:6887-6899, 2004.

Abstract

Specialized Monte Carlo simulations and the moment free energy (MFE) method are employed to study liquid-gas phase equilibria in size-disperse fluids. The investigation is made subject to the constraint of fixed polydispersity, i.e. the form of the `parent' density distribution $\rho^0(\sigma)$ of the particle diameters $\sigma$, is prescribed. This is the experimentally realistic scenario for e.g. colloidal dispersions. The simulations are used to obtain the cloud and shadow curve properties of a Lennard-Jones fluid having diameters distributed according to a Schulz form with a large (40%) degree of polydispersity. Good qualitative accord is found with the results from a MFE method study of a corresponding van der Waals model that incorporates size-dispersity both in the hard core reference and the attractive parts of the free energy. The results show that polydispersity engenders considerable broadening of the coexistence region between the cloud curves. The principal effect of fractionation in this region is a common overall scaling of the particle sizes and typical inter-particle distances, and we discuss why this effect is rather specific to systems with Schulz diameter distributions. Next, by studying a family of such systems with distributions of various widths, we estimate the dependence of the critical point parameters on $\delta$. In contrast to a previous theoretical prediction, size-dispersity is found to raise the critical temperature above its monodisperse value. Unusually for a polydisperse system, the critical point is found to lie at or very close to the extremum of the coexistence region in all cases. We outline an argument showing that such behaviour will occur whenever size polydispersity affects only the range, rather than the strength of the inter-particle interactions.

Full paper available as cond-mat/0406214
DOI: 10.1063/1.1788632

C B Holmes, M Fuchs, M E Cates and P Sollich. Glass transitions and shear thickening suspension rheology. Journal of Rheology, 49:237-269, 2005.

Abstract

We introduce a class of simple models for shear thickening and/ or `jamming' in colloidal suspensions. These are based on schematic mode coupling theory (MCT) of the glass transition, having a memory term that depends on a density variable, and on both the shear stress and the shear rate. (Tensorial aspects of the rheology, such as normal stresses, are ignored for simplicity.) We calculate steady-state flow curves and correlation functions. Depending on model parameters, we find a range of rheological behaviours, including `S-shaped' flow curves, indicating discontinuous shear thickening, and stress-induced transitions from a fluid to a nonergodic (jammed) state, showing zero flow rate in an interval of applied stress. The shear thickening and jamming scenarios that we explore appear broadly consistent with experiments on dense colloids close to the glass transition, despite the fact that we ignore hydrodynamic interactions. In particular, the jamming transition we propose is conceptually quite different from various hydrodynamic mechanisms of shear thickening in the literature, although the latter might remain pertinent at lower colloid densities. Our jammed state is a stress-induced glass, but its nonergodicity transitions have an analytical structure distinct from that of the conventional MCT glass transition.

Full paper available as cond-mat/0406422
DOI: 10.1122/1.1814114

P Mayer and P Sollich. Exact non-equilibrium fluctuation dissipation relations for multi-spin observables in the Glauber-Ising spin chain. Slow dynamics in complex systems, AIP Conference Proceedings 708(1):703-704, 2004.

Abstract

We investigate the relation between two-time. Multi-spin, correlation and response functions in the non-equilibrium dynamics of the Glauber-Ising chain quenched to zero temperature. We find fluctuation-dissipation relations qualitatively similar to those reported in various glassy materials, but quantitatively dependent on the chosen observable. Our results can be understood by considering separately the contributions from large wavevectors, which are at quasi-equilibrium and obey the fluctuation dissipation theorem (FDT), and from small wavevectors where a generalized fdt holds with a non-trivial fluctuation-dissipation ratio $X_\infty$. For spin observables, reflecting critical aspects of the t = 0 quench, we get $X_\infty = 1/2$ while defect observables produce $X_\infty = 0$, revealing the underlying ordered phase. Our results suggest that the definition of an effective temperature $T_{\rm eff} = T/X_\infty$ for non- equilibrated large length scales is generically possible in non-equilibrium critical dynamics.

Full paper available as Adobe PDF
DOI: 10.1063/1.1764270

S M Fielding and P Sollich. Fluctuation-dissipation relations in ageing and driven non-mean field glass models. Slow dynamics in complex systems, AIP Conference Proceedings 708(1):639-642, 2004.

Abstract

We study the fluctuation-dissipation theorem (FDT) in the glass phase of (1) Bouchaud's trap model and (2) its driven counterpart, the ''soft glassy theology'' model. We incorporate into the models an arbitrary observable m and obtain its correlation and response functions in closed form. A limiting non-equilibrium FDT plot (of correlator vs. response) is approached at long times for most choices of m. In contrast to standard mean field models, however, the plot, in general, (i) depends non trivially on the observable, (ii) has a continuously varying slope (even though there is a single scaling of relaxation times with age) and (iii) differs in the ageing and driven regimes. Despite this, all plots share the same limiting slope for well separated times, suggesting that a meaningful non-equilibrium effective temperature could apply in this limit. Beyond the trap model, we discuss more generally the status of FD temperatures in such non-mean field systems.

Full paper available as Adobe PDF
DOI: 10.1063/1.1764243

M Fasolo and P Sollich. Effects of colloid polydispersity on the phase behaviour of colloid-polymer mixtures. Journal of Chemical Physics, 122:074904, 2005.

Abstract

We study theoretically the equilibrium phase behaviour of a mixture of polydisperse hard-sphere colloids and monodisperse polymers, modelled using the Asakura-Oosawa model within the free volume approximation of Lekkerkerker et al. We compute full phase diagrams in the plane of colloid and polymer volume fractions, using the moment free energy method. The intricate features of phase separation in pure polydisperse colloids combine with the appearance of polymer-induced gas-liquid coexistence to give a rich variety of phase diagram topologies as the polymer-colloid size ratio and the colloid polydispersity are varied. Quantitatively, we find that polydispersity disfavours fluid-solid against gas-liquid separation, causing a substantial lowering of the threshold value above which stable two-phase gas-liquid coexistence appears. Phase splits involving two or more solids can occur already at low colloid concentration, where they may be kinetically accessible. We also analyse the strength of colloidal size fractionation. When a solid phase separates from a fluid, its polydispersity is reduced most strongly if the phase separation takes place at low colloid concentration and high polymer concentration, in agreement with experimental observations. For fractionation in gas-liquid coexistence we likewise find good agreement with experiment, as well as with perturbative theories for near-monodisperse systems.

Full paper available as cond-mat/0410374
DOI: 10.1063/1.1851978

M Fasolo and P Sollich. Effects of polymer polydispersity on the phase behaviour of colloid-polymer mixtures. Journal of Physics: Condensed Matter, 17(6):797-812, 2005.

Abstract

We study the equilibrium behaviour of a mixture of monodisperse hard sphere colloids and polydisperse non-adsorbing polymers at their $\theta$-point, using the Asakura-Oosawa model treated within the free-volume approximation. Our focus is the experimentally relevant scenario where the distribution of polymer chain lengths across the system is fixed. Phase diagrams are calculated using the moment free energy method, and we show that the mean polymer size $\xi_{\rm c}$ at which gas-liquid phase separation first occurs decreases with increasing polymer polydispersity $\delta$. Correspondingly, at fixed mean polymer size, polydispersity favours gas-liquid coexistence but delays the onset of fluid-solid separation. On the other hand, we find that systems with different $\delta$ but the same mass-averaged polymer chain length have nearly polydispersity-independent phase diagrams. We conclude with a comparison to previous calculations for a semi-grandcanonical scenario, where the polymer chemical potentials are imposed, which predicted that fluid-solid coexistence was over gas-liquid in some areas of the phase diagram. Our results show that this somewhat counter-intuitive result arose because the actual polymer size distribution in the system is shifted to smaller sizes relative to the polymer reservoir distribution.

Full paper available as cond-mat/0411701
DOI: 10.1088/0953-8984/17/6/002

P Sollich. Soft glassy rheology. In R G Weiss, P Terech, editors, Molecular Gels: Materials with Self-Assembled Fibrillar Networks, pages 161-192, Dordrecht, 2006. Springer.

Abstract

We review models for the rheology of soft glasses, a class of materials including e.g. emulsions, foams, colloidal glasses and possibly--but with substantial caveats--gels. The main focus is on the soft glassy rheology (SGR) model, and in particular on the occurrence of rheological aging effects. We first review appropriate definitions of rheological response functions suited to aging samples (in which time translation invariance is lost). These are then used to study aging effects within the SGR model. Its constitutive equations relate shear stress to shear strain among a set of elastic elements, with distributed yield thresholds, undergoing activated dynamics governed by a ``noise temperature", $x$. For $1<x<2$ there is a power-law fluid regime in which transients occur, but no aging. For $x<1$, the model has a macroscopic yield stress. So long as this yield stress is not exceeded, aging occurs, with a sample's apparent relaxation time being of the order of its own age. The (age-dependent) linear viscoelastic loss modulus $G''(\omega,t)$ rises as frequency is lowered, but falls with age $t$, so as to always remain less than $G'(\omega,t)$ (which is nearly constant). Significant aging is also predicted for the stress overshoot in nonlinear shear startup and for the creep compliance. We discuss an extension of the model to include a proper tensorial description of stress and strain, and survey some related rheological models that have recently been developed.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Nematic-nematic demixing in polydisperse thermotropic liquid crystals. Journal of Chemical Physics, 122:214911, 2005.

Abstract

We consider the effects of polydispersity on isotropic-nematic phase equilibria in thermotropic liquid crystals, using a Maier-Saupe theory with factorized interactions. A sufficient spread (approx. 50%) in the interaction strengths of the particles leads to phase separation into two or more nematic phases, which can in addition coexist with an isotropic phase. The isotropic-nematic coexistence region widens dramatically as polydispersity is increased, leading to re-entrant isotropic-nematic phase separation in some regions of the phase diagram. We show that similar phenomena will occur also for non-factorized interactions as long as the interaction strength between any two particle species is lower than the mean of the intra-species interactions.

Full paper available as cond-mat/0501421
DOI: 10.1063/1.1924604

A C C Coolen, R Kühn and P Sollich. Theory of Neural Information Processing. Oxford University Press, 2005.

Abstract

This book takes the reader on a grand tour of neural networks and the advanced information processing systems they have inspired, with an emphasis on the mathematical tools and techniques used to analyse them. Uniquely, it covers both the operation of recurrent neural networks as dynamical systems and the use of feedforward networks and related algorithms for learning and statistical inference, bringing out the mathematical connections between these areas. Topics discussed include: biological and model neurons; learning dynamics in perceptrons; multi-layer networks; recurrent neural networks as associative memories; unsupervised learning, vector quantization and self-organized maps; Bayesian techniques for supervised learning; Gaussian processes; support vector machines; entropy, information and applications to statistical inference; macroscopic analysis of network operation and learning; equilibrium statistical mechanics and applications to stationary network operation and learnability.

The book is based on the core lecture courses from a well-established M.Sc. programme in Information Processing and Neural Networks at King's College London, and is ideal for self-study, as a textbook for postgraduate or advance undergraduate courses, or as a reference volume for researches and practitioners in data mining, machine learning, neural information processing and artificial intelligence. Exercises are used throughout the text to advance and deepen the reader's knowledge, and notes on historical background and suggestions for further reading guide the student into the literature. All mathematical details are included; appendices provide further background material on e.g. probability theory, linear algebra and stochastic processes, making the book accessible to a wide audience.

P Sollich and C K I Williams. Understanding Gaussian process regression using the equivalent kernel. In J Winkler, N Lawrence and M Niranjan, editors, Deterministic and Statistical Methods in Machine Learning, Lecture Notes in Artificial Intelligence 3635, pages 199-210, Berlin, 2005. Springer.

Abstract

The equivalent kernel is a way of understanding how Gaussian process regression works for large sample sizes based on a continuum limit. In this paper we show how to approximate the equivalent kernel of the widely-used squared exponential (or Gaussian) kernel and related kernels. This is easiest for uniform input densities, but we also discuss the generalization to the non-uniform case. We show further that the equivalent kernel can be used to understand the learning curves for Gaussian processes, and investigate how kernel smoothing using the equivalent kernel compares to full Gaussian process regression.

Full paper available as gzip'ped postscript or Adobe PDF

P Sollich. Can Gaussian process regression be made robust against model mismatch? In J Winkler, N Lawrence and M Niranjan, editors, Deterministic and Statistical Methods in Machine Learning, Lecture Notes in Artificial Intelligence 3635, pages 211-228, Berlin, 2005. Springer.

Abstract

Learning curves for Gaussian process (GP) regression can be strongly affected by a mismatch between the `student' model and the `teacher' (true data generation process), exhibiting e.g. multiple overfitting maxima and logarithmically slow learning. I investigate whether GPs can be made robust against such effects by adapting student model hyperparameters to maximize the evidence (data likelihood). An approximation for the average evidence is derived and used to predict the optimal hyperparameter values and the resulting generalization error. For large input space dimension, where the approximation becomes exact, Bayes-optimal performance is obtained at the evidence maximum, but the actual hyperparameters (e.g. the noise level) do not necessarily reflect the properties of the teacher. Also, the theoretically achievable evidence maximum cannot always be reached with the chosen set of hyperparameters, and maximizing the evidence in such cases can actually make generalization performance worse rather than better. In lower-dimensional learning scenarios, the theory predicts--in excellent qualitative and good quantitative accord with simulations--that evidence maximization eliminates logarithmically slow learning and recovers the optimal scaling of the decrease of generalization error with training set size.

Full paper available as gzip'ped postscript or Adobe PDF

C Gold and P Sollich. Fast Bayesian Support Vector Machine parameter tuning with the Nystrom method. In International Joint Conference on Neural Networks (IJCNN) 2005, vols. 1-5, pages 2820-2825, New York, 2005. IEEE.

Abstract

We experiment with speeding up a Bayesian method for tuning the hyperparameters of a Support Vector Machine (SVM) classifier. The Bayesian approach gives the gradients of the evidence as averages over the posterior, which can be approximated using Hybrid Monte Carlo simulation (HMC). By using the Nystrom approximation to the SVM kernel, our method significantly reduces the dimensionality of the space to be simulated in the HMC. We show that this speeds up the running time of the HMC simulation from $O(n^2)$ (with a large prefactor) to effectively $O(n)$, where $n$ is the number of training samples. We conclude that the Nystrom approximation has an almost insignificant effect on the performance of the algorithm when compared to the full Bayesian method, and gives excellent performance in comparison with other approaches to hyperparameter tuning.

Full paper available as Adobe PDF
DOI: 10.1109/IJCNN.2005.1556372

P Mayer, P Sollich, L Berthier, J P Garrahan. Dynamic heterogeneity in the Glauber-Ising chain. Journal of Statistical Mechanics: Theory and Experiment, P05002, 2005.

Abstract

In a recent paper [P. Mayer et al., Phys. Rev. Lett. 93, 115701 (2004)] it was shown, by means of experiments, theory and simulations, that coarsening systems display dynamic heterogeneity analogous to that of glass formers. Here, we present a detailed analysis of dynamic heterogeneities in the Glauber-Ising chain. We discuss how dynamic heterogeneity in Ising systems must be measured through connected multi-point correlation functions. We show that in the coarsening regime of the Ising chain these multi-point functions reveal the growth of spatial correlations in the dynamics, beyond what can be inferred from standard two-point correlations. They have non-trivial scaling properties, which we interpret in terms of the diffusion-annihilation dynamics of domain walls. In the equilibrium dynamics of the Ising chain, on the other hand, connected multi-point functions vanish exactly and dynamic heterogeneity is not observed. Our results highlight the similarities between coarsening systems and glass formers.

Full paper available as cond-mat/0502271
DOI: 10.1088/1742-5468/2005/05/P05002

C Gold, A Holub and P Sollich. Bayesian approach to feature selection and parameter tuning for Support Vector Machine classifiers. Neural Networks, 18(5-6), 693-701, 2005.

Abstract

A Bayesian point of view of SVM classifiers allows the definition of a quantity analogous to the evidence in probabilistic models. By maximizing this one can systematically tune hyperparameters and, via automatic relevance determination (ARD), select relevant input features. Evidence gradients are expressed as averages over the associated posterior and can be approximated using Hybrid Monte Carlo (HMC) sampling. We describe how a Nyström approximation of the Gram matrix can be used to speed up sampling times significantly while maintaining almost unchanged classification accuracy. In experiments on classification problems with a significant number of irrelevant features this approach to ARD can give a significant improvement in classification performance over more traditional, non-ARD, SVM systems. The final tuned hyperparameter values provide a useful criterion for pruning irrelevant features, and we define a measure of relevance with which to determine systematically how many features should be removed. This use of ARD for hard feature selection can improve classification accuracy in non-ARD SVMs. In the majority of cases, however, we find that in data sets constructed by human domain experts the performance of non-ARD SVMs is largely insensitive to the presence of some less relevant features. Eliminating such features via ARD then does not improve classification accuracy, but leads to impressive reductions in the number of features required, by up to 75%.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1016/j.neunet.2005.06.044

D Barber and P Sollich. Stable Belief Propagation in Gaussian DAGs. In International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2007, vol. 2, pages 409-412, 2007. IEEE.

Abstract

We consider approximate inference in the important class of large Gaussian distributions corresponding to multiply-connected directed acylic networks. We show how Directed Belief Propagation can be implemented in a numerically stable manner by associating backward ($\lambda$) messages with an auxiliary variable, enabling intermediate computations to be carried out in moment form. We apply our method to the Fast Fourier Transform network with missing data, and show that the results are more accurate than those obtained using Undirected Belief Propagation on the equivalent Markov network.

Full paper available as gzip'ped postscript or Adobe PDF
DOI: 10.1109/ICASSP.2007.366259

N B Wilding, P Sollich and M Fasolo. Finite-size scaling and particle-size cutoff effects in phase-separating polydisperse fluids. Physical Review Letters, 95:155701, 2005.

Abstract

We study the liquid-vapor phase behaviour of a polydisperse fluid using grand canonical simulations and moment free energy calculations. The strongly nonlinear variation of the fractional volume of liquid across the coexistence region prevents naive extrapolation to detect the cloud point. We describe a finite-size scaling method which nevertheless permits accurate determination of cloud points and spinodals from simulations of a single system size. By varying a particle size cutoff we find that the cloud point density is highly sensitive to the presence of rare large particles; this could affect the reproducibility of experimentally measured phase behavior in colloids and polymers.

Full paper available as cond-mat/0506760
DOI: 10.1103/PhysRevLett.95.155701

N B Wilding and P Sollich. Liquid-vapour phase behaviour of a polydisperse Lennard-Jones fluid. Journal of Physics: Condensed Matter, 17(45):S3245-S3252, 2005.

Abstract

We describe a simulation study of the liquid-vapor phase behaviour of a model polydisperse fluid. Particle interactions are given by a Lennard-Jones potential in which polydispersity features both in the particle sizes and the amplitude of their interactions. We address the computationsl problem of accurately locating the cloud curve for such asystem using Monte Carlo simulations within the grand canonical ensemble. The strongly non-linear variation of the fractional volumes of the phases across the coexistence region precludes naive extrapolation to determine the cloud point density. Instead we propose an improved estimator for the cloud point location and use scaling arguments to predict its finite-size behaviour. Excellent agreement is found with the simulation results. Application of the method reveals that the measured cloud curve is highly sensitive to the presence of large particles even when they are extremely rare. This finding is expected to have implications for the reproducibility of experimentally measured phase diagrams in colloids and polymers.

Full paper available as gzip'ped postscript
DOI: 10.1088/0953-8984/17/45/008

A Garriga, P Sollich, I Pagonabarraga and F Ritort. Universality of fluctuation-dissipation ratios: the ferromagnetic model. Physical Review E, 72:056114, 2005.

Abstract

We calculate analytically the fluctuation-dissipation ratio (FDR) for Ising ferromagnets quenched to criticality, both for the long-range model and its short-range analogue in the limit of large dimension. Our exact solution shows that, for both models, $X^\infty=1/2$ if the system is unmagnetized while $X^\infty=4/5$ if the initial magnetization is non-zero. This indicates that two different classes of critical coarsening dynamics need to be distinguished depending on the initial conditions, each with its own nontrivial FDR. We also analyze the dependence of the FDR on whether local and global observables are used. These results clarify how a proper local FDR (and the corresponding effective temperature) should be defined in long-range models in order to avoid spurious inconsistencies and maintain the expected correspondence between local and global results; global observables turn out to be far more robust tools for detecting non-equilibrium FDRs.

Full paper available as cond-mat/0508243
DOI: 10.1103/PhysRevE.72.056114

P Mayer, S Léonard, L Berthier, J P Garrahan and P Sollich. Activated aging dynamics and negative fluctuation-dissipation ratios. Physical Review Letters, 96:030602, 2006.

Abstract

In glassy materials aging proceeds at large times via thermal activation. We show that this can lead to negative dynamical response functions and novel and well-defined violations of the fluctuation-dissipation theorem, in particular, negative fluctuation-dissipation ratios. Our analysis is based on detailed theoretical and numerical results for the activated aging regime of simple kinetically constrained models. The results are relevant to a variety of physical situations such as aging in glass-formers, thermally activated domain growth and granular compaction.

Full paper available as cond-mat/0508686
DOI: 10.1103/PhysRevLett.96.030602

A Annibale and P Sollich. Spin, bond and global fluctuation-dissipation relations in the non-equilibrium spherical ferromagnet. Journal of Physics A, 39:2853-2907, 2006.

Abstract

We study the out-of-equilibrium dynamics of the spherical ferromagnet after a quench to its critical temperature. We calculate correlation and response functions for spin observables which probe lengthscales much larger than the lattice spacing but smaller than the system size, and find that the asymptotic fluctuation-dissipation ratio (FDR) $X^\infty$ is the same as for local observables. This is consistent with our earlier results for the Ising model in dimension $d=1$ and $d=2$. We also check that bond observables, both local and long-range, give the same asymptotic FDR. In the second part of the paper the analysis is extended to global observables, which probe correlations among all $N$ spins. Here non-Gaussian fluctuations arising from the spherical constraint need to be accounted for, and we develop a systematic expansion in $1/\sqrt{N}$ to do this. Applying this to the global bond observable, i.e. the energy, we find that non-Gaussian corrections change its FDR to a nontrivial value which we calculate exactly for all dimensions $d>2$. Finally, we consider quenches from magnetized initial states. Here even the FDR for the global spin observable, i.e. the magnetization, is nontrivial. It differs from the one for unmagnetized states even in $d>4$, signalling the appearance of a distinct dynamical universality class of magnetized critical coarsening. For lower $d$ the FDR is irrational even to first order in $4-d$ and $d-2$, the latter in contrast to recent results for the transverse FDR in the $n$-vector model.

Full paper available as cond-mat/0510731
DOI: 10.1088/0305-4470/39/12/002

P Sollich. Trap models with slowly decorrelating observables. Journal of Physics A, 39:2573-2597, 2006.

Abstract

We study the correlation and response dynamics of trap models of glassy dynamics, considering observables that only partially decorrelate with every jump. This is inspired by recent work on a microscopic realization of such models, which found strikingly simple linear out-of-equilibrium fluctuation-dissipation relations in the limit of slow decorrelation. For the Barrat-Mezard model with its entropic barriers we obtain exact results at zero temperature $T$ for arbitrary decorrelation factor $\kappa$. These are then extended to nonzero $T$, where the qualitative scaling behaviour and all scaling exponents can still be found analytically. Unexpectedly, the choice of transition rates (Glauber versus Metropolis) affects not just prefactors but also some exponents. In the limit of slow decorrelation even complete scaling functions are accessible in closed form. The results show that slowly decorrelating observables detect persistently slow out-of-equilibrium dynamics, as opposed to intermittent behaviour punctuated by excursions into fast, effectively equilibrated states.

Full paper available as cond-mat/0601007
DOI: 10.1088/0305-4470/39/11/004

R L Jack, P Mayer and P Sollich. Mappings between reaction-diffusion and kinetically constrained systems: A+A $\leftrightarrow$ A and the FA model have upper critical dimension $d_{\rm c}=2$. Journal of Statistical Mechanics: Theory and Experiment, P03006, 2006.

Abstract

We present an exact mapping between two simple spin models: the Fredrickson-Andersen (FA) model and a model of annihilating random walks with spontaneous creation from the vacuum, A+A <-> 0. We discuss the geometric structure of the mapping and its consequences for symmetries of the models. Hence we are able to show that the upper critical dimension of the FA model is two, and that critical exponents are known exactly in all dimensions. These conclusions also generalise to a mapping between A+A <-> 0 and the reaction-diffusion system in which the reactions are branching and coagulation, A+A <-> A. We discuss the relation of our analysis to earlier work, and explain why the models considered do not fall into the directed percolation universality class.

Full paper available as cond-mat/0601529
DOI: 10.1088/1742-5468/2006/03/P03006

M Buzzacchi, P Sollich, N B Wilding and M Müller. Simulation estimates of cloud points of polydisperse fluids. Physical Review E, 73:046110, 2006.

Abstract

We describe two distinct approaches to obtaining cloud point densities and coexistence properties of polydisperse fluid mixtures by Monte Carlo simulation within the grand canonical ensemble. The first method determines the chemical potential distribution $\mu(\sigma)$ (with $\sigma$ the polydisperse attribute) under the constraint that the ensemble average of the particle density distribution $\rho(\sigma)$ matches a prescribed parent form. Within the region of phase coexistence (delineated by the cloud curve) this leads to a distribution of the fluctuating overall particle density n, p(n), that necessarily has unequal peak weights in order to satisfy a generalized lever rule. A theoretical analysis shows that as a consequence, finite-size corrections to estimates of coexistence properties are power laws in the system size. The second method assigns $\mu(\sigma)$ such that an equal peak weight criterion is satisfied for $p(n)$ for all points within the coexistence region. However, since equal volumes of the coexisting phases cannot satisfy the lever rule for the prescribed parent, their relative contributions must be weighted appropriately when determining $\mu(\sigma)$. We show how to ascertain the requisite weight factor operationally. A theoretical analysis of the second method suggests that it leads to finite-size corrections to estimates of coexistence properties which are exponentially small in the system size. The scaling predictions for both methods are tested via Monte Carlo simulations of a novel polydisperse lattice gas model near its cloud curve, the results showing excellent quantitative agreement with the theory.

Full paper available as cond-mat/0602043
DOI: 10.1103/PhysRevE.73.046110

N B Wilding, P Sollich, M Fasolo and M Buzzacchi. Phase behaviour and particle-size cutoff effects in polydisperse fluids Journal of Chemical Physics, 125:014908, 2006.

Abstract

We report a joint simulation and theoretical study of the liquid-vapor phase behaviour of a fluid in which polydispersity in the particle size couples to the strength of the interparticle interactions. Attention is focussed on the case in which the particles diameters are distributed according to a fixed Schulz form with degree of polydispersity $\delta=14\%$. The coexistence properties of this model are studied using grand canonical ensemble Monte Carlo simulations and moment free energy calculations. We obtain the cloud and shadow curves as well as the daughter phase density distributions and fractional volumes along selected isothermal dilution lines. In contrast to the case of size-independent interaction strengths (N.B. Wilding, M. Fasolo and P. Sollich, J. Chem. Phys. 121, 6887 (2004)), the cloud and shadow curves are found to be well separated, with the critical point lying significantly below the cloud curve maximum. For densities below the critical value, we observe that the phase behaviour is highly sensitive to the choice of upper cutoff on the particle size distribution. We elucidate the origins of this effect in terms of extremely pronounced fractionation effects and discuss the likely appearance of new phases in the limit of very large values of the cutoff.

Full paper available as cond-mat/0604548
DOI: 10.1063/1.2208358

M Buzzacchi, N B Wilding, P Sollich. Wetting transitions in polydisperse fluids. Physical Review Letters, 97:136104, 2006.

Abstract

The properties of the coexisting bulk gas and liquid phases of a polydisperse fluid depend not only on the prevailing temperature, but also on the overall parent density. As a result, a polydisperse fluid near a wall will exhibit density-driven wetting transitions inside the coexistence region. We propose a likely topology for the wetting phase diagram, which we test using Monte Carlo simulations of a model polydisperse fluid at an attractive wall, tracing the wetting line inside the cloud curve and identifying the relationship to prewetting.

Full paper available as cond-mat/0609118
DOI: 10.1103/PhysRevLett.97.136104

L Khoo, Z Cvetkovic, and P Sollich. Robustness of phoneme classification in different representation spaces. Proceedings of EUSIPCO 2006, Florence, Italy, September 4-8, 2006.

Abstract

The robustness of phoneme recognition using support vector machines to additive noise is investigated for three kinds of speech representation. The representations considered are PLP, PLP with RASTA processing, and a high-dimensional principal component approximation of acoustic waveforms. While the classification in the PLP and PLP/RASTA domains attains superb accuracy on clean data, the classification in the high-dimensional space proves to be much more robust to additive noise.

Full paper available as Adobe PDF

R Fantoni, D Gazzillo, A Giacometti and P Sollich. Phase behavior of weakly polydisperse sticky hard spheres: Perturbation theory for the Percus-Yevick solution. Journal of Chemical Physics, 125:164504, 2006.

Abstract

We study the effects of size polydispersity on the gas-liquid phase behaviour of mixtures of sticky hard spheres. To achieve this, the system of coupled quadratic equations for the contact values of the partial cavity functions of the Percus-Yevick solution is solved within a perturbation expansion in the polydispersity, i.e. the normalized width of the size distribution. This allows us to make predictions for various thermodynamic quantities which can be tested against numerical simulations and experiments. In particular, we determine the leading-order effects of size polydispersity on the cloud curve delimiting the region of two-phase coexistence and on the associated shadow curve; we also study the extent of size fractionation between the coexisting phases. Different choices for the size-dependence of the adhesion strengths are examined carefully; the Asakura-Oosawa model of a mixture of polydisperse colloids and small polymers is studied as a specific example.

Full paper available as cond-mat/0608714
DOI: 10.1063/1.2358136

D Gazzillo, A Giacometti, R Fantoni and P Sollich. Multicomponent adhesive hard sphere models and short-ranged attractive interactions in colloidal or micellar solutions. Physical Review E, 74:051407, 2006.

Abstract

We investigate the dependence of the stickiness parameters $t_{ij}=1/(12\tau _{ij})$ - where the $\tau _{ij}$ are the conventional Baxter parameters - on the solute diameters $\sigma _{i}$ and $\sigma _{j}$ in multicomponent sticky hard sphere (SHS) models for fluid mixtures of mesoscopic neutral particles. A variety of simple but realistic interaction potentials, utilized in the literature to model short-ranged attractions present in real solutions of colloids or reverse micelles, is reviewed. We consider: i) van der Waals attractions, ii) hard-sphere-depletion forces, iii) polymer-coated colloids, iv) solvation effects (in particular hydrophobic bonding and attractions between reverse micelles of water-in-oil microemulsions). We map each of these potentials onto an equivalent SHS model, by requiring the equality of the second virial coefficients. The main finding is that, for most of the potentials considered, the size-dependence of $t_{ij}(T,\sigma _{i},\sigma _{j})$ can be approximated by essentially the same expression, i.e. a simple polynomial in the variable $\sigma _{i}\sigma _{j}/\sigma _{ij}^{2}$, with coefficients depending on the temperature $T$, or - for depletion interactions - on the packing fraction $\eta _{0}$ of the depletant particles.

Full paper available as cond-mat/0611175
DOI: 10.1103/PhysRevE.74.051407

P Mayer and P Sollich. Aging in one-dimensional coagulation-diffusion processes and the Fredrickson-Andersen model. Journal of Physics A, 40:5823-5856, 2007.

Abstract

We analyse the aging dynamics of the one-dimensional Fredrickson-Andersen (FA) model in the nonequilibrium regime following a low temperature quench. Relaxation then effectively proceeds via diffusion limited pair coagulation (DLPC) of mobility excitations. By employing a familiar stochastic similarity transformation, we map exact results from the free fermion case of diffusion limited pair annihilation to DLPC. Crucially, we are able to adapt the mapping technique to averages involving multiple time quantities. This relies on knowledge of the explicit form of the evolution operators involved. Exact results are obtained for two-time correlation and response functions in the free fermion DLPC process. The corresponding long-time scaling forms apply to a wider class of DLPC processes, including the FA model. We are thus able to exactly characterise the violations of the fluctuation-dissipation theorem (FDT) in the aging regime of the FA model. We find nontrivial scaling forms for the fluctuation-dissipation ratio (FDR) $X = X(t_{\rm w}/t)$, but with a negative asymptotic value $X^\infty = -3\pi/(6\pi - 16) \approx -3.307$. While this prevents a thermodynamic interpretation in terms of an effective temperature, it is a direct consequence of probing FDT with observables that couple to activated dynamics. The existence of negative FDRs should therefore be a widespread feature in non mean-field systems.

Full paper available as cond-mat/0702618
DOI: 10.1088/1751-8113/40/22/005

S Léonard, P Mayer, P Sollich, L Berthier and J P Garrahan. Non-equilibrium dynamics of spin facilitated glass models. Journal of Statistical Mechanics: Theory and Experiment, P07017, 2007.

Abstract

We consider the dynamics of spin facilitated models of glasses in the non-equilibrium aging regime following a sudden quench from high to low temperatures. We briefly review known results obtained for the broad class of kinetically constrained models, and then present new results for the behaviour of the one-spin facilitated Fredrickson-Andersen and East models in various spatial dimensions. The time evolution of one-time quantities, such as the energy density, and the detailed properties of two-time correlation and response functions are studied using a combination of theoretical approaches, including exact mappings of master operators and reductions to integrable quantum spin chains, field theory and renormalization group, and independent interval and timescale separation methods. The resulting analytical predictions are confirmed by means of detailed numerical simulations. The models we consider are characterized by trivial static properties, with no finite temperature singularities, but they nevertheless display a surprising variety of dynamic behaviour during aging, which can be directly related to the existence and growth in time of dynamic lengthscales. Well-behaved fluctuation-dissipation ratios can be defined for these models, and we study their properties in detail. We confirm in particular the existence of negative fluctuation-dissipation ratios for a large number of observables. Our results suggest that well-defined violations of fluctuation-dissipation relations, of a purely dynamic origin and unrelated to the thermodynamic concept of effective temperatures, could in general be present in non-equilibrium glassy materials.

Full paper available as cond-mat/0703164
DOI: 10.1088/1742-5468/2007/07/P07017

Z Ghahramani, T L Griffiths, P Sollich. Bayesian nonparametric latent feature models. In J M Bernardo, M J Bayarri, J O Berger, A P Dawid, D Heckerman, A F M Smith and M West, editors, Bayesian Statistics 8, pages 201-225, 2007. Oxford University Press.

Abstract

We describe a flexible nonparametric approach to latent variable modelling in which the number of latent variables is unbounded. This approach is based on a probability distribution over equivalence classes of binary matrices with a finite number of rows, corresponding to the data points, and an unbounded number of columns, corresponding to the latent variables. Each data point can be associated with a subset of the possible latent variables, which we re- fer to as the latent features of that data point. The binary variables in the matrix indicate which latent feature is possessed by which data point, and there is a potentially infinite array of features. We derive the distribution over unbounded binary matrices by taking the limit of a distribution over $N \times K$ binary matrices as $K\to\infty$. We define a simple generative processes for this distribution which we call the Indian buffet process (IBP; Griffiths and Ghahramani, 2005, 2006) by analogy to the Chinese restaurant process (Aldous, 1985; Pitman, 2002). The IBP has a single hyperparameter which controls both the number of feature per object and the total number of fea- tures. We describe a two-parameter generalization of the IBP which has addi- tional flexibility, independently controlling the number of features per object and the total number of features in the matrix. The use of this distribution as a prior in an infinite latent feature model is illustrated, and Markov chain Monte Carlo algorithms for inference are described.

Full paper available as Adobe PDF

N B Wilding, P Sollich, M Buzzacchi. Polydisperse lattice-gas model. Physical Review E 77:011501, 2008.

Abstract

We describe a lattice-gas model suitable for studying the generic effects of polydispersity on liquid-vapor phase equilibria. Using Monte Carlo simulation methods tailored for the accurate determination of phase behaviour under conditions of fixed polydispersity, we trace the cloud and shadow curves for a particular Schulz distribution of the polydisperse attribute. Although polydispersity enters the model solely in terms of the strengths of the interparticle interactions, this is sufficient to induce the broad separation of cloud and shadow curves seen both in more realistic models and experiments.

Full paper available as arXiv:0709.0889
DOI: 10.1103/PhysRevE.77.011501

P Sollich. Weakly polydisperse systems: Perturbative phase diagrams that include the critical region. Physical Review Letters, 100:035701, 2008.

Abstract

The phase behaviour of a weakly polydisperse system, such as a colloid with a small spread of particle sizes, can be related perturbatively to that of its monodisperse counterpart. I show how this approach can be generalized to remain well-behaved near critical points, avoiding the divergences of existing methods and giving access to some of the key qualitative features of polydisperse phase equilibria. The analysis explains also why in purely size polydisperse systems the critical point is, unusually, located very near the maximum of the cloud and shadow curves.

Full paper available as arXiv:0709.1399
DOI: 10.1103/PhysRevLett.100.035701

R L Jack and P Sollich. Duality between random trap and barrier models. Journal of Physics A, 41:324001, 2008.

Abstract

We discuss the physical consequences of a duality between two models with quenched disorder, in which particles propagate in one dimension among random traps or across random barriers. We derive an exact relation between their diffusion fronts at fixed disorder, and deduce from this that their disorder-averaged diffusion fronts are exactly equal. We use effective dynamics schemes to isolate the different physical processes by which particles propagate in the models and discuss how the duality arises from a correspondence between the rates for these different processes.

Full paper available as arXiv:0710.1665
DOI: 10.1088/1751-8113/41/32/324001

N Betteridge, Z Cvetkovic and P Sollich. Phoneme classification in frequency subbands using ensemble methods. In 15th International Conference on Digital Signal Processing, pages 511-514, 2007. IEEE.

Abstract

Phoneme classification in frequency bands of acoustic waveforms is studied. The goal is to investigate whether separate classifications across a number of subband signals, combined using appropriate machine learning algorithms, can provide performance similar to classification performed directly on the original acoustic waveforms. If this is the case, then combining subband classifications might lead to speech recognition algorithms that are robust to linear filtering and narrow-band noise. We perform proof-of-concept experiments on three binary phoneme classification tasks of varying difficulty, using Support Vector Machine subband classifiers which are combined by simple and weighted voting techniques as well as stacked generalization methods. We find that combining subband classifiers improves performance and that the improvement becomes more marked as the number of subbands increases.

Full paper available as Adobe PDF
DOI: 10.1109/ICDSP.2007.4288631

M Ager, Z Cvetkovic, P Sollich and J Yousafzai. Robustness of phoneme classification using generative classifiers: Comparison of the acoustic waveform and PLP representations. Technical Report.

Abstract

The robustness of classification of isolated phoneme segments using generative classifiers is investigated for the acoustic waveform and PLP speech representations. Probabilistic PCA is used to fit a density to each phoneme class followed by maximum likelihood classification. The results show that although PLP performs best in quiet conditions, as the SNR decreases below $0$dB acoustic waveforms have a lower classification error. This is the case even though the waveform classifier is trained explicitly only on quiet data and is then modified by a simple transformation to account for the noise, whereas for PLP separate classifiers are trained for each noise condition. Even at $-18$dB SNR, multiclass performance of classification from waveforms is still significantly better than chance level. In addition the effect of time-alignment is tested and initial solution shown.

Full paper available as Adobe PDF

J Yousafzai, Z Cvetkovic, P Sollich and M Ager. Robustness of phoneme classification using Support Vector Machines: A comparison between PLP and acoustic waveform representations. Technical Report.

Abstract

Robustness of phoneme recognition to additive noise is investigated for PLP and acoustic waveform representations of speech using support vector machines (SVMs) combined via error-correcting code methods. While recognition in the PLP domain attains superb accuracy on clean data, it is significantly affected by mismatch between training and testing noise levels. The classification in the high-dimensional acoustic waveform domain, on the other hand, is more robust to additive noise. Moreover, these classifiers perform best when trained on clean data. We also show that the simpler structure of the waveform representation allows one to improve performance using custom-designed kernel functions.

Full paper available as Adobe PDF

J Yousafzai, M Ager, Z Cvetkovic and P Sollich. Discriminative and generative machine learning approaches towards robust phoneme classification. In Information Theory and Applications, 2008.

Abstract

Robustness of classification of isolated phoneme segments using discriminative and generative classifiers is investigated for the acoustic waveform and PLP speech representations. The two approaches used are support vector machines (SVMs) and mixtures of probabilistic PCA (MPPCA). While recognition in the PLP domain attains superb accuracy on clean data, it is significantly affected by mismatch between training and test noise levels. Classification in the high-dimensional acoustic waveform domain, on the other hand, is more robust in the presence of additive white Gaussian noise. We also show some results on the effects of custom-designed kernel functions for SVM classification in the acoustic waveform domain.

Full paper available as Adobe PDF
DOI: 10.1109/ITA.2008.4601091

A Annibale and P Sollich. Fluctuation-dissipation relations in critical coarsening: crossover from unmagnetized to magnetized initial states. Journal of Physics A, 41:135001, 2008.

Abstract

We study the non-equilibrium dynamics of the spherical ferromagnet quenched to its critical temperature, as a function of the magnetization of the initial state. The two limits of unmagnetized and fully magnetized initial conditions can be understood as corresponding to times that are respectively much shorter and much longer than a magnetization timescale, as in a recent field theoretical analysis of the $n$-vector model. We calculate exactly the crossover functions interpolating between these two limits, for the magnetization correlator and response and the resulting fluctuation-dissipation ratio (FDR). For $d>4$ our results match those obtained recently from a Gaussian field theory. For $d<4$, non-Gaussian fuctuations arising from the spherical constraint need to be accounted for. We extend our framework from the fully magnetized case to achieve this, providing an exact solution for the relevant integral kernel. The resulting crossover behaviour is very rich, with the asymptotic FDR $X^\infty$ depending non-monotonically on the scaled age of the system. This is traced back to non-monotonicities of the two-time correlator, themselves the consequence of large magnetization fluctuations on the crossover timescale. We correct a trivial error in our earlier calculation for fully magnetized initial states; the corrected FDR is consistent with renormalization group expansions to first order in $4-d$ for the longitudinal fluctuations of the O(n) model in the limit $n\to\infty$.

Full paper available as arXiv:0801.1381
DOI: 10.1088/1751-8113/41/13/135001

M Ager, Z Cvetkovic, P Sollich and Bin Yu. Towards robust phoneme classification: Augmentation of PLP models with acoustic waveforms. In Proceedings of European Signal Processing Conference, 2008.

Abstract

The robustness of classification of phoneme segments using generative classifiers is investigated for the PLP and acoustic waveform speech representations in the presence of white Gaussian noise. We combine the strengths of both representations, specifically the excellent classification accuracy of PLP in quiet conditions with the additional robustness of acoustic waveform classifiers. This is achieved using a convex combination of their respective log-likelihoods to produce a combined decision function. The resulting combined classifier is uniformly as accurate as PLP alone and is significantly more robust to the presence of additive noise during testing. Issues of noise modelling and time-invariant classification of acoustic waveforms are also considered with initial solutions used to improve accuracy.

Full paper available as Adobe PDF

J Yousafzai, Z Cvetkovic, P Sollich and Bin Yu. Combined PLP-acoustic waveform classification for robust phoneme recognition using Support Vector Machines. In Proceedings of European Signal Processing Conference, 2008.

Abstract

The robustness of phoneme classification to additive white Gaussian noise is investigated in acoustic waveform and PLP domains using support vector machines (SVMs). Classification in the PLP space gives excellent results at low noise level under matched training and testing conditions, but it is very sensitive to their mismatch. On the other hand, classification in the acoustic waveform domain is inferior at low noise levels, but exhibits a much more robust behaviour, and at high noise levels even with training on clean data significantly outperforms the classification in the PLP space with training under matched conditions. The two classifiers are then combined in a manner which attains the accuracy of PLP at low noise levels and significantly improves its robustness to additive noise.

Full paper available as Adobe PDF

F Rouyer, S Cohen-Addad, R Höhler, P Sollich and S M Fielding. The large amplitude oscillatory strain response of aqueous foam: Strain localization and full stress Fourier spectrum. European Physical Journal E, 27(3):309-321, 2008.

Abstract

We study the low frequency stress response of aqueous foams, subjected to oscillatory strain. As the strain amplitude is progressively increased starting from zero, the initially linear viscoelastic response becomes nonlinear as yielding sets in. To characterize this crossover from solid-like to liquid-like behaviour quantitatively, the full harmonic spectrum of the stress is measured. These results are compared to the soft glassy rheology model as well as to elastoplastic models. Moreover, to check for strain localization, we monitor the displacement profile of the bubbles at the free surface of the foam sample in a Couette cell using video microscopy. These observations indicate that strain localisation occurs close to the middle of the gap, but only at strain amplitudes well above the yield strain.

Full paper available as Adobe PDF
DOI: 10.1140/epje/i2008-10382-7

S M Fielding, M E Cates and P Sollich. Shear banding, aging and noise dynamics in soft glassy materials. Soft Matter, 2009.

Abstract

The `soft glassy rheology' (SGR) model gives an appealing account of the flow of nonergodic soft materials in terms of the local yield dynamics of mesoscopic elements. Newtonian, power-law, and yield-stress fluid regimes arise on varying a `noise temperature', $x$. Here we extend the model, to capture the idea that the noise is largely caused by yield itself. The extended model can account for the viscosity-bifurcation and shear-banding effects reported recently in a wide range of soft materials. A variant model may shed light on shear banding and strain-rate hysteresis seen in glassy star polymers solutions.

Full paper available as Adobe PDF
DOI: 10.1039/B812394M

P Sollich, S N Majumdar and A J Bray. Phase Transition in a Random Minima Model: Mean Field Theory and Exact Solution on the Bethe Lattice. Journal of Statistical Mechanics: Theory and Experiment, P11011, 2008.

Abstract

We consider the number and distribution of minima in random landscapes defined on non-Euclidean lattices. Using an ensemble where random landscapes are reweighted by a fugacity factor $z$ for each minimum they contain, we construct first a `two-box' mean field theory. This exhibits an ordering phase transition at $z_{\rm c}=2$ above which one box contains an extensive number of minima. The onset of order is governed by an unusual order parameter exponent $\beta=1$, motivating us to study the same model on the Bethe lattice. Here we find from an exact solution that for any connectivity $\mu+1>2$ there is an ordering transition with a conventional mean field order parameter exponent $\beta=1/2$, but with the region where this behaviour is observable shrinking in size as $1/\mu$ in the mean field limit of large $\mu$. We show that the behaviour in the transition region can also be understood directly within a mean field approach, by making the assignment of minima `soft'. Finally we demonstrate, in the simplest mean field case, how the analysis can be generalized to include both maxima and minima. In this case an additional first order phase transition appears, to a landscape in which essentially all sites are either minima or maxima.

Full paper available as arXiv:0807.4386
DOI: 10.1088/1742-5468/2008/11/P11011

R L Jack, P Sollich and P Mayer. Subdiffusive motion in kinetically constrained models. Physical Review E, 78:061107, 2008.

Abstract

We discuss a kinetically constrained model in which real-valued local densities fluctuate in time, as introduced recently by Bertin, Bouchaud and Lequeux. We show how the phenomenology of this model can be reproduced by an effective theory of mobility excitations propagating in a disordered environment. Both excitations and probe particles have subdiffusive motion, characterised by different exponents and operating on different time scales. We derive these exponents, showing that they depend continuously on one of the parameters of the model.

Full paper available as arXiv:0809.2897
DOI: 10.1103/PhysRevE.78.061107

A Annibale and P Sollich. Dynamic heterogeneities in critical coarsening: Exact results for correlation and response fluctuations in finite-sized spherical models. Journal of Statistical Mechanics: Theory and Experiment, P02064, 2009.

Abstract

We study dynamic heterogeneities in the out-of-equilibrium coarsening dynamics of the spherical ferromagnet after a quench from infinite temperature to its critical point. A standard way of probing such heterogeneities is by monitoring the fluctuations of correlation and susceptibility, coarse-grained over mesoscopic regions. We discuss how to define fluctuating coarse-grained correlations and susceptibilities in models where no quenched disorder is present. Our focus for the spherical model is on coarse-graining over the whole volume of $N$ spins, which requires accounting for ${\mathcal{O}}(N^{-1/2})$ non-Gaussian fluctuations of the spin variables. The latter are treated as a perturbation about the leading order Gaussian statistics. We obtain exact results for these quantities, which enable us to characterize the joint distribution of correlation and susceptibility fluctuations. We find that this distribution is qualitatively different, even for equilibrium above criti cality, from the spin-glass scenario where correlation and susceptibility fluctuations are linked in a manner akin to the fluctuation-dissipation relation between the average correlation and susceptibility. Our results show that coarsening at criticality is clearly heterogeneous above the upper critical dimension and suggest that, as in other glassy systems, there is a well-defined timescale on which fluctuations across thermal histories are largest. Surprisingly, however, neither this timescale nor the amplitude of the heterogeneities increase with the age of the system, as would be expected from the growing correlation length. Below the upper critical dimension, the strength of correlation and susceptibility fluctuations varies on a timescale proportional to the age of the system; the corresponding amplitude also grows with age, but does not scale with the correlation volume as might have been expected naively.

Full paper available as arXiv:0811.3168

M Ager, Z Cvetkovic and P Sollich. Robust phoneme classification: Exploiting the adaptability of acoustic waveform models. In Proceedings of European Signal Processing Conference, 2009.

Abstract

The robustness of classification of isolated phoneme segments using generative classifers is investigated for the acoustic waveform, MFCC and PLP speech representations. Gaussian mixture models with diagonal covariance matrices are used followed by maximum likelihood classification. The performance of noise adapted acoustic waveform models is compared with PLP and MFCC models that were adapted using noisy training set feature standardisation. In the presence of additive noise, acoustic waveforms have significantly lower classification error. Even for the unrealistic case where PLP and MFCC classifiers are trained and tested in exactly matched noise conditions acoustic waveform classifiers still outperform them. In both cases the acoustic waveform classifiers are trained explicitly only on quiet data and then modified by a simple transformation to account for the noise.

Full paper available as Adobe PDF

J Yousafzai, Z Cvetkovic and P Sollich. Custom-designed SVM kernels for improved robustness of phoneme classification. In Proceedings of European Signal Processing Conference, 2009.

Abstract

The robustness of phoneme classification to white Gaussian noise and pink noise in the acoustic waveform domain is investigated using support vector machines. We focus on the problem of designing kernels which are tuned to the physical properties of speech. For comparison, results are reported for the PLP representation of speech using standard kernels. We show that major improvements can be achieved by incorporating the properties of speech into kernels. Furthermore, the high-dimensional acoustic waveforms exhibit more robust behavior to additive noise. Finally, we investigate a combination of the PLP and acoustic waveform representations which attains better classification than either of the individual representations over a range of noise levels.

Full paper available as Adobe PDF

J Yousafzai, Z Cvetkovic and P Sollich. Tuning Support Vector Machines for robust phoneme classification with acoustic waveforms. In Proceedings of Interspeech 2009: 10th Annual Conference of the International Speech Communication Association, pages 2359-2362, 2009.

Abstract

This work focuses on the robustness of phoneme classification to additive noise in the acoustic waveform domain using support vector machines (SVMs). We address the issue of designing kernels for acoustic waveforms which imitate the state-of-the-art representations such as PLP and MFCC and are tuned to the physical properties of speech. For comparison, classification results in the PLP representation domain with cepstral mean-and-variance normalization (CMVN) using standard kernels are also reported. It is shown that our custom-designed kernels achieve better classification performance at high noise. Finally, we combine the PLP and acoustic waveform representations to attain better classification than either of the individual representations over the entire range of noise levels tested, from quiet condition up to -18dB SNR.

Full paper available as Adobe PDF

R L Jack and P Sollich. Duality symmetries and effective dynamics in disordered hopping models. Journal of Statistical Mechanics: Theory and Experiment, P11011, 2009.

Abstract

We identify a duality transformation in one-dimensional hopping models that relates propagators in general disordered potentials related by an up-down inversion of the energy landscape. This significantly generalises previous results for a duality between trap and barrier models. We use the resulting insights into the symmetries of these models to develop a real-space renormalisation scheme that can be implemented computationally and allows rather accurate prediction of propagation in these models. We also discuss the relation of this renormalisation scheme to earlier analytical treatments.

Full paper available as arXiv:0908.3492

P Sollich, M Urry and C Coti. Kernels and learning curves for Gaussian process regression on random graphs. In Y Bengio, D Schuurmans, J Lafferty, C K I Williams and A Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1723-1731, 2009.

Abstract

We investigate how well Gaussian process regression can learn functions defined on graphs, using large regular random graphs as a paradigmatic example. Random-walk based kernels are shown to have some non-trivial properties: within the standard approximation of a locally tree-like graph structure, the kernel does not become constant, i.e. neighbouring function values do not become fully correlated, when the lengthscale $\sigma$ of the kernel is made large. Instead the kernel attains a non-trivial limiting form, which we calculate. The fully correlated limit is reached only once loops become relevant, and we estimate where the crossover to this regime occurs. Our main subject are learning curves of Bayes error versus training set size. We show that these are qualitatively well predicted by a simple approximation using only the spectrum of a large tree as input, and generically scale with $n/V$, the number of training examples per vertex. We also explore how this behaviour changes for kernel lengthscales that are large enough for loops to become important.

Full paper available as Adobe PDF

P Sollich and N B Wilding. Crystalline phases of polydisperse spheres. Physical Review Letters, 104:118302, 2010.

Abstract

We use specialized Monte Carlo simulation methods and moment free energy calculations to provide conclusive evidence that dense polydisperse spheres at equilibrium demix into coexisting fcc phases, with more phases appearing as the spread of diameters increases. We manage to track up to four coexisting phases. Each of these is fractionated: it contains a narrower distribution of particle sizes than is present in the system overall. We also demonstrate that, surprisingly, demixing transitions can be nearly continuous, accompanied by fluctuations in local particle size correlated over many lattice spacings.

Full paper available as arXiv:0910.5085
DOI: 10.1103/PhysRevLett.104.118302

D J Ashton, N B Wilding and P Sollich. Fluid phase coexistence and critical behavior from simulations in the restricted Gibbs ensemble. Journal of Chemical Physics, 132:074111, 2010.

Abstract

The symmetrical restricted Gibbs ensemble (rge) is a version of the Gibbs ensemble in which particles are exchanged between two boxes of fixed equal volumes. It has recently come to prominence because-when combined with specialized algorithms-it provides for the study of near-coexistence density fluctuations in highly size-asymmetric binary mixtures. Hitherto, however, a detailed framework for extracting accurate estimates of critical point and coexistence curve parameters from rge density fluctuations has been lacking. Here we address this problem by exploiting an exact link between the rge density fluctuations and those of the grand canonical ensemble. In the subcritical region we propose and test a simple method for obtaining accurate estimates of coexistence densities. In the critical region we identify an observable that serves as a finite system size estimator for the critical point parameters, and present a finite-size scaling theory that allows extrapolation to the thermodynamic limit.

Full paper available as arXiv:0912.2280
DOI: 10.1063/1.3316208

M Ager, Z Cvetkovic and P Sollich. High-dimensional linear representations for robust speech recognition. In 2010 Information Theory and Applications Workshop (ITA 2010), pages 1-5, 2010.

Abstract

Phoneme classification is investigated in linear feature domains with the aim of improving the robustness to additive noise. Linear feature domains allow for exact noise adaptation and so should result in more accurate classification than representations involving nonlinear processing and dimensionality reduction. We develop a generative framework for phoneme classification using linear features. We first show results for a representation consisting of concatenated frames from the centre of the phoneme, each containing f frames. As no single f is optimal for all phonemes, we further average over models with a range of values of f. Next we improve results by including information from the entire phoneme. In the presence of additive noise, classification in this framework performs better than an analogous PLP classifier, adapted to noise using cepstral mean and variance normalisation, below 18dB SNR.

Full paper available as Adobe PDF
DOI: 10.1109/ITA.2010.5454172

M Ager, Z Cvetkovic and P Sollich. Phoneme classification in linear feature domains. Submitted for publication.

Abstract

Phoneme classification is investigated for linear feature domains with the aim of improving robustness to additive noise. In linear feature domains noise adaptation is exact, potentially leading to more accurate classification than representations involving non-linear processing and dimensionality reduction. A generative framework is developed for isolated phoneme classification using linear features. Initial results are shown for representations consisting of concatenated frames from the centre of the phoneme, each containing $f$ frames. As phonemes have variable duration, no single $f$ is optimal for all phonemes, therefore an average is taken over models with a range of values of $f$. Results are further improved by including information from the entire phoneme and transitions. In the presence of additive noise, classification in this framework performs better than an analogous PLP classifier, adapted to noise using cepstral mean and variance normalisation, below $18$dB SNR. Finally we propose classification using a combination of acoustic waveform and PLP log-likelihoods. The combined classifier performs uniformly better than either of the individual classifiers across all noise levels.

Full paper available as Adobe PDF

J Yousafzai, P Sollich, Z Cvetkovic and Bin Yu. Combined features and kernel design for noise robust phoneme classification using Support Vector Machines. IEEE Transactions on Audio, Speech, and Language Processing, 19:1396-1407, 2011.

Abstract

This work proposes methods for combining cepstral and acoustic waveform representations for a front end of support vector machine (SVM) based speech recognition systems that are robust to additive noise. The key issue of kernel design and noise adaptation for the acoustic waveform representation is addressed first. Cepstral and acoustic waveform representations are then compared on a phoneme classification task. Experiments show that the cepstral features achieve very good performance in low noise conditions, but suffer severe performance degradation in high noise. Classification in the acoustic waveform domain, on the other hand, although not as accurate in low noise, exhibits a more robust behavior in severe noise conditions. A combination of the cepstral and acoustic waveform representations achieves better classification performance than either of the individual representations over the entire range of noise levels tested, down to $-18$dB SNR, and even beats the performance achieved using cepstral features with standard SVM kernels for which training and testing is performed under matched noise conditions.

Full paper available as Adobe PDF
DOI: 10.1109/TASL.2010.2090657

P Sollich and R L Jack. Duality symmetries in driven one-dimensional hopping models. Progress of Theoretical Physics Supplement, 184:200-210, 2010.

Abstract

We consider some duality relations for models of non-interacting particles hopping on disordered one-dimensional chains. In particular, we discuss symmetries of bulk-driven barrier and trap models, and relations between boundary-driven and equilibrium models with related energy landscapes. We discuss the relationships between these duality relations and similar results for interacting many-body systems.

Full paper available as arXiv:0911.0208
DOI: 10.1143/PTPS.184.200

R L Jack and P Sollich. Large deviations and ensembles of trajectories in stochastic models. Progress of Theoretical Physics Supplement, 184:304-317, 2010.

Abstract

We consider ensembles of trajectories associated with large deviations of time-integrated quantities in stochastic models. Motivated by proposals that these ensembles are relevant for physical processes such as shearing and glassy relaxation, we show how they can be generated directly using auxiliary stochastic processes. We illustrate our results using the Glauber-Ising chain, for which biased ensembles of trajectories can exhibit ferromagnetic ordering. We discuss the relation between such biased ensembles and quantum phase transitions.

Full paper available as arXiv:0911.0211
DOI: 10.1143/PTPS.184.304

J P Garrahan, P Sollich and C Toninelli. Kinetically constrained models. In L Berthier, G Biroli, J-P Bouchaud, L Cipelletti and W van Saarloos, editors, pages 341-369, Oxford University Press, 2011.

Abstract

In this chapter we summarize recent developments in the study of kinetically constrained models (KCMs) as models for glass formers. After recalling the definition of the KCMs which we cover we study the possible occurrence of ergodicity breaking transitions and discuss in some detail how, before any such transition occurs, relaxation timescales depend on the relevant control parameter (density or temperature). Then we turn to the main issue: the prediction of KCMs for dynamical heterogeneities. We focus in particular on multipoint correlation functions and susceptibilities, and decoupling in the transport coefficients. Finally we discuss the recent view of KCMs as being at first order coexistence between an active and an inactive space-time phase.

Full paper available as Adobe PDF

J Yousafzai, Z Cvetkovic and P Sollich. Towards robust phoneme classification with hybrid features. In 2010 IEEE International Symposium on Information Theory Proceedings (ISIT), pages 1643-1647, 2010.

Abstract

In this paper, we investigate the robustness of phoneme classification to additive noise with hybrid features using support vector machines (SVMs). In particular, the cepstral features are combined with short term energy features of acoustic waveform segments to form a hybrid representation. The energy features are then taken into account separately in the SVM kernel, and a simple subtraction method allows them to be adapted effectively in noise. This hybrid representation contributes significantly to the robustness of phoneme classification and narrows the performance gap to the ideal baseline of classifiers trained under matched noise conditions.

Full paper available as Adobe PDF
DOI: 10.1109/ISIT.2010.5513345

J Yousafzai, Z Cvetkovic and P Sollich. Subband acoustic waveform front-end for robust speech recognition using support vector machines. In 2010 IEEE Spoken Language Technology Workshop (SLT), pages 253-258, 2010.

Abstract

A subband acoustic waveform front-end for robust speech recognition using support vector machines (SVMs) is developed. The primary issues of kernel design for subband components of acoustic waveforms and combination of the individual subband classifiers using stacked generalization are addressed. Experiments performed on the TIMIT phoneme classification task demonstrate the benefits of classification in frequency subbands: the subband classifier outperforms the cepstral classifiers in the presence of noise for signal-to-noise ratio (SNR) below 12dB.

Full paper available as Adobe PDF
DOI: 10.1109/SLT.2010.5700860

M Urry and P Sollich. Exact learning curves for Gaussian process regression on large random graphs. In J Lafferty and C K I Williams and J Shawe-Taylor and R S Zemel and A Culotta, editors, Advances in Neural Information Processing Systems 23, pages 2316-2324, 2010.

Abstract

We study learning curves for Gaussian process regression which characterise performance in terms of the Bayes error averaged over datasets of a given size. Whilst learning curves are in general very difficult to calculate we show that for discrete input domains, where similarity between input points is characterised in terms of a graph, accurate predictions can be obtained. These should in fact become exact for large graphs drawn from a broad range of random graph ensembles with arbitrary degree distributions where each input (node) is connected only to a finite number of others. Our approach is based on translating the appropriate belief propagation equations to the graph ensemble. We demonstrate the accuracy of the predictions for Poisson (Erdos-Renyi) and regular random graphs, and discuss when and why previous approximations of the learning curve fail.

Full paper available as Adobe PDF

N B Wilding and P Sollich. Phase behavior of polydisperse spheres: Simulation strategies and an application to the freezing transition. Journal of Chemical Physics, 133:224102, 2010.

Abstract

The statistical mechanics of phase transitions in dense systems of polydisperse particles presents distinctive challenges to computer simulation and analytical theory alike. The core difficulty, namely, dealing correctly with particle size fractionation between coexisting phases, is set out in the context of a critique of previous simulation work on such systems. Specialized Monte Carlo simulation techniques and moment free energy method calculations, capable of treating fractionation exactly, are then described and deployed to study the fluid-solid transition of an assembly of repulsive spherical particles described by a top-hat ``parent''distribution of particle sizes. The cloud curve delineating the solid-fluid coexistence region is mapped as a function of the degree of polydispersity $\delta$, and the properties of the incipient ``shadow'' phases are presented. The coexistence region is found to shift to higher densities as $\delta$ increases, but does not exhibit the sharp narrowing predicted by many theories and some simulations.

Full paper available as arXiv:1008.3068
DOI: 10.1063/1.3510534

P Sollich and N B Wilding. Polydispersity induced solid-solid transitions in model colloids. Soft Matter, 7:4472-4484, 2011.

Abstract

Specialized Monte Carlo simulation techniques and moment free energy method calculations, capable of treating fractionation exactly, are deployed to study the crystalline phase behaviour of an assembly of spherical particles described by a top-hat ``parent'' distribution of particle sizes. An increase in either the overall density or the degree of polydispersity is shown to generate a succession of phase transitions in which the system demixes into an ever greater number of face-centred cubic ``daughter'' phases. Each of these phases is strongly fractionated: it contains a much narrower distribution of particle sizes than is present in the system overall. Certain of the demixing transitions are found to be nearly continuous, accompanied by fluctuations in local particle size correlated over many lattice spacings. We explore possible factors controlling the stability of the phases and the character of the demixing transitions.

Full paper available as Adobe PDF
DOI: 10.1039/c0sm01367f

L Berthier, P Chaudhuri, C Coulais, O Dauchot and P Sollich. Suppressed compressibility at large scale in jammed packings of size-disperse spheres. Physical Review Letters, 106:120601, 2011.

Abstract

We analyze the large scale structure and fluctuations of jammed packings of size disperse spheres, produced in a granular experiment as well as numerically. While the structure factor of the packings reveals no unusual behavior for small wavevectors, the compressibility displays an anomalous linear dependence at low wavectors and vanishes when $q\to 0$. We show that such behavior occurs because jammed packings of size disperse spheres have no bulk fluctuations of the volume fraction and are thus hyperuniform, a property not observed experimentally before. Our results apply to arbitrary particle size distributions. For continuous distributions, we derive a perturbative expression for the compressibility that is accurate for polydispersity up to about 30%.

Full paper available as arXiv:1008.2899

J Yousafzai, Z Cvetkovic and P Sollich. On the potential for robust ASR with combined subband-waveform and cepstral features. Technical report.

Abstract

This work explores the potential for robust classification of phonemes in the presence of additive noise and linear filtering using high-dimensional features in the subbands of acoustic waveforms. The proposed technique is compared with state-of-the-art automatic speech recognition (ASR) front-ends on the TIMIT phoneme classification task using support vector machines (SVMs). The key issues of selecting the appropriate SVM kernels for classification in frequency subbands and the combination of individual subband classifiers using ensemble methods are addressed. Experiments demonstrate the benefits of the classification in the subbands of acoustic waveforms: it outperforms the standard cepstral front-end in the presence of noise and linear filtering for all signal-to-noise ratios (SNRs) below a crossover point between 12dB and 6dB. Combining the subband-waveform and cepstral classifiers achieves further performance improvements over both individual classifiers.

Full paper available as Adobe PDF

M Ager, Z Cvetkovic and P Sollich. Combined waveform-cepstral representation for robust speech recognition. 2011 IEEE International Symposium on Information Theory Proceedings (ISIT), pages 864-868, 2011.

Abstract

High-dimensional acoustic waveform representations are studied as a front-end for noise robust automatic speech recognition using generative methods, in particular gaussian mixture models and hidden markov models. The proposed representations are compared with standard cepstral features on phoneme classification and recognition tasks. While lower error rates are achieved using cepstral features at very low noise levels, the acoustic waveform representations are much more robust to noise. A convex combination of acoustic waveforms and cepstral features is then considered and it achieves higher accuracy than either of the individual representations across all noise levels.
DOI: 10.1109/ISIT.2011.6034260

A Baule and P Sollich. Singular features in noise-induced transport with dry friction. EPL, to be published.

Abstract

We present an exactly solvable nonlinear model for the directed motion of an object due to zero-mean fluctuations on a uniform featureless surface. Directed motion results from the effect of dry (Coulombic) friction coupled to asymmetric surface vibrations with Poissonian shot noise statistics. We find that the transport of the object exhibits striking non-monotonic and singular features: transport actually improves for larger dry friction up to a critical dry friction strength $\Delta^*$ and undergoes a transition to a unidirectional mode of motion at $\Delta^*$. This transition is indicated by a cusp singularity in the mean velocity of the object. Moreover, the stationary velocity distribution also contains singular features, such as a discontinuity and a delta peak at zero velocity. Our results suggest that dissipation can in fact enhance transport, which might be exploited in artificial small scale systems.

J K Yousafzai, Z Cvetkovic, P Sollich and M Ager. A high-dimensional subband speech representation and SVM framework for robust speech recognition. Submitted for publication.

Abstract

This work proposes a novel support vector machine (SVM) based robust automatic speech recognition (ASR) front-end that operates on an ensemble of the subband components of high-dimensional acoustic waveforms. The key issues of selecting the appropriate SVM kernels for classification in frequency subbands and the combination of individual subband classifiers using ensemble methods are addressed. The proposed front-end is compared with state-of-the-art ASR front-ends in terms of robustness to additive noise and linear filtering. Experiments performed on the TIMIT phoneme classification task demonstrate the benefits of the proposed subband based SVM representation: it outperforms the standard cepstral front-end in the presence of noise and linear filtering for signal-to-noise ratio (SNR) below 12-dB. A combination of the proposed front-end with a conventional representation such as MFCC yields further improvements over the individual front-ends across the full range of noise levels.

Full paper available as Adobe PDF

R Kühn and P Sollich. Spectra of empirical auto-covariance matrices. Submitted for publication.

Abstract

We compute spectra of sample auto-covariance matrices of second order stationary stochastic processes. We look at a limit in which both the matrix dimension $N$ and the sample size $M$ used to define empirical averages diverge, with their ratio $\alpha=N/M$ kept fixed. We find a remarkable scaling relation which expresses the spectral density $\rho(\lambda)$ of sample auto-covariance matrices for processes with dynamical correlations as a continuous superposition of appropriately rescaled copies of the spectral density $\rho^{(0)}_\alpha(\lambda)$ for a sequence of uncorrelated random variables. The rescaling factors are given by the Fourier transform $\hat C(q)$ of the auto-covariance function of the stochastic process. We also obtain a closed-form approximation for the scaling function $\rho^{(0)}_\alpha(\lambda)$. This depends on the shape parameter $\alpha$, but is otherwise universal: it is independent of the details of the underlying random variables, provided only they have finite variance. Our results are corroborated by numerical simulations using auto-regressive processes.

Full paper available as arXiv:1112.4877

M E Cates and P Sollich. Thermodynamic interpretation of soft glassy rheology models. Submitted for publication.

Abstract

Mesoscopic models play an important role in our understanding of the deformation and flow of amorphous materials. One such description, based on the Shear Transformation Zone (STZ) theory, has recently been re-formulated within a non-equilibrium thermodynamics framework, and found to be consistent with it. We show here that a similar interpretation can be made for the Soft Glassy Rheology (SGR) model. Conceptually this means that the "noise temperature" x, proposed phenomenologically in the SGR model to control the dynamics of a set of slow mesoscopic degrees of freedom, can consistently be interpreted as their actual thermodynamic temperature. (Because such modes are slow to equilibrate, this generally does not coincide with the temperature of the fast degrees of freedom and/or heat bath.) If one chooses to make this interpretation, the thermodynamic framework significantly constrains extensions of the SGR approach to models in which x is a dynamical variable. We assess in this light some such extensions recently proposed in the context of shear banding.

Full paper available as arXiv:1201.3275

P Sollich and A Barra. Notes on the polynomial identities in random overlap structures. Submitted for publication.

Abstract

In these notes we review first in some detail the concept of random overlap structure (ROSt) applied to fully connected and diluted spin glasses. We then sketch how to write down the general term of the expansion of the energy part from the Boltzmann ROSt (for the Sherrington-Kirkpatrick model) and the corresponding term from the RaMOSt, which is the diluted extension suitable for the Viana-Bray model. From the ROSt energy term, a set of polynomial identities (often known as Aizenman-Contucci or AC relations) is shown to hold rigorously at every order because of a recursive structure of these polynomials that we prove. We show also, however, that this set is smaller than the full set of AC identities that is already known. Furthermore, when investigating the RaMOSt energy for the diluted counterpart, at higher orders, combinations of such AC identities appear, ultimately suggesting a crucial role for the entropy in generating these constraints in spin glasses.

Full paper available as arXiv:1201.3483

Last updated Thu 1 Mar 2012 Contact: Peter Sollich

Back to my home page