King's College London
MSc in Information Processing and Neural Networks

Description of Core Courses

Neural networks (core course)
This introductory course covers the basics of neural information processing. Subjects include neuron models (graded response, McCulloch-Pitts, stochastic binary neurons, coupled oscillators), learning in layered neural networks (linear separability, perceptrons, error backpropagation, dynamics of learning) and the operation of recurrent neural networks (creation of attractors, analysis via Lyapunov functions, Hopfield model, analysis of dynamics).

Advanced neural networks (core course)
Radial basis functions and function approximation, overfitting, regularisation and generalisation. Self-organising maps and vector quantisation. Bayesian analysis of learning in layered networks. Introduction to Gaussian processes. Introduction to Support Vector Machines.

Description of Optional Lecture Courses

Mathematics

Information theory in neural networks
The first part covers the basics of Shannon's information theory: definitions and properties, with proofs, of the main tools for quantifying information, e.g. entropy, relative entropy, mutual information, and their link with coding theory. The second part describes applications of information-theoretic concepts to neural information processing, e.g. Boltzmann Machine learning, unsupervised learning by maximum information preservation, detection of coherent features, and learning based on information geometry (natural gradient).

Statistical mechanics of neural networks
This course covers applications of advanced tools from equilibrium and non-equilibrium statistical mechanics (mainly from the field of disordered magnetic systems) to analyse operation and learning in neural networks. The course concentrates on analytic solutions obtained by replica theory and the theory of stochastic processes, in the context of the operation of recurrent attractor neural networks and the dynamics of learning in layered neural networks.

Non-linear dynamics
This course covers various aspects of the behaviour and analysis of non-linear differential equations, such as phase-plane analysis, critical points and linearisation, integrability, periodic solutions, stability analysis and Lyapunov functions, bifurcation theory, attractors and chaotic systems.

Applied Probability and Stochastics
Probability spaces, random variables, distributions, independence, product spaces. Expectation and conditional expectation. Moments, generating functions, characteristic functions. Random processes, filtrations and stopping times. Martingales, Brownian motion and the Poisson process. Elements of Itô integration.

Introduction to Derivatives Pricing
Forward prices, discounting, arbitrage-free pricing; binomial trees, derivatives pricing in discrete time by use of binomial lattices, geometric Brownian motion, volatility and drift, martingales and conditional expectation, Itô calculus, hedging portfolios, replication, Black-Scholes model, put-call parity. Risk premium, risk-neutral measure.

Mathematics \& Statistics (LSE)

Computational learning theory
Machines and concepts, representability, logical machines representing classes of boolean formulae. Threshold machines. Learning algorithms, probable correctness of the monomial learning algorithm, the perceptron learning algorithm. Multilayer perceptrons and the algorithm for back-propagation. Probably approximately correct learning, VC-dimension, distribution dependent learning. Complexity theory, polynomial-time learning algorithms, machines representing graph-theoretical properties. Non-learnability, formal concepts.

Theory of Algorithms
The aim of the course is to provide an introduction to the theory of algorithms, data structures, and computational complexity: Basics of computer architecture and data representations. Introduction to programming in Java. Sorting and searching. Running times. Hash tables. Linked lists. Graphs and graph traversal algorithms. Polynomial-time algorithms. NP-complete problems.

Basic Time Series
Starting from the fundamental mathematical properties of time series, this course describes the modelling and forecasting methods as they have developed over the recent past, and their current use in practice.

Engineering

Communication theory
Source coding and channel digital signal coding. Digital filter design and Implementation. Transform processes, algorithms and applications.

Digital signal processing
This course provides an introduction to signal processing by digital methods, with emphasis on frequency-selective filtering. It is particularly suitable for those with an analogue electronics background who wish to become familiar with alternatives offered by digital techniques. The course includes a significant amount of hands-on experience using desk-top computers, in small groups or individually, supplementing the lectures and reinforcing the concepts taught.

Computer Science

Numerical analysis
This course covers the derivation and analysis of sequential numerical methods. Subjects include systems of linear, interpolation, least-squares approximation, orthogonal polynomials, numerical integration, non-linear algebraic equations, ordinary differential equations, and second-order equations.

Physics

Statistical Mechanics (NOT EXAMINABLE)
General formalism of equilibrium statistical mechanics (including entropy, derivation of Boltzmann distribution, partition function, discrete and continuum state spaces), thermodynamics, ideal gases, degenerate Fermi and Bose Gases, heat capacity, phonons and photons, multi-component systems, non-ideal systems.

[Return to MSc in Information Processing and Neural Networks]