Statistical and Machine Learning
Brief description
The course Statistical and Machine Learning Methods provides an insight into the components and algorithms of statistical and machine learning methods. Various approaches are presented on how characteristics can be learnt from data, either supervised or unsupervised, and how unknown patterns can be recognised. The techniques presented can be applied to a variety of classification and regression problems, whether for one-dimensional signals (e.g. language), two-dimensional (e.g. images) or symbolic data (e.g. texts, documents).
Lecture contents
- Introduction to classification methods, Bayesian and other decision rules
- Optimisation methods: Gradient descent, algorithmic differentiation, optimisation with boundary condition
- Linear classifiers: perceptron, support vector machines
- Deep neural networks
- Dimension reduction methods (PCA, LDA)
- Unsupervised learning: mixture distributions, clustering methods
Learning outcomes & professional competences
After attending the course, students will be able to
- select a suitable solution method for a given classification or regression problem
- Apply methods of supervised and unsupervised learning to new problems and critically evaluate the results of the learning process
- Have a basic understanding of machine learning methods
- Can use program libraries to implement classifiers (e.g. neural networks, support vector machines) and write their own programs
- can make a sensible choice for the dimension of the feature vector and the complexity of the classifier for a given set of training data
The students
- Have acquired skills in Python that they can also use outside this application domain
- Have an understanding of the principle of parsimony and can transfer it to other problems
- Can analyse a given classification or regression problem, synthesise a solution and then evaluate it on test data
- Can transfer the knowledge and skills gained in this course to other disciplines
- Can analyse more extensive tasks together in a group, break them down into subtasks and work on them in a solution-oriented manner
- Can assess the performance and limitations of machine learning methods
Methodical realisation
- Lectures with predominantly blackboard use, occasional slide presentation
- Classroom exercises with exercise sheets and demonstrations on the computer
- practical exercises with Python, in which students independently generate training and test data, work out solutions and implement and test learning procedures or classifiers and analyse results.
Recommended reading
- R.O. Duda, P.E. Hart and D.G. Stork: "Pattern Classification", 2nd Edition, Wiley, 2000
- I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, MIT Press, 2016
- S. Theodoridis: "Machine Learning", Academic Press, 2015
- K. Fukunaga, Statistical Pattern Recognition, Academic Press, 1990
- Hastie, Tibshirani, The Elements of Statistical Learning, Springer 2003
Exercises
Switched to panda.uni-paderborn.de. Files here are outdated, but kept for students who took the curse at that time.
Exercise files:
Tutorial:
Homework Assignment:
You are interested in: