Statistical and Machine Learning

Brief de­scrip­tion

The course Statistical and Machine Learning Methods provides an insight into the components and algorithms of statistical and machine learning methods. Various approaches are presented on how characteristics can be learnt from data, either supervised or unsupervised, and how unknown patterns can be recognised. The techniques presented can be applied to a variety of classification and regression problems, whether for one-dimensional signals (e.g. language), two-dimensional (e.g. images) or symbolic data (e.g. texts, documents).

Lec­ture con­tents

  • Introduction to classification methods, Bayesian and other decision rules
  • Optimisation methods: Gradient descent, algorithmic differentiation, optimisation with boundary condition
  • Linear classifiers: perceptron, support vector machines
  • Deep neural networks
  • Dimension reduction methods (PCA, LDA)
  • Unsupervised learning: mixture distributions, clustering methods

Learn­ing out­comes & pro­fes­sion­al com­pet­ences

After attending the course, students will be able to

  • select a suitable solution method for a given classification or regression problem
  • Apply methods of supervised and unsupervised learning to new problems and critically evaluate the results of the learning process
  • Have a basic understanding of machine learning methods
  • Can use program libraries to implement classifiers (e.g. neural networks, support vector machines) and write their own programs
  • can make a sensible choice for the dimension of the feature vector and the complexity of the classifier for a given set of training data

The students

  • Have acquired skills in Python that they can also use outside this application domain
  • Have an understanding of the principle of parsimony and can transfer it to other problems
  • Can analyse a given classification or regression problem, synthesise a solution and then evaluate it on test data
  • Can transfer the knowledge and skills gained in this course to other disciplines
  • Can analyse more extensive tasks together in a group, break them down into subtasks and work on them in a solution-oriented manner
  • Can assess the performance and limitations of machine learning methods

Meth­od­ic­al real­isa­tion

  • Lectures with predominantly blackboard use, occasional slide presentation
  • Classroom exercises with exercise sheets and demonstrations on the computer
  • practical exercises with Python, in which students independently generate training and test data, work out solutions and implement and test learning procedures or classifiers and analyse results.

Re­com­men­ded read­ing

  • R.O. Duda, P.E. Hart and D.G. Stork: "Pattern Classification", 2nd Edition, Wiley, 2000
  • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, MIT Press, 2016
  • S. Theodoridis: "Machine Learning", Academic Press, 2015
  • K. Fukunaga, Statistical Pattern Recognition, Academic Press, 1990
  • Hastie, Tibshirani, The Elements of Statistical Learning, Springer 2003

Clas­si­fic­a­tion

  • Course for Master students
  • ECTS: 6
  • Language: English
  • Semester: Summer term

Lec­ture notes

Script (Recent version accessible via https://panda.uni-paderborn.de):

Ex­er­cises

Switched to panda.uni-paderborn.de. Files here are outdated, but kept for students who took the curse at that time.

Exercise files:

Tutorial:

Homework Assignment:

Old exams

Lec­turer

business-card image

Prof. Dr. Reinhold Häb-Umbach

Communications Engineering / Heinz Nixdorf Institute

Head of Department of Communications Engineering

Write email +49 5251 60-3626

Train­ers

business-card image

Christoph Böddeker

Communications Engineering / Heinz Nixdorf Institute

Research & Teaching

Write email +49 5251 60-5288
business-card image

Thilo von Neumann

Communications Engineering / Heinz Nixdorf Institute

Research & Teaching

Write email +49 5251 60-5288