Theoretical Concepts of Machine Learning (2VO)
| Course no.: | 365.041 |
| Lecturer: | Ulrich Bodenhofer |
| Start: | Oct 7, 2009 |
| Time: | Wed 1:45-3:15pm/4:15pm |
| Changes: | No lectures on Nov 4 and 18; additional lectures on Nov 12 and Nov 26 (both 1:45pm-3:15pm in KG712) No lecture on Jan 13; additional lecture on Jan 14 (1:45pm-4:15pm in KG712) |
| Location: | KG 712 |
| Mode: | VO, 2-3h, weekly |
| Registration: | KUSSS |
| Written exam: | Thu, Feb 11, 2010, 3:30-4:30pm (register via KUSSS) |
| Oral exams: | upon individual appointment |
Motivation
Machine learning methods, i.e. methods that infer models/relationships by learning from data, are still gaining importance in various fields, such as, process modeling, speech and image processing, bioinformatics, and so forth. Their ability to cope with tasks for which no analytical model is available ideally complements classical approaches. One has to acknowledge, however, that machine learning methods also bear great risks if they are applied inappropriately. The given lecture provides a look behind the curtain of machine learning. The goal is to make students acquainted with the basic concepts and methods to analyze, evaluate and understand models created by machine learning. In the sequel, we will also have a closer look at support vector machines and neural networks from this foundational perspective.Contents
- Repetition of the basic concepts of machine learning
- Evaluation criteria and optimization strategies
- Statistical learning theory
- Support vector machines: advanced topics and applications
- Neural networks: short overview
Necessary Background
Parts of the lecture will be quite mathematical, so a profound background in calculus, probability and statistics is necessary. This should not be a problem for graduate students of mathematics, computer science, physics, mechatronics, and statistics. Prior knowledge of machine learning (e.g. attendance of Prof. Widmer's lecture "Machine Learning and Pattern Classification") is surely helpful, but not an absolute pre-requisite. Students of bioinformatics should take into account that there is a significant overlap with the lecture "Bioinformatics II: Theoretical Bioinformatics and Machine Learning".Course Material
Slides
- Introduction
pages i-xii; last update 2009-10-20;
download: Screen PDF (183KB) / Print PDF (182KB) - Unit 1: Introduction to Machine Learning
pages 1-30; last update 2009-09-17;
download: Screen PDF (722KB) / Print PDF (707KB) - Unit 2: Model Evaluation in Supervised Machine Learning
pages 31-116; last update 2009-10-28;
download: Screen PDF (1101KB) / Print PDF (1085KB) - Unit 3: Statistical Learning Theory
pages 116-167; last update 2009-11-26;
download: Screen PDF (429KB) / Print PDF (411KB) - Unit 4: Support Vector Machines
pages 168-302; last update 2010-01-28;
download: Screen PDF (6253KB) / Print PDF (6253KB) - Unit 5: Artificial Neural Networks
pages 303-329; last update 2009-10-13;
download: Screen PDF (611KB) / Print PDF (610KB)
© 2009 Ulrich Bodenhofer
This material, no matter whether in printed or electronic form, may be used for personal and educational use
only. Any reproduction of this material, no matter whether as a whole or in parts, no matter whether in printed or in
electronic form, requires explicit prior acceptance of the author.
Software demos
Notes for further reading
- Lecture Notes Bioinformatics II (PDF, 8MB)
Books recommended for further reading
- C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995. ISBN 0-19-853864-2. [link]
- R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. Second edition. John Wiley & Sons, 2001. ISBN 0-471-05669-3. [link]
- T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, 2001. ISBN 0-387-95284-5. [link]
- R. Herbrich. Learning Kernel Classifiers. MIT Press, 2002. ISBN 0-262-08306-X. [link]
- B. Schölkopf and A. J. Smola. Learning With Kernels. MIT Press, 2002. ISBN 0-262-19475-9. [link]
- V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 1995. ISBN 0-387-98780-0. [link]
- V. N. Vapnik. Statistical Learning Theory. John Wiley &Sons, 1998. ISBN 0-471-03003-1. [link]