- Introduction and Overview
- Linear classifiers
- Support vector machines
- Kernel methods
- Deep learning
- Regularization and Overfitting
- Dimensionality reduction
- Random forests
Machine Learning I - Theoretical Foundations (M, 8.0 LP)
|Module Number||Module Name||CP (Effort)|
|INF-75-50-M-5||Machine Learning I - Theoretical Foundations||8.0 CP (240 h)|
|CP, Effort||8.0 CP = 240 h|
|Position of the semester||1 Sem. in SuSe|
|Level|| Master (Entry Level)|
|Area of study||[INF-KI] Intelligent Systems|
|Reference course of study||[INF-88.79-SG] M.Sc. Computer Science|
|Type/SWS||Course Number||Title||Choice in |
|SL||SL is |
required for exa.
Machine Learning I - Theoretical Foundations
|P||84 h||156 h||
- About [INF-75-50-K-5]: Title: "Machine Learning I - Theoretical Foundations"; Presence-Time: 84 h; Self-Study: 156 h
- About [INF-75-50-K-5]:
The study achievement "[U-Schein] proof of successful participation in the exercise classes (ungraded)" must be obtained.
- It is a prerequisite for the examination for PL1.
Examination achievement PL1
- Form of examination: written exam (Klausur) (120-150 Min.)
- Examination Frequency: each semester
- Examination number: 67550 ("Machine Learning I")
Evaluation of grades
The grade of the module examination is also the module grade.
Competencies / intended learning achievements
- recognize machine-learning problems in their everyday life or work day
- find and implement solutions to ML problems
- understand the inner workings of ML algorithms
- describe concepts and formalisms of Machine Learning as a generic approach to a variety of disciplines, including image processing, robotics, computational linguistics and software engineering
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville: Deep learning. MIT Press, 2017.
- Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning. Vol. 1. Springer, Berlin: Springer series in statistics, 2001.
- John Shawe-Taylor and Nello Cristianini. Kernel methods for pattern analysis. Cambridge university press, 2004.
Requirements for attendance of the module (informal)
IMPORTANT: This course requires of students a profound knowledge and skill in the following areas of mathematics:
(a) Uni- und multivariate CALCULUS (derivatives, gradients, Jacobians, Hesse matrix, chain rule, Leibnitz notation, experience in computing Jacobians and gradients of simple functions in R^d, etc.),
(b) LINEAR ALGEBRA (vectors, matrices, eigen and singular-value decomposition, basic calculation rules with matrices, etc.).
Kaiserslautern CS BSc students studied this material in their undergrad in the courses 'Algebraic Structures' and 'Analysis', but oftentimes struggle to apply it in the context of machine learning. International students typically never studied a large chunk of the required material and are in most cases overwhelmed by high comparably high mathematical difficulty of the course. This has resulted in the past in very high dropout rates of the ML1 course. Both groups of students (national and international) are STRONGLY advised to study in the semester break before the course these mathematical foundations. (Former) Kaiserslautern BSC students are recommended to study their old math course material ('Algebraic Structures' and 'Analysis'). International CS students are recommended to study the MOOC 'Mathematics for Machine Learning': https://www.coursera.org/specializations/mathematics-machine-learning). Part 1 (Linear Algebra) and Part 2 (Multivariate Calculus) are sufficient, but Part 3 (PCA) is helpful too.
Furthermore, an excellent review of the math that we'll require is in Sections 2+4 in Goodfellow et al.: Deep Learning.
Moreover, students are required to have basic knowledge and experience in programming in one out of the following three languages: Python, Julia, R. We recommend Python. The exercise sheets need to be submitted in Latex; thus Latex is another requirement of this course.
2. Optional requirements (nice to have):
Furthermore, the following mathematical knowledge is helpful (although not mandatory required):
calculation rules with matrices (as much as possible from matrix cookbook: https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf)
basics of continuous optimization (convexity, nonlinear OPs, Lagrange duality, gradient descent)
3. Preparing for these requirements
We advise students that want to major in machine learning to elect mathematics as minor (Nebenfach) already during their BSc studies, which is an excellent preparation for this ML1 course. Students that want to specialize on ML in their MSc are furthermore highly advised to enroll, for their supplementary study area, in as many math courses as possible during their MSC. Areas of mathematics that go well with machine learning are: linear algebra, calculus, optimization, probability, and statistics.
All machine-learning course are an excellent fit also for students majoring in mathematics (or other quantitative study programs) with minor computer science.
For more information on the mathematical preliminaries of the ML courses, see:
Requirements for attendance of the module (formal)None
References to Module / Module Number [INF-75-50-M-5]
|Course of Study||Section||Choice/Obligation|
|[WIW-82.176-SG#2009] B.Sc. Business Administration and Engineering specialising in Computer Science (2009) ||[Fundamentals] Field of study: Computer Science||[WP] Compulsory Elective|
|[INF-88.79-SG] M.Sc. Computer Science||[Core Modules (non specialised)] Computer Science Theory||[WP] Compulsory Elective|
|[INF-88.79-SG] M.Sc. Computer Science||[Core Modules (non specialised)] Formal Fundamentals||[WP] Compulsory Elective|
|[INF-88.79-SG] M.Sc. Computer Science||[Specialisation] Specialization 1||[WP] Compulsory Elective|
|[EIT-88.A20-SG#2021] M.Sc. European Master in Embedded Computing Systems (EMECS) ||[Core Modules (non specialised)] Core Subjects||[WP] Compulsory Elective|
|[EIT-88.A20-SG#2021] M.Sc. European Master in Embedded Computing Systems (EMECS) ||[Free Elective Area] Elective Subjects||[W] Elective Module|
|[EIT-88.D55-SG#2021] M.Sc. Embedded Computing Systems (ESY) ||[Core Modules (non specialised)] Core Program||[WP] Compulsory Elective|
|[EIT-88.D55-SG#2021] M.Sc. Embedded Computing Systems (ESY) ||[Free Elective Area] Elective Subjects||[W] Elective Module|
|[INF-KI_Ba_V-MPOOL-4]||Specialization Bachelor TA Intelligent Systems|
|[INF-SIAK-DT-AI-MPOOL-6]||SIAK Certificate "Digital Transformation" - Modules INF "Artificial Intelligence"|
|[MV-MB-INF-2022-MPOOL-6]||Wahlpflichtmodule M.Sc. Maschinenbau mit angewandter Informatik 2022|
|[MV-MBINFO-MPOOL-6]||Wahlpflichtmodule Maschinenbau mit angewandter Informatik|