Module Handbook

  • Dynamischer Default-Fachbereich geändert auf INF

Module INF-75-50-M-5

Machine Learning I - Theoretical Foundations (M, 8.0 LP)

Module Identification

Module Number Module Name CP (Effort)
INF-75-50-M-5 Machine Learning I - Theoretical Foundations 8.0 CP (240 h)

Basedata

CP, Effort 8.0 CP = 240 h
Position of the semester 1 Sem. in SuSe
Level [5] Master (Entry Level)
Language [EN] English
Module Manager
Lecturers
Area of study [INF-KI] Intelligent Systems
Reference course of study [INF-88.79-SG] M.Sc. Computer Science
Livecycle-State [NORM] Active

Courses

Type/SWS Course Number Title Choice in
Module-Part
Presence-Time /
Self-Study
SL SL is
required for exa.
PL CP Sem.
4V+2U INF-75-50-K-5
Machine Learning I - Theoretical Foundations
P 84 h 156 h
U-Schein
ja PL1 8.0 SuSe
  • About [INF-75-50-K-5]: Title: "Machine Learning I - Theoretical Foundations"; Presence-Time: 84 h; Self-Study: 156 h
  • About [INF-75-50-K-5]: The study achievement "[U-Schein] proof of successful participation in the exercise classes (ungraded)" must be obtained.
    • It is a prerequisite for the examination for PL1.

Examination achievement PL1

  • Form of examination: written exam (Klausur) (120-150 Min.)
  • Examination Frequency: each semester
  • Examination number: 67550 ("Machine Learning I")

Evaluation of grades

The grade of the module examination is also the module grade.


Contents

  • Introduction and Overview
  • Linear classifiers
  • Support vector machines
  • Optimization
  • Kernel methods
  • Deep learning
  • Regularization and Overfitting
  • Regression
  • Clustering
  • Dimensionality reduction
  • Random forests

Competencies / intended learning achievements

After successfully completing the module, students will be able to:
  • recognize machine-learning problems in their everyday life or work day
  • find and implement solutions to ML problems
  • understand the inner workings of ML algorithms
  • describe concepts and formalisms of Machine Learning as a generic approach to a variety of disciplines, including image processing, robotics, computational linguistics and software engineering

Literature

Relevant literature (in order of descending importance):
  • Ian Goodfellow, Yoshua Bengio, and Aaron Courville: Deep learning. MIT Press, 2017.
  • Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning. Vol. 1. Springer, Berlin: Springer series in statistics, 2001.
  • John Shawe-Taylor and Nello Cristianini. Kernel methods for pattern analysis. Cambridge university press, 2004.

Requirements for attendance of the module (informal)

1. Mandatory Requirements:

IMPORTANT: This course requires of students a profound knowledge and skill in the following areas of mathematics:

(a) Uni- und multivariate CALCULUS (derivatives, gradients, Jacobians, Hesse matrix, chain rule, Leibnitz notation, experience in computing Jacobians and gradients of simple functions in R^d, etc.),

(b) LINEAR ALGEBRA (vectors, matrices, eigen and singular-value decomposition, basic calculation rules with matrices, etc.).

Kaiserslautern CS BSc students studied this material in their undergrad in the courses 'Algebraic Structures' and 'Analysis', but oftentimes struggle to apply it in the context of machine learning. International students typically never studied a large chunk of the required material and are in most cases overwhelmed by high comparably high mathematical difficulty of the course. This has resulted in the past in very high dropout rates of the ML1 course. Both groups of students (national and international) are STRONGLY advised to study in the semester break before the course these mathematical foundations. (Former) Kaiserslautern BSC students are recommended to study their old math course material ('Algebraic Structures' and 'Analysis'). International CS students are recommended to study the MOOC 'Mathematics for Machine Learning': https://www.coursera.org/specializations/mathematics-machine-learning). Part 1 (Linear Algebra) and Part 2 (Multivariate Calculus) are sufficient, but Part 3 (PCA) is helpful too.

Furthermore, an excellent review of the math that we'll require is in Sections 2+4 in Goodfellow et al.: Deep Learning.

Moreover, students are required to have basic knowledge and experience in programming in one out of the following three languages: Python, Julia, R. We recommend Python. The exercise sheets need to be submitted in Latex; thus Latex is another requirement of this course.

2. Optional requirements (nice to have):

Furthermore, the following mathematical knowledge is helpful (although not mandatory required):

calculation rules with matrices (as much as possible from matrix cookbook: https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf)

basics of continuous optimization (convexity, nonlinear OPs, Lagrange duality, gradient descent)

3. Preparing for these requirements

We advise students that want to major in machine learning to elect mathematics as minor (Nebenfach) already during their BSc studies, which is an excellent preparation for this ML1 course. Students that want to specialize on ML in their MSc are furthermore highly advised to enroll, for their supplementary study area, in as many math courses as possible during their MSC. Areas of mathematics that go well with machine learning are: linear algebra, calculus, optimization, probability, and statistics.

All machine-learning course are an excellent fit also for students majoring in mathematics (or other quantitative study programs) with minor computer science.

For more information on the mathematical preliminaries of the ML courses, see:

https://ml.cs.uni-kl.de/guide.html#plan

Requirements for attendance of the module (formal)

None

References to Module / Module Number [INF-75-50-M-5]

Course of Study Section Choice/Obligation
[WIW-82.176-SG#2009] B.Sc. Business Administration and Engineering specialising in Computer Science (2009) [2009] [Fundamentals] Field of study: Computer Science [WP] Compulsory Elective
[INF-88.79-SG] M.Sc. Computer Science [Core Modules (non specialised)] Computer Science Theory [WP] Compulsory Elective
[INF-88.79-SG] M.Sc. Computer Science [Core Modules (non specialised)] Formal Fundamentals [WP] Compulsory Elective
[INF-88.79-SG] M.Sc. Computer Science [Specialisation] Specialization 1 [WP] Compulsory Elective
[EIT-88.A20-SG#2021] M.Sc. European Master in Embedded Computing Systems (EMECS) [2021] [Core Modules (non specialised)] Core Subjects [WP] Compulsory Elective
[EIT-88.A20-SG#2021] M.Sc. European Master in Embedded Computing Systems (EMECS) [2021] [Free Elective Area] Elective Subjects [W] Elective Module
[EIT-88.D55-SG#2021] M.Sc. Embedded Computing Systems (ESY) [2021] [Core Modules (non specialised)] Core Program [WP] Compulsory Elective
[EIT-88.D55-SG#2021] M.Sc. Embedded Computing Systems (ESY) [2021] [Free Elective Area] Elective Subjects [W] Elective Module
Module-Pool Name
[INF-KI_Ba_V-MPOOL-4] Specialization Bachelor TA Intelligent Systems
[INF-SIAK-DT-AI-MPOOL-6] SIAK Certificate "Digital Transformation" - Modules INF "Artificial Intelligence"
[MV-MB-INF-2022-MPOOL-6] Wahlpflichtmodule M.Sc. Maschinenbau mit angewandter Informatik 2022
[MV-MBINFO-MPOOL-6] Wahlpflichtmodule Maschinenbau mit angewandter Informatik