Patrick PunNTU SingaporeLet’s connect

Teaching/MH4510

MH4510 · Previously taught undergraduate course

Statistical Learning and Data Mining

Undergraduate Mathematical Sciences

Fall, 2016–2019Back to teaching history

Historical course page. Course materials and assessment information for registered students are provided through NTULearn. External resource links may require NTU access.

Overview

[I no longer teach this course. However, this page is still kept for record.]

Brief Introduction

This is a 4AU (academic unit) applied mathematics/statistics course and suitable for fourth-year undergraduate students who have learned Linear Algebra, Multivariate Calculus, Elementary Probability and Statistics, R Language, and Regression Analysis. This course generalizes the content of Regression Analysis and explores two main topics in Data Mining: prediction and classification. It also introduces some basic methods for Association Rule Mining. In fact, many real-life problems can be categorized into these topics. In this course, we will emphasize on the applications and interpretations of various learning theories and encourage students to explore the applicability of statistical methods in real-life problems through group project.

Learning Objectives and Outcomes

This course gives an overall view of the modern statistical/machine learning techniques for mining massive datasets, ranging from generalized linear models, over model selection, to the state-of-the-art techniques like LASSO, neural networks, etc. This course will not only discuss individual algorithms and methods, but also tie principles and approaches together from a theoretical perspective. Moreover, students can gain a hands-on experience through team project. This course equips students with the necessary skills for being a data analyst.

Upon successful completion of this course, the students will be able to:
  1. Resolve data mining problems with various modern statistical techniques;
  2. Summarize the strengths and shortcomings of different techniques;
  3. Evaluate learning methods statistically and recommend the optimal one for applications;
  4. Implement the modern statistical techniques with statistical software such as R.

Assessment Scheme

  • Take-home Assignments (X%)
  • Group Project (Y%): click here for details
  • Final Examination (Z%)
In AY2016/17 and 2017/18, (X,Y,Z)=(20,25,55), but in AY2016/17, there was midterm exam instead of group project;
In AY2018/19, (X,Y,Z)=(15,30,55);
In AY2019/20 (co-instructed with Dr. Fedor Duzhin), (X,Y,Z)=(10,45,45).

Topics

  1. [3hrs] Introduction of the course/subject and Review of Matrix Calculus;
  2. [2hrs] Optimal Decision Rules and K-Nearest Neighbors (KNN) Methods;
  3. [3hrs] Linear Models for Regression;
  4. [3hrs] Generalized Linear Models for Classification;
  5. [4hrs] Resampling Methods (Cross-Validation & Bootstrap) for Model Assessment and Selection;
  6. [4hrs] Subset Selection and Regularization Methods (Ridge Regression & LASSO) for addressing Overfitting Problems;
  7. [4hrs] Deep Learning Basics: Artificial (Feed-forward) Neural Networks (ANN);
  8. [4hrs] Classification and Regression Trees (CART) and Ensemble Methods (Bagging & Boosting): Random Forests, AdaBoost;
  9. [3hrs] Support Vector Machines (SVM);
  10. [3hrs] Association Analysis (Apriori);
  11. [If time permits] Further Topics: Clustering Methods (K-means & Hierarchical), Introduction to Reinforcement Learning, etc.

References

Remark: The first two books can be downloaded for free from their websites. The third book provides some sample chapters downloadable at its website.

Major Difference between MH4510 and Other Courses

Dear NTU students, please read the following before you decide to take this elective course.

This course focuses on the idea and the implementation of various statistical learning approaches. Compared to other courses, there will be relatively less exam-type assignments. Instead, we will have many programming questions and a group project. The study mode of this course is different from that of the traditional math courses, where I refer the traditional way to "attend lecture->do assignments/practice->exam of similar questions as assignments". Students need to fully understand the approaches. To this end, students need to be hands-on. The involvement in the group project will be very helpful. That may explain why many students thought this course was difficult. Skipping the lectures/labs or trying to acquire all the content within a short period with video lectures will result in negative consequence. To most students, the course content is rich and difficult. Accidentally, from the past experience, many students who chose this course had quite high GPA.

I hope these information help you to judge whether this course is suitable for you.

Waiver Requests by NTU Singapore Students

Prerequisitie: MH2500 (Probability & Introduction to Statistics); MH3500 (Statistics); MH3510 (Regression Analysis); MH3511 (Data Analysis with Computers)

Rules for Prerequisitie Waiver Requests: There is no way to approve the waiver request for MH2500. For MH3500 and MH3511, waiver requests are possibly entertained, subject to good academic records. However, students without clearing MH3500 or MH3511 must work very hard to fully understand the statistical concepts mentioned in this course and the use of R for analyzing data. In general, waiver for MH3510 is impossible because MH4510 is an extension of MH3510. However, the following case may be considered: students with excellent academic records take both MH3510 and MH4510 simultaneously and are self-motivated to learn MH3510.

Rules for Timetable Clash Waiver Requests: Students have to provide valid reason(s) how they catch up the progress. Moreover, they need to provide a solution to how can they attend group project presentation in the last teaching week.

Important Message: Even though your waiver requests are approved, it does not mean that your current knowledge allows you to handle this course easily or missing some lectures/tutorials/labs does not matter. Instead, I trust in you that you are able to self-acquire the pre-requisitie knowledge and will catch up the progress by yourself.