COMP / ELEC / STAT 502

COMP / ELEC / STAT 502
Spring 2026

Schedule

The schedule of class topics, home works, tests, due dates, and handouts will come on-line in this table in a timely manner. The actual materials (lecture slides, other handouts, home work assignments) will be posted in Canvas for download. Should you have trouble with this web page please notify me and copy the TA(s).

The Lecture Notes and Handouts constitute the mandatory reading, and consist of two main ingredients:

Copies of my slides and other material that I compile for the class, posted in /Files/Lecture Notes in Canvas.
Chapters from course materials that Prof. Colin Fyfe put together for his graduate and senior courses at the University of the West of Scotland, Scotland. These will be posted in /Files/Other, mandatory text (or Files/Other, optional text).

Key to notation:
L#-"title" = my lecture slides # titled "title".
L#.x-"title" = Files related to lecture lecture L# (such as illustrations, details of equations, etc.)
CFi/k = Chapter k of Colin Fyfe's Volume i. Note that since I am using chapters from two different volumes two printed chapters may have the same number. Mark the volume # on your hardcopies.

Additional, highly recommended reading from classis text books is also listed. The references to those are given in the Course materials.
>>

L11-CompetitiveUnsupervisedLearning.h.pdf
L11.1-Self_Organizing_Maps-EM502.pdf
CF1/5.6-8

No.

Date

Topic

Lecture Notes & Handouts
mandatory reading

Homework

Upload an electronic copy to Canvas
by 12:30pm on the due date,
unless otherwise indicated.

Recommended Reading

1

Jan 13

Orientation, Introduction

On your own: Review of basics (L2-ReviewOfBasics.pdf). This will help you, along with exercises in HW02, to gauge your preparedness and to catch up on background if needed. Notions / formulae that I collected in Lecture 2 (L2) will come up in various contexts at various times in the course. Alternatively, you can look these up when the time comes :-)
L1-Introduction-502.pdf
L2-ReviewOfBasics.pdf
CF2/1

HW01 due 1/15.
This is a very light (not real) home work. You will only have to submit answers to my survey to assess backgrounds. Do this HW individually.

Form HW groups for HW2 and on. We will discuss HW groups in class.

Haykin Ch1
or Ham & Kostanic Ch 1, Ch2.1-3

2

Jan 15

Introduction, orientation, cont'd
Train a single PE

L2-ReviewOfBasics.pdf
L1.1-Training-a-single-PE.pdf
CF2/1, CF2/2

HW02 due 1/24
Part of HW02 problems may be fully graded,
others only for turning in and earnestly attempting them. See details in the posted HW02. The exact manner of grading will be determined according to grading help (TBD).
The longer time (overlapping with HW03) is given so that HW groups can form and establish their collaboration.

Haykin Ch10
or Cover & Thomas Ch1 and Ch9

3

Jan 20
Associative memory L4-AssociativeMemory.h.pdf and L4.x files
CF1/2 7aykin Ch2.11
or Ham & Kostanic Ch 3.2

4

Jan 22
Associative memory, cont'd updated L4-AssociativeMemory.h.pdf and L4.x files
CF1/2

Haykin Ch2.11
or Ham & Kostanic Ch 3.2

5

Jan 27
Associative memory(wrap-up)
Simple Perceptron updated L4-AssociativeMemory.h.pdf and L4.x files
CF1/2
L5-Perceptron.h.part1.pdf
CF1/3

HW03

Haykin Ch2.11
or Ham & Kostanic Ch 3.2

6
Jan 29 Simple Perceptron L5-Perceptrons.h.pdf and L.5x files
CF1/3
Ham & Kostanic Ch 2.5-2.6; or Haykin Ch3

7
Feb 3 Preliminary to Multilayer Perceptron
Multilayer Perceptron with Back Propagation L6.0-Train-a-2layer-network.h.pdf
L6-MultiLayerPerceptron.h.part1.pdf
CF1/4
Recommended: experiment with Jesse Hellem's HW03 code after you are done with your HW03. Nothing to turn in, just enjoy running it and exploring limits with the characters.ascii data set (all letters plus 10 digits). Code will be available from after the late HW03 deadline passed, on Feb 7 Ham & Kostanic Ch 3.3.1; or Haykin Ch4.1-4.4

8
Feb 5 Multilayer Perceptron, cont'd
Issues With Backprop L6-MultiLayerPerceptron.h.pdf
L7-IssuesWithMLPs.h.part1.pdf
CF1/4 HW04 Part I due Feb 11
This is Part I of HW04. Part II will post on Feb 11 and will be due on Feb 19 (so two weeks for HW04 altogether). We will give you feedback on Part I before Part II is due and, in addition to using the feedback to imrpove Part II, you will have a chance to revise and resubmit Part I along with Part II. See details with the Assignment in Canvas. Ham & Kostanic Ch 3.3.1; or Haykin Ch4.1-4.4

9
Feb 10 Issues With Backprop: Scaling, convergence, performance monitoring
Matrix BP; practical items (time permitting) L7-IssuesWithMLPs.h.pdf
L7.1-Vector-matrix-BP.h.pdf
L7.0x-HW04_sample-plots_xxx.pdf sample plots
L7.3_Sample-parameter-table_502.pdf CF1/4
Ham & Kostanic Ch 3.3.2 - 3.3.7; or Haykin Ch4.5-4.8

Feb 12 NO CLASS, Spring recess HW04 Part II due Feb 19
Part I will have to be re-submitted together with and Part II, with optional revision to Part I. See details with the Assignment(s) in Canvas.

10
Feb 17 Issues With Backprop: More practical items; cross-validation
TBA L7-IssuesWithMLPs.h.pdf
L7.1-Vector-matrix-BP.h.pdf
L7.2-BP-practical-items.h.pdf
L7.0x-HW04_sample-plots_xxx.pdf samples
L7.3_Sample-parameter-table_502.pdf
Ham & Kostanic Ch 3.3.2 - 3.3.7; or Haykin Ch4.5-4.8

11
Feb 19 Cross-validation;

Network Pruning: Optimal Brain Damage and Optimal Brain Surgeon L7.4-Cross-validation.h.pdf
L9-NetworkPruning.pdf
L9.1-note_OBS-derivations.pdf
L9.2-RecursiveEvaluation-of-InverseHessian.pdf HW05 due Feb 26 (BP application exercises including cross-validation) Haykin Ch 4.15 pp 222 - 226

12
Feb 24 Finish Network Pruning;

Unsupervised learning. Competitive learning: SCL, SOM Ham and Kostanic Ch 4.1-4.2; or Haykin CH9.1-6

13
Feb 26 Competitive learning: SOM, cont'd; visualization, interpretation L11-CompetitiveUnsupervisedLearning.h.pdf
update of L11.1-Self_Organizing_Maps-EM502.pdf
CF1/5.6-8 HW06 due March 7;

Review PCA in preparation for PCA networks on March 10.
PCA material in Canvas, in Supplements/PCA-Ham&Kostanic-pp396-399.pdf or in any other standard text Ham and Kostanic Ch 4.1-4.2; or Haykin CH9.1-6

14
Mar 3 QUIZ1
Material up to (including) Feb 24

Competitive learning, cont'd

Return to COMP 502 / ELEC 502 / STAT 502 home page

No.	Date	Topic	Lecture Notes & Handouts mandatory reading	Homework Upload an electronic copy to Canvas by 12:30pm on the due date, unless otherwise indicated.	Recommended Reading
1	Jan 13	Orientation, Introduction On your own: Review of basics (L2-ReviewOfBasics.pdf). This will help you, along with exercises in HW02, to gauge your preparedness and to catch up on background if needed. Notions / formulae that I collected in Lecture 2 (L2) will come up in various contexts at various times in the course. Alternatively, you can look these up when the time comes :-)	L1-Introduction-502.pdf L2-ReviewOfBasics.pdf CF2/1	HW01 due 1/15. This is a very light (not real) home work. You will only have to submit answers to my survey to assess backgrounds. Do this HW individually. Form HW groups for HW2 and on. We will discuss HW groups in class.	Haykin Ch1 or Ham & Kostanic Ch 1, Ch2.1-3
2	Jan 15	Introduction, orientation, cont'd Train a single PE	L2-ReviewOfBasics.pdf L1.1-Training-a-single-PE.pdf CF2/1, CF2/2	HW02 due 1/24 Part of HW02 problems may be fully graded, others only for turning in and earnestly attempting them. See details in the posted HW02. The exact manner of grading will be determined according to grading help (TBD). The longer time (overlapping with HW03) is given so that HW groups can form and establish their collaboration.	Haykin Ch10 or Cover & Thomas Ch1 and Ch9
3	Jan 20	Associative memory	L4-AssociativeMemory.h.pdf and L4.x files CF1/2		7aykin Ch2.11 or Ham & Kostanic Ch 3.2
4	Jan 22	Associative memory, cont'd	updated L4-AssociativeMemory.h.pdf and L4.x files CF1/2		Haykin Ch2.11 or Ham & Kostanic Ch 3.2
5	Jan 27	Associative memory(wrap-up) Simple Perceptron	updated L4-AssociativeMemory.h.pdf and L4.x files CF1/2 L5-Perceptron.h.part1.pdf CF1/3	HW03	Haykin Ch2.11 or Ham & Kostanic Ch 3.2
6	Jan 29	Simple Perceptron	L5-Perceptrons.h.pdf and L.5x files CF1/3		Ham & Kostanic Ch 2.5-2.6; or Haykin Ch3
7	Feb 3	Preliminary to Multilayer Perceptron Multilayer Perceptron with Back Propagation	L6.0-Train-a-2layer-network.h.pdf L6-MultiLayerPerceptron.h.part1.pdf CF1/4	Recommended: experiment with Jesse Hellem's HW03 code after you are done with your HW03. Nothing to turn in, just enjoy running it and exploring limits with the characters.ascii data set (all letters plus 10 digits). Code will be available from after the late HW03 deadline passed, on Feb 7	Ham & Kostanic Ch 3.3.1; or Haykin Ch4.1-4.4
8	Feb 5	Multilayer Perceptron, cont'd Issues With Backprop	L6-MultiLayerPerceptron.h.pdf L7-IssuesWithMLPs.h.part1.pdf CF1/4	HW04 Part I due Feb 11 This is Part I of HW04. Part II will post on Feb 11 and will be due on Feb 19 (so two weeks for HW04 altogether). We will give you feedback on Part I before Part II is due and, in addition to using the feedback to imrpove Part II, you will have a chance to revise and resubmit Part I along with Part II. See details with the Assignment in Canvas.	Ham & Kostanic Ch 3.3.1; or Haykin Ch4.1-4.4
9	Feb 10	Issues With Backprop: Scaling, convergence, performance monitoring Matrix BP; practical items (time permitting)	L7-IssuesWithMLPs.h.pdf L7.1-Vector-matrix-BP.h.pdf L7.0x-HW04_sample-plots_xxx.pdf sample plots L7.3_Sample-parameter-table_502.pdf CF1/4		Ham & Kostanic Ch 3.3.2 - 3.3.7; or Haykin Ch4.5-4.8
	Feb 12	NO CLASS, Spring recess		HW04 Part II due Feb 19 Part I will have to be re-submitted together with and Part II, with optional revision to Part I. See details with the Assignment(s) in Canvas.
10	Feb 17	Issues With Backprop: More practical items; cross-validation TBA	L7-IssuesWithMLPs.h.pdf L7.1-Vector-matrix-BP.h.pdf L7.2-BP-practical-items.h.pdf L7.0x-HW04_sample-plots_xxx.pdf samples L7.3_Sample-parameter-table_502.pdf		Ham & Kostanic Ch 3.3.2 - 3.3.7; or Haykin Ch4.5-4.8
11	Feb 19	Cross-validation; Network Pruning: Optimal Brain Damage and Optimal Brain Surgeon	L7.4-Cross-validation.h.pdf L9-NetworkPruning.pdf L9.1-note_OBS-derivations.pdf L9.2-RecursiveEvaluation-of-InverseHessian.pdf	HW05 due Feb 26 (BP application exercises including cross-validation)	Haykin Ch 4.15 pp 222 - 226
12	Feb 24	Finish Network Pruning; Unsupervised learning. Competitive learning: SCL, SOM		Ham and Kostanic Ch 4.1-4.2; or Haykin CH9.1-6
13	Feb 26	Competitive learning: SOM, cont'd; visualization, interpretation	L11-CompetitiveUnsupervisedLearning.h.pdf update of L11.1-Self_Organizing_Maps-EM502.pdf CF1/5.6-8	HW06 due March 7; Review PCA in preparation for PCA networks on March 10. PCA material in Canvas, in Supplements/PCA-Ham&Kostanic-pp396-399.pdf or in any other standard text	Ham and Kostanic Ch 4.1-4.2; or Haykin CH9.1-6
14	Mar 3	QUIZ1 Material up to (including) Feb 24 Competitive learning, cont'd

COMP / ELEC / STAT 502 Spring 2026

Schedule

COMP / ELEC / STAT 502
Spring 2026