# Singular Learning Theory

This is a seminar on Sumio Watanabe’s Singular Learning Theory, co-organised by Edmund Lau and Dan Murfet.

*Image from Sumio Watanabe’s homepage*.

## References

- S. Watanabe “Algebraic geometry and statistical learning theory”, 2009.
- E. Lau’s blog Probably Singular.

## Schedule

Each week there is a main session and a supplementary session. Dates are AEDT.

**13-1-22**(*Dan Murfet*): What is learning? Singularities and pendulums (video).**Supplementary**(*Edmund Lau*): The Fisher information matrix (video).

**20-1-22**(*Edmund Lau*): Fisher information, KL-divergence and singular models (video).**Supplementary**(*Liam Carroll*): Markov Chain Monte Carlo (video).

**27-1-22**(*Liam Carroll*): Neural networks and the Bayesian posterior (video)**Supplementary**(*Spencer Wong*): Rings, ideals and the Hilbert basis theorem (video).

**3-2-22**(*Spencer Wong*): From analytic to algebraic I (video).**Supplementary**(*Ken Chan*): Resolution of singularities (video).

**10-2-22**(*Dan Murfet*): Introduction to density of states (video, notes).**Supplementary**(*Spencer Wong*): Polynomial division (video).

**17-2-22**(*Spencer Wong*): From analytic to algebraic II (video).**Supplementary**: Working session 1 (video).

**24-2-22**(*Edmund Lau*): Free energy asymptotics (video)**Supplementary**: Working session 2 (video)

**3-3-22**(*Spencer Wong*): From analytic to algebraic III (video).**Supplementary**: Working session 3 (video).

**10-3-22**(*Tom Waring*): Regularly parametrised models (video).**17-3-22**(*Edmund Lau*): Bounding the partition function (video).**24-3-22**(*Edmund Lau*): The influence of sampling (video).**7-4-22**(*Edmund Lau*): Main Theorem 1 (video).**14-4-22**(*Edmund Lau*): Main Theorem 2 (video).**8-9-22**(*Matt Faruggia-Roberts*): Complexity of rank estimation (video).**15-9-22**(*Matt Faruggia-Roberts*): Piecewise-linear paths in equivalent networks (video).**22-9-22**(*various*) A minimal introduction to the geometry of tanh networks (video).

## Background reading

Some rough handwritten notes:

- Deep Learning Theory 1: Why deep learning theory?
- Deep Learning Theory 2: Thermodynamics of Singular Learning Theory
- Deep Learning Theory 3: Phase transitions
- Singular Learning Theory 4: Local RLCT
- Singular Learning Theory 5: Symmetry and RLCT
- Singular Learning Theory 6: Generalisation and Power Laws
- Singular Learning Theory 8: Calculations for feedforward networks
- Singular Learning Theory 12: Density of states
- Singular Learning Theory 13: Asymptotics of the free energy