Ma 148a: Geometric and Categorical Information Theory
Winter 2025, Caltech Math Department, Tuesday-Thursday 9:00-10:30 am, Room 187 Line Hall, Instructor:
Matilde Marcolli
(Regina Valluzzi, "Structure Evolution", ca 2010)
Brief Course Description
This class will present various ways in which geometry and physics
play a role in information theory. Topics will include information geometry;
algebraic and categorical structures of entropy; geometric aspects
of quantum information; methods of statistical physics in coding theory;
quantum field theory methods in the theory of computation.
Slides
Slides of classes will be added here:
Summary of Lectures
A brief summary of the material covered in each lecture will be posted here
- Tue Jan 7: convex sets, the n-simplex, convex bodies, cones, and their properties; categories, functors, and natural transformations; the simplex category, faces and degeneracy maps, and simplicial sets; the cube category and cubical sets
- Thu Jan 9: class cancelled
- Tue Jan 14: simplex and probabilities on finite sets, partial oder and Shur convexity, stochastic and bistochastic matrices, Perron-Frobenius theorem and invariant measures, category of finite probabilities, category FinStoch, category FinProb and relations, probabilitic pointed sets, probabilistic categories, zero object and categorical sum, statstical independence as product or coproduct
- Thu Jan 16: Shannon entropy, Khinchin axioms, Renyi entropy and Renyi dimension, Kullback-Leibler divergence, escort probabilities, statistical mechanics interpretation, box counting dimension and Renyi entropy
- Tue Jan 21: Tsallis entropy, q-analogs, Shannon entropy and multinomial coefficients, q-multinomial coefficients, geometries over finite fields and q-analogs, projective spaces, Grassmannians, flag varieties and multinomial coefficients, quadratic Tsallis entropy and asymptotic distribution, vector space valued stochastic processes, geometry over the "field with one element", Renyi entropy and the finite logarithm, finite polylogarithms and functional equations
- Thu Jan 23: categorical characterization of information loss and Shannon and Tsallis entropies, Fadeev version of Khinchin axioms, Fisher-Rao information metric, Christoffel symbols, conjugate connections, statistical manifolds, Amari-Chentsov tensor, deformed connections, Kullback-Leibler divergence and thermodynamics
- Tue Jan 28: divergence functions and Bregman generators, statistical manifolds and Shannon entropy, dual connections of divergence function, Pythagorean relation, projection theorem of information geometry, Frobenius manifolds and F-manifolds, occurrence in algebraic geometry and Gromov-Witten invariants, F-manifold structure of cones, statistical manifolds and WDVV equations
- Thu Jan 30: Hochschild cohomology, mutual information and Hochschild cohomology, finite information structures, probability functors and measurable functions, semigroup action and Hochschild cohomology, entropy functionals as Hoschschild cocycles
- Tue Feb 4: Kolmogorov complexity, relation to Shannon entropy, Kraft inequality for prefix-free codes, Levin probability distribution, Gell-Mann effective complexity, logical depth, integrated information
- Thu Feb 6: comparison of classical and quantum information, pure states and complex projective spaces, Bloch sphere, Fubini-Study metric on projective spaces, projective group action, combined systems tensor products and Serge embeddings, n-qbits and projective spaces as symmetric products of the projective line, Wigner theorem, density matrices, Kadison theorem, rotated Bloch vector
- Tue Feb 11: structure of the set of density matrices as a convex set, stratification in terms of U(N) orbits and simplexes of classical probabilities, von Neumann entropy, relative entropy in quantum information, Baker-Campbell-Hausdorff formula and the Fisher Rao metric, completely positive maps and quantum channels, projective measurements, Kraus form, Choi matrix, channel capacity, probabilistic categories in quantum information
- Thu Feb 13: tropical semiring, thermodynamic deformations, Khinchin axioms and algebraic properties (commutativity, unit, associativity), deformation by Shannon entropy and Maslov dequantization, relation to tropical geometry, deformation by Renyi and Tsallis entropy, non-extensive thermodynamics, semirings of functions, Cantor sets and multifractal decomposition, local dimension and local dynamical entropy, semiring deformation by Kullback-Leibler divergence, successor function in thermodynamical semirings
- Tue Feb 18: successor function and cumulants, operads, algebras over operads, operads of rooted trees, Ainfty-operads, DG operad, planar rooted trees, entropy operad, information algebra, binary guessing trees, multivariable tree-information measures, non-binary guessing trees, entropy functionals and Ainfty operad
- Thu Feb 20: Witt ring and thermodynamic semirings, thermodynamics in positive characteristic, density matrices and non-unital operad structures, range of operad composition maps and projective quantum measurements, quantum entropy functionals, tree-quantum-entropy functionals, operad of tree quantum channels
- Tue Feb 25: coding and information, Shannon coding and channel capacity, code parameters of error correcting codes, sigleton bound, Gilbert-Varshamov line and the Shannon Random Code Ensemble, statistical physics of coding/decoding, spoiling operations and asymptotic bound, estimates, computability question, asymptotic bound and Kolmogorov complexity, asymptotic bound as phase transition
- Thu Feb 27: spherical codes and sphere packings, kissing number, spherical codes from binary codes, code parameters for spherical codes, asymptotic bound, spoiling operations on spherical codes, Kolmogorov complexity
- Tue Mar 4: linear codes, central extensions, self-orthogonal codes, quantum codes, error operators, quantum stabilizer codes, CRSS algorithm, symplectic geometry over finite fields (char > 2), Heisenberg groups, Darboux form, CRSS and isotropic subspaces,
- Thu Mar 6: case of Reed-Solomon codes, perfect tensors and Lagrangians in general position, 3-qtritt example, symplectic geometry over finite fields (char =2), polarization, enhanced Lagrangians, tensor networks, HaPPY code, Moufang loops and codes, tensor networks and buildings.
- Tue Mar 11: Student Presentations
- Thu Mar 13: Student Presentations
Suggested readings: Books
The class has no official textbook, but there are some useful reference books
- M.Mezard, A.Montanari, "Information, physics and computation", Oxford
University Press, 2009.
- I.Bengtsson, K.Zyczkowski, "Geometry of quantum states", Cambridge
University Press, 2006.
- K.R. Parthasarathy, "Quantum Computation, quantum error
correcting codes and information theory", Narosa, 2006.
- Gilles Pisier, K. R. Parthasarathy, Vern Paulsen and Andreas Winter,
"The Functional Analysis of Quantum Information Theory", Springer 2015.
- S.Amari, H.Nagaoka, "Methods of Information Geometry" AMS and Oxford, 2000
Suggested papers
- Paolo Perrone, "Notes on Category Theory with examples from basic mathematics", arXiv:1912.10642
- Brendan Fong, David I Spivak, "Seven Sketches in Compositionality: An Invitation to Applied Category Theory", arXiv:1803.05316
- D.C.Brody, L.P.Hughston, "Geometric Quantum Mechanics", quant-ph/9906086
- Paolo Facchi, Ravi Kulkarni, V. I. Man'ko, Giuseppe Marmo, E. C. G. Sudarshan, Franco Ventriglia, "Classical
and Quantum Fisher Information in the Geometrical Formulation of Quantum Mechanics", arXiv:1009.5219
- Philippe Elbaz-Vincent, Herbert Gangl, Maxim Knotsevich, "On Poly(ana)logs I", arXiv:math/0008089
- Juan Pablo Vigneaux, "Information theory with finite vector spaces", arXiv:1807.05152
- Juan Pablo Vigneaux, "Generalized information structures and their cohomology", arXiv:1709.07807
- Pierre Badot, Daniel Bennequin, "The Homological Nature of Entropy", Entropy 2015, 17(5), 3253-3318
- John C. Baez, Tobias Fritz, Tom Leinster, "A Characterization of Entropy in Terms of Information Loss", arXiv:1106.1791
- Frank Nielsen, "An elementary introduction to information geometry",
arXiv:1808.08271
- Noemie C.Combe, Yuri I.Manin, "F-manifolds and geometry
of information", arXiv:2004.08808
- Claus Hertling, Yuri Manin, "Weak Frobenius manifolds", arXiv:math/9810132
- Noemie Combe, Yuri I. Manin, Matilde Marcolli, "Geometry of Information: classical and quantum aspects", arXiv:2107.08006
- Matilde Marcolli, "Gamma Spaces and Information", arXiv:1807.05314
- Yuri Manin, Matilde Marcolli, "Homotopy Theoretic and Categorical Models of Neural Information Networks", arXiv:2006.15136
- Greg Friedman,"An elementary illustrated introduction to simplicial sets", arXiv:0809.4221
- Matilde Marcolli, Ryan Thorngren, "Thermodynamic semirings",
arXiv:1108.2874
- Matilde Marcolli, Nicolas Tedeschi, "Entropy algebras and
Birkhoff factorization", arXiv:1412.0247
- Nihat Ay, Markus Mueller, Arleta Szkola, "Effective Complexity and its Relation to Logical Depth", arXiv:0810.5663
- Masafumi Oizumi, Naotsugu Tsuchiya, Shun-ichi Amari, "A unified framework for information integration based on information geometry", arXiv:1510.04455
- Yuri Manin, Matilde Marcolli, "Error-correcting codes
and phase transitions", arXiv:0910.5135
- Yuri Manin, Matilde Marcolli, "Kolmogorov complexity
and the asymptotic bound for error-correcting codes", arXiv:1203.0653
- Yuri Manin, "A computability challenge: asymptotic
bounds and isolated error-correcting codes",
arXiv:1107.4246
- Yuri Manin, "Complexity vs Energy: Theory of Computation and
Theoretical Physics", arXiv:1302.6695
- Yuri Manin, "Zipf's law and L. Levin's probability distributions",
arXiv:1301.0427
- Yuri I. Manin, Matilde Marcolli, "Asymptotic bounds for spherical codes", arXiv:1801.01552
- Noemie Combe, Yuri I. Manin, Matilde Marcolli, "Moufang Patterns and Geometry of Information", arXiv:2107.07486
- Matilde Marcolli, Christopher Perez, "Codes as fractals
and noncommutative spaces", arXiv:1107.5782
- Matilde Marcolli, John Napp, "Quantum computation and
real multiplication", arXiv:1312.3590
- Yuri Manin, "Renormalization and computation I:
motivation and background", arXiv:0904.4921,
- Yuri Manin, "Renormalization and Computation II:
Time Cut-off and the Halting Problem", arXiv:0908.3430.
- Colleen Delaney, Matilde Marcolli, "Dyson-Schwinger
equations in the theory of computation", arXiv:1302.5040
Suggested papers for student presentations:
(more material will be added here)
- John C. Baez, Mike Stay, "Physics, Topology, Logic and
Computation: A Rosetta Stone", arXiv:0903.0340
- Juan Pablo Vigneaux, "A homological characterization of generalized multinomial coefficients related to the entropic chain rule", arXiv:2003.02021
- Gy.Maksa, The general solution of a functional equation
related to the mixed theory of information, Aequationes
Mathematicae, Vol. 22 (1981), 90-96
pdf
- Philippe Elbaz-Vincent, Herbert Gangl, "Finite polylogarithms, their multiple analogues and the
Shannon entropy" pdf
- S. Amari, A. Chichoki, Information Geometry derived of
divergence functions, Bull. Polish Acad. Sci. Tech. Ser.,
Vol.58 (2010), No. 1, 183-195
pdf
- Yuri I. Manin, "F-manifolds with flat structure and Dubrovin's duality", arXiv:math/0402451
- Pierre Baudot, Monica Tapia, Daniel Bennequin, Jean-Marc Goaillard, "Topological Information Data Analysis", arXiv:1907.04242
- G. L. Litvinov, "The Maslov dequantization, idempotent and tropical mathematics: A brief introduction", arXiv:math/0507014
- Oleg Viro, "Dequantization of real algebraic geometry on logarithmic paper", arXiv:math/0005163
- A.Voronov, "The Ainfty operad and Ainfty algebras", pdf
- Frederic Chapoton, Muriel Livernet, "Pre-Lie algebras and the rooted trees operad", arXiv:math/0002069
- Martin Markl, "Operads and PROPs", arXiv:math/0601129
- Alain Connes, Caterina Consani, "Characteristic one, entropy and the absolute point", arXiv:0911.3537
- Oliver Lorscheid, "F1 for everyone", arXiv:1801.05337
- Young-Tak Oh, "q-deformation of Witt-Burnside rings", arXiv:math/0411353
- Henry Cohn, Yang Jiao, Abhinav Kumar, Salvatore Torquato, "Rigidity of spherical codes", arXiv:1102.5060
- A.R. Calderbank, E.M. Rains, P.W. Shor, N.J.A. Sloane, "Quantum Error Correction via Codes over GF(4)", arXiv:quant-ph/9608006
- A.R. Calderbank, E.M. Rains, P.W. Shor, N.J.A. Sloane, "Quantum Error Correction and orthogonal geometry", arXiv:quant-ph/9605005
- F. Pastawski, B. Yoshida, D. Harlow, and J. Preskill, "Holographic quantum error-correcting
codes: Toy models for the bulk/boundary correspondence", arXiv:1503.06237
- Matthew Heydeman, Matilde Marcolli, Sarthak Parikh, Ingmar Saberi, "Nonarchimedean Holographic Entropy from Networks of Perfect Tensors", arXiv:1812.04057
- Shamgar Gurevich, Ronny Hadani, "Quantization of symplectic vector spaces over finite fields", arXiv:0705.4556
- Shamgar Gurevich, Ronny Hadani, "The Weil Representation in Characteristic Two", arXiv:0808.1664
- Gunnar Carlsson, "Deloopings in algebraic K-theory" pdf
- Graeme Segal, "Categories and cohomology theories" pdf
- Bob Coecke, Tobias Fritz, Robert W. Spekkens, "A mathematical theory of resources", arXiv:1409.5531
- Tom Mainiero, "Homological Tools for the Quantum Mechanic", arXiv:1901.02011
- Yuri Manin, "Classical computing, quantum computing,
and Shor's factoring algorithm", arXiv:quant-ph/9903008
- S.Trebst, M.Troyer, Z.Wang, A.Ludwig, "A short introduction
to Fibonacci anyon models", arXiv:0902.3275
- Michael H. Freedman, Alexei Kitaev, Michael J. Larsen, Zhenghan Wang,
"Topological quantum computation", arXiv:quant-ph/0101025
- C.Nayak, S.Simon, A.Stern, M.Freedman, S.Sarma,
"Non-Abelian Anyons and Topological Quantum Computation", arXiv:0707.1889
- Eric Rowell, Richard Stong, Zhenghan Wang, "On classification of modular tensor categories", arXiv:0712.1377
- Colleen Delaney, Zhenghan Wang, "Symmetry defects and their application to topological quantum computing", arXiv:1811.02143
- Colleen Delaney, Eric C. Rowell, Zhenghan Wang, "Local unitary representations of the braid group and their applications to quantum computing", arXiv:1604.06429
- M.Redei, S.J.Summers, "Quantum Probability Theory", arXiv:quant-ph/0601158
- P.Jizba, "Information theory and generalized statistics"
pdf
- Bernard Field and Tapio Simula, "Introduction to topological quantum computation with non-Abelian anyons", Quantum Sci. Technol. 3 (2018) 045004
pdf
Syllabus
Requirements for the class: one oral
presentation on reading materials chosen from the list
provided above (first come first serve) and attendance
of (most) lectures.
Assigned student presentations:
The schedule of assigned persentations will be added here
- Mar 11 9:00-9:30: Milo, "TQFTs, Jones polynomial, Reshetikhin–Turaev invariants and Fibonacci anyons"
- Mar 11 9:30-10:00: Zeyu, "Categorical models of information networks"
- Mar 11 10:00-10:30: Jeff, "Quantum Computing Enhanced Sensing"
- Mar 11 10:30-11:00: Jun, (online) "Frobenius manifolds"
- Mar 11 11:00-11:30 Yaodong, (online) "Non-Abelian Anyons and Topological Quantum Computation"
- Mar 13 9:00-9:30: Rodion, "Quantum Probability Theory"
- Mar 13 9:30_10:00: Eliot, "Generalized multinomial coefficients and entropic chain rule"
- Mar 13 10:00-10:30: Shiri, "Quantum error correction, orthogonal geometry, codes over GL(4)"
- Mar 13 10:30-11;00 (ROOM 359 LH): Cameron, "Information cohomology and machine learning"
- Mar 13 11:00-11:30 (ROOM 359 LH): Caden, "Physics, Topology, Logic and Computation: A Rosetta Stone"