Episode

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!

https://www.pymc-labs.com/

My Intuitive Bayes Online Courses: https://www.intuitivebayes.com/

1:1 Mentorship with me: https://topmate.io/alex_andorra

How does the world of statistical physics intertwine with machine learning, and what groundbreaking insights can this fusion bring to the field of artificial intelligence?

In this episode, we delve into these intriguing questions with Marylou Gabrié. an assistant professor at CMAP, Ecole Polytechnique Paris.

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at bababrinkman.com/ !

Thank you to my Patrons for making this episode possible!

Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas, Luke Gorrie and Cory Kiser.

Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag ;)

Takeaways:

• Developing methods that leverage machine learning for scientific computing can provide valuable insights into high-dimensional probabilistic models.
• Generative models can be used to speed up Markov Chain Monte Carlo (MCMC) methods and improve the efficiency of sampling from complex distributions.
• The Adaptive Monte Carlo algorithm augmented with normalizing flows offers a powerful approach for sampling from multimodal distributions.
• Scaling the algorithm to higher dimensions and handling discrete parameters are ongoing challenges in the field.
• Open-source packages, such as Flow MC, provide valuable tools for researchers and practitioners to adopt and contribute to the development of new algorithms. The scaling of algorithms depends on the quantity of parameters and data. While some methods work well with a few hundred parameters, larger quantities can lead to difficulties.
• Generative models, such as normalizing flows, offer benefits in the Bayesian context, including amortization and the ability to adjust the model with new data.
• Machine learning and MCMC are complementary and should be used together rather than replacing one another.
• Machine learning can assist scientific computing in the context of scarce data, where expensive experiments or numerics are required.
• The future of MCMC lies in the exploration of sampling multimodal distributions and understanding resource limitations in scientific research.

Links from the show: https://learnbayesstats.com/episode/98-fusing-statistical-physics-machine-learning-adaptive-mcmc-marylou-gabrie/

Chapters:

00:00 Introduction and Background

01:20 Marie-Lou's Work and Interests

02:32 Journey into Statistical Physics and Machine Learning

04:51 Studying Deep Neural Networks with Statistical Mechanics

05:48 Using Machine Learning as a Tool

07:39 Using Generative Models to Speed up MCMC Methods

08:10 Adaptive Monte Carlo Augmented with Normalizing Flows

10:42 Challenges and Trade-offs in the Algorithm

15:04 When to Use the New Algorithm

18:15 Scaling the Algorithm to Higher Dimensions

20:49 Adopting New Algorithms and Open Source Package

22:42 Benchmarking the Algorithm

23:10 Handling Multimodal Distributions

25:01 Sampling Discrete Parameters

26:27 Flow MC Package and Contributions

31:00 Value of Open Source Packages

37:06 Running the Deep Neural Network and Bottlenecks

42:52 Scaling with Quantity of Parameters and Data

43:10 Impact of Quantity of Parameters

43:42 Amortization in Generative Models

45:12 Tuning Time and Running Times

47:05 Scaling the Algorithm

49:07 Machine Learning in Scarce Data

52:20 Machine Learning-Assisted Scientific Computing

53:16 Relevance of MCMC in the Future

57:35 Machine Learning Assisting Scientific Computing

01:05:39 Promising Areas of Research

01:08:23 Sampling Multimodal Distributions