Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!
Back in 2016, when I started dedicating my evenings and weekends to learning how to code and do serious stats, I was a bit lost… Where do I start? Which language do I pick? Why are all those languages just named with one single letter??
Then I found some stats classes by Justin Bois — and it was a tremendous help to get out of that wood (and yes, this was a pun). I really loved Justin’s teaching because he was making the assumptions explicit, and also explained them — which was so much more satisfying to my nerdy brain, which always wonders why we’re making this assumption and not that one.
So of course, I’m thrilled to be hosting Justin on the show today! Justin is a Teaching Professor in the Division of Biology and Biological Engineering at Caltech, California, where he also did his PhD. Before that, he was a postdoc in biochemistry at UCLA, as well as the Max Planck Institute in Dresden, Germany.
Most importantly for the football fans, he’s a goalkeeper — actually, the day before recording, he saved two penalty kicks… and even scored a goal! A big fan of Los Angeles football club, Justin is a also a magic enthusiast — he is indeed a member of the Magic Castle…
Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !
Thank you to my Patrons for making this episode possible!
Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, Adam Bartonicek, William Benton, Alan O’Donnell, Mark Ormsby, James Ahloy, Robin Taylor, Thomas Wiecki, Chad Scherrer, Nathaniel Neitzke, Zwelithini Tunyiswa, Elea McDonnell Feit, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Joshua Duncan, Ian Moran, Paul Oreto, Colin Caprani, George Ho, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Raul Maldonado, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Matthew McAnear, Michael Hankin, Cameron Smith, Luis Iberico, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Aaron Jones, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, David Haas, Robert Yolken and Or Duek.
Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag 😉
Links from the show:
- Justin’s website: http://bois.caltech.edu/index.html
- Justin on GitHub: https://github.com/justinbois/
- Justin’s course on Data analysis with frequentist inference: https://bebi103a.github.io/
- Justin’s course on Bayesian inference: https://bebi103b.github.io/
- LBS #6, A principled Bayesian workflow, with Michael Betancourt: https://learnbayesstats.com/episode/6-a-principled-bayesian-workflow-with-michael-betancourt/
- Physical Biology of the Cell: https://www.routledge.com/Physical-Biology-of-the-Cell/Phillips-Kondev-Theriot-Garcia-Phillips-Kondev-Theriot-Garcia/p/book/9780815344506
- Knowledge Illusion – Why We Never Think Alone: https://www.amazon.fr/Knowledge-Illusion-Never-Think-Alone/dp/039918435XThe
- Sustainable Energy – Without the Hot Air: https://www.amazon.com/Sustainable-Energy-Without-Hot-Air/dp/0954452933
- Information Theory, Inference and Learning Algorithms: https://www.amazon.com/Information-Theory-Inference-Learning-Algorithms/dp/0521642981
Abstract
Justin Bois did his Bachelor and PhD in Chemical Engineering before working as a Postdoctoral Researcher in Biological Physics, Chemistry and Biological Engineering. He now works as a Teaching Professor at the division of Biology and Biological Engineering at Caltech, USA.
He first got into Bayesian Statistics like many scientists in fields like biology or psychology, by wanting to understand what the statistics actually mean that he was using. His central question was “what is parameter estimation actually?”. After all, that’s a lot of what doing quantitative science is on a daily basis!
The Bayesian framework allowed him to find an answer and made him feel like a more complete scientist. As a teaching professor, he is now helping students of life sciences such as neuroscience or biological engineering to become true Bayesians.
His teaching covers what you need to become a proficient Bayesian analyst, from opening datasets to Bayesian inference. He emphasizes the importance of models implicit in quantitative research and shows that we do in most cases have a prior idea of an estimand’s magnitude.
Justin believes that we are naturally programmed to think in a Bayesian framework but still should mess up sometimes to learn that statistical techniques are fragile. You can find some of his teaching on his website.
Transcript
This transcript was generated automatically. Some transcription errors may have remained. Feel free to reach out if you’re willing to correct them.
[00:00:00] In 2016, when I started dedicating my evenings and weekends to learning how to code and do serious stats, I was a bit lost, to be honest. Where do I start? Which language do I speak? Why are all those languages just named with one single letter, like R or C? Then I found some stats classes by just in voice.
And it was a tremendous help to get out of that wood. And yes, this was a pun. I really enjoyed Justine’s teaching because he was making the assumptions explicit, and he also explained them, which was so much more satisfying to my minority brain, which always wonders why we’re making this assumption and not that one.
So of course, I’m thrilled to be hosting Justin on the show today. Justin is a teaching professor in the division of biology and biological engineering at Caltech, California, where he also did his PhD. Before that, he was a postdoc in biochemistry at UCLA as well as the Max Plan Institute in Tris, Germany.
Most importantly, for the football fans, Justin is a goalkeeper. [00:01:00] Actually, the day before recording, he saved two penalty, penalty, kicks, and even scored a goal. Yes, a big fan of Los Angeles’s football club. Justine is also a magic enthusiast. He is indeed a member of the Magic Castle. This is Learning Patient Statistics.
Ex episode 70, recorded September 2nd, 2022. Welcome to Learning Patient Statistics, a fortnightly podcast on Beijing Inference, The methods project in the People who Make Impossible. I’m your host, Alex Andora. You can follow me Twitter at ann underscore like the country. For any info about the podcast, learn base stats.com is lap less to be Show notes becoming corporate sponsor supporting lbs and Pat.
Unlocking base merge, everything is in there. That’s learn base dance.com. If with all that info, a model is still resisting you, or if you find my voice special, smooth and [00:02:00] want me to come and teach patient stats in company, then reach out at alex.andorra@pymc-labs.io or book call with me at learnbayesstats.com.
Thanks a lot folks. And best patient wish shes to you old. Let me show you how to be a good bla and change your predictions after taking information and, and if you’re thinking they’ll be less than amazing, let’s adjust those expectations. What’s a basian is someone who cares about evidence and doesn’t jump to assumptions based on intuitions and prejudice.
Abassian makes predictions on the best available info and adjusts the probability cuz every belief is provisional. And when I kick a flow, mostly I’m watching eyes widen. Maybe cuz my likeness lowers expectations of tight ryman. How would I know unless I’m Ryman in front of a bunch of blind men, drop in placebo controlled science like I’m Richard Feinman, just in boys.
Welcome to Learning Patient St Sticks. Thank you. Happy to be here. Yes. Very [00:03:00] happy to have you here because, well, you know that, but listeners do not. But you are actually one of the first people who introduced me back to, uh, statistics and programming in 2017 when I started my Carrie Shift. So it’s awesome to have you here today.
I’m glad my stuff helped you get going. That’s, that’s the point. That’s the goal. Yeah. Yeah, that’s really cool. And also, I’m happy to have learned how you pronounce your last name because in French, you know, that’s a French name. I dunno if you have some French origin, but in French it means, I know, I know it’s a French name, but it’s actually, as far as I understand, my family’s from Northern Germany and there’s a, a name there that’s spelled b e u s s, like, and it’s pronounced like in Germany, you say Boce.
And then it got anglicized, I think when I moved to the US but uh, I was actually recently, just this past summer in Luanne, Switzerland, and there was a giant wood recycling bin. With my name on it, , it said d i s. So I got my picture taken next to that. So yeah. Yeah. Lo Zen is in the French speaking part of Switzerland.[00:04:00]
That’s right. Cool. So we’re starting already with the origin story, so I love that cuz it’s actually always my first question. So how did you jump to the stats in biology worlds and like how Senior of a Pass read it? Well, I think the path that I had toward really thinking carefully about statistical inferences is a very common path among scientists, meaning scientists outside of data scientists and, and maybe also outside of really data rich branches of sciences such as astronomy.
So I studied chemical engineering as an undergraduate. It was a standard program. I didn’t really do any undergrad research or anything, but I got into a little bit of statistics when I had a job at Kraft Foods. After undergraduate where I worked at the statistician on doing some predictive modeling about, uh, some food safety issues.
And I thought it was interesting, but I sort of just, I was an engineer. I was making the product, I was implementing the stuff in the production facility and the statistician kind of took care of [00:05:00] everything else. I thought, I thought he was one of the coolest people in the company, . Um, but I didn’t really, you know, it didn’t really hook me in to really thinking about that.
But I went and did a PhD and my PhD really didn’t involve really much experimentation at all. I was actually doing computational modeling of like how nucleic acids get their structure and shape and things. And that was, it just didn’t really involve analysis of much data. Then in my post-doctoral studies, in my post-doctoral work, I was working with some experimentalists who had some data sets and they needed.
do estimates of parameters based on some theoretical models that I had derived or worked on. And I had done some stuff and you know, various lab classes and stuff, but it’s your standard thing. It’s like, ooh, I know how to do a cur fit. Meaning I can, I guess in the Python way I would do it, SciPi dot optimized dot cur fit.
Or you know, in MATLAB I could do at least squares or something like that. And, and I knew this idea of minimizing the sum of the square of the residuals and that’s gonna get you [00:06:00] a line that looks close to what your data points are. But the inference problems, the theoretical curves were actually a little bit say for some of ’em.
There was no close to form solution. They were actually solutions to differential equations. And so the actual theoretical treatment I had was a little bit more complicated. And so I needed to start to think a little bit more carefully about exactly how we’re going about estimating the parameters thereof.
Right? And so I kind of just started grabbing uh, books and I. Discovered quickly that I had no idea what I was doing, , and actually neither did anybody around me. And I don’t mean that pejoratively, it’s just, it’s a very common thing among the scient. A lot of people in the sciences that aren’t, that don’t work as much with data.
And perhaps it’s less common now, but it’s definitely more common than, you know, 10, 15, uh, years ago. And so I just kind of started looking into how we should actually think about the estimates of [00:07:00] parameters given a data set. And really what happened was the problem became crystallized for me, the problem of parameter estimation.
And I had never actually heard that phrase, perimeter estimation. To me. It was find the best fit per. If your curve goes through your data point, that means that you’re, the theory that you derived is probably pretty good. And of course, I didn’t think about what the word probably meant there. I, I only knew it colloquially, right?
And so, cuz I was focused on deriving what the theory is. And of course that’s a whole, hugely important part of, of the scientific enterprise. But once you get that theory arrived to try to estimate the parameters of that are present in that theory from measurement, that problem just became clear to me.
Once I had a clear problem statement, then I was able to start to think about how to solve it. And so the problem statement was, I have a theory that has a set of parameters. I want to try to figure out what the parameters are by taking [00:08:00] some measurements and checking for one set of parameters. The measurements would be different.
How do I find what parameters there are to, to give me this type, type of data that I observe. I intentionally just stated that awkwardly because that awkwardness there sort of made the, It’s funny, it made it clear to me that the problem was unclear . And, and so I, that’s what got me into a basian mode of thinking because it was hard for me to wrap my head around what it meant to do that thing that I’ve been doing all this time.
This minimizing some squares of residuals and trying to find the best fit parameter. And, you know, in retrospect now I’ve actually, you know, that I taught myself. Cause I didn’t really ever have a course in statistical inference or anything like that, say Okay. I was essentially doing a maximum likelihood estimation, which is a f way of doing prime destination.
And I, and I hadn’t actually thought about what that meant. I mean, I understand that now. We don’t really need to talk [00:09:00] about that since we’re talking about BA stuff now, but, and it was just harder for me to wrap my head around what that meant. And so I started reading. About the basing interpretation of probability, and it was really, it really just crystallized everything and made it clear, and then I could state the problem much more clearly.
The problem was I was trying to find a posterior probability density function for these parameters given the data, and that was just so much clearly stated in Baying framework, and then that kinda lit me on fire because I was like, Holy cow, this thing that we do so often in the scientific enterprise, I can actually state the question , right?
And I just thought that was such a profound moment, and then I was kind of hooked from there on out and I, I was concent trying to improve how I thought about these things. And yeah, so I did a lot of reading. I realized I just talked a lot. You probably have [00:10:00] some questions about some of the stuff I just said, so please.
Oh yeah, well wait. But, um, I mean, that’s good to have a, an overview like that. And so I guess that’s also like, it sounds like you were introduced to patient statistics at the same time as you were doing that deep dive into, wait, like, I’m not sure I understand what I’m using then. Oh, actually I don’t understand anything and then I have to learn about that.
But it seems that you, you were also introduced to patient stats at that same time, Is that right? Yeah, I think so. And I think this is actually sort of a classic way in which scientists come up with what it is that they want to study. Because instead you start poking around, you kind of don’t really know where the holes in your knowledge are.
And so what I saw was like just a giant hole in my knowledge and my toolbox, and I saw the hole and I said, All right, let’s fill it . And um, and so then I just started feeling around on how to do that. I see. And I am also curious as [00:11:00] to, and what motivated you to dive into the Beijing way of doing things?
I really do think it was the clarity. I think that, Okay. I think that arguing about like what interpretation or probability you wanna use is not the most fruitful way to spend one’s time. For me, it was really, it was just so much more intuitive. I felt like I could have this interpretation of probability that it’s, it’s a quantification of the plausibility of a logical conjecture of any logical conjecture gave me sort of the flexibility where I could think about like a parameter of a theory that I had derived and what value that might have, that I could think about that probabil.
directly, and that was just very much easier for me to understand, you know? Yeah. I looked at some books that took Frequentness approaches and, and I would read them and I would read them again, and then again, and yes, in the end I understood something and one [00:12:00] should not shy away from a certain method or a field or anything, just because it’s hard, difficult to understand.
I don’t think you should do that. At the same time though, the ease with which the, I could understand how the Bay and prouder estimation problem was structured and, and how you could, how you were ascribing probability distributions to parameter values. They also very, very pleasing structure of a basing problem, which is where you have a prior picture of, of what your parameter should look like.
That is then updated by the measurements you make via the likelihood to give you the posterior. that was really pleasing to me because that was exactly how I was thinking about when I’m doing science right. Like I have some rough idea of what I think the parameter might be, you know? So for example, the first one that I was really working on was I was measuring dissociation constants between two proteins.
And so [00:13:00] the quick with what that means is inside cells and outside actually, uh, proteins are binding with other proteins all the time, and the binding interactions are just central to all kinds. Life , let’s put, put it that way, a huge, huge part of life and a very common thing to do is to try to, as you start to look at new pieces of proteins as you wanna start to look at how they interact with with others.
And so I was looking at how these two specific proteins bound with each other and I was working with an experimentalist, her name is Amy Barocco. And she was doing some careful measurements of using a technique. I don’t, we won’t go into, it’s a biochemical technique. And she would get some data and I needed to measure what the dissociation constant is.
Association constant tells you how tight, how strong do these two things buy into each other. And so biochemists have a pretty good idea what the dissociation constant value is going to be. They know that it’s not gonna be so strong that it’s, that the force holding thing together is stronger than the force is holding a nucleus [00:14:00] together.
they know that. It’s not like atomic energy level forces, right? They also know that it’s so you, you have some prior knowledge on it, right? You have some idea what the energetics are and. And then you update those with the measurements. And it just made the whole process of what, what she was doing as an experimentalist, what I was doing as her collaborator, theoretical collaborator with her.
It just made it so much clearer to me when I started learning about the basing approach to parameter estimation, and I found that really exhilarating. Yeah. Yeah. So I love the like, kind of the two dimensions of this approach, which is like, on one hand you were really, uh, diving into the methods basically, and I’m trying to understand how do I know what I know about that basically.
And so, which reminds me of how Michael Beko, uh, for instance, entered the field also. On the other end, also a very practical approach of being like, well, it’s just like makes more sense [00:15:00] to me in my work and like I can more easily relate it to what I’m doing. Which is also a big, a big component of how people end up doing patient statistics.
So I find it’s more interesting. Yeah. Yeah. Actually, I don’t know if you went over his, his story, how he came to it when he had him on the podcast, but it’s very similar. You know, he was doing physics stuff and he was like, Wait, I need to figure out to do this. And it was the same thing for me. I was like, Wait, I, I don’t know.
I need to understand this. And you know, and one of the things that really I found that came up very nicely from doing it is the idea of you can pretty clearly see if you have identifiability problems. And some of the experiments that I’ve seen, even like measuring things like dissociation constants, there are definitely identifiability problems that I think a lot in the community is that meaning the biochemistry community is just completely unaware of.
And, and it’s because, you know, if, if you’re kind of taking the approach that you learn sort of by reading what other people did in papers throughout the years, [00:16:00] which is here’s the best fit parameter. Like I have seen papers where it’s sort of like how to guides on how to do a certain assay. And then they straight up tell you, they’re like, Oh yeah, you know, you plug this into.
I don’t know what, whatever software it is and it’s gonna spit back at you this number, which is gonna be your best fit parameter. And then that’s what you report. They like, they’ll literally say this in like technique papers, which is really, you know, a lot of times you can get away with that because you know the exact value of the parameter is close to that.
It’s fine for your operational purposes, that’s fine, but it can really bite you if you mess it. If you have something that’s like, for example, not identifiable. And so where the parameter value could be anything. And you might be saying, you know, based on your experiment, your experiment can’t resolve it.
And you’re saying to yourself, Oh yeah, the association constants one micromolar and it could easily be one nanomolar , right? And it’s just not gonna be identifiable in your assay. And so I started to see all these real benefits of [00:17:00] taking these approaches and it made me, I don’t know, I think it just made me a much better scientific thinker.
And to me that was just, You know, as scientists, we’re always trying to, you know, we always think very deeply about our problems and we’re really trying to understand, in my case, the living world. And the way in which we do that is by performing experiments, right? And making measurements and exploring with our eyes whether those eyes happen to be attached to microscopes or telescopes or whatever.
You know, it’s, we do this exploration and that’s the center part of it. But if we wanna interpret, if we wanna learn from what we observe, that’s where this data analysis comes in. And to me, it, I just felt like as I was learning how to do basing analysis, that this was something that was so central to the scientific method to actually the doing of science that I had overlooked up to that time.
And that, you know, like, you know, I’m a career scientist at that point. Now I’m really a career educator of [00:18:00] scientists. But then I was a career scientist, and it was like, to me it was just like this big moment because I felt that now I’m a much more complete scientist. Now that I have this, and yeah man, it was, uh, it was one of the, those few weeks I remember ’em very clearly.
I would, I was at UCLA and I would go to the student center and I would get a burrito and I would sit at a table with a burrito and I was reading James’ book, I don’t know if you’re familiar with ET Janes, which is very much on the scientific philosophy side of things. And just, and I would sit there and I would, I was just so, I was like, it was almost like I was high , meaning not on drugs, but like, I just felt like I was.
Exploring a whole new whole realm of science in general. It was really, really exciting. Yeah, yeah, yeah. No, I completely understand. Definitely have had and still have some of those moments from time to time. Yeah. Well actually you already [00:19:00] talked a bit about that, but let’s do like formal intro of uh, what you are doing note nowadays.
So like, uh, can you tell listeners, well basically define the work that you’re doing nowadays and, and the topics that you are particularly interested in. Yeah, so after I did that post, like I mentioned at ucla, I ended up coming to Caltech, which is. Really a place that I really love. It’s where I did my PhD.
Yeah, it’s an institution that’s just a wonderful place to be. You’re just surrounded by brilliant people and I got the opportunity to come back here as at the time my title was lecturer. My title now is teaching professor, which I think is more descriptive of what I do. So I am now not so much a scientist.
I do collaborate with scientists on various things, but you know, I don’t run my own research group. I am a teaching professor, so I teach a lot of courses. And the courses I teach, I’d say my students are roughly. More or less 50 50 undergraduates and then graduate students mostly in, uh, life sciences.
This is, uh, neuroscience. We have [00:20:00] a, an option called, uh, computation neuro systems, biochemistry, biology, bioengineering. So it’s mostly that sort of group of students and sort of my mission here is really to really train them to be world class scientists. And so my contributions to science now really come through training developing scientists.
Okay. So I teach a lot of different classes at Caltech, mostly to life scientists. That’s biologists, bioengineers. We have students in special options like biochemistry, molecular biophysics, uh, computational neuro systems, neurobiology, so all these different majors. And I teach, uh, undergrad and graduate students in that roughly 50 50 undergraduates and graduate students.
And so I teach a lot of different classes, you know, all with the aim of training, basically developing scientists and at, at the risk of being a little bit IMO about my institu. , our students here are really spectacular. They’re among the best in the world, and so it’s really a privilege to be able to train people of that [00:21:00] caliber.
It’s funny, I walk around this place and I’m teaching students and I know that most of them are a lot smarter than I am . Um, and it’s really real privilege and real fun. It’s real challenge and something that keeps my mind fresh and keeps me moving. And so I teach a lot of different classes. I think the one that your listeners are most interested in are a couple of classes I teach in data science type stuff.
So I teach like a two term course. So this will be a total of roughly 20 weeks combined. Uh, the first 10 weeks are. Really about things like exploratory data analysis, sort of data management, like how you, you know, keep track of files even, you know, we do a lot of practical things like even how do you open files of different formats and make informative graphics from them.
And then we spend roughly the last half of that class, like five weeks on some frequentist based approaches to statistical inference. The second term is entirely a course on BaZing inference. And so that’s 10 weeks on BaZing inference. And in both [00:22:00] courses, they’re really applied to problems that come about in biology.
We work with real data sets, many of them from Caltech, many of them from outside Caltech, from. Types of areas of biology. And so they really do inference on real data sets together. Um, I do also teach courses. I teach, uh, like this year I’m teaching a course. Uh, I teach a freshman lab course in biology, which is really fun.
I teach an electrical engineering lab course where they like build, uh, BioD devices, you know, basically the things that are going to get you the measurements that you’re then going to need to do, uh, the inferences on. I teach a course in, um, chemical kinetics and thermodynamics, which are important for like the physical side of studying life.
And I also teach a course. Um, I co-teach a course with, uh, Michael Elowitz on biological circuit design. So we look at how hooking up different genes, different cells with each other can give rise to function that’s, you know, greater than some of the parts. And so I teach a, a variety of things, but you know, kind of [00:23:00] central to all of them is you need to be able to use a computer to do calculation and then, and especially on, on the data science side.
And it turns out that actually the. Data science courses are actually my most heavily enrolled as well. Uh, there seems to be a lot of really demand for that. A lot of universities now have programs in data science and they are having, certainly students in the physical sciences and social sciences as well are learning things about statistical inferences and that sort of stuff.
I would say that a lot of the biology training is lagging a little bit on that, on the undergrad level. I, I think that’ll change the next five or 10 years. But, so there’s a lot of grad students who really want that training. They know they need it, and so they take my courses. Yeah. So many, uh, things. I, I have too many questions, so, but actually let’s, if you can, can you take an example from your work probably in biology to illustrate.
Like how patient stats are helpful there and also what you do [00:24:00] basically as a researcher in that, in that field. So the problems that I approach are, they’re really about how do I best train a student so they can tackle any of the problems that come at them. And that’s really what I, one of the things that I really do there.
Oh, well, that’s perfect because like, that’s also the goal of, of the podcast, like to be how do you learn basic stats? So like Yeah, for sure. Talk about that. That’s awesome. Right. So it’s important to choose a variety of data sets. And what I do is I, I actually have a few data sets that I really kind of keep coming back to, to help students learn how to.
More and more maybe sophisticated things and inference problems with them. And maybe I’ll, I’ll talk to you about one of my favorite data sets that I use in my classes, and we do like four or five different things with this one rather simple data set. Now, I know that many of your audience are not biologists, but the cool thing about this data set is that it’s [00:25:00] so simple and easy to set up that you can even understand if you’re not a biologist.
But here’s a little biology so you understand where it comes from. So this was an experiment that was done in the lab of Joe Howard at, uh, the Max Plunk Institute of Molecular Cell Biology and Genetics in, uh, Dresden in, in Germany. And the data were acquired by Melissa Gardner, who’s now a professor at the University of Minnesota and Mariani, who’s a professor at Vanderbilt University.
And what they were interested in is there’s, um, you know, inside all your cells are, well not all your cells, but inside many of your cells. There are these filaments. There are like proteins that are linked end to end, and they’re called microtubules. And it turns out that microtubules, they’re really, they don’t just sit there as a filmer.
They’re actually constantly growing and then shrinking. But the shrinking is actually kind of really a spectacular event. In fact, it has a name called Catastrophe because they shrink incredibly fast. It’s like they kind of just, they grow, grow, grow, grow, grow, and then boom, they’re gone. And so here was the, a very simple problem that Maria and Melissa were trying [00:26:00] to study, which is, okay, I’m gonna have my market tube go grow, and then it’s gonna undergo catastrophe.
What is the amount of time, How is the time it takes before it undergoes catastrophe distributed? Right? So we wanna know something about the probability distributions of the time to catastrophe. Now this sort of sounds like a failure analysis problem kind of, right? Or it sounds like, Well, the simplest model you might choose, and this is what I start with, with students, is, okay, let’s model.
The arrive this catastrophe as a pluss on process, meaning that you don’t know how long the thing’s grown, there’s no memory of how long it’s been growing. It’s a biochemical process that just can like randomly get triggered and then and will model as a pluss on process, which would mean that the time to catastrophe is exponentially distributed.
So now you’re gonna have an exponential likelihood. So basically what they measured was over and over again, they measured how long did it grow before it underwent catastrophe with these microtubules and they recorded that. And so we would take that data set and then [00:27:00] I give it to the students. We start with that and we say, Okay, we’ll take it to be exponential distributed.
We have one parameter. The exponential distribution has a single parameter, this so-called rate parameter, and let’s estimate that parameter as a BA and estimation problem. And so they worked that out. We also learned how to do things like posterior predictive checks. And this is where you make a, you basically look at could the model parametize.
By your posterior, where the parameters are drawn from the posterior distribution, can that generate the data set that you actually observe? And the answer is definitively no. And in fact, you see it right away. If you just plot the ecdf of the micro tubic catastrophe times, you see this inflection point.
So you know it can’t be a plus on process. So now the student’s gotta think, Okay, so what are we gonna do? We’re gonna have to come up with another model. And so one of the things I have them practice, I say, Okay, well let’s look at it off the shelf distribution. Okay? The exponential distribution is a special [00:28:00] case of a more flexible distribution called a gamma distribution.
The gamma distribution says, okay, instead of the time between arrival of posan processes is exponentially distributed. But how much time does it take, say, for two posan processes to arrive? Well, that’s gamma distributed with one of its parameters being two. The number of posan processes that had to arrive.
And then, you know, we’ll discuss this with the student. They’ll say, Okay, a scientist, does this make sense? And it’s like, well, no. Cause they have some biochemistry knowledge and like, Well, that really makes sense if you have like the same biochemical process has to happen to trigger catastrophe. But it’s probably different process.
They probably have different rates. And I think there is a distribution, there is a name for a distribution. I forget where it’s like the amount of time it takes for, let’s say we have three poss on processes that arrive and they have to arrive in succession with rates, beta one, beta two, and beta three.
Now I forgot what the name is because instead we say, All right, look, let’s do the work of deriving what the [00:29:00] distribution should be. What should be the likelihood? Let’s derive it. And then they go through that exercise and then they do parameter estimation on. Model. And then we bring that into model assessments.
We do all kinds of poster predictive checks. We do model comparison. And so what I’m showing the students is like, Look, with this very simple experiment, it’s literally just a list of times that they had to wait before they observed the micro tubic catastrophe on single microtubules. Very simple dataset.
There’s actually a lot to think about with the modeling. You know, then of course I’ll do things like I’ll give ’em data sets that I know are not that with models that I know are not identifiable , and they’ll have to work through that and they’ll have to find the non identifi abilities. But yeah, I mean, so when you ask like what problems I work on, for me as an instructor, it’s about what problems can a student learn the best from?
And in particular, I want those problems to sort of be, at least in their general area of science, which would. Biology or study of the living [00:30:00] world. Yeah. I look that, and I agree now that also I’m, I’m starting to teach more on my end. It’s something that I discovered is really important, like a good dataset.
When you find one you’re really happy, , it’s like, it’s true. I think it’s important to, it’s not just the data set that’s important. You have to think about what is the model, what is the question? And especially for pedagogies, can the question be clear? One of the things that I think is challenging with how statistical inferences sort of maybe taught in the undergrad or sort of used, I think in literature by people who aren’t thinking carefully about the inference problems, is it’s used in a way where it’s sort of like, uh, here’s an off the shelf thing with a name that I can use and report.
And one of the things that I really, and I think that’s actually really detrimental, really what you wanna have is you’re a scientist. You have to think, what question do I wanna ask? What question am I [00:31:00] approaching? And then I wanna design my experiment around that question. And the interpretation of the experiment really is often it’s, it’s done within that.
So I have a, a colleague, Rob Phillips, and he’s written several books and one of his books, which is one of my favorite of all time called Physical Biology of the Cell. And the first, it’s either, it might even be in the front matter, but at the latest, it’s in the very first chapter, he and his co-authors saying that book implicit in every quantitative measurement is in model.
And you know, yeah, we can talk about non-parametric statistics, we can talk about all that. I think as scientists, that is the case because what we’re trying to is very often the case. So what we’re trying to do is we’re trying to understand something more general about nature, how nature works, right? And so we do that in the context of models.
You’ve heard of your listeners, I’ve heard of many, these F equals ma, , you know, Newton’s Laws of motion. These are. Where we make measurements. And that’s how, actually, that’s how statistics [00:32:00] began, right? It began with Gauss and Lalo measuring movements of Jupiter, right? And, um, they’re trying to say something more general about Newton’s laws of gravity, right?
You can make the measurements, but we actually understand why these things move the way they do, because there’s a model there. And so for me, when we’re trying to teach people about statistical inference, the model side of it actually really does have to have, as you know, it’s as important as the actual data, the measurements themselves.
Yeah, and it’s like, it’s really that idea of generative modeling and patient workflow that we’ve talked about numerous time on, on this podcast and already with, uh, Michael Beko on episode six. So sure, I’ll put the link in the journals to that episode. Very glad that it’s still to date also, the book that, that you just mentioned, I think we should, uh, put that in the show notes for listeners because um, that’s something they may be interested in.
For sure. [00:33:00] Well, one of the things I’ll just say briefly is that Mike Betten Court in particular, he looms large in my courses. Uh, he’s been a really strong influence on me and on the way I think about bays and inferences. And I think that his approaches to how inference fits into the scientific enterprise are very.
And just extremely valuable. And so I think that he’s, uh, his contributions are, should be lauded. Uh, he’s great. And what’s the name of the book, uh, that you just mentioned? It’s called Physical Biology of the Cell, and it’s by Rob Phillips, Julie Tario Yon and Aon Garcia. And so, yeah, it’s called Physical Biology of the Cell.
Ah, yes. Physical Biology of the Cell. Perfect. Yeah. Rob Phillips, Indeed. Perfect. Yeah, I’ll put that in there. Show notes. Yeah. But I love that idea. In fact, I don’t know who said it first because I first heard it said by, uh, his co-author Julie Terrio. Um, but I [00:34:00] don’t know if Rob, It doesn’t matter. They did it collectively, I’m sure.
But this idea that implicit in every quantitative measurement is a. I think is really important for somebody as a scientist to think about. Yeah, no, exactly. And that’s something I often tell people, especially when they are learning about priors. Uh, you know, and they’re like, I don’t know what to choose.
Like, it’s just so hard. Like it’s just you use priors all the time. , it’s just like you forgot that you’re used, but you use priors all the time. It’s just like it’s implicit in here. Uh, the basin workflow makes it explicit, so it forces you to basically lay down your priors, but actually just think about that question and you’ll see that you have priors in that question.
It’s just that you didn’t think about that. Yeah. I think that one of the things about the BaZing approach is that it’s very intuitive and I think that the way we think about probability. That we can ascribe it not just to random variables, but to [00:35:00] any logical conjecture really sort of matches our intuition.
And what the prior does is it, it actually helps you realize No. Yeah. You do know something. Again. So Rob is also a heavy influence on me, um, in a different way than, than Mike is. But, uh, like be in court. But, uh, like that and corporate Rob Phillips, uh, one of the things that he pushes, and it’s a, it’s a course and I actually taught a course with him a few years ago on this, is this idea of, uh, sort of order of magnitude thinking that you actually know more about something than you think you do.
And I actually kind of use some of this ideas behind order, magnitude thinking and how we actually come up with priors. I’ll give you an example. Uh, and this is something that Rob and I do in our class, and I actually literally just did it last week at a high school. I, my brother’s a high school principal and I spoke at his school with students about science, and I did exactly this with high school students, so anybody can do this.
And I, I just asked him a question. I said, You know, how big is an e coli? e coli, you know, they know what e coli was. It’s a bacterium that lives in, in human [00:36:00] gut and in guts of most mammals. And I said, Okay, so how big is an e coli? So how, you know, how long is it? And they’re like, I have no idea. And of course my point is, is that they absolutely have an idea.
And I said, All right, so what we’re gonna do is we’re gonna do is called the bet the farm approach. So there’s a figure of speech in English called BET the Farm. So when, in the old days, actually for a lot of people today too, You know, if you had a farm that’s kind of all you had, that was your sustenance, that that’s what gave you your food, but that’s also what gave you your income.
And so the farm is basically all you have. And so would you be willing to bet your farm on something? And so I’m a big fan of the Los Angeles Football Club. They’re currently leading the, uh, major league soccer in the US Right now. They’re top of the leader board. So I guess it’s gonna put a date on on this, uh, podcast recording.
But if you asked me if I was gonna bet the farm, if they would win the mls, I would say no. I’m not willing to to bet that much. They, I think they will, but I’m not gonna bet the farm. Now, if you asked me would I bet the farm that the sun is gonna rise in the east tomorrow, [00:37:00] you know, there is a finite probability that it won’t, it’s exceedingly small, but I would bet the farm on that.
And so what I, so I said to the students, I said, Well, would you bet the farm that an e Coli sell is smaller than a kilometer? Like, Oh yeah, sure I would. I’m like, So then you know something, you know that it’s not bigger than a kilometer. Right. How about a meter? Okay. How about a centimeter? No. How about a millimeter?
Remember you can see a millimeter with your naked eye. No. Okay, so now we’re actually getting out and then I go to the other side. It’s like, all right, is it bigger than an angstrom that’s about the distance between the hydrogen and the carbon in a bond? Well, of course it’s gotta be bigger than that because it’s full of molecules, right?
And so you come from both sides with this order of magnitude. And then if you wanna get an estimate for it, you just take the geometric mean of the two sides you come from, right? But you can also do that for setting priors. So if I’m gonna make a measurement, so let’s say I’m gonna actually take measurements of the size of e coli cells.
I can do that with a microscope or something like that now, and I [00:38:00] wanna try to report what’s the typical size of an e coli cell? Well, you know, my posterior is gonna have a nice Gaussian distribution, probably something centered around two microns. Cause that’s how big they’re two micrometers. But for my prior, this is a common technique that I like to use for that, is I do this bet the farm approach, and I get.
Bounds on either end where I’m like, I would bet the farm that it’s smaller than the big one or bigger than the small one. And then I’ll essentially throw a log normal on it. You will say, Okay, so log base 10 of this parameter is going to be normally distributed. And I’ll say, Okay, I’ll leave say 5% of the mass probability mass out beyond where I bet the farm.
And now I got myself a nice prior, but the idea is, is that I asked these high school students and I said, How big is an coal I sell? And they said, I have no idea. And then instantly we were able to say, No, you do . You do have an idea . Right? And so I do like that, you know, I also think that we are naturally [00:39:00] sort of program to think in a BA framework.
And this came clear to. about seven years ago. So I live here near Pasadena and, and we have a, there’s a very nice garden called the Huntington Gardens in Pasadena, and if you’re ever in the area, I’d recommend visiting it. And uh, when my son was little, we used to like to go there early in the mornings on the weekends cuz little kids wake up at an unreasonable hour.
And so it’s a nice place to go out to the gardens and walk around. And so one Saturday, my son was two, he, and, and it was the first time he used the word probably right, it’s the first time he used it. And so I had him, he’s still in diaper actually. And I had him on his chaining table. It was actually his first word, , right?
It’s first word. No, the second word. They’re first word was Bates. That’s right. Something like that. Anyway, so I had him up on his changing table and I said to him, I said, Hey, what are we gonna do? And he looked at me and he said, Probably we’re going to the Huntington. And uh, I was just really struck by that.
I was like, Oh, [00:40:00] wow, this kid’s basian because he was he was thinking about exactly that Saturday. He wasn’t thinking about a very large number of repeats of that Saturday , right? No, he was thinking specifically about that day and he was ascribing a probability to an event, a probability to a logical conjecture, which was not really a random variable.
It’s not something that could be repeated over and over again. It, it was, that’s gonna vary from experiment to experiment. There’s only one experiment. It’s this day right now. Yeah. And uh, I was like, you know, I think we’re, we’re wired to think about probability in the Basian way. And that’s the moment where it dawned on me.
I’m like, ah, this is why I understood what I was doing when I started reading the books about basian approaches. Cause, because apparently I’d been thinking that way, probably like him. Since I was two . Oh yeah. I love that story. And I love, So like your reaction when you hear your son [00:41:00] saying probably is like, Oh yeah, exactly.
My wife, I, after I got him ready, I ran out and I told my wife that I worry sometimes about my wife that she might damage the muscles in her eyes from rolling her eyes so much after I tell her things like this . Yeah. I love that. Because like I think most people would like, you know, not notice the probably
Exactly, Exactly. You’re patie, right? That’s right. So proud of you, my son . Yeah. But actually, um, like something I wanted to ask you because you’re very passionate about education and teaching science, so I’m wondering if there are. Most important skills that you’re trying to instill in your students? Wow.
Okay. That’s a good question. It’s almost less of a skill. I asked this question. It’s a mentality that actually requires some skills. Really the mentality I want them to have is that, is to not be looking to statistical inference for just like an off the shelf [00:42:00] answer. This is an a tendency I think for a lot, especially in biology.
You know, I have some friends who in medicine, I have some friends who work in medical research and like the FDA has specific things that you’re supposed to do statistically with data as you’re going through like clinical trials and these sort of things where it’s like, this is what I’m supposed to do and then this is the answer that I get.
And I think one of the things that I’ve learned quite quickly as I started, especially getting into basian type of analysis, is that that just doesn’t fly. Every, you need to think about your experiments as a generative process. You need to remember what Rob and Julie and co-workers and co-authors said, which is that implicit in every measurement is a model.
And that model is different from measurement to measurement. You’re studying different things, right? And you’re studying them in different ways. You’re making them measurements with different instruments. And what I want them to have is the mentality that just like you don’t just do a standard experiment every [00:43:00] time.
You shouldn’t be doing a standard analysis every time. Like you shouldn’t just be looking for the quick thing that’s gonna tell you a yes or no or a, you know, that’s gonna give you a number and then you check to see how close to 0.05 it is. You know, this is, I’m try that, I wanna break and I do get a lot of students come in and say, How do I do this?
I need to do this for my paper. My advisor wants this. And so to me that’s the big problem because as I told you earlier, I discovered early on that the statistical in. process is a, an integral part of the scientific method. And just like you think carefully about an experiment, just like you make adjustments to experiments, to get better measurements, to get different measurements, you need to adjust your analysis as well.
The word that, uh, Mike Benko uses a lot is bespoke, which is kind of a fancy way of saying tailor made . And I think that that’s a great mentality. And so that’s what I’m trying to get them to do. Now, to your [00:44:00] question about skills, that’s the number one thing. Now, the problem is, is that they won’t do that if they’re not able to , right?
So you have to then give them the skills to be able to do that, and that is a real challenge because you have to find the right balance of rigor and breadth of coverage. To give them the toolbox they need with the understanding of the tools they need. When we’re actually, it’s weird statistical inference, you’d think it would be more mature, but it’s just, we’re just not there where there’s a lot we just don’t understand about how it works.
And so that has always been a challenge for me. My course changes every year based on my understanding, based on what I think is gonna be an easier way for them to grasp it based on what I think relative importance of things are. And based on where the literature has moved as statisticians, particularly applied statisticians are learning new things.
And so from the bays and inference point [00:45:00] of view, what are the skills that students need to think generatively? The first thing that they need to be able to do is they need to be able to codify their domain knowledge into a likelihood, which means that they need to understand what probability distributions are, what probability is, and they need to take what that implicit model is.
And they’re the ones who know it the best. They’re the ones who study e coli or they’re the ones who study how muscles work or a certain part of the brain or old faction or whatever. They know it, right? And so you have to help them put that in the language. A likelihood right from that bay serum kind of, it structures itself, right?
You put a prior on that and now you’re, you’re done. I always tell the students in like week two of my basing class, I’m like, I’ll write down a posterior, right? Cuz you know, we’ll specify likelihood and we’ll specify a prior, I’m like, Oh, here’s a posterior, it’s done . And um, I’m like, Oh, class ended early.
We don’t have [00:46:00] to do the final eight weeks. So everything else after constructing the posterior is making sense of it. And so constructing the posterior takes a lot of domain knowledge, which means, you know, to come up with the likelihood and the prior, but then it’s all about making sense of it. And what skills do you need to do that?
For me, the big one is really just to be able to sample out of a posterior distribution. And you know, we can talk a lot about the theoretical underpinnings of all this stuff, including how samplers work. I touched on that briefly, but ultimately you have to, you can’t have a full bulletproof understanding of everything that you’re doing.
Partly cuz people who are experts in this don’t have it. But, and so, you know, I really focus on how can we do sampling? How can we make sure that our sample is working as it should? How can we identify potential issues with inferences that we can get from our model? One of the things that, you know, I often remind students of is that, so one, I give them a problem that’s not [00:47:00] identifiable and the model that I give them is very well rooted in physical theory.
Like it’s using tried and true laws of thermodynamics and. It’s important for them to recognize, oh wait, I shouldn’t be adjusting my model so that I can get it to be identified. Yeah, I mean, there are some things you can do with priors and things like that if your priors way off or something. But fundamentally, this problem I give them, there’s, there’s just no way you can get a couple of the parameters.
And it’s because the experiment can’t resolve it. You don’t wanna start contorting your model to your experiment because then your models, like, you know, your model should be informing how you do your experiment. And if a parameter is simply not something you can identify through an experimental method you’re doing, you have to think of a different experiment to get at that parameter, right?
And so all those things, the, the skills that you need to do that I think are understanding probability and probability distributions, that you can construct the model. And then secondly [00:48:00] is to be good sample. You know, that’s really what my course is. It’s like model building and sampling and how you can do those effectively.
Yeah, yeah. So really still that idea of generative model building in, in a sense and Absolutely. Exactly. , Yes. Yeah. And also that mindset of like becoming okay with not knowing everything and failing and doubting and things like that, I think is extremely important. That’s actually something I really learned when I started learning how to program, how to code, because, so I don’t know how it works in US, for instance, but in France, the educational system is really built around.
Not failing . It’s like, it’s like the first failure that you have will make you off track. And then it’s gonna be extremely complicated to get back to like a, you know, royal track, which is like what everybody wants to do. Uh, like the equivalent of the Ivy League in the US if you want. It’s like, it’s like very some the, the grand of calls, [00:49:00] right?
Exactly. Yeah. I’m surprised you know that term. But yeah, I grew up that kind of stuff like cold coal, which is, that’s the Ivy League basically. It’s extremely hard to get in there. And there is just one way to get in there. I mean, now they start to diversify that a bit, which is good. But at my time, cause I’m very old, like there was like one.
Track to go there and one mistake would make you out of that. So you become extremely good at avoiding mistakes and you know, at like really applying what you’re supposed to do and do what you’re supposed to do, which in a way is good, but at some point it becomes problematic. And so something that I really learned really when I started programming was this idea of failing all the.
And becoming comfortable with, Well, it’s okay. I don’t know how to solve that yet. I’ll, I’ll understand. But right now I don’t know how to do that. And the fact that that line of code doesn’t work, it’s okay. It doesn’t mean my computer is gonna explode or [00:50:00] that my career is done, you know? And that I think is extremely important to teach students not only as a skill, but as a mindset.
Because it also improves your grade, improves your determination, improves your confidence. And I think also that’s why these kind of teaching not only. Stats, but the programmation programming also, it teaches you the scientific method, right? It’s like, I have a, a hypothesis about that. I’m gonna test it.
If it doesn’t work, I’m gonna change my prior. And I really love that because on not only teaches you skills, but also as you were saying, a very important mindset that you can use then in other areas of your life. Yeah. I especially like that latter point you made. And so I had mentioned that about identifiability issues, which means, okay, your, your model’s, you know, there’s no reason to assume that you’re, to think that your model has, you know, failed.
If you have identifiability issues, it’s just that you need a different assay. You need a different experiment to test it, to get those parameters. But on the other side is what you [00:51:00] mentioned, which is what happens if we start failing our posterior predictive checks? This is what this does. Is it that that actually raises the next scientific.
You know, it shows you where your holes and your knowledge are. Now, you need to figure that out either by, you know, kind of reassessing your theory and thinking carefully about some theoretical considerations, thinking carefully about what the next experiment is to see why it’s doing that. You know? And so, yeah, I think, you know, one of the things that I ask students is like, you know, I say, Oh, is your job as a scientist to verify hypotheses?
And it’s not, It’s to assault them . And, and that’s where you find, find out new things, you know. Yeah. No, that’s good. Yeah. And, and with respect to failure, by failure, I don’t, the word has such a negative connotation to it, but when you start doing statistical inference, you’re gonna mess up and there’s gonna be stuff that’s just, just hard.
I mean, and [00:52:00] that it does require you to keep trying and you learn along the way. And, uh, I have definitely encountered plenty of student frustration. Of course, they don’t often understand that I’ve had the same frustrations myself, , Um, that’s how you learn. Right. Yeah. Yeah, exactly. So, uh, I wanna be, uh, time is fine by, and I wanna be conscious of your time, so Yeah, I talk too much.
I’m sorry about that, . No, that’s very good actually. Like, I always like people who talk a lot should be on podcasts. Forget me. Why do you think I have a podcast? I suppose just like I cheat because I’m on each episode, you know? So I see. Yes. Okay. That’s how much I talk. So before closing up the show, with the two traditional questions, there is one other question about education that I like to ask you is, so we talked about the skills and so on, and we just talked about like, you’re gonna mess up when you start learning, actually.
Do you think that there are mistakes that [00:53:00] students have to make in order to really understand some statistical concepts? And if yes, um, which one? I use a phrase like when you wander down a dark. It’s leading you to nowhere, but you actually gain some life experience, um, and you know how to avoid those dark alleys.
So one of the things I think that they need to do is they need to mess up things to understand that statistical inference is fragile. That this, these techniques are, are fragile or brittle and they break. And so one of the one , so the, I give them little problems where I know they’re gonna hit something that’s gnarly , um, that’s gonna highlight how subtle and dastardly these problems can be.
And so, yeah, these are good. Yeah. So for the microtubule problem, it turns out that if you do that model, I told you where you work out, the number of, let’s say I have three steps, [00:54:00] actually. The, the example I give ’em is two, say it takes two. Event, two possum processes to arrive for catastrophe to happen.
And they can arrive and, and they’re different possum processes. So they have different rates. Okay. If you compute the, if you have an uninformed prior on the rates or you have a prior, that’s the same for the, uh, the two different rates, which you actually wouldn’t do cuz you would do, you’d have to be careful too.
You’d have to order them because there’s a label switching identifiability issue there. But let’s say that, So that gets ’em already the label switching. But anyway, let’s say you have not like totally uniform priors on those things and you’re gonna compute a maximize posterior probability estimate for the parameters.
They’ll always have the same rate for the map every time. And, and you can work that out analytically. Actually, in fact, you can do that pen and paper, You can work it out. You know, i, I, they, they often immediately resort to numerics and so then they’re gonna start to report them. They’re gonna be like, Oh yeah, it’s, one of them is 0.045.[00:55:00]
One and the other one’s 0.0452. Cuz there’s some numerical, right? You know, when they think, oh they’re, they’re very close, but they’re slightly different. And so that’s a mistake right there that they’ll, that I have them intentionally fall into. And it exposes a problem with only reporting basically the mode of the posterior.
Right. And so what I do is I do, I do kind of like, I don’t know it sounds mean, but I sort of set these little traps for them to like bump into, to discover themselves, you know? And so I wouldn’t say there’s any really one mistake, It’s the general idea that this whole process is brittle and that it’s very easy to break and to break subtly.
And so I want them to keep making all these little mistakes and so that at the end they’re very careful and that they also don’t overstate their results because there’s often subtleties that we don’t know about that are lurking. And you know, I had, I was using a data set, I forget all the details, but I was using a data set for four years.[00:56:00]
And finally I had a TA notice one, one of those little subtle details that I had missed for years. And you know, and I share that story with the students too. I’m like, Look, like I’m no dummy . You know, It’s like I’m careful, but you’re still gonna get nailed. It’s gonna happen. And so you just gotta remember that this is a precarious endeavor that you’re embarking on here to do this statistical.
Yeah. Yeah, yeah, yeah. I love that. And I do that also in my teaching, the workshops, for instance, that I, we do for, uh, with often, like, especially when I teach, um, priors in relation to generalized linear models. Uh, since you have the link functions, choosing priors is way less intuitive because of the link function.
And so often I start with a somewhat classic prior, you know, c normal signal equal stent, [00:57:00] which a lot of people just use like that. And then you look at the prior predictive checks and it looks awful. And, and there’s just those little things, but like, making the mistake will be much more powerful than the students, and they will remember that way more, or like further plus on regression, for instance, like writing the model, but omitting the exponential.
And so the model one sample, because it will start around zero and that means minus infinity probability. So p c won’t even start sampling, which is very dreaded error when you start learning patient statistics. So I do that in purpose, and then the students are like, I ask him, So what’s the problem here?
Our model is good, right? So what? What do you think is happening? So yeah, it sounds mean, And I mean, we are smiling if the people don’t see us, but we’re smiling. So yeah, that’s not good. But I think it’s a way better way to teach people instead of just telling them, Oh yeah, here, use the exponential link.
Absolutely. It can’t be prescriptive. That goes back to [00:58:00] my earliest point, which is do not want to teach a pres. They need to think about it. And it has to be bespoke modeling. And that takes effort and it takes training and it takes patience and it takes collaboration. It actually takes all the things that we value as virtues and scientists and we should embrace it.
Yeah. Yeah. And I particularly like that part also, uh, uh, about collaboration lately. I, I just read, um, a book called The Illusion of Knowledge. That’s by Steven Snowman. Yeah, The illusion of knowledge. Why we never Think Alone. A really good book. I’ll put that in the show notes and yeah, it’s a book basically about the strength of homo sapiens is to collaborate and like basically everything you think you know, actually you know it because like we all know it.
It’s a community of knowledge and like basically nobody is just on his island doing his thing. And I love that idea. It’s really, really a good book and, [00:59:00] and you can find that, especially in the open source community for instance, it’s like everybody like Christing on the shoulders of Giants all the time.
Doesn’t that make us all giants? I think so. Which is like right. Really big shoulders . Yeah. Yeah, that’s right. Yeah. Awesome. Well, Justine, I still have so many questions, but I think we have to call it the show. And before that though, I’ll ask you the two traditional questions. I’ll ask you everyone at the end of the show.
So first one, Is if you had limited time and resources, which problem would you try to solve? That’s a good question. Of course. For me, again, my mission is about training scientists, and I think the problem I would try to solve is the same problem that I had mentioned before, which is I want to break the habit of looking for prescriptive ways of doing things with the statistical inference.
I want. To me that’s a problem. and a ways to solving it is by education. If you’re asking more generally, I would say [01:00:00] climate change, but I think pretty much everyone would. So , Yeah. One of the most common answers for sure. Uh, but I’m not surprised by your answer, that’s for sure. Cool. And so second question, if you could have dinner with any great scientific mind, dead, alive, or fictional who be?
So that’s a interesting question and I was thinking about it and I have opinions for different reasons, but I’ll, for the purpose of this podcast, I’ll actually say David Mackay and I wonder if anybody has ever said him before. But I say him because in the early reading that I was doing about these things, his book was really useful to me.
The title of the book is Information Theory Inference and Learning Algorithms, and as a book from like 2003 where he really brought those three topics onto the same. Which is really featured essentially that as Bay and Probability, and it’s written in such a clear pedagogical and [01:01:00] actually even humorous at Points way.
And so I highlight that book because I think it’s really useful for learning about these ideas. And yeah, some of this stuff is dated because it’s about 20 years old now, so then the machine learning side of things, but the ideas and the unifying ideas of having those three areas, information theory, uh, machine learning and statistical inference, all kind of linked by the same ideas is really important.
Now, I mention him because his second book that he published is related to the first question, which is called Sustainable Energy Without All the Hot Air. And in that book, he uses all order of magnitude types of estimates to try to figure out how do we solve the, you know, the, the, basically the energy problem again, argued with this sharp, witty.
Direct, engaging, fun way. And I just think that if I had the chance to sit with him, I would be [01:02:00] intrigued, entertained, inspired. And I think that, that he was, uh, just really also a great scientific communicator and thinker. He died tragically at a very young age, uh, from cancer, uh, I think it was 48 when he passed away, uh, probably five or six years ago.
But anyway, that’s a for the listeners podcast, both those books are fantastic. One’s really probably gonna help you think about base and inference. The other one about one of the greatest technical challenges of our time. Yeah. So I need the one about, uh, renewable energy to the notes. I don’t remember the name of the first one.
Can you remind me? It’s called Information Theory, Inference and Learning Algorithm. It’s one of the early books that I was reading when I was getting into this area. Yeah. Okay. Yeah, I have it. This is added to the show notes people. Awesome. Uh, well Justine, thanks a lot for taking the time. I wanted to talk about spots and I takes, because I discovered that you’re very passionate about football and you’re even a goalkeeper and you [01:03:00] were playing yesterday and people will have heard in the introduction.
You did some exploits, uh, yesterday that are related in the introduction to this show. So well done. And I’m sure all listeners, uh, will join me in congratulating you. Thank you, . Yeah, it’s not every day that goalkeeper scores a goal, so well done. That’s true. That’s true. I hope you celebrated that as was required with the team
I, I did. Ah, okay. Although it was a little overshadowed cause my teammate scored four goals right after I did that. So on. Don’t steal the goalkeeper standard. That’s not cool. That’s right, That’s right. Yeah. Yeah. That’s not cool at all. Well, that will be for next episode, and as usual, I’ll put resources and a link to your website in the show notes for those who wanna dig deeper, especially your courses, you have two courses that are particularly relevant for the listeners.
So these are in the show notes. Thank you again, Justine, for taking the time and being on [01:04:00] this show. My pleasure. Thanks so much for having me. You bet. And um, You’re welcome for our next episode. Dedicated you with Goalkeepers Football. And mainly PSG . Oh boy. Okay. We’ll talk later about that . Perfect. See you, Justin.