Learning Bayesian Statistics

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!

Getting Daniel Lee on the show is a real treat — with 20 years of experience in numeric computation; 10 years creating and working with Stan; 5 years working on pharma-related models, you can ask him virtually anything. And that I did…

From joint models for estimating oncology treatment efficacy to PK/PD models; from data fusion for U.S. Navy applications to baseball and football analytics, as well as common misconceptions or challenges in the Bayesian world — our conversation spans a wide range of topics that I’m sure you’ll appreciate!

Daniel studied Mathematics at MIT and Statistics at Cambridge University, and, when he’s not in front of his computer, is a savvy basketball player and… a hip hop DJ — you actually have his SoundCloud profile in the show notes if you’re curious!

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !

Thank you to my Patrons for making this episode possible!

Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas and Luke Gorrie.

Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag 😉

Links from the show:

Abstract

by Christoph Bamberg

Our guest this week, Daniel Lee, is a real Bayesian allrounder and will give us new insights into a lot of Bayesian applications. 

Daniel got introduced to Bayesian stats when trying to estimate the failure rate of satellite dishes as an undergraduate student. He was lucky to be mentored by Bayesian greats like David Spiegelhalter, Andrew Gelman and Bob Carpenter. He also sat in on reading groups at universities where he learned about cutting edge developments – something he would recommend anyone to really dive deep into the matter.

He used all this experience working on Pk/Pd (Pharmacokinetics/ Pharmacodynamics) models. We talk about the challenges in understanding individual responses to drugs based on the speed with which they move through the body. Bayesian statistics allows for incorporating more complexity into those models for more accurate estimation.

Daniel also worked on decision making and information fusing problems for the military, such as identifying a plane as friend or foe through the radar of several ships.

And to add even more diversity to his repertoire, Daniel now also works in the world of sports analytics, another popular topic on our show. We talk about the state of this emerging field and its challenges.

Finally, we cover some STAN news, discuss common problems and misconceptions around Bayesian statistics and how to resolve them.

Transcript

This is an automatic transcript and may therefore contain errors. Please get in touch if you’re willing to correct them.

Transcript
Speaker:

Let me show you how to be a good peasy and

change your production.

2

00:00:03,812 --> 00:00:06,834

Getting Daniel Lee on the show is a real

treat.

3

00:00:06,954 --> 00:00:11,677

With 20 years of experience in numeric

computation, 10 years creating and working

4

00:00:11,677 --> 00:00:16,120

with Stan, 5 years working on

pharma-related models, you can ask him

5

00:00:16,120 --> 00:00:17,801

virtually anything.

6

00:00:17,942 --> 00:00:20,223

And that I did, my friends.

7

00:00:20,343 --> 00:00:25,247

From joint models for estimating oncology

treatment efficacy to PKPD models, from

8

00:00:25,247 --> 00:00:30,058

data fusion for US Navy applications to

baseball and football analytics.

9

00:00:30,058 --> 00:00:33,658

as well as common misconceptions or

challenges in the Bajan world, our

10

00:00:33,658 --> 00:00:38,960

conversation spans a wide range of topics

that I am sure you will appreciate.

11

00:00:39,440 --> 00:00:44,221

Daniel studied mathematics at MIT and

statistics at Cambridge University, and

12

00:00:44,221 --> 00:00:49,403

when he's not in front of his computer,

he's a savvy basketball player and a

13

00:00:49,403 --> 00:00:51,243

hip-hop DJ.

14

00:00:51,243 --> 00:00:55,844

You actually have his Soundcloud profile

in the show notes if you're curious.

15

00:00:56,065 --> 00:00:59,550

This is Learning Bajan Statistics, episode

96.

16

00:00:59,550 --> 00:01:01,830

recorded October 12, 2023.

17

00:01:04,711 --> 00:01:08,592

Hello, my dear patients.

18

00:01:08,752 --> 00:01:12,593

Some of you may know that I teach

workshops at Pimesy Labs to help you

19

00:01:12,593 --> 00:01:17,075

jumpstart your basic journey, but

sometimes the fully live version isn't a

20

00:01:17,075 --> 00:01:18,015

fit for you.

21

00:01:18,015 --> 00:01:22,616

So we are launching what we call the

Guided Learning Path.

22

00:01:22,616 --> 00:01:26,837

This is an extensive library of video

courses handpicked from our live

23

00:01:26,837 --> 00:01:27,717

workshops.

24

00:01:27,842 --> 00:01:30,943

that unlocks asynchronous learning for

you.

25

00:01:31,023 --> 00:01:34,965

From A-B testing to Gaussian processes,

from hierarchical models to causal

26

00:01:34,965 --> 00:01:39,328

inference, you can explore it all at your

own pace, on your own schedule, with

27

00:01:39,328 --> 00:01:40,848

lifetime access.

28

00:01:40,889 --> 00:01:44,851

If that sounds like fun and you too want

to become a vision modeler, feel free to

29

00:01:44,851 --> 00:01:49,373

reach out at alex.andorra at

primec-labs.com.

30

00:01:50,174 --> 00:01:53,255

And now, let's get nerdy with Daniel Lee.

31

00:01:58,006 --> 00:02:01,246

Daniel Lee, welcome to Learning Bayesian

Statistics.

32

00:02:01,246 --> 00:02:03,887

Hello.

33

00:02:04,267 --> 00:02:06,468

Yeah, thanks a lot for taking the time.

34

00:02:06,888 --> 00:02:12,009

I'm really happy to have you on the show

because I've followed your work for quite

35

00:02:12,009 --> 00:02:18,091

a long time now and I've always thought

that it'd be fun to have you on the show.

36

00:02:18,131 --> 00:02:20,412

And today was the opportunity.

37

00:02:20,412 --> 00:02:23,072

So thank you so much for taking the time.

38

00:02:23,512 --> 00:02:27,933

And so let's start writing.

39

00:02:29,587 --> 00:02:30,867

What are you doing?

40

00:02:32,609 --> 00:02:35,031

How would you define the work you're doing

nowadays?

41

00:02:35,031 --> 00:02:37,932

And what are the topics you are

particularly interested in?

42

00:02:38,513 --> 00:02:41,875

Yeah, so I just joined Zealous Analytics

recently.

43

00:02:41,995 --> 00:02:46,278

They're a company that does sports

analytics, mostly for professional teams.

44

00:02:46,539 --> 00:02:52,363

Although they're expanding to amateur

college teams as well.

45

00:02:52,363 --> 00:02:55,104

And what I get to do is...

46

00:02:56,522 --> 00:03:01,544

look at data and try to project how well

players are going to do in the future.

47

00:03:01,544 --> 00:03:06,506

That's the bulk of what I'm focused on

right now.

48

00:03:06,506 --> 00:03:07,806

That sounds like fun.

49

00:03:08,467 --> 00:03:16,990

Were you already a sports fan or is it

that mainly you're a good modeler and that

50

00:03:16,990 --> 00:03:19,531

was a fun opportunity that presented

itself?

51

00:03:20,031 --> 00:03:22,572

Yeah, I think both are true.

52

00:03:22,833 --> 00:03:24,966

I grew up playing a lot of basketball.

53

00:03:24,966 --> 00:03:27,447

I coached a little bit of basketball.

54

00:03:28,008 --> 00:03:30,030

Um, yeah.

55

00:03:30,030 --> 00:03:34,173

So I feel like I know the subject matter

of basketball pretty well.

56

00:03:34,173 --> 00:03:39,978

The other sports I know very little about,

but, um, uh, you know, combine that with

57

00:03:39,978 --> 00:03:41,540

being able to model data.

58

00:03:41,540 --> 00:03:44,762

It's actually a really cool opportunity.

59

00:03:46,724 --> 00:03:47,444

Yeah.

60

00:03:47,805 --> 00:03:52,128

And actually, how did you end up doing

what you're doing today?

61

00:03:52,349 --> 00:03:53,042

Because.

62

00:03:53,042 --> 00:03:57,325

I know you've got a very, very senior

path.

63

00:03:57,325 --> 00:04:06,011

So I'm really interested also in your kind

of origin story because, well, that's an

64

00:04:06,011 --> 00:04:06,691

interesting one.

65

00:04:06,691 --> 00:04:10,854

So how did you end up doing what you're

doing today?

66

00:04:10,854 --> 00:04:11,554

Yeah.

67

00:04:11,875 --> 00:04:19,240

So sports ended up happening because I

don't know, it actually started through

68

00:04:19,240 --> 00:04:20,100

stand.

69

00:04:20,161 --> 00:04:21,761

I didn't really have...

70

00:04:22,270 --> 00:04:26,472

an idea that I'd be working in sports

full-time professionally until this

71

00:04:26,472 --> 00:04:28,293

opportunity presented itself.

72

00:04:29,694 --> 00:04:35,376

And what ended up happening was I met the

founders of Zealous Analytics

73

00:04:35,497 --> 00:04:40,379

independently about a decade ago and the

company didn't start till 2019.

74

00:04:40,480 --> 00:04:42,320

So, you know, met them.

75

00:04:42,521 --> 00:04:43,921

Luke was at Harvard.

76

00:04:45,342 --> 00:04:51,625

Dan was at NYU and Doug at the time was

going to the Dodgers.

77

00:04:52,234 --> 00:04:59,116

And I talked to them independently about

different things and, you know, fast

78

00:04:59,116 --> 00:05:03,198

forward about 10 years and I happened to

be free.

79

00:05:03,238 --> 00:05:04,639

This opportunity came up.

80

00:05:04,639 --> 00:05:06,339

They're using Stan inside.

81

00:05:06,339 --> 00:05:10,781

They're using a bunch of other stuff too,

but it was a good time.

82

00:05:11,121 --> 00:05:19,965

And do you remember how you first got

introduced to Bayesian methods and also

83

00:05:19,965 --> 00:05:21,985

why they stuck with you?

84

00:05:22,746 --> 00:05:23,526

Yeah.

85

00:05:24,547 --> 00:05:29,910

So there are actually two different times

that I got introduced to Bayesian methods.

86

00:05:29,910 --> 00:05:32,332

The first was I was working in San Diego.

87

00:05:32,352 --> 00:05:34,533

This is after my undergraduate degree.

88

00:05:35,754 --> 00:05:41,839

We were working on trying to estimate when

hardware would fail and we're talking

89

00:05:41,839 --> 00:05:46,201

about modems and things that go with

satellite dishes.

90

00:05:46,201 --> 00:05:49,003

So they happen to be somewhere that's hard

to

91

00:05:52,618 --> 00:05:58,701

spread across and when one of those pieces

go down, it's actually very costly to

92

00:05:59,782 --> 00:06:02,744

repair, especially when you don't have a

part available.

93

00:06:03,045 --> 00:06:08,709

So we started using graphical models and

using something called Weka to build

94

00:06:08,709 --> 00:06:11,190

graphical models and do Bayesian

computation.

95

00:06:11,190 --> 00:06:14,853

This was all done using graphical models

and it was all discrete.

96

00:06:15,493 --> 00:06:21,957

That was the first time I got introduced

to Bayesian statistics.

97

00:06:22,230 --> 00:06:24,030

It was very simple at the time.

98

00:06:25,771 --> 00:06:30,733

What ended up happening after that was I

went to grad school at Cambridge, did part

99

00:06:30,733 --> 00:06:35,035

three mathematics and ended up taking all

the stats courses.

100

00:06:35,475 --> 00:06:42,518

And that's where I really saw Bayesian

statistics, learned MCMC, learned how bugs

101

00:06:42,518 --> 00:06:46,660

was built using the graphical models and

conjugacy.

102

00:06:47,280 --> 00:06:50,141

And then...

103

00:06:50,206 --> 00:06:54,949

Yeah, so that was the real introduction to

Bayesian modeling.

104

00:06:54,949 --> 00:06:55,269

Yeah.

105

00:06:55,269 --> 00:07:08,518

And actually I'm curious because,

especially in any content basically where

106

00:07:08,738 --> 00:07:16,303

we talk about, so how do you end up doing

what you're doing and stuff like that,

107

00:07:16,303 --> 00:07:18,765

there is kind of a hindsight

108

00:07:20,114 --> 00:07:24,557

it looks obvious how you ended up doing

what you're doing.

109

00:07:24,557 --> 00:07:26,819

And that almost seems easy.

110

00:07:27,099 --> 00:07:31,243

But I mean, at least in my case, that

wasn't, you know, it's like you always

111

00:07:31,243 --> 00:07:36,427

have obstacles along the way and so on,

which is not necessarily negative, right?

112

00:07:36,427 --> 00:07:45,114

We have that really good saying in French

that says basically, what's the obstacle,

113

00:07:45,114 --> 00:07:49,037

the obstacles in front of you makes you

114

00:07:49,870 --> 00:07:51,330

grow, basically.

115

00:07:51,490 --> 00:07:56,351

It's a very hard thing to translate, but

basically that's the substance.

116

00:07:56,771 --> 00:08:03,233

So yeah, I'm just curious about your own

path.

117

00:08:03,233 --> 00:08:09,715

How senior was it to get to where you are

right now?

118

00:08:18,662 --> 00:08:23,424

I've always believed in learning from

failures or learning from experiences

119

00:08:23,424 --> 00:08:24,745

where you don't succeed.

120

00:08:24,865 --> 00:08:27,026

That's where you gain the most knowledge.

121

00:08:27,026 --> 00:08:29,307

That's where you get to learn where your

boundary is.

122

00:08:31,609 --> 00:08:37,432

If you want to know about the path to how

I became where I'm at now, let's see.

123

00:08:38,433 --> 00:08:40,354

I guess I could go all the way back to

high school.

124

00:08:40,354 --> 00:08:42,935

I grew up just outside of Los Angeles.

125

00:08:44,176 --> 00:08:45,397

In high school...

126

00:08:46,898 --> 00:08:49,700

I had a wonderful advisor named Sanzha

Kazadi.

127

00:08:49,700 --> 00:08:58,566

He was a PhD student at Caltech and he ran

a research program for high school kids to

128

00:08:58,566 --> 00:08:59,526

do basic research.

129

00:08:59,526 --> 00:09:05,050

So starting there, I learned to code and

was working on the traveling salesman

130

00:09:05,050 --> 00:09:05,911

problem.

131

00:09:08,773 --> 00:09:13,296

From there, I went to MIT, talking about

failures.

132

00:09:13,296 --> 00:09:15,737

I tried to be a physics major going in.

133

00:09:15,858 --> 00:09:21,121

I failed physics three times in the first

year, so I couldn't.

134

00:09:21,121 --> 00:09:22,782

I ended up being a math major.

135

00:09:24,483 --> 00:09:28,386

And it was math with computer science, so

it was really close to a theoretical

136

00:09:28,386 --> 00:09:31,948

computer science degree, doing some

operational research as well.

137

00:09:33,470 --> 00:09:37,112

At the end of MIT, I wasn't doing so well

in school.

138

00:09:37,112 --> 00:09:39,894

I was trying to apply to grad school, and

that wasn't happening.

139

00:09:40,114 --> 00:09:42,996

Got a job in San Diego.

140

00:09:43,316 --> 00:09:45,497

MIT alum hired me.

141

00:09:46,074 --> 00:09:51,218

That's where I started working for three

and a half years in software, a little bit

142

00:09:51,218 --> 00:09:52,138

of computation.

143

00:09:52,138 --> 00:09:57,922

So a lot of it was translating algorithms

to production software, working on

144

00:09:57,922 --> 00:10:05,568

algorithms and went through a couple of

companies with the same crew, but we just

145

00:10:05,568 --> 00:10:07,388

kind of bounced around a little bit.

146

00:10:09,110 --> 00:10:14,333

At the end of that, I ended up going back

to Cambridge for...

147

00:10:14,354 --> 00:10:18,557

a one year program called part three

mathematics.

148

00:10:18,557 --> 00:10:21,278

It's also a master's degree.

149

00:10:23,460 --> 00:10:26,001

I got there not knowing anything about

Cambridge.

150

00:10:26,342 --> 00:10:31,085

I didn't do enough research, obviously.

151

00:10:32,806 --> 00:10:37,490

For the American viewers, people, the

system is completely different.

152

00:10:37,490 --> 00:10:40,992

There's no midterms, no nothing.

153

00:10:41,112 --> 00:10:43,153

You have three trimesters.

154

00:10:43,490 --> 00:10:47,453

You take classes in each of them and you

take two weeks of exams at the end.

155

00:10:47,453 --> 00:10:49,414

And that determines your fate.

156

00:10:49,815 --> 00:10:56,180

And, um, I got to Cambridge and I couldn't

even understand anything in the syllabus

157

00:10:56,180 --> 00:10:58,622

other than the stuff in statistics.

158

00:10:58,802 --> 00:11:02,445

Mind you, I hadn't done an integral in

three years, right?

159

00:11:02,445 --> 00:11:03,906

Integral derivative.

160

00:11:04,147 --> 00:11:06,449

I didn't know what the normal distribution

was.

161

00:11:06,449 --> 00:11:08,070

And I go to Cambridge.

162

00:11:08,070 --> 00:11:09,711

Those are the only things I can read.

163

00:11:10,012 --> 00:11:11,593

So I'm teaching myself.

164

00:11:11,793 --> 00:11:12,373

Um,

165

00:11:13,278 --> 00:11:18,241

measure theory while learning all these

new things that I've never seen and

166

00:11:18,602 --> 00:11:20,583

managed to squeak out passing.

167

00:11:20,804 --> 00:11:22,465

So happy.

168

00:11:24,187 --> 00:11:29,391

At the end of that, I asked David

Spiegelhalter, who happened to just come

169

00:11:29,391 --> 00:11:35,096

back to Cambridge, that was his first year

back in the stats department, who I should

170

00:11:35,096 --> 00:11:35,637

talk to.

171

00:11:35,637 --> 00:11:40,540

This is, so when I say I learned bugs,

he's, he had a course on

172

00:11:41,866 --> 00:11:45,688

applied Beijing statistics, which was

taught in wind bugs.

173

00:11:46,068 --> 00:11:50,371

And he would literally show us which

buttons to click and in which order, in

174

00:11:50,371 --> 00:11:51,792

order for it not to crash.

175

00:11:51,792 --> 00:11:53,613

So that was fun.

176

00:11:54,274 --> 00:11:56,936

But he told me, he told me I should talk

to Andrew Gelman.

177

00:11:56,936 --> 00:12:05,321

Um, so I ended up, uh, talking to Andrew

Gelman and working with Andrew from 2009

178

00:12:05,902 --> 00:12:10,625

to 2016 and that's how I really got into

Beijing stats.

179

00:12:10,625 --> 00:12:11,205

Um,

180

00:12:11,462 --> 00:12:13,242

After Cambridge, I knew theory.

181

00:12:13,242 --> 00:12:14,822

I hadn't seen any data.

182

00:12:15,143 --> 00:12:22,965

Working for Andrew, I saw a bunch of data

and actually how to really work with data.

183

00:12:24,705 --> 00:12:26,566

Since then I've run a startup.

184

00:12:26,566 --> 00:12:28,346

We try to take Stan.

185

00:12:28,506 --> 00:12:31,587

So Stan's an open source probabilistic

programming language.

186

00:12:33,508 --> 00:12:39,149

In 2017, a few of us thought there was a

good opportunity for making a business

187

00:12:39,149 --> 00:12:39,702

around it.

188

00:12:39,702 --> 00:12:41,582

very much like time C labs.

189

00:12:42,003 --> 00:12:46,065

And, you know, we try to make a horizontal

platform for it.

190

00:12:46,065 --> 00:12:48,866

And at that time, there wasn't enough

demand.

191

00:12:49,147 --> 00:12:55,950

So we pivoted and ended up estimating

models for writing very complicated models

192

00:12:55,950 --> 00:12:58,291

and estimating things for the farm

industry.

193

00:12:59,112 --> 00:13:08,737

And then since then I've like I left the

company in 2020 at the end of 2021.

194

00:13:09,278 --> 00:13:13,940

I consulted for a bit, just random

projects, and then ended up with Celus.

195

00:13:14,100 --> 00:13:17,362

So that's how I got to today.

196

00:13:17,362 --> 00:13:18,142

Yeah.

197

00:13:18,242 --> 00:13:18,883

Man.

198

00:13:18,883 --> 00:13:24,366

Yeah, thanks a lot for that exhaustive

summary, I'd say, because that really

199

00:13:24,366 --> 00:13:29,608

shows how random usually paths are, right?

200

00:13:29,829 --> 00:13:37,393

And I find that really inspiring also for

people who are a bit upstream in their

201

00:13:37,393 --> 00:13:38,693

carrier path.

202

00:13:39,062 --> 00:13:44,504

could be looking at you as a role model

and could be intimidated by thinking that

203

00:13:44,504 --> 00:13:51,128

you had everything figured out from when

you were 18 years old, right?

204

00:13:51,148 --> 00:13:55,811

Just getting out of high school, which was

not the case from what I understand.

205

00:13:55,811 --> 00:14:01,093

And that's really reassuring and

inspiring, I think, for a lot of people.

206

00:14:01,093 --> 00:14:03,255

Yeah, definitely not.

207

00:14:03,255 --> 00:14:08,297

I could tell you going to career fairs at

the end of my undergraduate degree,

208

00:14:08,994 --> 00:14:15,057

people will look at my math degree and not

even really look at my resume.

209

00:14:15,057 --> 00:14:21,500

Because my GPA was low, my grades were bad

as a student, and also, who needs a bad

210

00:14:21,500 --> 00:14:22,361

mathematician?

211

00:14:22,361 --> 00:14:24,322

That makes no sense anywhere.

212

00:14:24,322 --> 00:14:29,384

So that limited what I was doing, but at

the end it all worked out.

213

00:14:31,866 --> 00:14:32,206

Yeah, yeah, yeah.

214

00:14:32,206 --> 00:14:36,869

Now you made an agreement in a way, our

path, our seminar, except for me, that was

215

00:14:36,869 --> 00:14:38,634

a GPA in business school.

216

00:14:38,634 --> 00:14:40,835

So business school and political science.

217

00:14:40,835 --> 00:14:43,077

Political science, I did have decent

grades.

218

00:14:43,077 --> 00:14:50,001

Business school, it really depended on

what the course was about.

219

00:14:50,001 --> 00:14:55,545

Because when I was not interested in the

course, yeah, that showed.

220

00:14:56,125 --> 00:14:58,186

For sure, that showed in the GPA.

221

00:14:59,247 --> 00:15:06,552

But yeah, and I find that also super

interesting because in your path, there is

222

00:15:06,552 --> 00:15:08,513

also so many amazing

223

00:15:08,830 --> 00:15:15,234

people you've met along the way and that

it seems like these people were also your

224

00:15:15,234 --> 00:15:16,655

mentors at some point.

225

00:15:16,655 --> 00:15:20,738

So Yeah, do you want to talk a bit more

about that?

226

00:15:21,599 --> 00:15:31,866

Yeah, I've um, I've been really fortunate

You know as I was going through so Not you

227

00:15:31,866 --> 00:15:36,409

know, I haven't had very many formal

mentors that were great and by that I mean

228

00:15:36,409 --> 00:15:37,109

like

229

00:15:37,298 --> 00:15:40,701

advisors that were assigned to me through

schools.

230

00:15:41,903 --> 00:15:50,032

They tend to see what I do and discount my

abilities because of my inability to do

231

00:15:50,032 --> 00:15:51,453

really well at school.

232

00:15:51,734 --> 00:15:54,136

So that's what it is.

233

00:15:54,136 --> 00:15:58,621

But there were a bunch of people that

really did sort of shape my career.

234

00:16:01,454 --> 00:16:04,435

The, you know, working for Andrew Gelman

was great.

235

00:16:04,435 --> 00:16:06,976

He's, um, he trusted me.

236

00:16:06,976 --> 00:16:12,258

Like he, for me, he was a really, he

trusted me with a lot.

237

00:16:12,258 --> 00:16:12,478

Right.

238

00:16:12,478 --> 00:16:19,461

So he's, he was able to, um, just set me

loose on a couple of problems to start.

239

00:16:19,841 --> 00:16:22,102

And he never micromanages.

240

00:16:22,102 --> 00:16:27,884

So he just let me go for some that's a

really difficult place to be, um, without

241

00:16:27,884 --> 00:16:30,045

having guidance in a difficult problem.

242

00:16:30,045 --> 00:16:30,685

But.

243

00:16:31,138 --> 00:16:35,581

For someone like me, that was absolutely

fine and encouraging.

244

00:16:36,703 --> 00:16:40,166

You know, and working with Andrew and I

worked really closely with Bob Carpenter

245

00:16:40,166 --> 00:16:47,934

for a long time and that was really great

because he has such a depth of knowledge

246

00:16:47,934 --> 00:16:55,020

and also humility that, I don't know,

it's, it's fun working with Bob.

247

00:16:56,286 --> 00:17:00,569

Some of the other times that I've really

gotten to grow in my career, we're sitting

248

00:17:00,569 --> 00:17:03,050

in on some amazing reading groups.

249

00:17:03,371 --> 00:17:05,532

So there are two that come to mind.

250

00:17:06,433 --> 00:17:12,597

At Columbia, Dave Bly runs a reading group

for his group and got to sit in.

251

00:17:12,597 --> 00:17:17,441

And those are phenomenal because they

actually go deep into papers and really,

252

00:17:17,721 --> 00:17:24,105

really get at the content of the paper,

what it's doing, what the research is.

253

00:17:24,270 --> 00:17:28,612

trying to infer what's going on, where the

research is going next.

254

00:17:29,753 --> 00:17:36,898

But that really helped expand my horizon

for things that I wasn't seeing while

255

00:17:36,898 --> 00:17:39,260

working in Andrew's group.

256

00:17:39,260 --> 00:17:42,742

So it was just, you know, much more

machine learning oriented.

257

00:17:43,042 --> 00:17:48,445

And in a similar vein at Cambridge, I was

able to sit in on Zubin Karamani's group.

258

00:17:48,806 --> 00:17:51,708

Don't know why he let me, but he let me

just sit in.

259

00:17:51,848 --> 00:17:54,329

I was group reading groups and

260

00:17:54,774 --> 00:17:57,615

He had a lot of good people there at the

time.

261

00:17:57,736 --> 00:18:01,678

That was when Carl Rasmussen was there

working on his book.

262

00:18:01,718 --> 00:18:08,864

Um, David Knowles, uh, I don't know who

else, but just sitting there reading about

263

00:18:08,864 --> 00:18:13,047

these papers, reading these techniques,

people presenting their own work inside

264

00:18:13,047 --> 00:18:14,147

the reading group.

265

00:18:14,147 --> 00:18:19,071

Um, yeah, my encouragement would be if you

have a chance to go sit in on reading

266

00:18:19,071 --> 00:18:20,372

groups, go join them.

267

00:18:20,372 --> 00:18:23,213

It's actually a good way, especially if

it's not in your.

268

00:18:23,666 --> 00:18:24,648

area of focus.

269

00:18:24,648 --> 00:18:30,239

It's a good way to learn and make

connections to literature that otherwise

270

00:18:30,239 --> 00:18:32,181

would be very hard to read on your own.

271

00:18:34,826 --> 00:18:36,966

Yeah, I mean, completely agree with that.

272

00:18:36,966 --> 00:18:41,707

And yeah, it feels like a dream team of

mentors you've had.

273

00:18:41,887 --> 00:18:42,608

I'm really jealous.

274

00:18:42,608 --> 00:18:46,589

Like David Spiegelhalter, Andrew Gellman,

Bob Carpenter, all those people.

275

00:18:46,589 --> 00:18:47,629

It's absolutely amazing.

276

00:18:47,629 --> 00:18:51,610

And I've had the chance of interviewing

them on the podcast.

277

00:18:51,610 --> 00:18:57,192

So I will definitely link to those

episodes in the show notes.

278

00:18:57,192 --> 00:18:59,112

And yeah, completely agree.

279

00:19:04,950 --> 00:19:10,673

Today, I would definitely try and do it

with Andrew, because I've talked with him

280

00:19:10,673 --> 00:19:11,514

quite a lot already.

281

00:19:11,514 --> 00:19:16,037

And yeah, it's really inspiring.

282

00:19:16,037 --> 00:19:18,819

And that's really awesome.

283

00:19:19,119 --> 00:19:25,163

And yeah, I completely agree that in

general, that's something that I'm trying

284

00:19:25,163 --> 00:19:25,484

to do.

285

00:19:25,484 --> 00:19:28,065

And that's also where I started the

podcast in a way.

286

00:19:28,065 --> 00:19:34,114

Surrounding yourself with smarter people

than you is usually a good thing.

287

00:19:34,114 --> 00:19:35,274

good way to go.

288

00:19:36,435 --> 00:19:40,577

And definitely me, I've had the chance

also to have some really amazing mentors

289

00:19:40,577 --> 00:19:41,838

along my way.

290

00:19:41,958 --> 00:19:49,622

People like Ravin Kumar, Thomas Vicky,

Osvaldo Martin, Colin Carroll, Austin

291

00:19:49,622 --> 00:19:50,483

Rushford.

292

00:19:50,483 --> 00:19:54,205

Well, Andrew Ganneman also with everything

he's produced.

293

00:19:54,925 --> 00:19:59,388

And yeah, Adrian Zabolt also absolutely

brilliant.

294

00:19:59,388 --> 00:20:00,368

Luciano Paz.

295

00:20:00,368 --> 00:20:02,769

All these people basically in the times

he...

296

00:20:02,994 --> 00:20:08,997

world who helped me when I was really

starting and not even knowing about Git

297

00:20:08,997 --> 00:20:14,861

and taking a bit of their free time to

review my PRs and help me along the way.

298

00:20:14,861 --> 00:20:18,743

That's just really incredible.

299

00:20:18,743 --> 00:20:24,426

So yeah, what I encourage people to do

when they really start in that domain is

300

00:20:25,387 --> 00:20:27,728

much more than trying to find a...

301

00:20:29,710 --> 00:20:36,073

an internship that shines on us, trying to

really find a community where you'll be

302

00:20:36,073 --> 00:20:39,795

surrounded by smart and generous people.

303

00:20:39,995 --> 00:20:43,117

That's usually going to help you much more

than a name on the CV.

304

00:20:43,117 --> 00:20:45,158

Absolutely.

305

00:20:46,799 --> 00:20:56,165

And so actually, I'd like to talk a bit

about some of the Pharma-related models

306

00:20:56,165 --> 00:20:57,525

you've worked on.

307

00:20:58,442 --> 00:21:00,883

You've worked on so many topics.

308

00:21:00,883 --> 00:21:03,325

It's really hard to interview you.

309

00:21:03,325 --> 00:21:08,608

But a kind of model I'm really curious

about, also because we work on that at

310

00:21:08,608 --> 00:21:11,670

labs from time to time, is farmer-related

models.

311

00:21:11,670 --> 00:21:17,833

And in particular, can you explain how

Bayesian methods are used in estimating

312

00:21:17,833 --> 00:21:20,655

the efficacy of oncology treatments?

313

00:21:20,975 --> 00:21:24,817

And also, what are PKPD models?

314

00:21:26,038 --> 00:21:27,859

Yeah, let's start with PKPD models.

315

00:21:27,859 --> 00:21:32,001

So PKPD stands for pharmacometric

pharmacodynamic models.

316

00:21:32,001 --> 00:21:42,067

And these models, the pharmacokinetics

describe, so we take drug and it goes into

317

00:21:42,067 --> 00:21:42,928

the body.

318

00:21:43,588 --> 00:21:48,011

You can model that using, you know, you

know how much drug goes in the body.

319

00:21:48,011 --> 00:21:52,213

And then at some point it has to exit the

body through.

320

00:21:53,182 --> 00:21:54,622

absorption through something, right?

321

00:21:54,622 --> 00:21:55,962

So your liver can take it out.

322

00:21:55,962 --> 00:21:59,243

It'll go into your bloodstream, whatever.

323

00:21:59,884 --> 00:22:01,464

That's the kinetics part.

324

00:22:01,464 --> 00:22:03,925

You know that the drug went in and it

comes out.

325

00:22:03,925 --> 00:22:05,686

So you can measure the blood at different

times.

326

00:22:05,686 --> 00:22:09,367

You can measure different parts of the

body to get an estimate of how much is

327

00:22:09,367 --> 00:22:10,107

left.

328

00:22:10,187 --> 00:22:14,128

You can estimate how that works.

329

00:22:15,289 --> 00:22:18,449

The pharmacodynamic part is the more

difficult part.

330

00:22:18,726 --> 00:22:23,610

So each person responds differently to the

drug depending on what's inside the drug

331

00:22:23,610 --> 00:22:26,351

and how much concentration is in the body.

332

00:22:26,832 --> 00:22:32,877

You and I could take the same dose of

ibuprofen and we're going to ask each

333

00:22:32,877 --> 00:22:40,563

other how you feel and that number is, I

don't know, is it on a scale of 1 to 10?

334

00:22:41,144 --> 00:22:46,147

You might be saying a 3, I might be saying

a 4 just based on what we feel.

335

00:22:46,148 --> 00:22:47,929

There are other measurements there that...

336

00:22:48,966 --> 00:22:56,227

sometimes you can measure that's more

directly tied to the mechanism, but most

337

00:22:56,227 --> 00:23:00,489

of the time it's a few hops away from the

actual drug entering the bloodstream.

338

00:23:00,489 --> 00:23:08,971

So the whole point of pharmacokinetic,

pharmacodynamic modeling is just measuring

339

00:23:09,311 --> 00:23:13,092

drug goes in, drug goes out, what's the

effect.

340

00:23:19,182 --> 00:23:25,164

trials and in design of how much dose to

give people.

341

00:23:25,164 --> 00:23:30,087

So if you give someone double the dosage,

are they actually gonna feel better?

342

00:23:30,087 --> 00:23:35,709

Is the level of drug gonna be too high

such that there are side effects, so on

343

00:23:35,709 --> 00:23:37,910

and so forth.

344

00:23:37,910 --> 00:23:44,953

The way Bayesian methods play out here is

that if we, you know, just

345

00:23:45,026 --> 00:23:46,046

really broad step.

346

00:23:46,046 --> 00:23:52,211

If you take a step back, the last

generation of models, assume that everyone

347

00:23:52,211 --> 00:23:56,895

came from, you were trying to estimate a

population mean for all these things.

348

00:23:57,095 --> 00:24:01,399

So you're trying to take individuals and

individual responses and try to get the

349

00:24:01,399 --> 00:24:06,943

mean parameters of a, usually a

parameterized model of how the kinetics

350

00:24:06,943 --> 00:24:09,605

works and then the dynamics works.

351

00:24:15,102 --> 00:24:19,863

it'd be better if we had hierarchical

models that assumed that there was a, you

352

00:24:19,863 --> 00:24:26,306

know, a mean but each person's individual

and that could describe the dynamics for

353

00:24:26,306 --> 00:24:30,968

each person a little better than it can

for just using plugging in the overall.

354

00:24:31,568 --> 00:24:37,250

So to do that, you kind of ended up

needing Bayesian models.

355

00:24:37,451 --> 00:24:41,012

But on top of that, the other reason why

Bayesian models are really popular for

356

00:24:41,012 --> 00:24:43,353

this stuff right now is that...

357

00:24:47,762 --> 00:24:56,185

The people that study these models have a

lot of expertise in how the body works and

358

00:24:56,185 --> 00:24:58,106

how the drugs work.

359

00:24:58,446 --> 00:25:02,848

And so they've been wanting to incorporate

more and more complexity into the models,

360

00:25:03,148 --> 00:25:10,891

which is very difficult to do inside the

setting of certain packages that limit the

361

00:25:10,891 --> 00:25:11,892

flexibility.

362

00:25:11,892 --> 00:25:15,926

There's a lot of flexibility that you can

put in, but there's always a limit.

363

00:25:15,926 --> 00:25:17,566

to that flexibility.

364

00:25:18,207 --> 00:25:27,114

And that's where Stan and other tools like

PyMC are coming into play now, not just

365

00:25:27,114 --> 00:25:33,138

for the Bayesian estimates, but really for

the ability to create models that are more

366

00:25:33,138 --> 00:25:35,520

complex.

367

00:25:35,520 --> 00:25:39,303

And that are generative in particular?

368

00:25:39,303 --> 00:25:43,365

These are, because people are trying to

really understand

369

00:25:43,546 --> 00:25:46,607

for these types of studies, they're trying

to understand what happens.

370

00:25:46,607 --> 00:25:49,248

Like, what's the best dosage to give

people?

371

00:25:49,329 --> 00:25:52,330

Should it be scaled based on the size of

the human?

372

00:25:53,951 --> 00:25:55,192

What happens?

373

00:25:55,192 --> 00:25:57,653

You know, it's a lot of what happens.

374

00:25:57,653 --> 00:26:02,556

Can you characterize what's going to

happen if you give it to a larger

375

00:26:02,556 --> 00:26:03,336

population?

376

00:26:03,336 --> 00:26:09,480

You know, you've seen some variability

inside the smaller trial.

377

00:26:09,480 --> 00:26:10,760

What happens next?

378

00:26:13,162 --> 00:26:14,362

Yeah, fascinating.

379

00:26:14,482 --> 00:26:26,189

And so it seems to me that it's kind of a

really great use case for patient stats,

380

00:26:26,189 --> 00:26:26,509

right?

381

00:26:26,509 --> 00:26:31,212

Because, I mean, you really need a lot of

domain knowledge here.

382

00:26:31,212 --> 00:26:32,832

You want that in the model.

383

00:26:33,693 --> 00:26:38,836

You probably also have good ideas of

priors and so on.

384

00:26:39,316 --> 00:26:41,117

But I'm wondering what are the main

385

00:26:41,294 --> 00:26:44,075

challenges when you work on that kind of

model?

386

00:26:45,817 --> 00:26:52,562

The main challenges, I think, some of the

challenges have to do with at least when I

387

00:26:52,562 --> 00:26:53,162

was working there.

388

00:26:53,162 --> 00:26:56,484

So mind you, I didn't work directly for a

pharma company.

389

00:26:56,545 --> 00:27:01,748

We had a startup where we were building

these models and selling to pharma.

390

00:27:05,294 --> 00:27:12,890

One of the issues is that there's a lot of

historic...

391

00:27:16,150 --> 00:27:18,611

very good reasons for using older tools.

392

00:27:18,611 --> 00:27:20,152

They don't move as fast, right?

393

00:27:20,152 --> 00:27:24,715

So you've got regulators, you've got

people trying to be very careful and

394

00:27:24,715 --> 00:27:25,596

conservative.

395

00:27:25,596 --> 00:27:30,739

So trying out new methods on the same

data, if it doesn't produce results that

396

00:27:30,739 --> 00:27:38,205

they're used to, it's a little harder to

do there than it is, let's say in sports,

397

00:27:38,205 --> 00:27:38,345

right?

398

00:27:38,345 --> 00:27:45,489

In sports, no one's gonna die if I predict

something wrong next year.

399

00:27:46,766 --> 00:27:51,350

If you use a model that's completely

incompatible with the data in pharma and

400

00:27:51,350 --> 00:27:55,173

it gives you bad results, bad things do

happen sometimes.

401

00:27:57,516 --> 00:27:59,537

So anyway, things move a little slower.

402

00:28:01,259 --> 00:28:06,444

The other thing is that most people are

not trained in understanding Bayesian

403

00:28:06,444 --> 00:28:06,984

stats yet.

404

00:28:06,984 --> 00:28:13,109

You know, I do think that there's a

difference...

405

00:28:14,246 --> 00:28:20,707

in understanding Bayesian statistics from

a theoretic, like on paper point of view,

406

00:28:20,708 --> 00:28:25,269

and actually being a pragmatic modeler of

data.

407

00:28:25,569 --> 00:28:30,390

Um, and right now I think there's a

turning point, right?

408

00:28:30,390 --> 00:28:37,353

I think the world is catching up and the

ability to model is spreading, uh, a lot

409

00:28:37,353 --> 00:28:40,613

wider and the, um,

410

00:28:42,018 --> 00:28:47,135

So anyway, I think that's part of that is

happening in farm as well.

411

00:28:50,130 --> 00:28:53,131

Yeah, yeah, for sure.

412

00:28:53,131 --> 00:28:59,733

Yeah, these kind of models, I really find

them fascinating because they are both

413

00:29:01,014 --> 00:29:04,836

quite intricate and complicated from a

statistical standpoint.

414

00:29:04,836 --> 00:29:08,977

So you really learn a lot when you work on

them.

415

00:29:09,317 --> 00:29:12,899

And at the same time, they are extremely

useful and helpful.

416

00:29:12,899 --> 00:29:18,561

And usually, they are about extremely

fascinating projects that have a deep

417

00:29:18,561 --> 00:29:19,481

impact.

418

00:29:22,396 --> 00:29:27,259

on people, basically it's helping directly

people who I find them absolutely

419

00:29:27,259 --> 00:29:27,860

fascinating.

420

00:29:27,860 --> 00:29:32,983

I mean, I can tell you that specifically,

the place where I had difficulty working

421

00:29:32,983 --> 00:29:36,766

in PTA-PD models was that I didn't

understand the biology enough.

422

00:29:36,766 --> 00:29:43,510

So there are these terms, these constants,

these rate constants that describe

423

00:29:43,751 --> 00:29:45,872

elimination of the drug through the liver.

424

00:29:46,153 --> 00:29:47,658

And because I don't

425

00:29:47,658 --> 00:29:51,220

don't know biology well enough, I don't

know what's a reasonable range.

426

00:29:51,681 --> 00:29:55,525

And, you know, people that study the

biology know this off the back, off the

427

00:29:55,525 --> 00:30:00,430

top of their head because they've studied

the body, but they can't, you know, most

428

00:30:00,430 --> 00:30:07,616

aren't able to work with a system like

STAND well enough to write the model down.

429

00:30:07,616 --> 00:30:11,500

And it's that mismatch that makes it

really tough because then, you know,

430

00:30:11,500 --> 00:30:12,160

there's...

431

00:30:13,486 --> 00:30:17,327

Some in some of the conversations we had

in that world, it's, you know, why aren't

432

00:30:17,327 --> 00:30:18,467

you using a Jefferies prior?

433

00:30:18,467 --> 00:30:20,948

Why aren't you using a non-informative

prior?

434

00:30:21,268 --> 00:30:26,109

But on the flip side, it's like, if that

rate constant is 10 million, is that

435

00:30:26,109 --> 00:30:26,649

reasonable?

436

00:30:26,649 --> 00:30:27,650

No, it's not.

437

00:30:27,650 --> 00:30:29,530

It has to be between like zero and one.

438

00:30:29,530 --> 00:30:34,612

So we should be, you know, like for me,

it's if we put priors there, that limit

439

00:30:34,612 --> 00:30:41,093

that, that makes the modeling side of it a

lot easier, but you know, as someone that

440

00:30:41,386 --> 00:30:48,999

didn't understand the biology well enough

to make those claims, it made the modeling

441

00:30:48,999 --> 00:30:53,525

much, much more difficult and harder to

explain as well.

442

00:30:55,626 --> 00:30:56,446

Yeah, yeah, yeah.

443

00:30:56,446 --> 00:30:57,446

Yeah, definitely.

444

00:30:58,126 --> 00:31:02,287

And the biology of those models is

absolutely fascinating, but really, really

445

00:31:02,287 --> 00:31:03,187

intriguing.

446

00:31:04,748 --> 00:31:13,150

And also, you've also worked on something

that's called data fusion for US Navy

447

00:31:13,150 --> 00:31:13,971

applications.

448

00:31:13,971 --> 00:31:16,991

So that sounds very mysterious.

449

00:31:18,292 --> 00:31:21,613

How did Bayesian statistics contribute to

these projects?

450

00:31:21,613 --> 00:31:24,533

And what were some of the challenges you

faced?

451

00:31:24,974 --> 00:31:27,375

Unfortunately, I didn't know Bayesian

stats at the time.

452

00:31:27,375 --> 00:31:29,537

This was when I first started working.

453

00:31:29,677 --> 00:31:32,299

But, you know, data fusion's actually...

454

00:31:32,319 --> 00:31:33,720

We should have used Bayesian stats.

455

00:31:33,720 --> 00:31:37,603

If I was working on a problem now, it

should be done with Bayesian stats.

456

00:31:37,603 --> 00:31:38,383

The...

457

00:31:39,164 --> 00:31:44,868

Just the problem in a nutshell, if you

imagine you have an aircraft carrier, it

458

00:31:44,868 --> 00:31:49,111

can't move very fast, and what it has is

about a dozen ships around it.

459

00:31:49,612 --> 00:31:51,033

All of them have radars.

460

00:31:51,033 --> 00:31:52,673

All of them point at the same thing.

461

00:31:53,054 --> 00:31:55,936

If you're sitting on the aircraft carrier

trying to make decisions about what's

462

00:31:55,936 --> 00:31:58,037

coming at you, what to do next.

463

00:31:58,078 --> 00:32:01,800

If there's a single plane coming at you,

that's one thing.

464

00:32:02,121 --> 00:32:08,626

If all the 12 ships around you, you know,

hit that same thing with the radar and it

465

00:32:08,626 --> 00:32:12,329

says that there are 12 things coming at

you because things are slightly jittered,

466

00:32:12,670 --> 00:32:13,991

that's bad news, right?

467

00:32:13,991 --> 00:32:17,694

So, you know, if they're not identifying

themselves.

468

00:32:17,694 --> 00:32:22,697

So the whole problem is, is there enough

information there where you can...

469

00:32:23,026 --> 00:32:28,405

accurately depict what's happening based

on multiple pieces of data.

470

00:32:30,006 --> 00:32:31,066

Hmm.

471

00:32:31,066 --> 00:32:31,986

Okay.

472

00:32:32,387 --> 00:32:33,568

Yeah, that sounds pretty fun.

473

00:32:33,568 --> 00:32:35,509

And indeed, yeah, lots of uncertainty.

474

00:32:35,509 --> 00:32:38,290

So, and I'm guessing you don't have a lot

of data.

475

00:32:38,510 --> 00:32:43,313

And also, it's the kind of experiments you

cannot really remake and remake.

476

00:32:43,954 --> 00:32:48,356

So, your patient stats would be helpful

here, I'm guessing.

477

00:32:48,776 --> 00:32:52,419

Yeah, it's, it's always the edge cases

that are tough, right?

478

00:32:52,419 --> 00:32:57,886

It's, if the, if the, if the plane or the

ship that's coming at you,

479

00:32:57,886 --> 00:33:01,627

says who they are, identifies themselves,

and follows normal protocol.

480

00:33:01,627 --> 00:33:07,609

It's an easy problem, like you have the

identifier, but it's when that stuff's

481

00:33:07,609 --> 00:33:08,350

latent, right?

482

00:33:08,350 --> 00:33:10,931

People hide it intentionally.

483

00:33:11,291 --> 00:33:13,432

Then you have to worry about what's going

on.

484

00:33:15,373 --> 00:33:21,395

The really cool thing there was a guy I

worked for, Clay Stannick, had come up

485

00:33:21,395 --> 00:33:22,875

with a way to

486

00:33:28,222 --> 00:33:31,963

of each of the radar pictures and just

stack them on top of each other.

487

00:33:31,963 --> 00:33:40,245

If you do that, if you see a high

intensity, then it means that the pictures

488

00:33:40,245 --> 00:33:40,965

overlap.

489

00:33:40,965 --> 00:33:45,086

And if there's no high intensity, then it

means the pictures don't overlap.

490

00:33:45,106 --> 00:33:47,727

And the nice thing is that that's rotation

invariant.

491

00:33:47,727 --> 00:33:53,229

So it really just helps with the alignment

problem because everyone's looking at the

492

00:33:53,229 --> 00:33:55,109

same picture from different angles.

493

00:33:58,782 --> 00:34:02,563

Yeah, yeah, it's super interesting also.

494

00:34:02,563 --> 00:34:03,563

I love that.

495

00:34:03,563 --> 00:34:07,384

And you haven't had the opportunity to

work again on that kind of models now that

496

00:34:07,384 --> 00:34:09,405

you're an Asian expert?

497

00:34:09,405 --> 00:34:11,825

No.

498

00:34:11,825 --> 00:34:13,626

Well, you've heard it, folks.

499

00:34:13,786 --> 00:34:18,547

If you have some model like that who are

entertaining you, feel free to contact him

500

00:34:18,987 --> 00:34:23,429

or me, and I will contact him for you if

you want.

501

00:34:23,429 --> 00:34:26,069

So actually.

502

00:34:26,778 --> 00:34:31,503

I'm curious, you know, in general, because

you've worked with so many people and in

503

00:34:31,503 --> 00:34:32,904

so many different fields.

504

00:34:33,385 --> 00:34:38,892

I wonder if you picked up some common

misconceptions or challenges that people

505

00:34:38,892 --> 00:34:46,140

face when they try to apply vision stats

to real world problems and how you think

506

00:34:46,140 --> 00:34:47,881

we can overcome them.

507

00:34:49,638 --> 00:34:53,414

Yeah, that's an interesting question.

508

00:34:59,698 --> 00:35:10,282

I think working with Dan, well, yeah, I

think the common error is that we don't

509

00:35:10,282 --> 00:35:12,903

build our models complex enough.

510

00:35:14,003 --> 00:35:19,365

They don't describe the phenomenon well

enough to really explain the data.

511

00:35:19,846 --> 00:35:28,249

And I think that's where, that's the most

common problem that we have.

512

00:35:31,366 --> 00:35:38,151

Yeah, the thing that I use the most, that

I get the most mileage out of is actually

513

00:35:38,151 --> 00:35:43,556

putting on either a measurement model or

just adding a little more complexity to

514

00:35:43,556 --> 00:35:46,938

model and it starts working way better.

515

00:35:46,959 --> 00:35:51,803

In pharmacometrics specifically, I

remember we started asking, how do you

516

00:35:51,803 --> 00:35:52,963

collect the data?

517

00:35:53,884 --> 00:35:56,987

What sort of ways is the measurement

wrong?

518

00:35:57,127 --> 00:36:00,229

And we just modeled that piece and put it

into the same

519

00:36:00,350 --> 00:36:04,372

parametric forms of the model and

everything started fitting correctly.

520

00:36:04,672 --> 00:36:08,434

It's like, cool, I should do that more

often.

521

00:36:08,615 --> 00:36:13,658

So yeah, I think if I was to think about

that, that's sort of the thing.

522

00:36:13,658 --> 00:36:20,502

The other thing is, I guess people try to

apply Bayesian stats, Bayesian models to

523

00:36:20,502 --> 00:36:24,765

everything, and it's not always

applicable.

524

00:36:25,982 --> 00:36:30,184

I don't know if you're actually going to

be able to fit a true LLM using MCMC.

525

00:36:30,184 --> 00:36:32,425

Like I think that'd be very, very

difficult.

526

00:36:32,826 --> 00:36:36,948

Um, so it's okay to not be Bayesian for

that stuff.

527

00:36:41,758 --> 00:36:43,938

Yeah.

528

00:36:43,938 --> 00:36:45,439

So that's interesting.

529

00:36:45,439 --> 00:36:53,501

So nothing about priors or about model

fitting or about model time sampling of

530

00:36:53,501 --> 00:36:54,421

the models.

531

00:36:56,362 --> 00:36:58,442

No, I mean, they're all related, right?

532

00:36:58,442 --> 00:37:00,883

The worst the model fits.

533

00:37:01,163 --> 00:37:09,405

So when a model doesn't actually match the

data, at least running in Stan, it tends

534

00:37:09,405 --> 00:37:10,105

to.

535

00:37:10,586 --> 00:37:15,068

overinflate the amount of time it takes,

the diagnostics look bad.

536

00:37:15,409 --> 00:37:21,494

A lot of things get fixed once you start

putting in the right level of complexity

537

00:37:21,494 --> 00:37:22,834

to match the data.

538

00:37:22,834 --> 00:37:26,897

But you know, that's yeah.

539

00:37:26,897 --> 00:37:31,721

I mean, is it MCMC is definitely slower

than running optimization?

540

00:37:32,181 --> 00:37:32,782

That's true.

541

00:37:32,782 --> 00:37:33,162

Yeah.

542

00:37:33,162 --> 00:37:34,042

No, for sure.

543

00:37:38,270 --> 00:37:43,493

Yeah, I'm asking because as I'm teaching a

lot, these are recurring themes.

544

00:37:43,913 --> 00:37:46,815

I mean, it really depends where people are

coming from.

545

00:37:46,815 --> 00:37:53,260

But you have recurring themes where that

can be kind of a difficulty for people.

546

00:37:54,000 --> 00:37:59,624

Something I've seen that's pretty common

is understanding the different types of

547

00:37:59,624 --> 00:38:00,245

distributions.

548

00:38:00,245 --> 00:38:07,449

So prior predictive samples and prior

samples, how do they differ?

549

00:38:07,586 --> 00:38:10,908

Posterior samples, post-hereditary

samples, what's the difference between all

550

00:38:10,908 --> 00:38:11,828

of that?

551

00:38:12,269 --> 00:38:21,515

That's definitely a topic of complexity

that can trigger some difficulty for

552

00:38:22,075 --> 00:38:22,616

people.

553

00:38:22,616 --> 00:38:25,517

And I mean, I think that's quite normal.

554

00:38:25,538 --> 00:38:29,661

I remember personally, it took me a few

months to really understand that stuff

555

00:38:29,661 --> 00:38:31,561

when I started learning Baystance.

556

00:38:32,883 --> 00:38:35,144

And now with my educational content,

557

00:38:37,686 --> 00:38:45,770

decrease that time for people so that they

maybe make the same mistakes as me, but

558

00:38:45,770 --> 00:38:48,711

they realize it's faster than I did.

559

00:38:49,572 --> 00:38:50,773

That's kind of the objective.

560

00:38:50,773 --> 00:38:53,534

Yeah, that's really good.

561

00:38:53,534 --> 00:38:57,056

So what other things do you see that

people are struggling with?

562

00:38:57,056 --> 00:39:03,159

Or do you have, you know, what are some of

the common themes right now?

563

00:39:03,540 --> 00:39:05,300

I mean, priors a lot.

564

00:39:06,926 --> 00:39:11,769

priors is extremely common, especially if

people come from the classic machine

565

00:39:11,769 --> 00:39:18,775

learning framework, where it's really hard

for them to choose a prior.

566

00:39:19,936 --> 00:39:30,345

And actually something I've noticed is two

ways of thinking about them that allows

567

00:39:30,345 --> 00:39:36,609

them to kind of be less anxious about

choosing a prior.

568

00:39:36,622 --> 00:39:44,745

which is one, making them realize that

having flat priors doesn't mean not having

569

00:39:44,745 --> 00:39:45,605

priors.

570

00:39:45,665 --> 00:39:51,048

And so the fact that they were using flat

priors before by default in a class

571

00:39:51,048 --> 00:39:54,369

equalization regression, for instance,

that's a prior.

572

00:39:54,409 --> 00:39:55,610

That's already an assumption.

573

00:39:55,610 --> 00:40:01,412

So why would you be less comfortable

making another assumption, especially if

574

00:40:01,412 --> 00:40:03,993

it's more warranted in that case?

575

00:40:03,993 --> 00:40:04,554

So.

576

00:40:04,554 --> 00:40:09,955

Basically trying to see these idea of

priors along a slider, you know, a

577

00:40:09,955 --> 00:40:16,057

gradient where you would have like the

extreme left would be the completely flat

578

00:40:16,057 --> 00:40:21,098

priors, which lead to a completely overfit

model that has a lot of variance in the

579

00:40:21,098 --> 00:40:22,038

predictions.

580

00:40:22,158 --> 00:40:26,960

And then at the other end of the slider,

extreme right would be the completely

581

00:40:26,960 --> 00:40:31,841

biased model where your priors would

basically be, you know, either a point or

582

00:40:31,841 --> 00:40:33,194

completely outside of

583

00:40:33,194 --> 00:40:36,555

the realm of the data and then you cannot

update, basically.

584

00:40:37,115 --> 00:40:39,336

But that would be a completely underfit

model.

585

00:40:39,536 --> 00:40:44,438

So in a way, the priors are here to allow

you to navigate that slider.

586

00:40:44,438 --> 00:40:49,640

And why would you always want to be to the

extreme left of the slider, right?

587

00:40:50,341 --> 00:40:52,642

Because in the end, you're already making

a choice.

588

00:40:52,642 --> 00:40:59,364

So why not thinking a bit more

exhaustively and clearly about the choice,

589

00:40:59,364 --> 00:41:01,705

explicitly about the choices you're

making.

590

00:41:02,558 --> 00:41:09,860

Yeah, that already usually helps them to

make them feel less guilty about choosing

591

00:41:09,860 --> 00:41:10,740

prior.

592

00:41:11,020 --> 00:41:12,080

So that's interesting.

593

00:41:12,080 --> 00:41:13,561

Yeah, absolutely.

594

00:41:13,561 --> 00:41:19,143

And so to go on that point a little bit,

that's what I'm trying to say with the

595

00:41:19,143 --> 00:41:19,983

complexity of the model.

596

00:41:19,983 --> 00:41:26,685

It's like, if we just assume normal things

a lot of times, but sometimes things

597

00:41:26,685 --> 00:41:27,325

aren't normal.

598

00:41:27,325 --> 00:41:28,725

There's more variance than normal.

599

00:41:28,725 --> 00:41:29,365

So.

600

00:41:29,726 --> 00:41:32,766

making something a t-distribution

sometimes fixes it.

601

00:41:33,547 --> 00:41:40,449

Just understanding the prior predictive,

the posterior, the posterior predictive

602

00:41:40,449 --> 00:41:44,650

draws also summarizing those, looking at

the data really helps.

603

00:41:45,891 --> 00:41:54,393

One thing that I think for anyone trying

to do models in production, one thing to

604

00:41:54,393 --> 00:41:55,873

know is that

605

00:41:57,598 --> 00:42:04,641

models, the programs that you write,

either in PyMC or Stan, the quality of the

606

00:42:04,641 --> 00:42:08,963

fit is not just the program itself, it's

the program plus the data.

607

00:42:08,963 --> 00:42:12,324

If you swap out the data and it has

different properties than the one that you

608

00:42:12,324 --> 00:42:17,086

trained it on before, it might actually

have worse properties or better

609

00:42:17,086 --> 00:42:17,766

properties.

610

00:42:17,766 --> 00:42:22,608

And we can see this with like non-centered

parameterization and different variance

611

00:42:22,608 --> 00:42:25,130

components being estimated in weird ways.

612

00:42:25,130 --> 00:42:29,832

if you just blindly assume that you can go

and take your model that fit on one data

613

00:42:29,832 --> 00:42:32,454

and just blindly productionize it.

614

00:42:32,454 --> 00:42:35,676

It doesn't quite work that way yet,

unfortunately.

615

00:42:35,676 --> 00:42:39,398

Yeah, yeah, yeah.

616

00:42:39,799 --> 00:42:40,399

For sure.

617

00:42:40,399 --> 00:42:49,304

And also, another prompt that I use to

help them understand a bit more,

618

00:42:49,545 --> 00:42:51,325

basically, why we're using...

619

00:42:52,066 --> 00:42:57,207

generative models and why that means

making assumptions and how to make them

620

00:42:57,207 --> 00:43:01,948

and being more comfortable making

assumptions is, well, imagine that you had

621

00:43:01,948 --> 00:43:05,749

to bet on every decision that your model

is making.

622

00:43:07,290 --> 00:43:12,531

Wouldn't you want to use all the

information you have at your fingertips,

623

00:43:13,011 --> 00:43:14,792

especially with the internet now?

624

00:43:14,992 --> 00:43:18,953

It's not that hard to find some

information about the parameters of any

625

00:43:18,953 --> 00:43:21,553

model you're working on and find a

pattern.

626

00:43:21,794 --> 00:43:26,838

somewhat informed prior because you don't

need, you know, there is no best prior so

627

00:43:26,838 --> 00:43:32,322

you don't need the perfect prior because

it's a prior, you have the data so it's

628

00:43:32,322 --> 00:43:36,085

going to be updated anyways and if you

have a lot of data it's going to be washed

629

00:43:36,085 --> 00:43:42,851

out so but you know if you had to bet on

any decision you're making or that your

630

00:43:42,851 --> 00:43:46,206

model is making wouldn't you want you to

use

631

00:43:46,206 --> 00:43:50,427

all the information you have available

instead of just throwing your hands in the

632

00:43:50,427 --> 00:43:53,668

air and being like, oh, no, I don't know

anything, so I'm going to use flat priors

633

00:43:53,668 --> 00:43:54,528

everywhere.

634

00:43:55,108 --> 00:43:56,668

You really don't know anything?

635

00:43:57,008 --> 00:43:59,109

Have you searched on Google?

636

00:43:59,109 --> 00:44:01,149

It's not that far.

637

00:44:01,890 --> 00:44:09,072

So yeah, that usually also helps when you

frame it in the context of basically

638

00:44:09,072 --> 00:44:14,773

decision-making with an incentive, which

here would be money.

639

00:44:16,502 --> 00:44:21,703

betting for your life, then, well, it

would make sense, right, to use any bit of

640

00:44:21,703 --> 00:44:24,244

information that you can put your hands

on.

641

00:44:24,244 --> 00:44:30,565

So why won't you do it here?

642

00:44:30,565 --> 00:44:37,987

Actually, I'm curious with your extensive

experience in the modeling world, do you

643

00:44:37,987 --> 00:44:43,309

have any advice you would give to someone

looking to start a career in computational

644

00:44:43,309 --> 00:44:45,909

Bayesian stats or data science in general?

645

00:44:47,898 --> 00:44:58,921

Yeah, my, my advice would probably to go

try to go deeper in one subject or not one

646

00:44:58,921 --> 00:45:05,463

subject, go deeper in one dimension than

you're comfortable going.

647

00:45:06,843 --> 00:45:12,205

If you want to get into like actually

building out tools, go deep, understand

648

00:45:12,205 --> 00:45:15,205

how PyMC works, understand how Stan works,

try to

649

00:45:15,238 --> 00:45:18,539

actually submit pull requests and figure

out how things are done.

650

00:45:19,300 --> 00:45:23,522

If you want to get into modeling, go start

understanding what the data is.

651

00:45:23,522 --> 00:45:24,562

Go deep.

652

00:45:24,843 --> 00:45:29,566

Don't just stop at, you know, I have data

in a database.

653

00:45:29,566 --> 00:45:31,326

Go ask how it's collected.

654

00:45:31,326 --> 00:45:35,789

Figure out what the chain actually is to

get the data to where it is.

655

00:45:36,009 --> 00:45:40,932

Going deep in that way, I think, is going

to get you pretty far.

656

00:45:40,932 --> 00:45:44,353

It'll give you a better understanding of

how certain things are.

657

00:45:44,470 --> 00:45:50,779

You never know when that knowledge

actually comes into play and will help

658

00:45:50,779 --> 00:45:51,139

you.

659

00:45:51,139 --> 00:45:53,301

But a lot of the...

660

00:45:54,910 --> 00:45:57,095

Yeah, that would be my advice.

661

00:45:57,095 --> 00:46:03,049

Just go deeper than maybe your peers or

maybe people ask you to.

662

00:46:06,514 --> 00:46:07,955

Yeah, that's a really good point.

663

00:46:09,476 --> 00:46:15,019

Yeah, I love it and that's true that I was

thinking, you know, in the people around

664

00:46:15,019 --> 00:46:21,043

me, usually, yeah, it's that kind of

people who stick to it with that passion,

665

00:46:21,043 --> 00:46:29,668

who are in the place they want it to be at

because, well, they also have that passion

666

00:46:29,668 --> 00:46:30,288

to start with.

667

00:46:30,288 --> 00:46:32,989

That's really important.

668

00:46:33,022 --> 00:46:37,404

I remember someone recently asked me like,

should they focus on machine learning,

669

00:46:37,404 --> 00:46:42,407

Beijing stats, is Beijing stats going to

go away, is AI taking over?

670

00:46:42,908 --> 00:46:48,891

And my answer to that, I think was pretty

much along the lines of go and learn any

671

00:46:48,891 --> 00:46:50,252

of them really well.

672

00:46:50,252 --> 00:46:54,755

If you don't learn any of them really

well, then you'll just be following

673

00:46:54,755 --> 00:46:58,437

different things and be bouncing back and

forth and you'll miss everything.

674

00:46:58,437 --> 00:46:59,397

But if you...

675

00:46:59,910 --> 00:47:04,233

end up like Bayesian stats has been around

for a while and I don't think it's going

676

00:47:04,233 --> 00:47:05,233

to go away.

677

00:47:05,233 --> 00:47:10,157

But if you bounce from Bayesian stats, try

to go to ML, try to go to deep learning

678

00:47:10,157 --> 00:47:16,701

without actually really investing enough

time into any of those, when it comes down

679

00:47:16,701 --> 00:47:22,065

to having a career in this stuff, you're

going to find yourself like a little short

680

00:47:22,065 --> 00:47:25,928

of expertise to distinguish yourself from

other people.

681

00:47:25,928 --> 00:47:28,849

So that, you know, that's...

682

00:47:29,706 --> 00:47:33,328

That's where this advice mentality is

coming from.

683

00:47:35,369 --> 00:47:36,950

Especially just starting out.

684

00:47:36,950 --> 00:47:41,193

I mean, there's so many things to look at

right now that, you know, it's, it's hard

685

00:47:41,193 --> 00:47:42,613

to keep track of everything.

686

00:47:45,370 --> 00:47:46,811

Yeah, no, for sure.

687

00:47:46,811 --> 00:47:48,592

That's definitely a good point, too.

688

00:47:48,592 --> 00:47:58,617

And actually, in your opinion, currently,

what are the main sticking points in the

689

00:47:58,617 --> 00:48:01,619

Bayesian workflow that you think we can

improve?

690

00:48:02,339 --> 00:48:07,362

All of us in the community of

probabilistic programming languages, core

691

00:48:07,362 --> 00:48:12,885

developers, Stan, IMC, and all those PPLs,

what do you think are those sticking

692

00:48:12,885 --> 00:48:13,665

points?

693

00:48:14,066 --> 00:48:18,034

would benefit from some love from all of

us?

694

00:48:19,559 --> 00:48:23,847

Oh, that's a good question.

695

00:48:26,334 --> 00:48:32,478

You know, in terms of the workflow, I

think just usability can get better.

696

00:48:32,679 --> 00:48:35,101

We can, we can do a lot more from that.

697

00:48:35,101 --> 00:48:38,744

Um, with that said, it's, it's hard.

698

00:48:38,744 --> 00:48:42,786

Like the tools that we're talking about

are pretty niche.

699

00:48:43,207 --> 00:48:50,873

And so it's, it's not like there are, um,

millions and millions of users of our

700

00:48:50,873 --> 00:48:56,157

techniques, so it's, you know, the, it's

just hard to do that.

701

00:48:56,658 --> 00:49:03,104

Um, but you know, the, the thing that I

run into a lot are transformations of prom

702

00:49:03,104 --> 00:49:09,690

and I really wish that we end up with, um,

reparameterizations of problems

703

00:49:09,690 --> 00:49:13,974

automatically such that it fits well with

the method that you choose.

704

00:49:14,434 --> 00:49:20,660

Um, if we could do that, then life would

be good, but, uh, you know, I think that's

705

00:49:20,660 --> 00:49:22,221

a hard problem to tackle.

706

00:49:24,010 --> 00:49:26,311

Yeah, I mean, for sure.

707

00:49:26,311 --> 00:49:33,074

Because, and that's also something I've

started to look into and hopefully in the

708

00:49:33,074 --> 00:49:36,776

coming weeks, I'll be able to look into it

for our Prime C.

709

00:49:36,776 --> 00:49:42,339

Precisely, I was talking about that with

Ricardo Viera, where we were thinking of,

710

00:49:42,339 --> 00:49:53,566

you know, having user wrapper classes on

some, on some distributions, you know.

711

00:49:53,566 --> 00:49:58,910

normal beta-gap with the classic

reparameterization, where instead of

712

00:49:58,910 --> 00:50:04,834

letting the users, I mean, making the

users have to reparameterize by hand

713

00:50:04,834 --> 00:50:11,580

themselves, you could just ask Climacy to

do pm.normal non-centered, for instance,

714

00:50:11,580 --> 00:50:13,221

and do that for you.

715

00:50:13,361 --> 00:50:16,343

In other words, that'd be really cool.

716

00:50:16,343 --> 00:50:19,265

So of course, these are always...

717

00:50:19,482 --> 00:50:24,543

bigger PRs than you suspect when you start

working on them.

718

00:50:25,043 --> 00:50:27,024

But that definitely would be a fun one.

719

00:50:27,024 --> 00:50:32,285

So, and then that'd be a fun project I'd

like to work on in the coming weeks.

720

00:50:32,285 --> 00:50:35,806

But we'll see how that goes with open

source.

721

00:50:35,806 --> 00:50:41,308

That's always very dependent on how much

work you have to do before to actually pay

722

00:50:41,308 --> 00:50:48,249

your rent and then see how much time you

can afford to dedicate to

723

00:50:48,614 --> 00:50:53,428

open source, but hopefully I'll be able to

make that happen and that'd be definitely

724

00:50:53,428 --> 00:50:54,269

super fun.

725

00:50:57,578 --> 00:51:01,620

And actually talking about the future

developments, I'm always curious about

726

00:51:01,620 --> 00:51:02,521

Stan.

727

00:51:02,521 --> 00:51:08,044

What do you folks have on your roadmap,

especially some exciting developments that

728

00:51:08,044 --> 00:51:10,905

you've seen in the works for the future of

Stan?

729

00:51:11,686 --> 00:51:16,468

So I actually haven't, I don't know what's

coming up on the roadmap too much.

730

00:51:17,049 --> 00:51:21,991

Lately, I've been focused on working on my

new job and so that's good.

731

00:51:21,991 --> 00:51:26,373

But a couple of the interesting things are

Pathfinder just made it in.

732

00:51:26,498 --> 00:51:35,425

It's a new VI algorithm, which I believe

addresses some of the difficulties with

733

00:51:35,425 --> 00:51:36,486

ADVI.

734

00:51:36,706 --> 00:51:38,228

So that should be interesting.

735

00:51:38,228 --> 00:51:44,233

And finally tuples should land if it

hasn't already landed inside the scan

736

00:51:44,233 --> 00:51:45,153

language.

737

00:51:45,554 --> 00:51:53,100

So that means that you can return from a

function multiple returns, which should be

738

00:51:53,100 --> 00:51:55,021

better for efficiency in writing.

739

00:51:55,410 --> 00:51:58,032

things down in the language.

740

00:51:58,172 --> 00:52:04,657

Other than that, it's like, you know,

there's always activity around new

741

00:52:04,657 --> 00:52:08,140

functionality in Stan and making things

faster.

742

00:52:08,781 --> 00:52:15,226

And the, you know, interface, the work on

the interface is where it makes it a lot

743

00:52:15,226 --> 00:52:18,389

easier to operate Stan is always good.

744

00:52:18,389 --> 00:52:23,973

So there's command-stan-r command-stan-pi

that really do a lot of the heavy lifting.

745

00:52:24,293 --> 00:52:24,893

Yeah.

746

00:52:26,599 --> 00:52:29,001

Yeah, super fun.

747

00:52:29,001 --> 00:52:35,505

For sure, I didn't know Pathfinder was

there, but definitely super cool.

748

00:52:35,505 --> 00:52:37,166

Have you used it yourself?

749

00:52:37,327 --> 00:52:40,349

And is there any kind of model you'd

recommend using it on?

750

00:52:40,869 --> 00:52:42,230

No, I haven't used it myself.

751

00:52:42,230 --> 00:52:47,674

But there is a model that I'm working on

at Zellis that I do want to use it on.

752

00:52:47,674 --> 00:52:50,936

So we're doing, we call it.

753

00:52:52,726 --> 00:52:54,686

component skill projection models.

754

00:52:54,686 --> 00:53:00,429

So you have observations of how players

are doing for many measurements, and then

755

00:53:00,429 --> 00:53:05,291

you have that over years, and you can

imagine that there are things that you

756

00:53:05,291 --> 00:53:10,433

don't observe about them that kind of, you

know, there's a function that you apply to

757

00:53:10,433 --> 00:53:15,915

the underlying latent skill that then

produces the output.

758

00:53:15,915 --> 00:53:20,116

And, you know, over time you're trying to

estimate over time what that does.

759

00:53:20,417 --> 00:53:22,217

And so for something like that,

760

00:53:22,770 --> 00:53:26,961

I think using an approximate solution

would probably be really good.

761

00:53:30,667 --> 00:53:30,827

Yeah.

762

00:53:30,827 --> 00:53:34,870

Do you already have a tutorial page on

this 10 website that we can refer people

763

00:53:34,870 --> 00:53:37,632

to for that episode's show notes?

764

00:53:38,513 --> 00:53:39,234

I'm not sure.

765

00:53:39,234 --> 00:53:40,215

I could send it to you, though.

766

00:53:40,215 --> 00:53:43,597

I believe there's a Pathfinder paper out

in the archives.

767

00:53:43,878 --> 00:53:45,719

Bob Carpenter's on it.

768

00:53:45,719 --> 00:53:46,920

OK, yeah, for sure.

769

00:53:47,260 --> 00:53:53,085

Yeah, add that to the show notes, and I'll

make sure to put that on the website when

770

00:53:53,085 --> 00:53:57,289

your episode goes out, because I'm sure

people are going to be curious about that.

771

00:53:57,289 --> 00:53:58,889

Yeah.

772

00:53:59,294 --> 00:54:07,940

And more generally, are there any emerging

trends or developments in Bayesian stats

773

00:54:07,940 --> 00:54:12,262

that you find particularly exciting or

promising for future applications?

774

00:54:14,064 --> 00:54:25,512

No, but I do feel like the adoption of

Bayesian methods and modeling, there's

775

00:54:25,512 --> 00:54:27,570

still time for that to spread.

776

00:54:27,570 --> 00:54:35,233

especially in the world now where LLMs are

the biggest rage and it's, you know, LLMs

777

00:54:35,233 --> 00:54:40,855

are being applied everywhere, but I still

think that there's space for more places

778

00:54:40,855 --> 00:54:47,178

to use really smart, complex models with

limited data.

779

00:54:49,099 --> 00:54:53,461

So with the, with all these tools, I just

think that, you know, more industries need

780

00:54:53,461 --> 00:54:55,761

to catch on and start using them.

781

00:54:58,090 --> 00:54:59,311

Yeah, I see.

782

00:54:59,311 --> 00:55:02,213

Already, I'm pretty impressed by what you

folks do at Zillus.

783

00:55:02,213 --> 00:55:05,156

That sounds really funny and interesting.

784

00:55:05,156 --> 00:55:13,823

And actually, one of their most recent

episodes I did, episode 91, with Max

785

00:55:13,823 --> 00:55:17,486

Gebel, was talking about European football

analytics.

786

00:55:17,486 --> 00:55:20,589

And I'm really surprised.

787

00:55:20,589 --> 00:55:25,573

So I don't know if you folks at Zillus

work already on the European market, but

788

00:55:25,573 --> 00:55:26,833

I'm really impressed.

789

00:55:27,390 --> 00:55:33,931

I'm pretty impressed in how mature the US

market is on that front of spots

790

00:55:33,931 --> 00:55:34,932

analytics.

791

00:55:35,572 --> 00:55:41,814

And on the contrary, how at least

continental Europe is really, really far

792

00:55:41,814 --> 00:55:44,414

behind that curve.

793

00:55:44,635 --> 00:55:47,175

I am both impressed and appalled.

794

00:55:47,175 --> 00:55:51,917

I'm curious what you know about that.

795

00:55:51,917 --> 00:55:54,577

I don't think anyone's that far behind

right now.

796

00:55:55,554 --> 00:55:59,536

So I know you had Jim Albert on the show

too, and I heard both of those.

797

00:55:59,536 --> 00:55:59,857

Right.

798

00:55:59,857 --> 00:56:08,143

And the, the thing that I'm really excited

about right now is making all the models

799

00:56:08,143 --> 00:56:10,104

more complex, right?

800

00:56:10,104 --> 00:56:17,049

So I think that, you know, we probably

have some of the more advanced models or

801

00:56:17,049 --> 00:56:21,632

at least up to industry standard in a lot

of them and like more complex than others

802

00:56:21,953 --> 00:56:23,458

when I, you know, I just got here.

803

00:56:23,458 --> 00:56:28,439

got to the company and when I look at it,

I think there's like another order of

804

00:56:28,439 --> 00:56:32,900

complexity that we can get to using the

tools that already exist.

805

00:56:32,900 --> 00:56:34,101

And that's where I'm excited.

806

00:56:34,101 --> 00:56:36,841

It's the data is out there.

807

00:56:36,922 --> 00:56:40,762

It's been collected for, you know, five

years, 10 years.

808

00:56:41,083 --> 00:56:42,623

Uh, there's new tracking data.

809

00:56:42,623 --> 00:56:44,364

That's, you know, that that's happening.

810

00:56:44,364 --> 00:56:48,905

So there's more data coming out, more

fidelity of data, but even using the data

811

00:56:48,905 --> 00:56:50,685

that we have, um,

812

00:56:51,754 --> 00:56:56,415

A lot of the models that people are

fitting are at the summary level

813

00:56:56,415 --> 00:56:57,615

statistics.

814

00:56:58,255 --> 00:57:01,076

And that's great and all.

815

00:57:01,076 --> 00:57:06,718

We're making really good things that

people can use using that level of

816

00:57:06,718 --> 00:57:07,858

information.

817

00:57:08,278 --> 00:57:14,820

But we can be more granular about that and

write more complex models and have better

818

00:57:15,160 --> 00:57:21,541

understanding of the phenomenon, like how

these metrics are being generated.

819

00:57:22,236 --> 00:57:26,529

And I think that's, for me, that's what's

exciting right now.

820

00:57:28,766 --> 00:57:31,287

Yeah.

821

00:57:31,287 --> 00:57:36,371

And that's what I've seen too, mainly in

Europe, where now you have amazing

822

00:57:36,371 --> 00:57:37,871

tracking data.

823

00:57:38,412 --> 00:57:39,693

Really, really good.

824

00:57:39,693 --> 00:57:43,896

In football, I don't know that much

because unfortunately I haven't had any

825

00:57:44,016 --> 00:57:46,978

insight peeking that I've had for rugby.

826

00:57:47,278 --> 00:57:50,900

And I mean, that tracking data is

absolutely fantastic.

827

00:57:51,121 --> 00:57:53,262

It's just that people don't do models on

them.

828

00:57:54,203 --> 00:57:58,370

They just do descriptive statistics.

829

00:57:58,370 --> 00:58:01,712

which is already good, but they could do

so much from that.

830

00:58:01,712 --> 00:58:09,639

But for now, I haven't been successful

explaining to them what they would get

831

00:58:09,639 --> 00:58:10,700

with models.

832

00:58:11,721 --> 00:58:19,728

And something that I'm guessing is that

there is probably not enough competitive

833

00:58:19,728 --> 00:58:25,453

pressure on this kind of usage of data.

834

00:58:25,453 --> 00:58:26,773

Because I mean,

835

00:58:28,094 --> 00:58:32,897

Unless they are very special, a sports

team is never going to come to you as a

836

00:58:32,897 --> 00:58:35,419

data scientist and tell you, hey, we need

models.

837

00:58:35,419 --> 00:58:39,582

Because they don't really know what the

difference between a mean and a model

838

00:58:39,582 --> 00:58:41,103

actually is.

839

00:58:41,103 --> 00:58:48,548

So usually these kinds of data analytics

are sold by companies here in Europe.

840

00:58:48,548 --> 00:58:55,873

And well, from a company standpoint, they

don't have a lot of competitive pressure.

841

00:58:56,074 --> 00:59:01,998

Why would you invest in writing models

which are hard to develop and takes time

842

00:59:01,998 --> 00:59:02,998

and money?

843

00:59:03,239 --> 00:59:10,964

Whereas you can just, you know, sell raw

data that then you do stat desk on.

844

00:59:11,224 --> 00:59:15,407

And that costs way less and still you're

ahead of the competition with that.

845

00:59:15,867 --> 00:59:16,468

Kind of makes sense.

846

00:59:16,468 --> 00:59:19,690

So yeah, I don't know.

847

00:59:19,730 --> 00:59:22,612

I'm curious what you've seen and I think

the competitive pressure is way higher in

848

00:59:22,612 --> 00:59:25,153

the US, which also explains why you are.

849

00:59:25,806 --> 00:59:30,467

trying to squeeze even more information

from your data with more complex models.

850

00:59:31,527 --> 00:59:32,347

Yeah.

851

00:59:33,188 --> 00:59:38,249

I think you've described sort of the path

of a lot of data analytics going into a

852

00:59:38,249 --> 00:59:44,211

lot of industries, which is like, the

first thing that lands is there exists

853

00:59:44,211 --> 00:59:46,391

data, let's go collect data.

854

00:59:47,031 --> 00:59:51,153

Let's go summarize data, and then someone

will take that and sell it to the people

855

00:59:51,153 --> 00:59:52,153

that collected the data.

856

00:59:52,153 --> 00:59:54,073

And that's cool.

857

00:59:54,730 --> 00:59:58,791

And I always think the next iteration of

that is taking that data and doing

858

00:59:58,791 --> 01:00:01,232

something useful and deriving insight.

859

01:00:01,492 --> 01:00:09,295

The thing that baseball has done really

well was linking, um, runs to outcomes

860

01:00:09,295 --> 01:00:11,296

that they cared about winning games.

861

01:00:11,296 --> 01:00:11,636

Right.

862

01:00:11,636 --> 01:00:14,397

It's like you increase your runs, you win

games.

863

01:00:14,458 --> 01:00:15,978

You decrease your runs, you lose games.

864

01:00:15,978 --> 01:00:16,178

Right.

865

01:00:16,178 --> 01:00:17,859

It's pretty simple.

866

01:00:18,459 --> 01:00:23,381

Um, so this is where it's, you know, even

I'm having trouble right now too.

867

01:00:23,381 --> 01:00:24,621

It's, it's, um,

868

01:00:25,794 --> 01:00:30,197

for basketball, like you shoot slightly

higher percentage, you're gonna score a

869

01:00:30,197 --> 01:00:33,140

little more, but does that actually

increase your wins?

870

01:00:33,460 --> 01:00:33,781

Yeah.

871

01:00:33,781 --> 01:00:37,524

And that's really tough to do in the

context of five on five.

872

01:00:37,524 --> 01:00:42,188

If you're talking about rugby, you got, is

it nine on nine or is it 11?

873

01:00:42,568 --> 01:00:44,150

It's 15.

874

01:00:44,150 --> 01:00:45,952

15, right?

875

01:00:45,952 --> 01:00:48,434

Classic European rugby is 15, yeah.

876

01:00:48,434 --> 01:00:51,016

Like the World Cup that's happening right

now.

877

01:00:51,016 --> 01:00:52,918

So if you got 15 players, like...

878

01:00:52,918 --> 01:00:55,059

What's the impact of replacing one player?

879

01:00:55,059 --> 01:00:58,100

And it starts getting a lot harder to

measure.

880

01:01:00,682 --> 01:01:07,906

So I do think that there's, so even from

where I'm sitting, it seems like there's a

881

01:01:07,906 --> 01:01:12,368

lot of hype around collecting data and

just visualizing data and understanding

882

01:01:12,368 --> 01:01:13,448

what's there.

883

01:01:13,669 --> 01:01:20,052

And people hope that a cool result will

come out by just looking at data, which I

884

01:01:20,052 --> 01:01:22,653

do hope that it will happen.

885

01:01:22,870 --> 01:01:30,174

But as soon as the lowest line fruit is

picked, the next thing has to be models.

886

01:01:30,174 --> 01:01:32,836

And yeah.

887

01:01:32,836 --> 01:01:34,317

Yeah, exactly.

888

01:01:34,317 --> 01:01:35,178

Completely agree with that.

889

01:01:35,178 --> 01:01:40,241

And I think it's for now, it's still a bit

too early for Europe for now.

890

01:01:40,241 --> 01:01:44,304

It's going to come, but we can have

already really good success by just doing

891

01:01:44,304 --> 01:01:46,926

stat desk, because a lot of people are

just not doing it.

892

01:01:46,926 --> 01:01:52,889

And so recruiting and training just based

on gut instinct.

893

01:01:53,026 --> 01:01:56,666

which is not useless but can definitely be

improved.

894

01:01:57,947 --> 01:02:02,548

You know, one of the other things about

sport that's really difficult is that,

895

01:02:03,308 --> 01:02:06,709

when we talk about models, we assume

everything is normally distributed.

896

01:02:06,709 --> 01:02:12,571

We assume that the central limit there and

holds or the law of large numbers and all

897

01:02:12,571 --> 01:02:15,552

these things are average.

898

01:02:15,872 --> 01:02:20,233

When you talk about the highest level of

sport, you're talking about the tail end

899

01:02:20,233 --> 01:02:22,073

of the tail end of the tail end.

900

01:02:22,386 --> 01:02:23,967

And that is not normal.

901

01:02:23,967 --> 01:02:26,129

And I'm seeing somebody to model.

902

01:02:26,129 --> 01:02:28,470

This is where, like I said, I'm really

excited.

903

01:02:28,470 --> 01:02:34,115

It's not everywhere, but a lot of times we

do assume that's normal normality

904

01:02:34,115 --> 01:02:34,955

assumptions.

905

01:02:34,955 --> 01:02:38,678

And I don't think they're normal.

906

01:02:38,678 --> 01:02:42,021

And I think if we actually model that

properly, we're going to actually see some

907

01:02:42,021 --> 01:02:43,261

better results.

908

01:02:44,723 --> 01:02:46,384

But it's early days for me.

909

01:02:46,384 --> 01:02:47,004

So.

910

01:02:48,246 --> 01:02:49,766

Yeah, it's actually a good point.

911

01:02:49,766 --> 01:02:50,506

Yeah.

912

01:02:50,506 --> 01:02:55,368

I hadn't thought of that, but yeah, it

definitely makes sense because then you

913

01:02:55,368 --> 01:02:59,870

get to scenarios which are really the

extreme by definition, because even the

914

01:02:59,870 --> 01:03:05,192

people you have in your sample are

extremely talented people already.

915

01:03:05,192 --> 01:03:14,596

So you cannot model that team the same way

as you would model the football team from

916

01:03:14,596 --> 01:03:15,736

around the corner.

917

01:03:18,366 --> 01:03:18,986

Awesome, Daniel.

918

01:03:18,986 --> 01:03:22,548

Well, it's already been a long time, so I

don't want to take too much of your time.

919

01:03:22,609 --> 01:03:28,173

But before asking you the last two

questions, I'm wondering if you have a

920

01:03:29,034 --> 01:03:34,939

personal anecdote or example to share of a

challenging problem you encountered in

921

01:03:34,939 --> 01:03:39,682

your research or teaching related to

Bayesian stats and how you were able to

922

01:03:39,682 --> 01:03:40,563

navigate through it.

923

01:03:40,563 --> 01:03:41,183

Oh, um...

924

01:03:50,994 --> 01:03:52,380

in teaching.

925

01:03:55,023 --> 01:03:56,193

I don't know.

926

01:03:56,193 --> 01:03:57,061

That one's a tough one.

927

01:03:57,061 --> 01:03:57,964

It's um...

928

01:04:00,830 --> 01:04:02,730

Yeah.

929

01:04:02,730 --> 01:04:02,870

I...

930

01:04:02,870 --> 01:04:03,370

It's a different one.

931

01:04:03,370 --> 01:04:05,691

Okay, here's one of the toughest ones

was...

932

01:04:05,751 --> 01:04:09,092

Just kind of knowing when to give up.

933

01:04:09,312 --> 01:04:18,874

So, going back to a workshop I taught

maybe in like 2013, 2012, around Stan.

934

01:04:19,935 --> 01:04:25,196

I remember someone had walked in with a

laptop that was like a 20-pound laptop.

935

01:04:25,196 --> 01:04:30,737

That was like 10 years old at that point

and was I think running a 32-bit Windows.

936

01:04:31,294 --> 01:04:35,878

and asking for help on how to run Stan on

this thing.

937

01:04:35,878 --> 01:04:39,441

I'm going to try to give up.

938

01:04:42,023 --> 01:04:44,485

Sometimes you just need better tools.

939

01:04:46,968 --> 01:04:48,249

It's a good point.

940

01:04:48,249 --> 01:04:50,531

Yeah, for sure.

941

01:04:50,531 --> 01:04:53,234

That's very true.

942

01:04:53,234 --> 01:04:57,737

That's also something actually they want

to...

943

01:04:57,830 --> 01:05:01,211

a message that I want to give to all the

people using Pimc.

944

01:05:01,452 --> 01:05:08,936

Please install Pimc with Mamba and not

Beep because Mamba is doing things really

945

01:05:08,936 --> 01:05:13,698

well, especially with the compiler, the C

compiler, and that will just make your

946

01:05:13,698 --> 01:05:15,199

life way easier.

947

01:05:15,199 --> 01:05:18,000

So I know we repeat that all the time.

948

01:05:18,000 --> 01:05:19,481

It's in the readme.

949

01:05:19,521 --> 01:05:25,164

It's in the readme of the workshops we

teach at Pimc Labs, and yet people still

950

01:05:25,164 --> 01:05:26,024

install

951

01:05:27,638 --> 01:05:30,679

So if you really have to install with

peep, then do it.

952

01:05:30,679 --> 01:05:32,600

Otherwise, just use MambaForge.

953

01:05:32,600 --> 01:05:33,200

It's amazing.

954

01:05:33,200 --> 01:05:36,901

You're not going to have any problems and

it's going to make your life easier.

955

01:05:37,562 --> 01:05:42,684

There is a reason why all the Pimc card

developers ask you that as a first

956

01:05:42,684 --> 01:05:47,485

question anytime you tell them, so I have

a problem with my Pimc install.

957

01:05:48,126 --> 01:05:49,346

Did you use Mamba?

958

01:05:50,367 --> 01:05:55,169

So yeah, it was just a general public

announcement that you made me think about

959

01:05:55,169 --> 01:05:56,809

that Daniel, thanks a lot.

960

01:05:58,335 --> 01:06:02,101

Okay, before letting you go, I'm gonna ask

you the last two questions I ask every

961

01:06:02,101 --> 01:06:03,783

guest at the end of the show.

962

01:06:04,685 --> 01:06:09,513

First one, if you had unlimited time and

resources, which problem would you try to

963

01:06:09,513 --> 01:06:10,173

solve?

964

01:06:12,338 --> 01:06:21,305

My, I would try to solve the income

disparity in the US and what that gets

965

01:06:21,305 --> 01:06:22,005

you.

966

01:06:23,107 --> 01:06:24,968

I'm thinking mostly health insurance.

967

01:06:24,968 --> 01:06:28,311

I think it's really bad here in the US.

968

01:06:30,633 --> 01:06:35,157

You just need resources to have health

insurance and it should be basic.

969

01:06:35,157 --> 01:06:36,358

It's a basic necessity.

970

01:06:36,358 --> 01:06:40,841

So working on some way to fix that would

be awesome.

971

01:06:41,562 --> 01:06:44,404

unlimited time and energy.

972

01:06:44,404 --> 01:06:47,586

Yeah, I mean, definitely a great answer.

973

01:06:47,586 --> 01:06:52,870

First one, we get that, but totally agree,

especially from a European perspective,

974

01:06:52,870 --> 01:06:58,714

it's always something that looks really

weird to you when you're coming to the US.

975

01:06:58,714 --> 01:07:00,495

It's super complicated.

976

01:07:00,495 --> 01:07:01,576

Also, yeah.

977

01:07:01,576 --> 01:07:07,340

One of the things, like, working in pharma

was like, realizing that a lot of the R&D

978

01:07:07,340 --> 01:07:09,281

budget is coming from

979

01:07:10,302 --> 01:07:15,005

you can call it overpayment from the

American system.

980

01:07:15,005 --> 01:07:20,869

And so if you still want new drugs that

are better, it's got to come from

981

01:07:20,869 --> 01:07:22,971

somewhere, but not sure where.

982

01:07:23,171 --> 01:07:23,872

It's a tough problem.

983

01:07:23,872 --> 01:07:25,333

Yeah, yeah, yeah.

984

01:07:25,333 --> 01:07:26,373

I know for sure.

985

01:07:27,775 --> 01:07:33,859

And second question, if you could have

dinner with a great scientific mind, dead,

986

01:07:33,859 --> 01:07:36,401

alive, or fictional, who would it be?

987

01:07:37,802 --> 01:07:40,384

That one, like I thought about this for a

while.

988

01:07:40,384 --> 01:07:44,227

And you know, the normal cast of

characters came up, Andrew, Delman, Bob

989

01:07:44,227 --> 01:07:46,129

Carpenter, Matt Hoffman.

990

01:07:46,129 --> 01:07:51,053

But the guy that I would actually sit down

with is Sean Frayn.

991

01:07:51,253 --> 01:07:53,675

You probably haven't heard of him.

992

01:07:54,076 --> 01:07:55,837

He's an American inventor.

993

01:07:57,098 --> 01:08:02,723

He has a company called Looking Glass

Factory that does 3D holographic displays

994

01:08:03,003 --> 01:08:04,864

without the need of a headset.

995

01:08:06,703 --> 01:08:10,891

He happens to have been my college

roommate and my big brother and my

996

01:08:10,891 --> 01:08:13,115

fraternity at New Delta at MIT.

997

01:08:13,115 --> 01:08:16,661

And I haven't caught up with him in a long

time.

998

01:08:16,661 --> 01:08:19,265

So that's a guy I would go sit down with.

999

01:08:21,858 --> 01:08:25,080

That sounds like a very fun dinner.

Speaker:

01:08:25,721 --> 01:08:27,802

Well, thanks a lot, Daniel.

Speaker:

01:08:28,383 --> 01:08:30,104

This was really, really cool.

Speaker:

01:08:30,385 --> 01:08:35,769

I'm happy because I had so many questions

for you and so many different topics, but

Speaker:

01:08:35,889 --> 01:08:37,611

we managed to get that in.

Speaker:

01:08:37,611 --> 01:08:40,213

So yeah, thank you so much.

Speaker:

01:08:41,074 --> 01:08:45,918

As usual, I put resources in a link to

your website in the show notes for those

Speaker:

01:08:45,918 --> 01:08:47,299

who want to dig deeper.

Speaker:

01:08:47,539 --> 01:08:50,741

Thanks again, Daniel, for taking the time

to be on this show.

Speaker:

01:08:50,972 --> 01:08:54,849

You had to be easy change your predictions

Previous post
Next post