Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!
Getting Daniel Lee on the show is a real treat — with 20 years of experience in numeric computation; 10 years creating and working with Stan; 5 years working on pharma-related models, you can ask him virtually anything. And that I did…
From joint models for estimating oncology treatment efficacy to PK/PD models; from data fusion for U.S. Navy applications to baseball and football analytics, as well as common misconceptions or challenges in the Bayesian world — our conversation spans a wide range of topics that I’m sure you’ll appreciate!
Daniel studied Mathematics at MIT and Statistics at Cambridge University, and, when he’s not in front of his computer, is a savvy basketball player and… a hip hop DJ — you actually have his SoundCloud profile in the show notes if you’re curious!
Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !
Thank you to my Patrons for making this episode possible!
Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas and Luke Gorrie.
Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag 😉
Links from the show:
- Daniel on Linkedin: https://www.linkedin.com/in/syclik/
- Daniel on Twitter: https://twitter.com/djsyclik
- Daniel on GitHub: https://github.com/syclik
- Daniel’s DJ profile: https://soundcloud.com/dj-syclik
- LBS #91, Exploring European Football Analytics, with Max Göbel: https://learnbayesstats.com/episode/91-exploring-european-football-analytics-max-gobel/
- LBS #85, A Brief History of Sports Analytics, with Jim Albert: https://learnbayesstats.com/episode/85-brief-history-sports-analytics-jim-albert/
- Daniel about GPTs in Probabilistic Programming: https://www.youtube.com/watch?v=KUuSwLMFPHM
- LBS #50, Ta(l)king Risks & Embracing Uncertainty, with David Spiegelhalter: https://learnbayesstats.com/episode/50-talking-risks-embracing-uncertainty-david-spiegelhalter/
- LBS #76, The Past, Present & Future of Stan, with Bob Carpenter: https://learnbayesstats.com/episode/76-past-present-future-of-stan-bob-carpenter/
- LBS #27, Modeling the US Presidential Elections, with Andrew Gelman & Merlin Heidemanns: https://learnbayesstats.com/episode/27-modeling-the-us-presidential-elections-with-andrew-gelman-merlin-heidemanns/
- LBS #20, Regression and Other Stories, with Andrew Gelman, Jennifer Hill & Aki Vehtari: https://learnbayesstats.com/episode/20-regression-and-other-stories-with-andrew-gelman-jennifer-hill-aki-vehtari/
Abstract
Our guest this week, Daniel Lee, is a real Bayesian allrounder and will give us new insights into a lot of Bayesian applications.
Daniel got introduced to Bayesian stats when trying to estimate the failure rate of satellite dishes as an undergraduate student. He was lucky to be mentored by Bayesian greats like David Spiegelhalter, Andrew Gelman and Bob Carpenter. He also sat in on reading groups at universities where he learned about cutting edge developments – something he would recommend anyone to really dive deep into the matter.
He used all this experience working on Pk/Pd (Pharmacokinetics/ Pharmacodynamics) models. We talk about the challenges in understanding individual responses to drugs based on the speed with which they move through the body. Bayesian statistics allows for incorporating more complexity into those models for more accurate estimation.
Daniel also worked on decision making and information fusing problems for the military, such as identifying a plane as friend or foe through the radar of several ships.
And to add even more diversity to his repertoire, Daniel now also works in the world of sports analytics, another popular topic on our show. We talk about the state of this emerging field and its challenges.
Finally, we cover some STAN news, discuss common problems and misconceptions around Bayesian statistics and how to resolve them.
Transcript
This is an automatic transcript and may therefore contain errors. Please get in touch if you’re willing to correct them.
Transcript
Let me show you how to be a good peasy and
change your production.
2
Getting Daniel Lee on the show is a real
treat.
3
With 20 years of experience in numeric
computation, 10 years creating and working
4
with Stan, 5 years working on
pharma-related models, you can ask him
5
virtually anything.
6
And that I did, my friends.
7
From joint models for estimating oncology
treatment efficacy to PKPD models, from
8
data fusion for US Navy applications to
baseball and football analytics.
9
as well as common misconceptions or
challenges in the Bajan world, our
10
conversation spans a wide range of topics
that I am sure you will appreciate.
11
Daniel studied mathematics at MIT and
statistics at Cambridge University, and
12
when he's not in front of his computer,
he's a savvy basketball player and a
13
hip-hop DJ.
14
You actually have his Soundcloud profile
in the show notes if you're curious.
15
This is Learning Bajan Statistics, episode
96.
16
recorded October 12, 2023.
17
Hello, my dear patients.
18
Some of you may know that I teach
workshops at Pimesy Labs to help you
19
jumpstart your basic journey, but
sometimes the fully live version isn't a
20
fit for you.
21
So we are launching what we call the
Guided Learning Path.
22
This is an extensive library of video
courses handpicked from our live
23
workshops.
24
that unlocks asynchronous learning for
you.
25
From A-B testing to Gaussian processes,
from hierarchical models to causal
26
inference, you can explore it all at your
own pace, on your own schedule, with
27
lifetime access.
28
If that sounds like fun and you too want
to become a vision modeler, feel free to
29
reach out at alex.andorra at
primec-labs.com.
30
And now, let's get nerdy with Daniel Lee.
31
Daniel Lee, welcome to Learning Bayesian
Statistics.
32
Hello.
33
Yeah, thanks a lot for taking the time.
34
I'm really happy to have you on the show
because I've followed your work for quite
35
a long time now and I've always thought
that it'd be fun to have you on the show.
36
And today was the opportunity.
37
So thank you so much for taking the time.
38
And so let's start writing.
39
What are you doing?
40
How would you define the work you're doing
nowadays?
41
And what are the topics you are
particularly interested in?
42
Yeah, so I just joined Zealous Analytics
recently.
43
They're a company that does sports
analytics, mostly for professional teams.
44
Although they're expanding to amateur
college teams as well.
45
And what I get to do is...
46
look at data and try to project how well
players are going to do in the future.
47
That's the bulk of what I'm focused on
right now.
48
That sounds like fun.
49
Were you already a sports fan or is it
that mainly you're a good modeler and that
50
was a fun opportunity that presented
itself?
51
Yeah, I think both are true.
52
I grew up playing a lot of basketball.
53
I coached a little bit of basketball.
54
Um, yeah.
55
So I feel like I know the subject matter
of basketball pretty well.
56
The other sports I know very little about,
but, um, uh, you know, combine that with
57
being able to model data.
58
It's actually a really cool opportunity.
59
Yeah.
60
And actually, how did you end up doing
what you're doing today?
61
Because.
62
I know you've got a very, very senior
path.
63
So I'm really interested also in your kind
of origin story because, well, that's an
64
interesting one.
65
So how did you end up doing what you're
doing today?
66
Yeah.
67
So sports ended up happening because I
don't know, it actually started through
68
stand.
69
I didn't really have...
70
an idea that I'd be working in sports
full-time professionally until this
71
opportunity presented itself.
72
And what ended up happening was I met the
founders of Zealous Analytics
73
independently about a decade ago and the
company didn't start till 2019.
74
So, you know, met them.
75
Luke was at Harvard.
76
Dan was at NYU and Doug at the time was
going to the Dodgers.
77
And I talked to them independently about
different things and, you know, fast
78
forward about 10 years and I happened to
be free.
79
This opportunity came up.
80
They're using Stan inside.
81
They're using a bunch of other stuff too,
but it was a good time.
82
And do you remember how you first got
introduced to Bayesian methods and also
83
why they stuck with you?
84
Yeah.
85
So there are actually two different times
that I got introduced to Bayesian methods.
86
The first was I was working in San Diego.
87
This is after my undergraduate degree.
88
We were working on trying to estimate when
hardware would fail and we're talking
89
about modems and things that go with
satellite dishes.
90
So they happen to be somewhere that's hard
to
91
spread across and when one of those pieces
go down, it's actually very costly to
92
repair, especially when you don't have a
part available.
93
So we started using graphical models and
using something called Weka to build
94
graphical models and do Bayesian
computation.
95
This was all done using graphical models
and it was all discrete.
96
That was the first time I got introduced
to Bayesian statistics.
97
It was very simple at the time.
98
What ended up happening after that was I
went to grad school at Cambridge, did part
99
three mathematics and ended up taking all
the stats courses.
100
And that's where I really saw Bayesian
statistics, learned MCMC, learned how bugs
101
was built using the graphical models and
conjugacy.
102
And then...
103
Yeah, so that was the real introduction to
Bayesian modeling.
104
Yeah.
105
And actually I'm curious because,
especially in any content basically where
106
we talk about, so how do you end up doing
what you're doing and stuff like that,
107
there is kind of a hindsight
108
it looks obvious how you ended up doing
what you're doing.
109
And that almost seems easy.
110
But I mean, at least in my case, that
wasn't, you know, it's like you always
111
have obstacles along the way and so on,
which is not necessarily negative, right?
112
We have that really good saying in French
that says basically, what's the obstacle,
113
the obstacles in front of you makes you
114
grow, basically.
115
It's a very hard thing to translate, but
basically that's the substance.
116
So yeah, I'm just curious about your own
path.
117
How senior was it to get to where you are
right now?
118
I've always believed in learning from
failures or learning from experiences
119
where you don't succeed.
120
That's where you gain the most knowledge.
121
That's where you get to learn where your
boundary is.
122
If you want to know about the path to how
I became where I'm at now, let's see.
123
I guess I could go all the way back to
high school.
124
I grew up just outside of Los Angeles.
125
In high school...
126
I had a wonderful advisor named Sanzha
Kazadi.
127
He was a PhD student at Caltech and he ran
a research program for high school kids to
128
do basic research.
129
So starting there, I learned to code and
was working on the traveling salesman
130
problem.
131
From there, I went to MIT, talking about
failures.
132
I tried to be a physics major going in.
133
I failed physics three times in the first
year, so I couldn't.
134
I ended up being a math major.
135
And it was math with computer science, so
it was really close to a theoretical
136
computer science degree, doing some
operational research as well.
137
At the end of MIT, I wasn't doing so well
in school.
138
I was trying to apply to grad school, and
that wasn't happening.
139
Got a job in San Diego.
140
MIT alum hired me.
141
That's where I started working for three
and a half years in software, a little bit
142
of computation.
143
So a lot of it was translating algorithms
to production software, working on
144
algorithms and went through a couple of
companies with the same crew, but we just
145
kind of bounced around a little bit.
146
At the end of that, I ended up going back
to Cambridge for...
147
a one year program called part three
mathematics.
148
It's also a master's degree.
149
I got there not knowing anything about
Cambridge.
150
I didn't do enough research, obviously.
151
For the American viewers, people, the
system is completely different.
152
There's no midterms, no nothing.
153
You have three trimesters.
154
You take classes in each of them and you
take two weeks of exams at the end.
155
And that determines your fate.
156
And, um, I got to Cambridge and I couldn't
even understand anything in the syllabus
157
other than the stuff in statistics.
158
Mind you, I hadn't done an integral in
three years, right?
159
Integral derivative.
160
I didn't know what the normal distribution
was.
161
And I go to Cambridge.
162
Those are the only things I can read.
163
So I'm teaching myself.
164
Um,
165
measure theory while learning all these
new things that I've never seen and
166
managed to squeak out passing.
167
So happy.
168
At the end of that, I asked David
Spiegelhalter, who happened to just come
169
back to Cambridge, that was his first year
back in the stats department, who I should
170
talk to.
171
This is, so when I say I learned bugs,
he's, he had a course on
172
applied Beijing statistics, which was
taught in wind bugs.
173
And he would literally show us which
buttons to click and in which order, in
174
order for it not to crash.
175
So that was fun.
176
But he told me, he told me I should talk
to Andrew Gelman.
177
Um, so I ended up, uh, talking to Andrew
Gelman and working with Andrew from 2009
178
to 2016 and that's how I really got into
Beijing stats.
179
Um,
180
After Cambridge, I knew theory.
181
I hadn't seen any data.
182
Working for Andrew, I saw a bunch of data
and actually how to really work with data.
183
Since then I've run a startup.
184
We try to take Stan.
185
So Stan's an open source probabilistic
programming language.
186
In 2017, a few of us thought there was a
good opportunity for making a business
187
around it.
188
very much like time C labs.
189
And, you know, we try to make a horizontal
platform for it.
190
And at that time, there wasn't enough
demand.
191
So we pivoted and ended up estimating
models for writing very complicated models
192
and estimating things for the farm
industry.
193
And then since then I've like I left the
company in 2020 at the end of 2021.
194
I consulted for a bit, just random
projects, and then ended up with Celus.
195
So that's how I got to today.
196
Yeah.
197
Man.
198
Yeah, thanks a lot for that exhaustive
summary, I'd say, because that really
199
shows how random usually paths are, right?
200
And I find that really inspiring also for
people who are a bit upstream in their
201
carrier path.
202
could be looking at you as a role model
and could be intimidated by thinking that
203
you had everything figured out from when
you were 18 years old, right?
204
Just getting out of high school, which was
not the case from what I understand.
205
And that's really reassuring and
inspiring, I think, for a lot of people.
206
Yeah, definitely not.
207
I could tell you going to career fairs at
the end of my undergraduate degree,
208
people will look at my math degree and not
even really look at my resume.
209
Because my GPA was low, my grades were bad
as a student, and also, who needs a bad
210
mathematician?
211
That makes no sense anywhere.
212
So that limited what I was doing, but at
the end it all worked out.
213
Yeah, yeah, yeah.
214
Now you made an agreement in a way, our
path, our seminar, except for me, that was
215
a GPA in business school.
216
So business school and political science.
217
Political science, I did have decent
grades.
218
Business school, it really depended on
what the course was about.
219
Because when I was not interested in the
course, yeah, that showed.
220
For sure, that showed in the GPA.
221
But yeah, and I find that also super
interesting because in your path, there is
222
also so many amazing
223
people you've met along the way and that
it seems like these people were also your
224
mentors at some point.
225
So Yeah, do you want to talk a bit more
about that?
226
Yeah, I've um, I've been really fortunate
You know as I was going through so Not you
227
know, I haven't had very many formal
mentors that were great and by that I mean
228
like
229
advisors that were assigned to me through
schools.
230
They tend to see what I do and discount my
abilities because of my inability to do
231
really well at school.
232
So that's what it is.
233
But there were a bunch of people that
really did sort of shape my career.
234
The, you know, working for Andrew Gelman
was great.
235
He's, um, he trusted me.
236
Like he, for me, he was a really, he
trusted me with a lot.
237
Right.
238
So he's, he was able to, um, just set me
loose on a couple of problems to start.
239
And he never micromanages.
240
So he just let me go for some that's a
really difficult place to be, um, without
241
having guidance in a difficult problem.
242
But.
243
For someone like me, that was absolutely
fine and encouraging.
244
You know, and working with Andrew and I
worked really closely with Bob Carpenter
245
for a long time and that was really great
because he has such a depth of knowledge
246
and also humility that, I don't know,
it's, it's fun working with Bob.
247
Some of the other times that I've really
gotten to grow in my career, we're sitting
248
in on some amazing reading groups.
249
So there are two that come to mind.
250
At Columbia, Dave Bly runs a reading group
for his group and got to sit in.
251
And those are phenomenal because they
actually go deep into papers and really,
252
really get at the content of the paper,
what it's doing, what the research is.
253
trying to infer what's going on, where the
research is going next.
254
But that really helped expand my horizon
for things that I wasn't seeing while
255
working in Andrew's group.
256
So it was just, you know, much more
machine learning oriented.
257
And in a similar vein at Cambridge, I was
able to sit in on Zubin Karamani's group.
258
Don't know why he let me, but he let me
just sit in.
259
I was group reading groups and
260
He had a lot of good people there at the
time.
261
That was when Carl Rasmussen was there
working on his book.
262
Um, David Knowles, uh, I don't know who
else, but just sitting there reading about
263
these papers, reading these techniques,
people presenting their own work inside
264
the reading group.
265
Um, yeah, my encouragement would be if you
have a chance to go sit in on reading
266
groups, go join them.
267
It's actually a good way, especially if
it's not in your.
268
area of focus.
269
It's a good way to learn and make
connections to literature that otherwise
270
would be very hard to read on your own.
271
Yeah, I mean, completely agree with that.
272
And yeah, it feels like a dream team of
mentors you've had.
273
I'm really jealous.
274
Like David Spiegelhalter, Andrew Gellman,
Bob Carpenter, all those people.
275
It's absolutely amazing.
276
And I've had the chance of interviewing
them on the podcast.
277
So I will definitely link to those
episodes in the show notes.
278
And yeah, completely agree.
279
Today, I would definitely try and do it
with Andrew, because I've talked with him
280
quite a lot already.
281
And yeah, it's really inspiring.
282
And that's really awesome.
283
And yeah, I completely agree that in
general, that's something that I'm trying
284
to do.
285
And that's also where I started the
podcast in a way.
286
Surrounding yourself with smarter people
than you is usually a good thing.
287
good way to go.
288
And definitely me, I've had the chance
also to have some really amazing mentors
289
along my way.
290
People like Ravin Kumar, Thomas Vicky,
Osvaldo Martin, Colin Carroll, Austin
291
Rushford.
292
Well, Andrew Ganneman also with everything
he's produced.
293
And yeah, Adrian Zabolt also absolutely
brilliant.
294
Luciano Paz.
295
All these people basically in the times
he...
296
world who helped me when I was really
starting and not even knowing about Git
297
and taking a bit of their free time to
review my PRs and help me along the way.
298
That's just really incredible.
299
So yeah, what I encourage people to do
when they really start in that domain is
300
much more than trying to find a...
301
an internship that shines on us, trying to
really find a community where you'll be
302
surrounded by smart and generous people.
303
That's usually going to help you much more
than a name on the CV.
304
Absolutely.
305
And so actually, I'd like to talk a bit
about some of the Pharma-related models
306
you've worked on.
307
You've worked on so many topics.
308
It's really hard to interview you.
309
But a kind of model I'm really curious
about, also because we work on that at
310
labs from time to time, is farmer-related
models.
311
And in particular, can you explain how
Bayesian methods are used in estimating
312
the efficacy of oncology treatments?
313
And also, what are PKPD models?
314
Yeah, let's start with PKPD models.
315
So PKPD stands for pharmacometric
pharmacodynamic models.
316
And these models, the pharmacokinetics
describe, so we take drug and it goes into
317
the body.
318
You can model that using, you know, you
know how much drug goes in the body.
319
And then at some point it has to exit the
body through.
320
absorption through something, right?
321
So your liver can take it out.
322
It'll go into your bloodstream, whatever.
323
That's the kinetics part.
324
You know that the drug went in and it
comes out.
325
So you can measure the blood at different
times.
326
You can measure different parts of the
body to get an estimate of how much is
327
left.
328
You can estimate how that works.
329
The pharmacodynamic part is the more
difficult part.
330
So each person responds differently to the
drug depending on what's inside the drug
331
and how much concentration is in the body.
332
You and I could take the same dose of
ibuprofen and we're going to ask each
333
other how you feel and that number is, I
don't know, is it on a scale of 1 to 10?
334
You might be saying a 3, I might be saying
a 4 just based on what we feel.
335
There are other measurements there that...
336
sometimes you can measure that's more
directly tied to the mechanism, but most
337
of the time it's a few hops away from the
actual drug entering the bloodstream.
338
So the whole point of pharmacokinetic,
pharmacodynamic modeling is just measuring
339
drug goes in, drug goes out, what's the
effect.
340
trials and in design of how much dose to
give people.
341
So if you give someone double the dosage,
are they actually gonna feel better?
342
Is the level of drug gonna be too high
such that there are side effects, so on
343
and so forth.
344
The way Bayesian methods play out here is
that if we, you know, just
345
really broad step.
346
If you take a step back, the last
generation of models, assume that everyone
347
came from, you were trying to estimate a
population mean for all these things.
348
So you're trying to take individuals and
individual responses and try to get the
349
mean parameters of a, usually a
parameterized model of how the kinetics
350
works and then the dynamics works.
351
it'd be better if we had hierarchical
models that assumed that there was a, you
352
know, a mean but each person's individual
and that could describe the dynamics for
353
each person a little better than it can
for just using plugging in the overall.
354
So to do that, you kind of ended up
needing Bayesian models.
355
But on top of that, the other reason why
Bayesian models are really popular for
356
this stuff right now is that...
357
The people that study these models have a
lot of expertise in how the body works and
358
how the drugs work.
359
And so they've been wanting to incorporate
more and more complexity into the models,
360
which is very difficult to do inside the
setting of certain packages that limit the
361
flexibility.
362
There's a lot of flexibility that you can
put in, but there's always a limit.
363
to that flexibility.
364
And that's where Stan and other tools like
PyMC are coming into play now, not just
365
for the Bayesian estimates, but really for
the ability to create models that are more
366
complex.
367
And that are generative in particular?
368
These are, because people are trying to
really understand
369
for these types of studies, they're trying
to understand what happens.
370
Like, what's the best dosage to give
people?
371
Should it be scaled based on the size of
the human?
372
What happens?
373
You know, it's a lot of what happens.
374
Can you characterize what's going to
happen if you give it to a larger
375
population?
376
You know, you've seen some variability
inside the smaller trial.
377
What happens next?
378
Yeah, fascinating.
379
And so it seems to me that it's kind of a
really great use case for patient stats,
380
right?
381
Because, I mean, you really need a lot of
domain knowledge here.
382
You want that in the model.
383
You probably also have good ideas of
priors and so on.
384
But I'm wondering what are the main
385
challenges when you work on that kind of
model?
386
The main challenges, I think, some of the
challenges have to do with at least when I
387
was working there.
388
So mind you, I didn't work directly for a
pharma company.
389
We had a startup where we were building
these models and selling to pharma.
390
One of the issues is that there's a lot of
historic...
391
very good reasons for using older tools.
392
They don't move as fast, right?
393
So you've got regulators, you've got
people trying to be very careful and
394
conservative.
395
So trying out new methods on the same
data, if it doesn't produce results that
396
they're used to, it's a little harder to
do there than it is, let's say in sports,
397
right?
398
In sports, no one's gonna die if I predict
something wrong next year.
399
If you use a model that's completely
incompatible with the data in pharma and
400
it gives you bad results, bad things do
happen sometimes.
401
So anyway, things move a little slower.
402
The other thing is that most people are
not trained in understanding Bayesian
403
stats yet.
404
You know, I do think that there's a
difference...
405
in understanding Bayesian statistics from
a theoretic, like on paper point of view,
406
and actually being a pragmatic modeler of
data.
407
Um, and right now I think there's a
turning point, right?
408
I think the world is catching up and the
ability to model is spreading, uh, a lot
409
wider and the, um,
410
So anyway, I think that's part of that is
happening in farm as well.
411
Yeah, yeah, for sure.
412
Yeah, these kind of models, I really find
them fascinating because they are both
413
quite intricate and complicated from a
statistical standpoint.
414
So you really learn a lot when you work on
them.
415
And at the same time, they are extremely
useful and helpful.
416
And usually, they are about extremely
fascinating projects that have a deep
417
impact.
418
on people, basically it's helping directly
people who I find them absolutely
419
fascinating.
420
I mean, I can tell you that specifically,
the place where I had difficulty working
421
in PTA-PD models was that I didn't
understand the biology enough.
422
So there are these terms, these constants,
these rate constants that describe
423
elimination of the drug through the liver.
424
And because I don't
425
don't know biology well enough, I don't
know what's a reasonable range.
426
And, you know, people that study the
biology know this off the back, off the
427
top of their head because they've studied
the body, but they can't, you know, most
428
aren't able to work with a system like
STAND well enough to write the model down.
429
And it's that mismatch that makes it
really tough because then, you know,
430
there's...
431
Some in some of the conversations we had
in that world, it's, you know, why aren't
432
you using a Jefferies prior?
433
Why aren't you using a non-informative
prior?
434
But on the flip side, it's like, if that
rate constant is 10 million, is that
435
reasonable?
436
No, it's not.
437
It has to be between like zero and one.
438
So we should be, you know, like for me,
it's if we put priors there, that limit
439
that, that makes the modeling side of it a
lot easier, but you know, as someone that
440
didn't understand the biology well enough
to make those claims, it made the modeling
441
much, much more difficult and harder to
explain as well.
442
Yeah, yeah, yeah.
443
Yeah, definitely.
444
And the biology of those models is
absolutely fascinating, but really, really
445
intriguing.
446
And also, you've also worked on something
that's called data fusion for US Navy
447
applications.
448
So that sounds very mysterious.
449
How did Bayesian statistics contribute to
these projects?
450
And what were some of the challenges you
faced?
451
Unfortunately, I didn't know Bayesian
stats at the time.
452
This was when I first started working.
453
But, you know, data fusion's actually...
454
We should have used Bayesian stats.
455
If I was working on a problem now, it
should be done with Bayesian stats.
456
The...
457
Just the problem in a nutshell, if you
imagine you have an aircraft carrier, it
458
can't move very fast, and what it has is
about a dozen ships around it.
459
All of them have radars.
460
All of them point at the same thing.
461
If you're sitting on the aircraft carrier
trying to make decisions about what's
462
coming at you, what to do next.
463
If there's a single plane coming at you,
that's one thing.
464
If all the 12 ships around you, you know,
hit that same thing with the radar and it
465
says that there are 12 things coming at
you because things are slightly jittered,
466
that's bad news, right?
467
So, you know, if they're not identifying
themselves.
468
So the whole problem is, is there enough
information there where you can...
469
accurately depict what's happening based
on multiple pieces of data.
470
Hmm.
471
Okay.
472
Yeah, that sounds pretty fun.
473
And indeed, yeah, lots of uncertainty.
474
So, and I'm guessing you don't have a lot
of data.
475
And also, it's the kind of experiments you
cannot really remake and remake.
476
So, your patient stats would be helpful
here, I'm guessing.
477
Yeah, it's, it's always the edge cases
that are tough, right?
478
It's, if the, if the, if the plane or the
ship that's coming at you,
479
says who they are, identifies themselves,
and follows normal protocol.
480
It's an easy problem, like you have the
identifier, but it's when that stuff's
481
latent, right?
482
People hide it intentionally.
483
Then you have to worry about what's going
on.
484
The really cool thing there was a guy I
worked for, Clay Stannick, had come up
485
with a way to
486
of each of the radar pictures and just
stack them on top of each other.
487
If you do that, if you see a high
intensity, then it means that the pictures
488
overlap.
489
And if there's no high intensity, then it
means the pictures don't overlap.
490
And the nice thing is that that's rotation
invariant.
491
So it really just helps with the alignment
problem because everyone's looking at the
492
same picture from different angles.
493
Yeah, yeah, it's super interesting also.
494
I love that.
495
And you haven't had the opportunity to
work again on that kind of models now that
496
you're an Asian expert?
497
No.
498
Well, you've heard it, folks.
499
If you have some model like that who are
entertaining you, feel free to contact him
500
or me, and I will contact him for you if
you want.
501
So actually.
502
I'm curious, you know, in general, because
you've worked with so many people and in
503
so many different fields.
504
I wonder if you picked up some common
misconceptions or challenges that people
505
face when they try to apply vision stats
to real world problems and how you think
506
we can overcome them.
507
Yeah, that's an interesting question.
508
I think working with Dan, well, yeah, I
think the common error is that we don't
509
build our models complex enough.
510
They don't describe the phenomenon well
enough to really explain the data.
511
And I think that's where, that's the most
common problem that we have.
512
Yeah, the thing that I use the most, that
I get the most mileage out of is actually
513
putting on either a measurement model or
just adding a little more complexity to
514
model and it starts working way better.
515
In pharmacometrics specifically, I
remember we started asking, how do you
516
collect the data?
517
What sort of ways is the measurement
wrong?
518
And we just modeled that piece and put it
into the same
519
parametric forms of the model and
everything started fitting correctly.
520
It's like, cool, I should do that more
often.
521
So yeah, I think if I was to think about
that, that's sort of the thing.
522
The other thing is, I guess people try to
apply Bayesian stats, Bayesian models to
523
everything, and it's not always
applicable.
524
I don't know if you're actually going to
be able to fit a true LLM using MCMC.
525
Like I think that'd be very, very
difficult.
526
Um, so it's okay to not be Bayesian for
that stuff.
527
Yeah.
528
So that's interesting.
529
So nothing about priors or about model
fitting or about model time sampling of
530
the models.
531
No, I mean, they're all related, right?
532
The worst the model fits.
533
So when a model doesn't actually match the
data, at least running in Stan, it tends
534
to.
535
overinflate the amount of time it takes,
the diagnostics look bad.
536
A lot of things get fixed once you start
putting in the right level of complexity
537
to match the data.
538
But you know, that's yeah.
539
I mean, is it MCMC is definitely slower
than running optimization?
540
That's true.
541
Yeah.
542
No, for sure.
543
Yeah, I'm asking because as I'm teaching a
lot, these are recurring themes.
544
I mean, it really depends where people are
coming from.
545
But you have recurring themes where that
can be kind of a difficulty for people.
546
Something I've seen that's pretty common
is understanding the different types of
547
distributions.
548
So prior predictive samples and prior
samples, how do they differ?
549
Posterior samples, post-hereditary
samples, what's the difference between all
550
of that?
551
That's definitely a topic of complexity
that can trigger some difficulty for
552
people.
553
And I mean, I think that's quite normal.
554
I remember personally, it took me a few
months to really understand that stuff
555
when I started learning Baystance.
556
And now with my educational content,
557
decrease that time for people so that they
maybe make the same mistakes as me, but
558
they realize it's faster than I did.
559
That's kind of the objective.
560
Yeah, that's really good.
561
So what other things do you see that
people are struggling with?
562
Or do you have, you know, what are some of
the common themes right now?
563
I mean, priors a lot.
564
priors is extremely common, especially if
people come from the classic machine
565
learning framework, where it's really hard
for them to choose a prior.
566
And actually something I've noticed is two
ways of thinking about them that allows
567
them to kind of be less anxious about
choosing a prior.
568
which is one, making them realize that
having flat priors doesn't mean not having
569
priors.
570
And so the fact that they were using flat
priors before by default in a class
571
equalization regression, for instance,
that's a prior.
572
That's already an assumption.
573
So why would you be less comfortable
making another assumption, especially if
574
it's more warranted in that case?
575
So.
576
Basically trying to see these idea of
priors along a slider, you know, a
577
gradient where you would have like the
extreme left would be the completely flat
578
priors, which lead to a completely overfit
model that has a lot of variance in the
579
predictions.
580
And then at the other end of the slider,
extreme right would be the completely
581
biased model where your priors would
basically be, you know, either a point or
582
completely outside of
583
the realm of the data and then you cannot
update, basically.
584
But that would be a completely underfit
model.
585
So in a way, the priors are here to allow
you to navigate that slider.
586
And why would you always want to be to the
extreme left of the slider, right?
587
Because in the end, you're already making
a choice.
588
So why not thinking a bit more
exhaustively and clearly about the choice,
589
explicitly about the choices you're
making.
590
Yeah, that already usually helps them to
make them feel less guilty about choosing
591
prior.
592
So that's interesting.
593
Yeah, absolutely.
594
And so to go on that point a little bit,
that's what I'm trying to say with the
595
complexity of the model.
596
It's like, if we just assume normal things
a lot of times, but sometimes things
597
aren't normal.
598
There's more variance than normal.
599
So.
600
making something a t-distribution
sometimes fixes it.
601
Just understanding the prior predictive,
the posterior, the posterior predictive
602
draws also summarizing those, looking at
the data really helps.
603
One thing that I think for anyone trying
to do models in production, one thing to
604
know is that
605
models, the programs that you write,
either in PyMC or Stan, the quality of the
606
fit is not just the program itself, it's
the program plus the data.
607
If you swap out the data and it has
different properties than the one that you
608
trained it on before, it might actually
have worse properties or better
609
properties.
610
And we can see this with like non-centered
parameterization and different variance
611
components being estimated in weird ways.
612
if you just blindly assume that you can go
and take your model that fit on one data
613
and just blindly productionize it.
614
It doesn't quite work that way yet,
unfortunately.
615
Yeah, yeah, yeah.
616
For sure.
617
And also, another prompt that I use to
help them understand a bit more,
618
basically, why we're using...
619
generative models and why that means
making assumptions and how to make them
620
and being more comfortable making
assumptions is, well, imagine that you had
621
to bet on every decision that your model
is making.
622
Wouldn't you want to use all the
information you have at your fingertips,
623
especially with the internet now?
624
It's not that hard to find some
information about the parameters of any
625
model you're working on and find a
pattern.
626
somewhat informed prior because you don't
need, you know, there is no best prior so
627
you don't need the perfect prior because
it's a prior, you have the data so it's
628
going to be updated anyways and if you
have a lot of data it's going to be washed
629
out so but you know if you had to bet on
any decision you're making or that your
630
model is making wouldn't you want you to
use
631
all the information you have available
instead of just throwing your hands in the
632
air and being like, oh, no, I don't know
anything, so I'm going to use flat priors
633
everywhere.
634
You really don't know anything?
635
Have you searched on Google?
636
It's not that far.
637
So yeah, that usually also helps when you
frame it in the context of basically
638
decision-making with an incentive, which
here would be money.
639
betting for your life, then, well, it
would make sense, right, to use any bit of
640
information that you can put your hands
on.
641
So why won't you do it here?
642
Actually, I'm curious with your extensive
experience in the modeling world, do you
643
have any advice you would give to someone
looking to start a career in computational
644
Bayesian stats or data science in general?
645
Yeah, my, my advice would probably to go
try to go deeper in one subject or not one
646
subject, go deeper in one dimension than
you're comfortable going.
647
If you want to get into like actually
building out tools, go deep, understand
648
how PyMC works, understand how Stan works,
try to
649
actually submit pull requests and figure
out how things are done.
650
If you want to get into modeling, go start
understanding what the data is.
651
Go deep.
652
Don't just stop at, you know, I have data
in a database.
653
Go ask how it's collected.
654
Figure out what the chain actually is to
get the data to where it is.
655
Going deep in that way, I think, is going
to get you pretty far.
656
It'll give you a better understanding of
how certain things are.
657
You never know when that knowledge
actually comes into play and will help
658
you.
659
But a lot of the...
660
Yeah, that would be my advice.
661
Just go deeper than maybe your peers or
maybe people ask you to.
662
Yeah, that's a really good point.
663
Yeah, I love it and that's true that I was
thinking, you know, in the people around
664
me, usually, yeah, it's that kind of
people who stick to it with that passion,
665
who are in the place they want it to be at
because, well, they also have that passion
666
to start with.
667
That's really important.
668
I remember someone recently asked me like,
should they focus on machine learning,
669
Beijing stats, is Beijing stats going to
go away, is AI taking over?
670
And my answer to that, I think was pretty
much along the lines of go and learn any
671
of them really well.
672
If you don't learn any of them really
well, then you'll just be following
673
different things and be bouncing back and
forth and you'll miss everything.
674
But if you...
675
end up like Bayesian stats has been around
for a while and I don't think it's going
676
to go away.
677
But if you bounce from Bayesian stats, try
to go to ML, try to go to deep learning
678
without actually really investing enough
time into any of those, when it comes down
679
to having a career in this stuff, you're
going to find yourself like a little short
680
of expertise to distinguish yourself from
other people.
681
So that, you know, that's...
682
That's where this advice mentality is
coming from.
683
Especially just starting out.
684
I mean, there's so many things to look at
right now that, you know, it's, it's hard
685
to keep track of everything.
686
Yeah, no, for sure.
687
That's definitely a good point, too.
688
And actually, in your opinion, currently,
what are the main sticking points in the
689
Bayesian workflow that you think we can
improve?
690
All of us in the community of
probabilistic programming languages, core
691
developers, Stan, IMC, and all those PPLs,
what do you think are those sticking
692
points?
693
would benefit from some love from all of
us?
694
Oh, that's a good question.
695
You know, in terms of the workflow, I
think just usability can get better.
696
We can, we can do a lot more from that.
697
Um, with that said, it's, it's hard.
698
Like the tools that we're talking about
are pretty niche.
699
And so it's, it's not like there are, um,
millions and millions of users of our
700
techniques, so it's, you know, the, it's
just hard to do that.
701
Um, but you know, the, the thing that I
run into a lot are transformations of prom
702
and I really wish that we end up with, um,
reparameterizations of problems
703
automatically such that it fits well with
the method that you choose.
704
Um, if we could do that, then life would
be good, but, uh, you know, I think that's
705
a hard problem to tackle.
706
Yeah, I mean, for sure.
707
Because, and that's also something I've
started to look into and hopefully in the
708
coming weeks, I'll be able to look into it
for our Prime C.
709
Precisely, I was talking about that with
Ricardo Viera, where we were thinking of,
710
you know, having user wrapper classes on
some, on some distributions, you know.
711
normal beta-gap with the classic
reparameterization, where instead of
712
letting the users, I mean, making the
users have to reparameterize by hand
713
themselves, you could just ask Climacy to
do pm.normal non-centered, for instance,
714
and do that for you.
715
In other words, that'd be really cool.
716
So of course, these are always...
717
bigger PRs than you suspect when you start
working on them.
718
But that definitely would be a fun one.
719
So, and then that'd be a fun project I'd
like to work on in the coming weeks.
720
But we'll see how that goes with open
source.
721
That's always very dependent on how much
work you have to do before to actually pay
722
your rent and then see how much time you
can afford to dedicate to
723
open source, but hopefully I'll be able to
make that happen and that'd be definitely
724
super fun.
725
And actually talking about the future
developments, I'm always curious about
726
Stan.
727
What do you folks have on your roadmap,
especially some exciting developments that
728
you've seen in the works for the future of
Stan?
729
So I actually haven't, I don't know what's
coming up on the roadmap too much.
730
Lately, I've been focused on working on my
new job and so that's good.
731
But a couple of the interesting things are
Pathfinder just made it in.
732
It's a new VI algorithm, which I believe
addresses some of the difficulties with
733
ADVI.
734
So that should be interesting.
735
And finally tuples should land if it
hasn't already landed inside the scan
736
language.
737
So that means that you can return from a
function multiple returns, which should be
738
better for efficiency in writing.
739
things down in the language.
740
Other than that, it's like, you know,
there's always activity around new
741
functionality in Stan and making things
faster.
742
And the, you know, interface, the work on
the interface is where it makes it a lot
743
easier to operate Stan is always good.
744
So there's command-stan-r command-stan-pi
that really do a lot of the heavy lifting.
745
Yeah.
746
Yeah, super fun.
747
For sure, I didn't know Pathfinder was
there, but definitely super cool.
748
Have you used it yourself?
749
And is there any kind of model you'd
recommend using it on?
750
No, I haven't used it myself.
751
But there is a model that I'm working on
at Zellis that I do want to use it on.
752
So we're doing, we call it.
753
component skill projection models.
754
So you have observations of how players
are doing for many measurements, and then
755
you have that over years, and you can
imagine that there are things that you
756
don't observe about them that kind of, you
know, there's a function that you apply to
757
the underlying latent skill that then
produces the output.
758
And, you know, over time you're trying to
estimate over time what that does.
759
And so for something like that,
760
I think using an approximate solution
would probably be really good.
761
Yeah.
762
Do you already have a tutorial page on
this 10 website that we can refer people
763
to for that episode's show notes?
764
I'm not sure.
765
I could send it to you, though.
766
I believe there's a Pathfinder paper out
in the archives.
767
Bob Carpenter's on it.
768
OK, yeah, for sure.
769
Yeah, add that to the show notes, and I'll
make sure to put that on the website when
770
your episode goes out, because I'm sure
people are going to be curious about that.
771
Yeah.
772
And more generally, are there any emerging
trends or developments in Bayesian stats
773
that you find particularly exciting or
promising for future applications?
774
No, but I do feel like the adoption of
Bayesian methods and modeling, there's
775
still time for that to spread.
776
especially in the world now where LLMs are
the biggest rage and it's, you know, LLMs
777
are being applied everywhere, but I still
think that there's space for more places
778
to use really smart, complex models with
limited data.
779
So with the, with all these tools, I just
think that, you know, more industries need
780
to catch on and start using them.
781
Yeah, I see.
782
Already, I'm pretty impressed by what you
folks do at Zillus.
783
That sounds really funny and interesting.
784
And actually, one of their most recent
episodes I did, episode 91, with Max
785
Gebel, was talking about European football
analytics.
786
And I'm really surprised.
787
So I don't know if you folks at Zillus
work already on the European market, but
788
I'm really impressed.
789
I'm pretty impressed in how mature the US
market is on that front of spots
790
analytics.
791
And on the contrary, how at least
continental Europe is really, really far
792
behind that curve.
793
I am both impressed and appalled.
794
I'm curious what you know about that.
795
I don't think anyone's that far behind
right now.
796
So I know you had Jim Albert on the show
too, and I heard both of those.
797
Right.
798
And the, the thing that I'm really excited
about right now is making all the models
799
more complex, right?
800
So I think that, you know, we probably
have some of the more advanced models or
801
at least up to industry standard in a lot
of them and like more complex than others
802
when I, you know, I just got here.
803
got to the company and when I look at it,
I think there's like another order of
804
complexity that we can get to using the
tools that already exist.
805
And that's where I'm excited.
806
It's the data is out there.
807
It's been collected for, you know, five
years, 10 years.
808
Uh, there's new tracking data.
809
That's, you know, that that's happening.
810
So there's more data coming out, more
fidelity of data, but even using the data
811
that we have, um,
812
A lot of the models that people are
fitting are at the summary level
813
statistics.
814
And that's great and all.
815
We're making really good things that
people can use using that level of
816
information.
817
But we can be more granular about that and
write more complex models and have better
818
understanding of the phenomenon, like how
these metrics are being generated.
819
And I think that's, for me, that's what's
exciting right now.
820
Yeah.
821
And that's what I've seen too, mainly in
Europe, where now you have amazing
822
tracking data.
823
Really, really good.
824
In football, I don't know that much
because unfortunately I haven't had any
825
insight peeking that I've had for rugby.
826
And I mean, that tracking data is
absolutely fantastic.
827
It's just that people don't do models on
them.
828
They just do descriptive statistics.
829
which is already good, but they could do
so much from that.
830
But for now, I haven't been successful
explaining to them what they would get
831
with models.
832
And something that I'm guessing is that
there is probably not enough competitive
833
pressure on this kind of usage of data.
834
Because I mean,
835
Unless they are very special, a sports
team is never going to come to you as a
836
data scientist and tell you, hey, we need
models.
837
Because they don't really know what the
difference between a mean and a model
838
actually is.
839
So usually these kinds of data analytics
are sold by companies here in Europe.
840
And well, from a company standpoint, they
don't have a lot of competitive pressure.
841
Why would you invest in writing models
which are hard to develop and takes time
842
and money?
843
Whereas you can just, you know, sell raw
data that then you do stat desk on.
844
And that costs way less and still you're
ahead of the competition with that.
845
Kind of makes sense.
846
So yeah, I don't know.
847
I'm curious what you've seen and I think
the competitive pressure is way higher in
848
the US, which also explains why you are.
849
trying to squeeze even more information
from your data with more complex models.
850
Yeah.
851
I think you've described sort of the path
of a lot of data analytics going into a
852
lot of industries, which is like, the
first thing that lands is there exists
853
data, let's go collect data.
854
Let's go summarize data, and then someone
will take that and sell it to the people
855
that collected the data.
856
And that's cool.
857
And I always think the next iteration of
that is taking that data and doing
858
something useful and deriving insight.
859
The thing that baseball has done really
well was linking, um, runs to outcomes
860
that they cared about winning games.
861
Right.
862
It's like you increase your runs, you win
games.
863
You decrease your runs, you lose games.
864
Right.
865
It's pretty simple.
866
Um, so this is where it's, you know, even
I'm having trouble right now too.
867
It's, it's, um,
868
for basketball, like you shoot slightly
higher percentage, you're gonna score a
869
little more, but does that actually
increase your wins?
870
Yeah.
871
And that's really tough to do in the
context of five on five.
872
If you're talking about rugby, you got, is
it nine on nine or is it 11?
873
It's 15.
874
15, right?
875
Classic European rugby is 15, yeah.
876
Like the World Cup that's happening right
now.
877
So if you got 15 players, like...
878
What's the impact of replacing one player?
879
And it starts getting a lot harder to
measure.
880
So I do think that there's, so even from
where I'm sitting, it seems like there's a
881
lot of hype around collecting data and
just visualizing data and understanding
882
what's there.
883
And people hope that a cool result will
come out by just looking at data, which I
884
do hope that it will happen.
885
But as soon as the lowest line fruit is
picked, the next thing has to be models.
886
And yeah.
887
Yeah, exactly.
888
Completely agree with that.
889
And I think it's for now, it's still a bit
too early for Europe for now.
890
It's going to come, but we can have
already really good success by just doing
891
stat desk, because a lot of people are
just not doing it.
892
And so recruiting and training just based
on gut instinct.
893
which is not useless but can definitely be
improved.
894
You know, one of the other things about
sport that's really difficult is that,
895
when we talk about models, we assume
everything is normally distributed.
896
We assume that the central limit there and
holds or the law of large numbers and all
897
these things are average.
898
When you talk about the highest level of
sport, you're talking about the tail end
899
of the tail end of the tail end.
900
And that is not normal.
901
And I'm seeing somebody to model.
902
This is where, like I said, I'm really
excited.
903
It's not everywhere, but a lot of times we
do assume that's normal normality
904
assumptions.
905
And I don't think they're normal.
906
And I think if we actually model that
properly, we're going to actually see some
907
better results.
908
But it's early days for me.
909
So.
910
Yeah, it's actually a good point.
911
Yeah.
912
I hadn't thought of that, but yeah, it
definitely makes sense because then you
913
get to scenarios which are really the
extreme by definition, because even the
914
people you have in your sample are
extremely talented people already.
915
So you cannot model that team the same way
as you would model the football team from
916
around the corner.
917
Awesome, Daniel.
918
Well, it's already been a long time, so I
don't want to take too much of your time.
919
But before asking you the last two
questions, I'm wondering if you have a
920
personal anecdote or example to share of a
challenging problem you encountered in
921
your research or teaching related to
Bayesian stats and how you were able to
922
navigate through it.
923
Oh, um...
924
in teaching.
925
I don't know.
926
That one's a tough one.
927
It's um...
928
Yeah.
929
I...
930
It's a different one.
931
Okay, here's one of the toughest ones
was...
932
Just kind of knowing when to give up.
933
So, going back to a workshop I taught
maybe in like 2013, 2012, around Stan.
934
I remember someone had walked in with a
laptop that was like a 20-pound laptop.
935
That was like 10 years old at that point
and was I think running a 32-bit Windows.
936
and asking for help on how to run Stan on
this thing.
937
I'm going to try to give up.
938
Sometimes you just need better tools.
939
It's a good point.
940
Yeah, for sure.
941
That's very true.
942
That's also something actually they want
to...
943
a message that I want to give to all the
people using Pimc.
944
Please install Pimc with Mamba and not
Beep because Mamba is doing things really
945
well, especially with the compiler, the C
compiler, and that will just make your
946
life way easier.
947
So I know we repeat that all the time.
948
It's in the readme.
949
It's in the readme of the workshops we
teach at Pimc Labs, and yet people still
950
install
951
So if you really have to install with
peep, then do it.
952
Otherwise, just use MambaForge.
953
It's amazing.
954
You're not going to have any problems and
it's going to make your life easier.
955
There is a reason why all the Pimc card
developers ask you that as a first
956
question anytime you tell them, so I have
a problem with my Pimc install.
957
Did you use Mamba?
958
So yeah, it was just a general public
announcement that you made me think about
959
that Daniel, thanks a lot.
960
Okay, before letting you go, I'm gonna ask
you the last two questions I ask every
961
guest at the end of the show.
962
First one, if you had unlimited time and
resources, which problem would you try to
963
solve?
964
My, I would try to solve the income
disparity in the US and what that gets
965
you.
966
I'm thinking mostly health insurance.
967
I think it's really bad here in the US.
968
You just need resources to have health
insurance and it should be basic.
969
It's a basic necessity.
970
So working on some way to fix that would
be awesome.
971
unlimited time and energy.
972
Yeah, I mean, definitely a great answer.
973
First one, we get that, but totally agree,
especially from a European perspective,
974
it's always something that looks really
weird to you when you're coming to the US.
975
It's super complicated.
976
Also, yeah.
977
One of the things, like, working in pharma
was like, realizing that a lot of the R&D
978
budget is coming from
979
you can call it overpayment from the
American system.
980
And so if you still want new drugs that
are better, it's got to come from
981
somewhere, but not sure where.
982
It's a tough problem.
983
Yeah, yeah, yeah.
984
I know for sure.
985
And second question, if you could have
dinner with a great scientific mind, dead,
986
alive, or fictional, who would it be?
987
That one, like I thought about this for a
while.
988
And you know, the normal cast of
characters came up, Andrew, Delman, Bob
989
Carpenter, Matt Hoffman.
990
But the guy that I would actually sit down
with is Sean Frayn.
991
You probably haven't heard of him.
992
He's an American inventor.
993
He has a company called Looking Glass
Factory that does 3D holographic displays
994
without the need of a headset.
995
He happens to have been my college
roommate and my big brother and my
996
fraternity at New Delta at MIT.
997
And I haven't caught up with him in a long
time.
998
So that's a guy I would go sit down with.
999
That sounds like a very fun dinner.
Speaker:
Well, thanks a lot, Daniel.
Speaker:
This was really, really cool.
Speaker:
I'm happy because I had so many questions
for you and so many different topics, but
Speaker:
we managed to get that in.
Speaker:
So yeah, thank you so much.
Speaker:
As usual, I put resources in a link to
your website in the show notes for those
Speaker:
who want to dig deeper.
Speaker:
Thanks again, Daniel, for taking the time
to be on this show.
Speaker:
You had to be easy change your predictions