Episode

Today we are doing something a little different.

A proper philosophy of science episode.

And one of my favorite conversations in a while.

My guest is Vaden Masrani, a machine learning researcher, turned consultant, and the co-host of the Increments podcast with Ben Chugg, Andreas Munk from episode 155, Put Us in

Touch, and I am so glad he did.

Vaden has a sharp, unusual position.

He loves Bayesian statistics and he's deeply critical of Bayesian epistemology.

So we spend the episode walking that line.

Why is it wonderful to use Bayes' theorem when you have data and a model you can falsify?

And why does it go wrong the moment you start assigning probabilities to one-off future events with nothing to count?

From there, we get into the problem of induction, Kal Popper, and critical rationalism.

Why criticism rather than falsifiability sits at the foundation of knowledge and how all of this shows up in stand-up comedy, stoicism, and even the downfall of Sam Bankman-Fried.

It is playful, contrarian, and it made me think a lot.

I had more questions actually than we had time for, so I am going on Vaden's show to keep the conversation going.

Stay tuned for that.

This is Learning Basion Statistics, episode 160, recorded by

May twenty, twenty twenty six.

Let me show you how to be a good basian.

Change your predictions after taking information.

And if you think it now be less than amazing, let's adjust those expectations.

What's a Bayesian?

It's someone who cares about evidence.

Welcome to Learning Bayesian Statistics, a podcast about Bayesian inference, the methods, the projects, and the people who make it possible.

I'm your host, Alex Andora.

You can follow me on Twitter at Alex underscore Andora.

like the country for any info about the show.

Learnbase stats.com is Lab Plus2B.

Show notes, becoming a corporate sponsor, unlocking Bayesian merch, supporting the show on Patreon.

Everything is in there.

That's LearnBaseTats.com.

If you're interested in one-on-one mentorship, online courses, or statistical consulting, feel free to reach out and book a call at topmate.io slash alex underscore and Dora.

See you around, folks, and best Bayesian wishes to you all.

Vaden Masrani, welcome to Learning Bayesian Statistics.

Nice to be here.

Yes, I'm I'm very happy to have you on the show.

you were recommended to me by Andreas Munk from episode 155.

So of course Andreas' episode will be in the related uh episodes for that one.

but that's how we met, and I'm I'm very happy that we did because I think we have a fun episode in front of uh us today.

Before that though, as usual, what is your origin story?

You know, uh what are you doing nowadays?

And how did you end up doing that?

Yeah.

Um so I know Andreas from grad school.

We did our PhDs together at UBC in a probabilistic machine learning reading group.

Uh not reading uh lab, rather.

Um and how I got there is kind of an interesting story.

So I um my undergrad was in physics, um, and so near the end of my undergrad.

I was working for um the Atlas experiment, which is the one that discovered the Higgs boson.

Gosh, in twenty twelve, twenty thirteen.

To be extremely clear, I had nothing to do with the discovery of the Higgs boson.

I was just just an undergrad making making plots.

but my kind of my summer project was to train neural networks to detect a particular kind of particle decay.

Um and through that process I realized I was way more interested in the neural networks side of things than the particle decay.

side of things.

So I kind of pivoted and that did my master's in um ML at UBC as well, more on the applied side of um side of the research.

Um and then that led me to working in Tokyo in a Bayesian ML group there with Emtiyaz Khan at RIKEN doing variational inference uh kinds of uh of work.

Um and I just love that experience so much that I decided as uh a masochist to do the whole PhD experience.

And um and that was a lot of fun.

and so that kind of takes us to a couple of years ago.

Um so after that I was doing some research at one of the big companies and um didn't love industry research as much, so I jumped from that about two years ago, year and a half ago,

uh, and started a consulting company.

Um and that's kind of what I've been doing ever since.

And so a lot of industry projects, not so much Bayesian statistics anymore, um, but lots lots on the philosophy side of things.

Um I should probably also mention

So in the sometime during the PhD, uh maybe first year of COVID, I started a podcast with a friend of mine.

Um, and in that space we talk a lot about Bayesian things, um also the philosophy of science and history and and all these various subjects.

And and so that's been just a extremely valuable source of intellectual nourishment because you don't have any of the publish or parish constraints.

You can just freely explore ideas at your um

uh kind of following your interests and and so that's what I'm spending most of my time these days doing last because consulting m kind of the nine to five and then podcasting on

the five to nine.

Yeah.

we'll we'll talk a bit more about your podcast.

That's the the increments podcast with uh Ben Chugg.

Um so definitely folks should check it out.

But we'll we'll talk about about it in uh later in the show.

And of course you will have uh everything in the show notes people if you wanna

Check out uh Falan's podcast.

I definitely encourage you to.

And on YouTube also well you'll have that on on his you this episode will also be on his YouTube channel, thanks to the magic of uh YouTube collaborations.

Um so yeah, definitely check that out.

On the consulting side, uh actually, so if I remember correctly, your company is um is called Sophia Consulting, right?

yeah.

Sofia consulting.

So two questions.

What do you guys do?

And also is my guess right that the Sophia is in relationship to philosophy and coming from the Greek word for wisdom, or is it unrelated?

No, it's totally related.

Nice catch.

You're one of the few to to catch that.

Yeah.

so yeah, Sophia means lover of wisdom.

Um and I kind of liked

introducing a nod to wisdom compared to say intelligence, which is a vastly overused word these days.

and so yeah, what we do is essentially just working with companies on their real world problems.

Um and so maybe the easiest way is to explain one of the projects we just concluded.

So there's a local company called AquaEye, and they make an underwater sonar gun that detects drowning swimmers.

Um and so you have a diver

out at five meters, ten meters, up to fifty meters, uh and you're bouncing sonar off of them.

Um, and then you do a lot of data cleaning and then you train a a fast CNN so that it works on device, essentially.

Um, and we just released the Aqua I Pro model, which um has moved over from random forests to CNNs.

And so that's been one of the major projects I've been uh working on for the last couple of years almost.

Um that's kind of in the space of

machine learning um in AI.

So I know that these terms are overloaded and mean um many different things to different people, but I am these days using AI to refer to chatbots and anything that's generative,

whereas ML is more the traditional make a prediction.

Um so the AquaI was one of the major ML projects we've been working on.

Um on the AI side of things, I was working with another local company called Huge that's making a smart home assistant

that lives on device entirely locally, so you can trust it with uh more personal data because uh none of that data would have to leave um leave the the box.

Um so those are just kind of two of the projects that I work on.

But in general, I just really love the real world problem solving component of consulting, which is very different than um publishing papers, for instance.

Um as I'm sure many of your listeners know, publishing papers is extremely important and difficult.

but there's all sorts of

problems that come online when you're dealing with a real data set that you've actually had to collect and actually have to clean up.

and so those kinds of challenges have been just loads of fun to work on these days, because you're actually able to save lives and and make a positive impact on the world.

Wow, okay.

This is super cool.

So in ease yeah, I'm not I'm not gonna argue with you that podcasts are a great way to stay on

up to date with the research with a lot more freedom and and same for that kind of research that you just talked about.

Um definitely resonates with me.

And also I'm curious for Sophia AI, do you like what are you always taking these kind of projects on or is that something like that's um like is that project representative of

what you do or do you usually do other kind of consulting

Um yeah.

I'm just curious how that works because I used to do uh professional statistical consulting, so I'm always curious about other people.

Yeah.

so the word consulting can mean a lot of different things from right charging a hundred grand for a PowerPoint presentation to working in the in the weeds and in programming.

uh and then obviously there's like the strategic uh direction side of things if the company's uh newer.

Um

And so it kind of depends on the project.

Um, I like to my think I like to think of myself as a bit of a jack of all traits in the sense of having the research background to be able to um determine what kind of projects

are feasible compared to uh we wanna train a bot to predict the stock market kinds of projects which just aren't feasible.

Um and so sometimes the the projects are more at the high level of of working with the founder and offering like strategic direction about okay, if you wanna do X, well you're

gonna have to do A, B, and C first, and what's the

the the risks associated with A, B, and C.

Um, if the company is a bit more advanced, if it's been around for a long time and they have a more concrete research problem, then often I'm the person that they reach to when

they need to figure out exactly how to run the experiments, how to uh validate the models, how to make sure that you haven't accidentally overfit and given yourself extremely um

unrealistic positive results, which which is uh something that's easy to make that kind of mistake when you're new to this.

So it very much depends on the client and and what their needs are.

But um one of the things I just love so much about let's say um machine learning data science is that data gives you uh sorry, uh knowing how to work with data gives you an all

access hall pass into everyone else's industry.

So I don't need to know much about how sonar works, but if I know enough about Fourier and signal stuff, then I can learn that.

there's another company I was working for, um that

was doing predictive analytics for heavy machinery in the Amazon rainforest.

And so it was super cool to learn all about trucks and how trucks and and stuff work.

so I I like the I like the fact that each job is different.

Each job requires really understanding um the unique problem situation that the company is in and what their goals are, and then being able to offer some both strategic direction,

but then also highly technical um instructions too.

To their team if if if they already have engineers or doing the first round of POCs and and prototypes to demonstrate that the idea is vi viable.

So it totally ranges.

Um I have yet to charge 100 grand for a PowerPoint presentation, but I'm hoping such opportunities will present themselves.

But but in general it ranges from the actually doing it, programming POCs all the way up to offering recommendations about which direction the the company should uh should pursue.

Okay.

Okay.

Basically the whole range you you could do, but you you of course do a lot of uh of modeling because that's that's out of your background.

Yeah.

Okay.

Interesting.

Um and and that makes sense also with my with my personal experience.

Um yeah.

Sounds like you guys are doing very very interesting work.

Well done on on doing that.

How many how many are you now in the company?

Just one baby.

Um so we um

There's another idea which it's a bit too early to discuss right now, but um for that idea we're I say we because I'm an academic at heart and I cringe saying I, but Um but so

there's an idea that I'm potentially pursuing V C funding for and I'm gonna go the more startup route and if we d go that direction, then definitely we'll be hiring and stuff.

But um but right now I like being agile, versatile and being able to pick up projects as they come.

Um there's a bit of a difficulty when you wanna scale and

the consulting world because you need to have enough steady clients coming in that you can um then guarantee steady work for the people you hire.

And right now it's been just a lean kind of uh just me and my laptop and just a bunch of networking and stuff.

But um but if we decide to do the VC route, then I'll be kind of going in more of the hiring hiring direction.

Yeah.

Makes sense.

Um so good luck with that and uh hopefully you get you get what you want.

Um

If we focus on the Bayesian side of things now, um what drew you to to Bayesian methods in the first place?

Yeah, so that's a good question.

Um what drew me to Bayesian methods was partially just how beautiful the math was.

so there's loads of applications in Bayesian statistics as you and your audience are quite familiar with, but what I found personally exciting is just how deep the research

opportunities are.

Um and

It's interesting 'cause in industry I've found that there have been less applications than I would have kind of guessed being in academia.

But the applications in industry are are very much at like the fundamental scientific level.

So protein discovery or obviously diffusion models are are a big one.

Um and so I liked liked the um the coupling of pretty math with real world applications.

Um and

My master's was in a bit of a different domain.

It was um about predicting whether someone sorry, predicting the stage of um a patient's Alzheimer's uh diagnosis based on uh speech analysis.

Um and so very kind of just uh mm black and white ML problem.

You get a bunch of text, you compute features, you do parse trees, and you count morphemes and and all this, and then you train classifier.

and so that was a great um

experience to to kind of get uh a good understanding of like the applications of ML.

But through that experience I didn't feel like I had a good understanding of how these algorithms actually worked.

Um there's a major difference between uh pulling off say a logistic regression model from SK Learn versus actually like deriving the updates and and doing it on the whiteboard.

Um and so the the pivot into Bayesian stuff was largely motivated by

liking the applications and liking the opportunity to really dive deep into into how a lot of this stuff worked.

and then that's uh that's what led to the PhD and yeah.

And so that's kinda how I got into it.

Okay.

Okay.

So that was pretty, pretty early in your career, right?

It was even b bef before even working.

Yeah.

So it was at the end of my masters, I took a late internship.

And that was my opportunity to decide if I wanted to continue in the academia route or make the jump to to industry.

Um I actually did two back to back internships.

One was at Samsung and the other was at in in Tokyo.

Actually opposite way around, Tokyo than Samsung.

and those two back to back experiences definitively convinced me that I want to stay in academia land.

I didn't like the working at a megacorp uh aspect so much.

it was nice to have

Like well funded coffee machines and lunch stations and stuff.

But um but I much preferred like the the difficult whiteboard sessions that um that the Tokyo experience um offered.

And so um that kind of sealed the deal and then I decided to do the PhD and uh and then never looked back.

Yeah.

Okay.

Yeah.

Very cool.

and actually something you you distinguish a lot.

in your own work and and podcast is distinguishing Besian statistics from Bayesian epistemology.

So this is a very practical podcast.

So we don't I mean sometimes we talk about it like episode 50 with uh David Spiegelhalder, episode 51 with uh Aubrey Clayton.

So we do mention philosophy and epistemology from time to time, but um we are usually more a uh

Practitioners podcasts, so I'm very interested to hear about you today because can you for listeners who haven't heard Framed it that way before, can you tell us what the actual

difference is between statistics and epistemology?

Yeah.

so that's a big question.

Um I'll just lay my cards and my biases out on the table.

Um

'cause I am extremely in favor of Bayesian statistics because of the myriad of applications that Bayesian stats um has.

Um just in researching this episode.

I was just like, Chat GPT, give me some just examples.

So obviously SLAM and Kalman filters and VAEs is all well understood.

MCMC, that's all well understood, but there is a nice example of I guess Air France 447 went down about a couple decades ago.

And Bayesian stats was used to be able to recover the the the wreckage.

So huge amounts of applications.

That's great.

Um Bayesian epistemology I'm extremely uh critical of.

Um and I don't like it as an epistemology.

Um I obviously haven't defined it yet.

But but that's kind of the the position I'm coming from.

And so um I'm going to give you an opinionated take on what Bayesian statistics or Bayesian epistemology is.

and then we should probably talk about um

epistemology in general, because I think Bayesian epistemology is best understood, contrasted against another epistemology.

but so if I can take a bit of a uh roundabout way to get there, the there's a podcast episode with Toby Ord and Sam Harris.

Um and this podcast episode was came out maybe five or six years ago.

Um and in this episode

Ord was talking about his most recent uh book called The Precipice at the time.

And he was talking about existential risks.

Um and so he was talking about uh so what's the probability that we will die from an asteroid?

uh or what's the probability that we'll die from a volcanic eruption?

Um and so you can assign numbers to these because you have a data set to to look at.

So you can look at the number of super volcanoes in the geological record over the last

couple hundred millennia and then you can start doing some basic take account divided by time to get probability associated with it.

but then he switched and he started talking about what's the probability that we will all die from superintelligence in the next hundred years.

and then he assigned a probability to that too.

But if you are a statistician, if you're a Bayesian statistician, I think your first question should be, well where are the statistics?

What's the what's

What are we putting into these models?

How can you come up with a probability of something that's going to happen in the future when there's nothing to count?

Right.

Um, and in particular, uh, how is it a valid move to compare probabilities derived without data um against probabilities that come from data?

Because in this episode, um, Ord was essentially making the point that we have a one in, I believe in this book it's one in

10 chance of extinction due to superintelligence.

And he contrasted that against uh one in, I'm just gonna make up a number, one in uh a billion chance of of earthquake.

Um so that got my little radar binging, my little bullshit detector binging.

Because um if you have a model and you train that model on zero data, how is that a model?

Where like where is the statistics?

So uh the answer to your question, the kind of the tongue in cheek answer is that Bayesian epistemology is Bayesian statistics minus the statistics.

It's just the equations.

Um how that actually works in practice and why some philosophers, particularly ones that come out of Oxford these days, um, why that they consider that to be a valid move, where

all other statisticians, Bayesian or frequentists, would say, Hold on, if you are drawing a scatter plot and you do a line of best fit on an empty whiteboard

With no dots, you're just doing lines on a whiteboard.

How is that a model?

So the answer to your question, uh at a surface level is that Bayesian epistemology is what happens when you take Bayes theorem way too seriously and you forget about the data.

and then you start coming up with numbers and you kind of um convince yourself that these numbers are legitimate because you're calling them probabilities when in fact they're made

up, they're made up out of your head, and you then are making decisions based on uh made up numbers which you've called the probability.

Now

if there was someone who was a Bayesian epistemologist on this podcast, they would strongly disagree with my uh description of this.

Um and so uh from their perspective, which we sh we can maybe steel man in in in a bit here, they have a different story to tell.

Um so I I I do want to be clear to the audience that I'm giving a biased accounting.

and so maybe I'll I'll pause there to let you kind of um poke and prod at some of the stuff I said, but um but it is an interesting question.

Why are some

philosophy departments okay with using probabilistic estimates that don't come from any data.

And the answer to that is due to a hundred years of literature that can all be kind of considered to be Bayesian epistemology.

Okay.

Yeah, thanks.

That's a very clear um it's a very clear uh explanation of of the difference.

Uh I really find that interesting.

So I spend most of my time thinking about statistics and models and not not so much about philosophy and epistemology.

But so I do have a few questions.

It and I I'll mainly ref try and rephrase what you told me to make sure I understood.

So please let me know if I didn't.

The main thing that m bothers you about epist Bayesian epistemology is that

is in the scenarios where you don't have data to update your beliefs.

Is that correct?

Um I would frame that as that's the most immediate example of the problems with Bayesian epistemology.

That's one that I think a listener can grok without having to talk about Cox's theorem or why, for example, we uh conflate uh

probability distribution with the psychological phenomenon of having beliefs in the first place.

there's questions to about why that connection even makes sense.

so there's a lot of philosophical reasons why I don't think it's gonna work.

but I like to start with a clear example of of how it can get us into trouble.

Um and so I guess my my my comments there is that that's just the tip of the iceberg.

But but that's one one reason because I think it can um

uh trick people into making decisions that are poorly informed because they're using a lot of math and and using a lot of math is not sufficient to making a good decision.

But Bayes theorem s divorced from the data when it's just Bayes theorem, that I think can be quite um enticing to people and it can lead them lead them astray.

So that would be how I do fully agree with the dangers of basically math washing.

I and it's used a lot in in politics and and

Basically like people who want to convince you of something they think is true and then they just try to um to just shoe in what they already believe into a math package to give

it the the illusion of objectivity.

what

I'm so I'm less disturbed by you are by the fact of using Bayes' theorem without priors without data, sorry.

Because so one thing I thought I understood, but I may be completely wrong on that, is in the Bayesian epistemology framework, you define probability not as a long term frequency.

But as a degree of belief.

So if you accept that definition of probability, then using Beige Theorem without data is just regurgitating your priors.

And to me, that's not too much of a problem of an issue if you say so.

Like, so I think my my problem is more when people do that thing, and for instance, your your example, I think, with the Sam Harris podcast and

The 10% probability that we get wiped out by Gen AI.

I think I'll listen to that podcast actually.

Is that a recent one?

No, it was older.

That was the one of the conversations that got me interested in this subject in the first place.

But I would say it's probably four or five years old.

by this one.

Okay.

Yeah.

So Yeah, because I I've seen I've also seen a report like that about what the impact of Gen AI would be in twenty twenty seven or things like that.

And

They were doing a lot of things like that, where it's like, I don't know, we're like, you don't have any data to really back that up.

So you're basically taking an assumption and then just thinking about how we could get there.

Which is fine as so, and again, to me, this is fine as a thought experiment.

The problem is that sometimes you can get convinced by you can convince yourself that actually

This is gonna happen and forget that you base that on an assumption.

Or in a less um let's say um in a less nice way of thinking about that, it would be more like Mali's, where it's be I'm gonna package that as something that is actually objective,

and instead of saying it's just reflecting my degrees of belief about something that can happen, i.e.

my priors.

I'm gonna say this is actually a probability of long term e so you basically in the communication of the output, you swap the definition of defin of probability that you had

at the beginning.

And I would say that is what really bothers me here.

Because if you stop that, well, I just my priors are are that, and so I think the probability of Jen AI wiping us out is ten percent.

and it's my belief.

Okay, like you

You didn't really apply Bayes theorem, it's just the basis of Bayes theorem, but okay.

Well I mean, why not?

Um So yeah, I'll I'll stop here because um I I talked a lot already.

Yeah, no totally.

So I I think we're we're um sh we we share intuition.

Uh I would uh maybe ag agree and sharpen what you said, which is that um I've no problem whatsoever with someone saying my opinion

is that we will be wiped out next year w with a one in ten chance.

Or um my gut feeling is that we will be wiped out.

Or my what's your prior on the question of whether or not you're gonna get the job next week?

Well my my hunch is that it's going to be it it's pretty likely.

That's all totally fine and great.

Um the difficulty or the where the the trap lies, I think, is that when one starts talking in terms of priors

And then when when Sanjay's talking in terms of actual probabilities, now you're comparing two things that have extremely different origins.

So for instance, um we can come up with uh a probability that um COVID-19 is going to let's say uh cause 10% of the population to be hospitalized, right?

And so how can we come up with that?

We can use Bayesian methods.

So we could use like approximate Bayesian computation, we could get a giant simulator.

Um actually a project I worked on, where you get a big simulator that literally has a computer program to say what's the probability that you're gonna go to work and you're

gonna touch something and uh so we're talking like a hundred thousand lines of C code with like deep assumptions based on the spread of disease and how a city works and it's

something that grad students have been working on for fifteen years to be able to come up with these models.

And that gives you a probability.

And you say, okay, the probability

is one in ten.

I'm just gonna keep using the same numbers.

And then someone else says, well my prior is one in thirteen.

And now the fact that the first group is is using Bayes' theorem and uh practicing Bayesian statistics, but they're still counting stuff.

They're still coming up with frequencies and the frequencies are Monte Carlo runs, right?

Um

And that number is then compared to someone's gut feeling.

But people forget that they're switching between these two extremely different assumptions about what the nature of probability is.

And so if you license yourself to use a subjective um interpretation of probability, um, and then you kind of forget that and then you compare it to counting asteroids or or

looking at the MCMC runs from a giant simulator.

Um now you're comparing someone's gut feelings to

thirty years of research, right?

Um and this is exactly what Toby Ord did in his book and it's escaped even someone as as uh wise as Sam Harris.

because uh if I just tell you what the probability is, then you think, okay, got it.

That's some objective thing that people have have calculated.

When in fact, no no no, probabilities are not all made equal.

And probabilities derive from uh models, um derive from data.

Like even when we talk about a prior

Like you can absolutely do empirical priors where your priors are informed by data.

So nothing is intrinsically um data independent about priors because there are techniques to inform your priors um if you're taking a Bayesian statistical approach.

Uh and so um my my main rub against Bayesian epistemology is that I just don't think that is how knowledge is produced and how we actually make scientific progress.

Um, I think we make that through um Karl Popper's um philosophy, Conjectures and Refutations, which I'm I'm sure we'll go into at s at some point.

Um so my my biggest problem is is that uh I just don't think that this accurately describes how knowledge is produced.

But my more immediate problems and the more applied problems are that it allows people to compare numbers that have extremely different origins and then potentially make extremely

uh bad decisions based on this.

So one

extremely salient example is Sam Bankman-Fried.

So if you read the book that came out um by uh Going Infinite, I I forgot the author's name at the moment.

same guy who did the big short.

Um you can just read that Sam Bankman-Fried just viewed everybody and everything as just walking probability distributions.

Um and now he's in jail.

And he's in jail because he has uh certain um he made certain decisions based on uh

faulty epistemology um that that wouldn't have happened uh had he not been um drunk off probabilities, shall we say?

Yeah.

And so let's try and make that a bit concrete.

So basically let's let's go back to the part the part you like, the Bayesian statistics part.

When a working statistician fits a model with priors and a likelihood, what work

is the Bayesian machinery doing that you think is legitimate and useful?

um yeah, so Bayesian statistics for your audience will know this for sure, but uh Bayesian statistics is differentiated from say a frequentist statistics in the sense that it uses

random variables as your parameters.

Right.

Um and so that gives you um a lot of uh freedom to incorporate in knowledge that doesn't really f

uh have an entry point uh via other other means.

So for example, if we are trying to uh get an estimate of the average height of people in Canada, say.

well that's a really nice domain for Bayesian statistics, because you can put a prior over what you think the height's gonna be.

You know the height's not gonna be zero.

You know the height's not gonna be a hundred feet.

and so that's just a really nice way to to uh

incorporate in some just domain knowledge into your problem such that the the answer you get um is is making use of of this common sense information we have about the problem.

Um and that's all fantastic.

And in particular it's fantastic because you can get a model, a Bayesian model, and then you can compare it to your data and you can be like, oh, my model is wrong.

This was a bad model.

I need to go back and I need to reassess.

um And so to the extent that Bayesian methods in when used in statistics are falsifiable.

And you can use data to figure out if your modeling assumptions were bogus or or or not bogus.

I think it's it's fantastic.

so that's just one example, but you can talk about how variational autoencoders work.

Um you can talk about uh MCMC and and there's all sorts of real-world problems for which this assumption that your random variables um sorry, that your parameters are random

variables makes a lot of sense and it's been validated and it works in practice.

Um

I wouldn't say that it's better or worse than frequentist statistics.

I would say it's just a different kind of statistics.

And any statistician these days is going to be practicing both and they're gonna know uh about the kinds of problems for which, say, a bootstrap estimator or or other frequentist

estimators of uncertainty are appropriate, and other problem domains where Bayesian statistics are appropriate.

So from the perspective of like the Bayesian frequentist debate as applied

To like engineering problems.

No dog in that fight.

I think both have their pros and their cons.

Um where I have more of a dog in that fight is when it comes to epistemology and when it comes to saying things like um the way that Newton was able to come up with his theories

was using uh Bayes theorem in his head somehow.

Um and so when it's used as a model of scientific discovery, that's where I think one can start running into uh difficulties, quite quite um quite severe ones, in fact.

Yeah.

Mm-hmm.

Yeah, so let's let's talk about that.

What's the central thing concretely on that that Bayesian epistemology is a general framework for reasoning under uncertainty gets wrong, according to you?

If I may take a bit of a historical lens on that.

so on our podcast we talk about Karl Popper a lot.

He's a philosopher that had a big impact on me.

Um and he also was uh writing basically was born in nineteen two and he died in nineteen ninety-four, meaning he was writing throughout the development of uh measure theory.

He was writing when all of these probabilists were were really working on on probability theory.

That was when like the Banach-Tarski paradox was discovered.

That was when Kolmogorov started working.

and so uh throughout the twentieth century there is a huge amount of development uh on

probability theory.

Um and a sizable percentage of the these economists and philosophers, um not so much the mathematicians, they were kind of just working on their measure theory, um, wanted to use

this as a framework to explain where knowledge comes from.

and uh and that's essentially where Popper says, hold hold your horses here, fellas.

Um I don't think it works that way.

and uh

So maybe one way to to enter this conversation is to um think about Newton's theories when they were first I guess developed in the 1600s and 1700s.

when they were developed, they were basically seen as like we have finally found true knowledge.

Like this again, this is pre Einstein, so this is before we realized that Newton's theories were wrong.

And Newton series just gave us this like insane amount of uh predictive um power, an insane amount of engineering capabilities.

It allowed us to figure out how to shoot cannonballs, it allowed us to explain um how the moon stays up in in its um its orbit.

and it so just it told us so much, not only about the things you can immediately see, but it told us stuff about stuff we can't see, stuff that's like in another galaxy.

It explains how stars would planets would orbit stars.

and so this was the problem that a lot of philosophers were trying to figure out.

So how is it possible that human beings w r coming up with scribbles on their piece of paper in some place in Europe, um, are able to discover uh general truths about the entire

cosmos, truths that they haven't themselves seen or don't have immediate access to.

So you can't touch a force.

You can't

see a force, you can see a a bowling ball fall.

You can hypothesize that it falls because of a force, but you can't actually see it, right?

and so uh the traditional answer for how this comes about um is via this process called induction.

and so induction in the the Pauperian worldview is like a a dirty word.

Induction is something that the

We Pauparians don't like very much.

But what is induction?

Induction in this case simply means knowledge is acquired via repetition.

So the way that uh Newton um was able to discover his laws was by repeatedly observing um apples falling from trees, say.

Um now Bayesian statistics, Bayesian statistics is just a four sorry, Bayesian epistemology, Bayesian epistemology.

is just a a f uh a form of induction.

It's it's this it was an attempt to fix what is called the problem of induction.

And the problem of induction is just that um all these philosophers said that you gain knowledge by repeated observations, and yet it's logically impossible that seeing an apple

fall a whole bunch of times can tell us a general truth about what's happening in the Andromeda galaxy.

Um and so uh philosophers from like Francis Bacon

to all the way to Bertrand Russell um were wrestling and and David Hume is the guy who who named the problem of induction.

They were wrestling with this apparent contradiction.

And the contradiction is on one hand, all the philosophers saying that we get knowledge because of repeated observations, and yet logically it's impossible to get any knowledge

about something far away based on immediately what you see.

So then Bayesian epistemology was an attempt to patch this.

So it was an attempt to say

Okay, sure.

Maybe we can't be absolutely certain um that uh Newton's laws are true given the things we've seen, but maybe we can be probably certain, or maybe we can be um less uncertain.

and this is uh what a lot of work has been uh spent on is trying to basically use probability theory to patch the problem of induction.

Um now.

so your question was like what's what's the big problem with this and why why don't why don't I think that this works?

Um well so for a number of reasons.

So in the Bayesian framework, this is um repeated observations, so you call them repeated uh so you see a bunch of evidence, right?

and now when I say evidence, I'm not meaning a row of a CSV, right?

So a row of a CSV is totally legit, that's that's data.

But

The Bayesian epistemologists meant is just you just see it.

You just open your eyes and you just see evidence.

Um but the question is, well, what evidence?

So what counts as evidence only counts in light of the theory.

So you have to have your theory first, and then based on your theory, you can figure out what counts as evidence.

But it it's a circularity or it's an infinite regress.

If you're gonna say that your theory comes from evidence, then that just doesn't make any

Um so there's one major issue, which is that when Bayesian epistemologists try to explain w where Newton's theories come from, they start with a prior over H um and then you have

your likelihood, evidence given H, but you've already kicked the can down the road because the question is where does the pri where does H come from?

And so H can't come from evidence.

It has to come from some other process.

Um so it's one problem.

with it.

I think the other major problem with Bayesian epistemology is that it encourages you to find evidence in support of your hypothesis.

um Every astrologer on the planet, every conspiracy theorist on the planet looks to find evidence in support of their theory.

It's extremely easy to find evidence that supports your view that homeopathy is going to cure your cold.

It's extremely easy to find further

evidence that 9 11 was an inside job.

If all you're looking for is evidence to support your preferred view, then you're gonna find a lot of that quite easily.

what Popper did, and we can punt on talking about Popper um as long as possible, but he inevitably rears his rears his uh his head when um this subject comes up comes up.

Um but Popper's whole point was that no no no the scientific mind is not looking to find evidence in c that confirms the hypothesis

It's always trying to find evidence to disconfirm your hyp your preferred hypothesis.

Um and so I think to summarize, my main problems with Bayesian epistemology is that one, logically, it just doesn't make any sense.

Um d you can't explain where a hypothesis comes from by talking about evidence in support of it because the evidence only works given the hypothesis.

Um and two, it produces dogmatism.

it produces this

this idea that you should only look for stuff that supports your your view.

Um and then three, which I haven't mentioned yet, but we can go into it, is that as soon as you start trying to patch the problem of induction with math on top of it, you're

basically building an edifice on a logical impossibility.

And then infinite numbers of paradoxes emerge.

So one of the major paradoxes in Bayesian epistemology is something called um Hempel's paradox or the Raven paradox, which some of your listeners may be familiar with this, but

is this idea that um if we're trying to find evidence in support of the theory that all ravens are black, all ravens are black.

So that is equivalent to if a thing is a raven, then it is black.

And as soon as you have a if then statement, you can take the contrapositive, which is if a thing is not black, then it's not a raven.

And then that would um count logically it would be equivalent to um seeing a black raven.

So Hempel's paradox or the Raven paradox is just the idea that any green shoe you look at or any orange uh ball or any uh was it San Pelgrino cup that's not black would be in

support of your hypothesis that all ravens are black.

So how can you possibly claim to have knowledge and understanding about how ravens work by looking at green shoes?

Um so Bayesian epistemology, because it is so fundamentally subjective, because it really encourages you to go inwards and think about your beliefs, and it doesn't encourage you to

go outwards and experiment, and it doesn't encourage you to go outwards and actively find information that goes against your beliefs and actually try to disconfirm your beliefs.

Um, I think it causes people to run in circles.

I think it causes people to become extremely dogmatic, as Sam Bangman Fried I think is evidence of.

Um and I think it logically just doesn't make any sense.

and so those are the main main reasons wh uh why I am not a big fan of Bayesian pistology, but still I'm a huge fan of Bayesian statistics.

And that line is something that I constantly have to walk in the other direction uh with like our audience on the podcast, because some people who are extremely anti Bayesian now

will wanna write off all of Bayesian stats too, and have to like, No, no, no, hold on.

Bayesian statistics is great because you have data, because you can be falsified by your data and your and your models, but Bayesian pistology.

is less um less good, shall we say.

So yeah.

Yeah.

Right.

Right.

So the main like m one of the most potent points in your critique is that whate uh whatever the name actually of your framework of epistemology, whether that's Bayesian or

anything else, it needs to be falsifiable and it needs to have falsifiability as a first class citizen in it.

Otherwise you can't really know

when you're producing knowledge or you're just producing more evidence for your prior beliefs.

Is that correct?

With a few asterisks.

Um so the um the first asterisk is that that's absolutely true for what we consider to be science.

So a scientific hypothesis has to be falsifiable.

Um but philosophy is not falsifiable.

there's all sorts of ideas that aren't falsifiable, but yet are still extremely valuable.

Um and so uh Popper's philosophy he called it critical rationalism.

Um or inverted you can say it's all about rational criticism.

So the generalization of falsifiability is criticism.

and so Popper's epistemology is uh takes criticism as a first-class citizen, not falsifiability.

So

falsifiability um would be a kind of criticism.

In particular it's an empirical kind of criticism.

Um where um if you are so lucky as to come up with a theory that can in fact be tested, then your whole goal is to try to falsify that theory.

but there's all sorts of theories for which they just don't have tests.

Um the integral calculus how can you falsify the integral calculus?

Well you

can't run experiments to falsify it, but you can absolutely uh criticize it.

You can absolutely discover things inside it that don't make any sense, such as what I mentioned earlier, the Banach-Tarski paradox, and and so Popper's epistemology takes

criticism as the fundamental unit, the most important thing.

Um and then various types of criticism um apply to various kinds of domains.

So in the scientific domain

Um experimental tests are are one of the the most important kinds of criticism, but also peer review.

Also the peer review process, the just uh explaining your idea to a friend on a podcast and having them be like, huh?

What does that mean?

That's a kind of criticism that's entirely valid because the whole goal is to error correct um all of the bugs in your thinking.

It's not to demonstrate that what you believe to be true has high probability, for example.

Um and so falsifiability is is what people know Popper for.

um most because that's one of the his like most important contributions.

But that contribution he made when he was sixteen.

and that was kinda like one of the first things he did.

But then what he went on to do was generalize that concept so that it applies not only to to science but to philosophy and and metaphysics and um art as well.

Art criticism is just as valid or stand up comedy.

Um so you can

You can find domains for which they get immediate real world feedback and criticism.

Um and those are the domains that are the ones that are going to uh make much more progress because they're the ones that have uh some sort of feedback mechanism that tells

them when they're wrong.

And so that's the most important thing.

Experimental tests are a kind of that, but it's not the only form of that.

Hmm.

Okay, okay.

That's fascinating.

So basically critical and I was gonna ask you about

about Popper and critical rationalism.

So so it's great that you did this already.

Um and basically that philosophy includes falsifiability with experiments in science, but it's not reduced to that in the sense that criticism will critical rationalism will be

helpful for other domains like art, stand-up comedy as you were saying.

Where you cannot falsify but you can criticize rationally.

Exactly.

What I would like to understand better is how does critical rationalism work concretely when you cannot make experiments?

So that can have to do with stand-up comedy, for instance, which I'm a big fan of.

It's one of my hobby.

So definitely take that as an example, please.

But also

Still in science you have um you have cases where you cannot run experiments, but you can make inferences.

Causal inference on quasi experiments, uh for instance, is is an example.

How would that work here?

so inferences is a word that uh has a specific meaning for machine learning people.

Um basically when you're not in training mode, you're in inference mode.

But maybe for the

people outside of ML, often inference just means like some secondary conclusion that is not immediately apparent, but yet can be inferred or guessed or deduced, or somehow you

get from A, which is the immediately obvious thing, to some B, which is less obvious.

Um so I'm gonna take uh take inference to mean that more general.

notion if if if that's uh a fair way to uh understand your your question.

Um and so if your question is how do we get from A to B?

How do we uh start with some information like it's raining outside to an inference like I should bring my umbrella, right?

Um what Popper says is essentially he doesn't care.

and that's gonna not be as vacuous as I as as I as it initially sounds.

So you could get from A to B because B logically follows from A.

So it could be A is I think that DNA is wound as a double helix.

I'm gonna think really hard and deduce that that would lead to an experiment where I shine a light on it in a particular way and it would make a particular shadow pattern on on the

wall.

So that would be like a deductive inference.

And that works in some cases.

It's

not most cases.

Um sometimes uh you're lucky enough that you can logically deduce secondary information.

other times you just need to make a wild crazy guess.

Um there is no mathematical or formulaic way to get from A to B.

It's just a conjecture.

Um a can you can take a lot of acid and go into a float tank for all Popper cares.

Um come up with some idea.

Um but what but Popper

did and what he his his um his move here was that everyone at the time was trying to figure out like a reliable way to go from a to b always.

And Popper said, I don't care.

All I care about is once you have B, how can we see if B is correct or if it's not correct?

So his whole framing is on error correction and it's not on a specific method that will reliably produce truth.

Right.

Um and so once you make that reframing, then all of a sudden a whole bunch of um moves open up to you, which is one of them is that w however you get your idea, I don't really

care as long as you can present your idea in such a way that it's clear to the person you're speaking to what you're trying to say, so they can understand it and then they can

criticize it.

Um so kind of the the the moves that are are looked down upon by by Popper would be to have an idea that is so vague

such that the person who you're explaining it to can't understand what you're saying and then can't even criticize it.

so if you reframe the way that we get from A to B as being less important than figuring out how to remove the errors from B, then you can start making a huge amount of progress

in all sorts of different domains.

So to go to stand up comedy, for instance, um

You could use stand-up comedy as an experiment, and is the experiment is quite simple.

Are people going to laugh when I say this thing?

If they don't laugh, the experiment has failed.

If they do laugh, then experiment has uh not been proven to be always funny.

it it it might not be funny in another context, but at least um you've attempted to falsify your view or criticize your view of the humorous nature of your your joke.

Um the Bayesian epistemologists would instead think of it as okay.

given um all the information in a person's head, how can you reliably produce funny jokes over and over and over and over and over again?

Like how can we come up with a a process to produce that funny joke?

Um, and that's I think uh the one of the central reframings um that Popper's work provides the the um the the reader, which is that it doesn't matter.

It doesn't matter how you get from A to B.

Like there obviously are differences between deductive logic and uh conjectural um

inferences.

But at the end of the day, all that matters is uh if you can criticize, if you can critique, if you can falsify, if you can error correct, if you can uh remove the mistakes

with the new piece of information.

Um and so Popper sometimes is described as um either a fallibilist, i.e.

the idea that all people are always fallible, or a negativist, uh the idea that all that matters is the elimination of error.

Um

Contrast it against the positivists who are positively trying to come up with a formula, a mechanism, an AI, um a training procedure that will reliably produce new new knowledge.

And so that's the the difference.

Yeah.

Okay, okay.

Super interesting.

Um so yeah, for the stand up example, that's interesting.

The problem is that so the main the main issue I would have with that is I mean the main question is not an issue.

My main question is feasibility in the sense that it's not that easy to run that experiment when you do stand up.

Like if you're on stage and you're not a known comedian, you go to open mics.

Open mics are a very weird sample of people who go there in the audience, and the audience is very variable.

So someone who is very well known, like let's say Ricky Gervais, if he goes to an open mic to test

um to test material.

It's somewhat random because usually they don't say Ricky Travay is gonna be there, but he knows his audience and he already knows what people who finds his joke funny are.

Who people uh who find his joke funny are.

The problem if you're a completely anonymous stemda comedian who just does random open mics

Is that you cannot really repeat the experiment because you don't have the same people in front of you, or at least the same the the same audience with somewhat similar audience

characteristics.

So you kind of repeat the experiment, but not really.

Like the only thing that doesn't change is the joke.

But the audience changes and it really depends on that.

So yeah, here I'm I'm a bit

Like my question is okay, like I see what you what poppers mean popper means on paper.

I'm not sure how feasible it is in all cases.

And a second question I have is okay, so it's critical rationalism.

I think we have covered the critical part.

And I think the rational part is very important too.

And so what makes a critic rational?

Because I'm guessing this is extremely important to Popper's framework, knowing

um the rest of I mean some of the rest of his production.

Mm-hmm.

Uh second question is easier and then let's do the first question after that.

Uh what makes a person rational is the willingness to listen to criticism.

Uh that's it.

It's criticism is not easy to receive.

I don't like getting it.

No one likes getting it.

but the the rational mind is the one who will um earnestly listen

to someone tearing their life's work apart and say thank you afterwards.

and that's it.

So rationality is is in Popper's worldview, um, and I would say my worldview too, in the sense that I'm extremely influenced by him, um, is is simply the willingness to receive

criticism lovingly and um in a spirit of appreciation.

Um and so someone I think Popper's um

was able to distill his entire philosophy into one simple motto, which is I may be wrong and you may be right, but together we'll get closer to truth.

and that's it.

And so it's all well and good to talk about um so now to move to your um your first uh example, which is that um if Ricky Gervais goes to a show of his, he's getting a biased

sample.

Right?

That was I think part of part partly what you're saying.

Um

And how can this be a good experiment um if he has a biosample?

Because the people who are listening to him are already biased to like his work and and so it's not really a great experiment.

Um I would say 100% agree with you.

Not a great experiment from the standards of rigor that say a um microbiologist would have to undergo in order to determine if their drug works for the whole population.

Of course, for that kind of problem

You absolutely need much more rigor than a a biased sample.

But if you reframe Gervais and the stand up comedian as not running a thorough unbiased A B test as like Meta would have to do or um or anyone doing drug discovery would have to

do, but just getting some more criticism.

then all of a sudden uh the uh biased sample doesn't matter too much.

um what matters is that it's just yet another round of of criticism.

And so, um, maybe to f fill it out a little bit, so you were uh bringing up this idea that it's not very feasible in practice because the stand-up comedian can't randomly sample

every person from every demographic and every uh geographic location.

Um absolutely they can't.

but what they can do is set up their life so that every conversation they have

They're doing a little bit of trial and error.

They're trying out a new joke here and there and they're saying, okay, that one landed a little bit.

Oh, didn't land.

Okay.

So I just tried to make my four year old kid laugh.

Didn't work.

Tried at my wife.

That didn't work either.

but this one made both of them laugh.

Okay, sweet.

Now maybe I'm gonna take this joke and I'm gonna try it at a few like off Broadway um comedy sellers, like like the the comedy seller.

Um and this is in fact how these comedians do it.

They they are constantly workshopping their jokes.

and then they'll go to a few small places where they're not really well known.

And then over about a year of this, they'll whittle down ten hours of material into like a tight forty five minutes.

and that whole year is a whole Pauperian year of subjecting themselves to um criticism from wherever it comes.

And so um thinking about experiments as a kind of criticism.

rather than criticism as a kind of experiment, which I think is what how you were thinking about it, um, is I think very important because when it comes to criticism, um doesn't

really matter if your s sample sizes are biased, as long as you have a lot of criticism.

and just the more of it you get, the better your work will become.

Of course, um nothing I'm saying would apply to to drug discovery.

Because there one kind of criticism is like

Uh you can't just test this drug on like your nephew and your wife because people will k you'll get people killed.

And so one of the kinds of criticism there is that you absolutely need to be thinking about like unbiased samples and having large sample sizes and stuff because the stakes are

so much higher, right?

Um and and so uh so that that was like one of the big aha moments um for me because I've always loved science.

I've been like a science nerd from from day one.

And like I remember

Learning about like the scientific method and like, I don't know, grade five or grade six.

It's that little like flow chart thing where you have like a hypothesis and then you do a little experiment and then you analyze your results and then it comes up with a new

hypothesis.

that is like the the same level of resolution as like the Bohr model of an atom where atoms or like electrons are floating around like this.

Um obviously it doesn't actually work like a solar system.

You have like a quantum cloud.

Um

And science doesn't actually work in this little circle.

Um science works by criticism first and foremost, where experimental testing is just one mode of criticism.

But but the reframing um that I found so valuable was just this like, shit, I need to set up my whole life to be able to receive criticism from as many people as I can.

And that means like not being a jerk.

Because if you're being a jerk, then people aren't gonna want to talk to you.

And if they're not gonna talk to you, then you're not gonna get criticism very well.

Or

setting up a podcast or in a Discord environment such that everyone feels comfortable like tearing each other's ideas apart in a constructive way.

Right.

and so how you set up your life and how you tune these dials such that you don't have so much criticism that everything is is destroyed and the community is broken and and your

life just sucks because it's not working very well.

Um to the opposite, which is even worse, which is like a Trumpian figure where no one can criticize him because he's gotten rid of every possible

person in this orbit who would ever be uh offer a critical ear.

And so that's the kind of the the the reframing which I think is extremely important.

Mm-hmm.

Yeah.

Completely agree.

This is this is really fascinating.

Um really love it.

And I think I I got much closer to to what you want to mean.

Um so a stand up example was was really really helpful because like this is close this is literally what I do actually what you describe is that I go to open mics but I also

workshop jokes on friends, they just don't know it.

exactly.

Yeah.

Yeah.

And then you and you're looking at their facial expressions, right?

And you're like, that one didn't land.

That's criticism, right?

even if they're not intending it.

It's just it's it's like another way is like feedback, just some sort of information signal from the world.

And note how this is intrinsically a um anthetical to the Bayesian epistemologist.

Because they think that the most important thing

is making sure your beliefs are well calibrated, right?

Your priors are well calibrated.

Um this is this is a intrinsically um inward view of epistemology.

It's not an outward view.

it's not a let's collect feedback from the world.

It's let me go away into my study for 16 hours and think really, really hard about all the evidence I've seen in the last fifteen years and then write a little probability on a

piece of paper and then publish a book based on that.

Um it's it's not about feedback from the world.

It's it's uh again, epistemology, not statistics.

Yeah.

Yeah, yeah, yeah.

Yeah.

I I have to say I don't care too much about Bayesian epistemology.

Yeah, totally fair.

Yeah, because like my goal my goal is usually not to calibrate my beliefs, but to update them and to see if I need to update them.

And so basically having an evolving mental model, that's like basically what I care about uh on whatever thing I do.

And to

evolve that mental model, you have to test it and do small experiments as controlled as you can, as we were talking about, but sometimes you can't, and also jokes for instance,

are extremely weird objects because it also it not only depends on the receiver, it also depends on the person making the joke, what he looks like, uh how he says the joke.

where he says it at what time.

It's a weird object.

But you know, like you can you can make progress anyways.

It's just it's gonna take a bit more feedback loops um than than a controlled experiment.

but yeah so and I think so I think it's actually very close to Bayesian statistics what what you're saying because the whole idea is we start from somewhere it's a bit like um

MCMC sampling.

Right?

We start from somewhere in the space, in the posterior space, but as iterations go, we get feedback and we get much closer to what the posterior distribution actually is.

Um so that's how I I think to put that in and review it in my head in a very simplified way.

Um but yeah, my Popper is really is really a fascinating author.

Uh I I got introduced to him mainly through falsification and

basically the foundation of epistemology, right?

Especially through um as you mentioned earlier, pseudoscience and basically trying to understand how you can let's say handle people who come with this kind of of claims.

Um and because it's so so intuitive.

Like the problem of pseudoscience is that they often have extremely

Good and intuitive thesis is like that's why it's so attractive to most of people.

It's like, oh yes, right.

Of course that that would work like that.

It I mean it makes sense.

And so yeah.

I was just gonna say and and note how much um evidence they have for their hypothesis when evidence from the the pseudoscientist or from the Bayesian pathologist.

Um

it's it's an infinite amount.

You can always come up with more examples of why your theory is correct.

And to the extent that um Bayesian like Bayes theorem encourages Bayesian philosophy, which then encourages Bayesian epistemology, which then encourages people to just act this

way in practice.

This is the exact antithesis of of um the the the critical um mindset where um so i I guess it's just an interesting example of how some dusty old equations

written four hundred years ago can turn into um the practices and behaviors that I claim lead to pseudoscience and to um conspiracy theories.

Mm-hmm.

Mm-hmm.

Yeah, I don't so I'm gonna butcher the example but the the point stands.

That's why I'm talking about it.

But I don't remember if it's Popper or if it's maybe Cal Sagan, maybe Sagan.

Who took that example of like to show you that you cannot prove a negative.

Like taking the example of you cannot prove to me that there is that there doesn't exist a giant teapot orbiting I think it was Neptune or whatever planet.

Yeah.

So yeah, you seem to be knowing that example, so please take it away because you're gonna make it much more justice than I I will.

Yeah.

Um

So that's Bertrand Russell's teapot, famous example.

Um and I believe he was um bringing that up in relation to his um atheism.

I'm very much an atheist as well.

And uh the arguments that he was arguing against were well prove to me, Bertrand, Bertie Boy, that God doesn't exist.

Prove that this thing d isn't there.

And he basically said, Well, you you can't prove a negative.

You can't prove the non-existence of something.

and uh and so that's related to um critical rationalism in in a number of ways.

One is is taking it in its positive um framing is that many ideas, many of which are are good and valid, are not falsifiable.

you can't prove them one way or or the other.

Um

In a more negative framing it's this idea that um if someone doesn't want to hear criticism, if someone puts forth challenges that are impossible to solve, then you can

never puncture through the dogma.

Um and so in that in that example I think he he kinda highlights two nice Pauperian concepts.

Um, even though Burton Russell himself was uh inductivist, and so he was someone that Popper argued about or argued with a bunch on on those that regard.

But

but yeah.

Um actually and another thing with Bertrand is it's interesting.

Um so uh Russell himself um I think is a beautiful example of the critical mindset because him and um Alfred North Whitehead, I believe his name is, um, came up with uh this like

one thousand page tome that was trying to um put all of mathematics um on a firm foundation of set theory.

And so they put hundreds and hundreds of hours into trying to um prove that all mathematics can be derived from a small set of axioms.

Um and then this annoying guy named Kurt Gödel showed that their entire life's work was impossible.

Um and what they did was say thank you.

That's it the you've just answered my question.

They didn't um start attacking Kurt Gödel, like imagine if they were on Twitter these days.

but it's it's beautiful example of um the f falsif the fallible attitude and this idea that uh you should be grateful when someone is kind enough to spend time to think about

your ideas and offer you criticism of them.

Um and and their uh their uh jettisoning of a thousand pages of of work given a proof that showed they were wrong is I think a really nice example of that.

Yes.

Yeah, yeah, yeah, for sure.

and actually something that reminds me

From what you were saying about the rationality of the criticism defined by Popper.

If I understood correctly, that's only on the eyes of the criticized, right?

Not the criticizee.

So basically it becomes rational just because I take it rationally if somebody criticized me, uh like criticizes one of my jokes, for instance.

And

Not because the critique is not rational.

Is that correct?

It is correct.

Yeah.

You you as much as I would like to be able to uh control the dogmatism of other people, uh as much as I try, I I can't do that unfortunately.

Yep.

Um all I can do is control the way that I act and I receive criticism.

Um and sometimes when you're speaking with uh a fine gentleman like yourself, both participants are just

naturally trying to get towards the truth and um and all criticism is offered in that in that spirit.

Um other times the interlocutor is not like that and they just hate you.

Or I was on uh the Doom Debates podcast, which if any listeners want to get a good example of what Bayesian epistemology looks like these days in practice, I would listen to Doom

Debates.

Um I was a bit of a jerk on that podcast.

I was in a bad bit of a bad mood and there was three hundred uh YouTube commenters

Who were all tearing into me.

and it hurt.

It definitely hurt because I thought I did well in that podcast, but my attitude was kind of just I had a bit of a bad attitude.

Um, and so yeah, it would be so nice if I could shout at each of these commenters and tell them to be more rational, but I'll take criticism where I can get it.

and even if it was a bit harder to to hear, it was still extremely valuable for me.

and was one of the episodes that I learned from the most um because of how much that particular criticism hurt.

But again.

Criticism in any form is is is valuable and so so I definitely adopted.

Yeah that changed changed a lot of things based on that.

But but yeah, unfortunately you can't control other people.

You can just control yourself.

And I think realizing that criticism is like the most valuable thing that you could possibly receive from anyone in any form, um, I think is is such a powerful reframing and

it um it's been extremely um important in my own life.

Yeah.

Definitely.

Yeah, but I mean couldn't agree more.

And so I'm obviously asking you that because of yeah, the like

Social media basically in the tendency of people to be extremely radicalized on there.

And it's just like, yeah, but what do you do if somebody is just giving you a critique which is adominum?

You know, like they have nothing interesting to tell you and they just attack you instead of attacking the idea.

sure.

So and the you know, the scientist in me wants to find a framework to handle that and just like, you know, have an answer to that.

But then I think

Popper's answer to it is actually much better and much more useful.

And actually, I think here probably the Stoics had that answer before Popper.

I see a lot of resemblance with Stoic philosophy here and like Epictetus, where it's like an insult just becomes an insult because you're interpreting it that way.

Um But if you don't take the ins what's meant as an insult as an insult, it is not an insult.

And actually it's gonna be much better because it's gonna make the insulter much, much more uh angry at you because uh they'll see they basically cannot cannot get at you.

and and also the idea in stoic philosophy that well actually, you know, um the obstacle is the way, so the critique you got is actually always useful because it's a critic, but it

It's not binary.

The usefulness of a critic is not binary.

It's a spectrum.

And maybe it's at zero if it's just an ad aluminium attack.

Maybe it's at one point five percent, where it's like, okay, most of it is garbage.

but here there is one part that's actually that's interesting.

Um I could actually get better with that.

Wherever whatever the source or form of the criticism was.

And I think that that goes directly into into Popper's uh philosophy as as you were saying.

Yeah, stoicism, um and critical rationalism or fallibil fallibilism pair beautifully together.

Absolutely.

Yeah.

the stoicism i is less about like where does knowledge come from and more about just how do you live the good life, how do you do with less?

How are you o if you were to lose your house tomorrow, are you gonna be okay?

and I think the kind of uh equanimity that stoicism can teach um allows you to receive criticism much better.

Um

My favorite line of from one of the stoic philosophers um is something like, uh Stoicism teach I got this from a a lecturer online whose name I'm blanking on at the moment.

I can try to add the show notes.

but at the end of the lecture he said, uh Sto the Stoics teach us that every man will die, but not every man will die whining.

And I just lied.

Yeah.

Um and yeah, every man will be criticized, but not every man will be criticized and whine about it afterwards, right?

And yeah, and so yeah, with regards to like ad hominems and um and just when people a attack you, um of course nothing prevents the critical rationalist from criticizing the

criticism.

ad infinitum.

And like sure, like where does the phrase like critical thinking come from?

Well it's a derivative of of Popper's work, right?

Um but yeah, so sometimes the criticism you receive just is low quality and they're just attacking you as

uh you as the person.

Um so often that I will tend to ignore.

Or like I guess one danger of talking about critical rationalism um and a trap that I've arguably fallen into in this conversation is like deifying criticism and pretending like

it's this outerworldly thing that never hurts.

But no, of course it's gonna hurt.

Like the the deeper the criticism is, like sometimes it really hurts.

And so especially when you're in like an extreme

Extremely critical debate with somebody um and emotions are high and tempers are are are uh overflowing and this has happened to me many times on on the podcast, um listeners of

increments will will know what I'm referring to.

but but yeah, a little bit of stoicism added on top and just a bit of a realization that everyone is human and that like even if someone has given you a big paragraph online and

three quarters of it is ad hominems, but maybe like one quarter of it is actually valid.

then you, the person receiving it, can focus their conversation on the germ of truth um and get them to sharpen their own criticism of you.

Um if you are able to control your emotions.

Um and I say that as someone who is extremely not able to control my emotions sometimes.

But it's a lifelong project.

But but yeah.

And then so um so basically in a nutshell, Popper's philosophy is criticism is at the foundation.

Everything you think about the scientific method

comes out of that.

Peer review, posting your like your paper on Twitter.

Like I remember I was much more scared to post a paper on Twitter than I was to submit it to peer review because I knew that Twitter was going to give me way harsher criticism.

And so like you figure out how to set up your work such that you get like the right amount of criticism at the right stages because sometimes an idea is so early that like if I was

to explain it to you, you would be like, ah, I think it does it's not going to work at all.

And so you

kind of want think about titrating in the criticism at the right stages.

You want to think about like, okay, there's a wall of YouTube commenters that are all telling me I'm a terrible person.

Um, but there's like four or five people whose opinions I really do respect and I'm going ask them to see what's valid there.

And so framing your entire life around how do I um take advantage of the fact that the world is going to criticize me whether or not I like it.

And so I may as well harness that that energy in a effective way.

is the game of figuring out how to do science.

Um and that's that's um one of the his central insights and why he calls his life's work critical rationalism.

Yeah.

Yeah I would I would even argue not only do science uh the white right, but mostly life uh exactly.

Exactly.

Yeah, no uh just if I can echo that, because I don't want to give the impression that this is just what us stodgy academic y science people do.

It's just as much what the

The roofer does when they're trying to figure out why your roof is leaking, just as much what your kid does when they're trying to learn to ride a bike and then they fall and

that's a little bit of error correction.

They should have done it differently.

Um and so there is no distinction in the Pauperian worldview between the scientist, the human being, the artist, the comedian, the UFC fighter.

Uh it's all it's all just learning from the world as as uh as as best you can.

Yeah.

Yeah.

Yeah.

Yeah.

Love it.

Love it.

Um fascinating.

and actually, you know,

So most of my work is trying to take the latest state of the art science and trying to distill it to people who either don't have the time or the inclination to um read the

papers or talk to the researchers and and and just yeah, like doing that.

And I think it's very important because uh papers are fine, but nobody reads them only scientists and they just read each other, you know, but normal people don't and so

We need people to take the science and just push it out there because otherwise m most of the population is just gonna say you stay with uh old beliefs and uh that's not what you

want in the end.

You want the science to be applied and not only written.

And so that's why I also I love a lot stand up comedy.

You've got a lot of very good stand up comedians who actually kind of do, you know, um epistemology like that.

I think

Ricky Gervais is actually a good one.

Uh Jimmy Carr, I think, is a good one.

They often are very rational, so that's interesting because I think they have to be very analytical.

And a a a very good show for that also that I often cite is the Big Bang Theory.

and Sheldon in particular has some very good snippets, you know, sometimes.

And and I think it's great.

Like and like there are there are two which comes to mind for our conversation, which is one.

Um that he so Penny makes a joke about Nebraska.

She's from Nebraska and I don't know, like another state that Nebraska Nebraska looks down on.

And she makes that joke in the apartment of Sheldon in Pasadena, California, and nobody laughs.

And and then she's like, Oh well, I guess that joke is only funny in Nebraska.

And then Sheldon says, you're gonna say that.

With the data at hand you can only say that that joke is not funny here.

Which is exactly what you were saying.

Exactly.

You there's no way to generalize beyond the local and a reliable end of the And I think that's awesome because like everybody can understand that.

And yet it's a very profound concept of scientific thinking.

And I think this is a great distillation of scientific principle for everybody.

Um which is extremely, extremely precious.

Um and another one I I really love that's also related to what we're saying, you know, but the with the teapot and the burden of the proof is basically on the people saying that God

exists.

Is um so Sheldon's mother in the show is very religious, so she basically tells him at some point that uh to prove to him that God doesn't exist.

And then Sheldon like, well

Imagine that I told you that I believe there is a giant invisible man in this room that influences your behavior.

Um the burden of proof wouldn't wouldn't be on you to disprove me.

It'd be on me to prove you that I'm actually right.

And it's ex it's yeah, another example of of of a great way to distinguish what we've been talking about.

Exactly.

Totally.

I love it.

Yeah.

Um so

Actually I still had a lot of questions for you, but it's it's getting late.

Uh so I don't wanna I don't wanna take too much of your time.

And um since I'm gonna come on your show at some point, actually I can keep some of those questions and if you want I can I can ask them to you and it's just gonna start spark some

some interesting uh interesting conversation between us.

What what do you say?

Yeah, sounds great.

Yeah, I'm in I'm in no rush, so yeah.

Awesome, so I let's do that.

I'll I'll keep these uh these questions on my computer and then when I come on your show I can I can actually uh ask them to you and then we'll just talk about that.

I have some some some good one about a a bit more concrete, you know, one about a paper by Andrew Gelman.

So yeah, just uh interesting stuff.

and also yeah, and two other questions that actually I think you're gonna like about falsifying your beliefs.

So

But that's just a teaser for you.

Um before you go next time on, next time on.

Exactly.

So before you go though, um, I do wanna talk a bit about your show, Increments Podcast with Panchug.

You've been running that for a while now, uh, and so I can definitely say congratulations because it's not an an easy job.

what's the elevator pitch?

Who is the audience?

And

What made you want to start a Philosophie of Science Podcast in the first place?

Yeah, what's the elevator pitch?

I don't know what the elevator pitch is, so I'm still trying to figure that out.

Um typically I just say it's applied philosophy, um, or it's a philosophy uh and science and history and everything in between.

Um The elevator pitch is that so I mentioned a little bit about Popper, um how he was writing from in the 20th century.

and if we just think a little bit about

all of the things that happen in the 20th century.

So we have Marxism, we have Freudianism, we have Einstein, we have Turing, we have Kolmogorov, we have um Dawkins and the Selfish Gene, and we have Turing, if I already

mentioned him.

Um so to study Popper is to study everything, because he was in the weeds with all of these subjects.

and so

I like to think of epistemology as like the master subject, this the center of the Venn diagram.

and so on the podcast, we are kind of just exploring outwards from the center.

So we have an ongoing series where we're um going through chapter uh by chapter of conjectures and refutations.

Um in between that we spend a lot of time talking about um AI and in particular

reasons why um b artificial general intelligence is not um going to happen in our lifetimes.

Um controversial claims, see next episode.

we spent a lot of time talking about effective altruism um in a bit of a negative way.

Uh effective altruism is a strong hotbed for Bayesian epistemology and for a newer philosophy called long-termism, which I'm quite critical of and I can maybe share a few

blog posts.

um to put in the show notes where where I um criticized one of William MacAskill's papers and then he ended up taking it down and removing all the quotes that I had quoted from

him.

we talk about open societies, so we haven't talked at all about Popper's political philosophy, but um that's a huge current.

Um and then we just have random episodes discussing kind of current cultural trends.

So um

Jonathan Heidt's work, um, this idea that if you give screens to your kids, you're gonna rot their brains.

Um, my daughter has an iPad and she's loving it, and it's been the best thing ever.

So extremely pro screen time, for example.

Um, we had an episode about recycling and why recycling doesn't actually work very well.

Um so we try to uh go from the the extreme niche philosophy discussions about what truth is, how probability works.

What certainty means?

How does logic work?

How do you go from logic to um to the real world?

Like what's the connection between logic and the real world?

All the way down to the super mundane, like, does recycling work?

Um, what's this notion of the patriarchy?

Is the patriarchy uh a valid concept?

Um, and everything in between.

And um both Ben and I, um I should have s shout out Ben.

So Ben's finishing his PhD at CMU, he's like

statistician extraordinaire.

He's doing fundamental work on e-values, which you should totally have Ben on, because I think your um your audience, which is who's primed for technical discussions, would love

to to to hear some of the stuff that he's working on.

Is uh Mike Jordan, for example, is on his committee, so it deeply into the the L community there.

Um and him and I have extremely short attention spans and so we get bored of subjects pretty quickly.

And so we like to keep moving on to new things and the world right now is um not boring, shall we say.

So there's plenty of stuff to talk about.

and those are the kinds of subjects we touch on.

Why did I start the podcast?

Why do we start the podcast?

it was started um during COVID when all of a sudden I couldn't do my favorite thing in the world, which is going to the bar and arguing with all my friends about politics.

Um and so when that stopped, I was like, Well

This is not acceptable.

And so Ben and I started our little conversations, uh gosh, almost five years ago now.

And uh we just had our hundredth episode a couple weeks ago and that was a n nice um accomplishment.

And yeah, we're looking forward to the next hundred.

I've no idea what the feature's gonna hold, but just chasing our interests as they come along.

Well done.

Well done.

Yeah.

this is a very interesting show and I didn't know about it before Andreas put us in put us in in touch.

but yeah, have to say um

Definitely you have a new listener here and this is exactly this kind of of uh nerdy, serious and entertaining content that I love, that I try to do here, and that uh I hope

the world had more of.

So definitely thanks a lot for doing that.

And and I'm sure a lot of my listeners are are gonna enjoy it.

So uh folks, it's gonna be in the show notes.

Um make sure to um give a listen to to Vadens and Vadens, sorry, uh and and Ben's podcast.

That's that's the French guy in me, you know, wanting to to get out because I can never let let him out when I'm in the US.

Um so sometimes I have to talk like that, like a real a real French.

and and yeah, so

Look at that.

Um I'm gonna be on the show.

So it's it's the it's the mark of a show that's not really high quality with uh guest screening, you know, but still other other uh people who've been on Vadence Vader's show

are are really I vouch for them.

So yeah.

So yeah.

thank you so much.

No, no, you bet, you bet.

you're doing a lot of of good work, so so I'm really happy to uh to help you guys on that.

And um well

Let's call it a show, Vaden.

I'll come on your show and ask you my other questions.

Before we go though, I have two questions ask you at the end of the show.

before you go.

You know, ask that to to all my guests.

So so that I have a distribution of of answers, right?

Um so first question, what's your favorite thing about Bayesian epistemology?

I'm kidding.

It inc well no, they gotta answer the question.

Um it encourages people to learn more about math.

I think math is awesome.

And if that's a good entry point in for some people, then I think it's a great way to to to start learning about uh math and equations.

Yeah.

Okay.

Love it, love it.

um and yeah, that that's a good answer.

Uh but that was a joke.

The actual first question is if you had unlimited time and resources, which problem would you try to solve?

Ooh, nice question.

I if I had unlimited time and resources I think the problem of education is really interesting.

I think that uh figuring out how to um create environments where people are free to explore their own interests and explore their curiosity, um while also being realistic

about the fact that there's a short supply of um qualified teachers and that

parents need a place to put their kids while they go to work um is an extremely interesting question and one that I think if we were to um make progress on that, it would

just unlock an infinite amount of unbounded potential.

But I think there's a lot of um a lot of wasted opportunities that come from um people not having the freedom to just explore their their own interests and curiosity as much as as I

think that everyone should be able to.

So maybe that.

Yeah.

Yeah.

Yeah.

Yeah.

Um very very on point answer uh with with your brand, so I am not surprised.

Um and second question I think I know the answer to that one, but we'll see.

If you could have dinner with any great scientific mind, dead, alive, or fictional, who would it be?

Okay, so popper's the easy answer, but I'm not gonna say popper.

Um Okay.

Richard Feynman just to make me wrong.

Yeah.

Richard Feynman, because

I also play music on the side and I heard he's a insanely good bongo player or was an insanely good b bongo player.

And I'm attracted to the the kind of mind that loves science but also loves people and human interaction and humor and joy and life and drinking and just having a good time.

Um and uh I find that that intersection is in short supply.

Um and so Richard DeFeynman would be definitely somebody I'd love to

Go for dinner with no.

Okay.

Yeah.

Yeah.

Love it.

and yeah, that dinner table is getting uh crowded.

We've we've got a few guests already.

Uh so we'll we'll need to find um a bigger restaurant, but but let's make it happen nonetheless.

Yeah.

Sounds great.

Yeah.

Awesome.

Awesome.

Well, uh Vadin, thank you so much for for coming on the show.

there will be a few links.

Folks in the in the show notes for that episode, also a few related episodes.

Feel free to to check them out.

I think they're gonna be very interesting.

Hope you enjoyed that episode.

That was uh somewhat of a of a change of pace uh of pace from the from the the classic uh technical episodes, which I like to do from time to time.

And and I hope you you did too.

And on that note, Vaden, um

Well, I'll I'll see you soon, but mainly thank you so much for taking the time and being on this show.

Thank you for everything.

This was a fantastic conversation and had a blast and looking forward to part two.

I'm happy to hear that.

This has been another episode of Learning Bayesian Statistics.

Be sure to rate, review, and follow the show on your favorite podcatcher and visit LearnBaseTats.com for more resources about today's topics, as well as access to more

episodes to help you reach true Bayesian state of mind.

That's LearnBaseTats.com.

Our theme music is GoodBayan by Baba Brinkman.

Fit MCLAS and Megaran.

Check out his awesome work at Bebabrinkman.com.

I'm your host.

Alexandora.

You can follow me on Twitter at Alex underscore andor like the country.

You can support the show and unlock exclusive benefits by visiting patreon.com slash learnbase tax.

Thank you so much for listening and for your support.

You're truly a good baby and change your predictions after taking information.

And if you're thinking I'll be less than amazing, let's adjust those expectations.

Let me show you how to be a good daisy.

Change calculations after taking fresh dating.

Those predictions that your brain is making.

Let's get them on a solid foundation.