MATH FOLKS -- explain the flaw in this reasoning (Doomsday Hypothesis)

[Tycho- Taylor's Version]

Okay, the whole set-up of this thing seems irrational to me, but I can't figure out where exactly the problem lies. Apparently the disagreement over that fact is pretty widespread, so I'm not alone. Maybe someone here can shed light on the situation.

From the article here: http://rationallyspeaking.blogspot.ca/2013/01/its-not-all-doom-and-gloom.html

This is the original line of reasoning:

Suppose that, after a devastating explosion at a large fireworks factory, you are called in to investigate. Forensic professionals are able to discern that the cause of the blast was an unstable batch of explosives being used for the company’s latest product, the “Blam-O 3000” – already on the market, which you now know to be essentially a bunch of ticking timebombs. Unfortunately, the only records of how many Blam-O’s had been sold happened to burn up in the explosion.

However, at a convenience store in a city on the other side of the country, your agents find one Blam-O with serial number 112. What does this tell you about the total number N of Blam-O’s on the market?

Trivially, it gives you a lower bound on the total number of Blam-O’s – namely, 112. But we can do a little bit better than that. Assuming that the unit found at the convenience store was selected randomly from the population N of all Blam-O’s, it seems very unlikely that N = 10 million, and much more likely that N = 1,000.

This is because if there were 10 million units, the chances of finding a serial number in the hundreds are only 999/10e6 = 0.1% – whereas if there were only 1,000 units, it becomes an effective certainty.

And now (less trivially) applied to a Doomsday scenario:

The Doomsday Argument (DA) seeks to show that knowing how long it’s been since the dawn of human life tells you something about the number of humans who will have ever existed. The argument was first put forth by Brandon Carter, but has been popularized by John Leslie and Nick Bostrom, whose primer on the subject is well worth reading. I will follow the terminology and numbers in Bostrom’s primer, for the sake of inter-comparability.

Putting it into the context of the above “Blam-O” problem, we are trying to estimate N, the number of humans who will have ever lived (analogous to the number of Blam-O’s on the market), based on knowing our birth rank R (analogous to a serial number). Our birth rank is just the number of humans from the beginning of the species to our birth; for example, if Genesis were true, Eve would have a birth rank R=2. Based on what we know of hominid evolution, according to Bostrom, the birth rank of any human living today is roughly R = 60 billion.

Now, following the presentation in Bostrom’s primer, we consider two hypotheses about the life expectancy of the human species: DoomSoon and DoomLate. For simplicity, we will treat them as the only two possibilities on offer; although that isn’t true, it doesn’t affect the broad direction of the underlying logic, and it makes the math easier.

DoomSoon is the hypothesis that we are already around halfway through our existence as humans, and that we will be wiped out by some catastrophe just as the number of humans N who ever existed reaches 200 billion. DoomLate is the hypothesis that the human species will live much longer – the number of humans who will have ever existed under this hypothesis is 200 trillion.

The prior odds you assign to these hypotheses depend on your evaluation of existential risks, discussed briefly above; let us say that you think O(DoomSoon) = 50:1 against, which implies (since we stipulate that DoomSoon and DoomLate are mutually exclusive and exhaustive hypotheses) that O(DoomLate) = 1:50 against = 50:1 in favor.

What evidence does our birth rank of R = 60 billion give us about which of these worlds we are in? Well, given that DoomSoon is true, the probability of finding yourself living at birth rank 60 billion or less is P(R<60b|DoomSoon) = 60e9/200e9 = 30%.

Meanwhile, the probability of finding yourself at rank 60 billion or less, given that DoomLate is true, is P(R<60b|DoomLate) = 60e9/200e12 = 0.03%.

The evidence thus favors DoomSoon over its competitor DoomLate by a margin of 30/0.03 = 1,000. This gives posterior odds on DoomSoon of O(DoomSoon|R<60b) = (1:50)*1000 = 20:1 in favor. As we step back from our dubious simplifying assumptions, it should still be clear that the DA strongly favours hypotheses which put the end of humanity earlier over those which put it later. This is our qualitative conclusion.

So what's the problem here, or is there one at all?

 

[John Stockton]

I don't think it's a problem rather an explanation. This is how I saw it: if you are dealing with an ordered list (1,2,3,4,...,n), and you randomly select a single item whose order is given, but do not know how many total items are part of the list, it's much more likely that the total number of items is closer to the lower bound given by the item than a far away number.

E.g. let's say I have a box of numbered balls, with balls being numbered 1,2,3,4,...,n and I don't know how many total balls there are. I then take a random ball and it's numbered 5. If the total number of balls is n=5, you have a 1 in 5 chance of selecting that ball randomly, which is quite probable. If n=100, then you'd have a 1 in 100 chance of selecting that ball and only a 9 in a 100 chance of even selecting a single digit number. So it seems more probable that the total number of balls is closer to 5 than 100.

 

[MortalWombat]

I'm calling biased experiment. P(R<60b|DoomSoon) is actually 1. P(R<60b|DoomLate) = 1. When you're picking blamo's, you know the buggers exist. When you're picking humans, you know they don't.

If you go with P = 1, then you end up with odds of 1:50 and 50:1, ie your original assumption. That's because your experiment yielded diddley squat.

 

[gotBooched]

I'm calling biased experiment. P(R<60b|DoomSoon) is actually 1. P(R<60b|DoomLate) = 1. When you're picking blamo's, you know the buggers exist. When you're picking humans, you know they don't.

If you go with P = 1, then you end up with odds of 1:50 and 50:1, ie your original assumption. That's because your experiment yielded diddley squat.

Get it, boi!

 

[MortalWombat]

Get it, boi!

Is that an upvote or a downvote? I am feeling on very shaky ground having voiced an opinion on something I have very little understanding of. Like that gameshow host thing still does my head in, let alone

 

[MortalWombat]

oooh, that embed didn't work


here's a link: http://en.wikipedia.org/wiki/Bayes'_theorem

 

[adam199]

Pretty interesting to think about. The problem here lies in the fact that outside factors are ignored. You can't predict the doomsday of a species based simply on the number of members that have existed, and the unlikelihood that the total members will be exponentially higher.

The growth of a species is almost always exponential. two turn into four, four turn into eight, and so on. The first two members of any species never think that there will be billions of their own kind. I'd say it's always safe to assume that any growing, healthy species is closer to zero than their upper-bound, based simply on exponential growth.

The question here is whether our species is growing healthily. The flaw here is in the fact that this problem only factors the number of members of our species so far. God's not going to hurl an asteroid at Earth just to prove a basic probability question true.

 

[tkotom]

well the blam o is a bad example since the 112 serial number could refer to the mixture's code, the type of casing or material used or a number of other things.. what's to say that it is number 112 of blam o's

 

[def1]

it seems like two different premises to me.

On one hand you have blamos which have stopped production.

On the other hand you have people who are still being born (produced).

But Bayesian statistics is over my head so i dunno

 

[Tycho- Taylor's Version]

I don't think it's a problem rather an explanation. This is how I saw it: if you are dealing with an ordered list (1,2,3,4,...,n), and you randomly select a single item whose order is given, but do not know how many total items are part of the list, it's much more likely that the total number of items is closer to the lower bound given by the item than a far away number.

E.g. let's say I have a box of numbered balls, with balls being numbered 1,2,3,4,...,n and I don't know how many total balls there are. I then take a random ball and it's numbered 5. If the total number of balls is n=5, you have a 1 in 5 chance of selecting that ball randomly, which is quite probable. If n=100, then you'd have a 1 in 100 chance of selecting that ball and only a 9 in a 100 chance of even selecting a single digit number. So it seems more probable that the total number of balls is closer to 5 than 100.

It seems pretty straightforward, I'm not sure why exactly my gut reaction is so against it. I've never been good with probabilities. Maybe it's just because the conclusion is so intuitive: it's less likely that there will be many humans that it is that there will be fewer humans.

But I'm not really sure why I think that either, besides the typical ecological constraints. Maybe this is the mathematical reason spelled out in plain English.

I'm calling biased experiment. P(R<60b|DoomSoon) is actually 1. P(R<60b|DoomLate) = 1. When you're picking blamo's, you know the buggers exist. When you're picking humans, you know they don't.

If you go with P = 1, then you end up with odds of 1:50 and 50:1, ie your original assumption. That's because your experiment yielded diddley squat.

Can you explain this a little bit further? I'm not good at this stuff.

This is from the article as well:

Critique 4: Richard&#8217;s objection. An RS commenter named Richard got me thinking about the DA. His objection has to do with the proper assignment of priors in the DA, not directly with the validity of the Bayesian update. According to him, although there may be a 1000-to-1 likelihood ratio for DoomSoon over DoomLate, this effect is exactly counteracted by one&#8217;s priors on DoomSoon and DoomLate. DoomLate, on this view, should have a greater prior (200e12/200e9 = 1,000 times bigger, to be precise), essentially because you are likelier to find yourself in a larger group of people than a smaller one (thus, I think Richard&#8217;s objection ends up being equivalent to the self-indicating assumption).

As Richard says, Bostrum&#8217;s reply here is that if there are both larger and smaller populations, we should expect to find ourselves in a large population with a large birth rank; the fact that we do not shows that the above priors should already have been adjusted. I tentatively agree with Richard that it looks like there might be something circular going on here: we are using the DA to determine our priors, to which we then apply the DA?

Will read the link too.

 

[Tycho- Taylor's Version]

well the blam o is a bad example since the 112 serial number could refer to the mixture's code, the type of casing or material used or a number of other things.. what's to say that it is number 112 of blam o's

Hmm.. I think we can assume for the sake of the example that the serial number indicates the position of production.

it seems like two different premises to me.

On one hand you have blamos which have stopped production.

On the other hand you have people who are still being born (produced).

But Bayesian statistics is over my head so i dunno

Yea I dunno either, but here are a couple other critiques that may have something to do with your comment:

Critique 2: We somehow know a priori that we are among the earliest humans. E.g., we expect most humans to be cyborgs, but we are not cyborgs, therefore, we are known to be early.

 

[TheQu**]

http://arxiv.org/pdf/0806.3538v1.pdf

 

[ocean size]

I like the Bayes theorem approach - it is something I think about more and more in my field. I just don't see that it applies here. When you are pulling a random blamo out of a population you know there is some finite number - looking at the number of humans you know we are is not the same thing - we don't know how long all of the given factors will allow ourselves to keep us around. I am not a good enough statistician to know what the problem it is but it seems like a violation of assumptions.

The blamo thing would be more comparable to picking a human throughout the human-rank of anatomically modern humans and putting odds on what the highest rank is now.

Think of it this way - Rank a person's days and set odds on how long they will live. Do it in a developed country with a good population growth rate. Say you pick a toddler, and this concept says they will most likely live to 3 or 5 instead of 76 or 90, since they are two now.

No, bullshit, a whole lot of factors goes into setting how long they can expect to live, which is the 76 to 90.

I'm linking to three different places:
Objections to the hypothesis: http://mind.oxfordjournals.org/content/107/426/403.full.pdf
A refutation of those objections: http://www.anthropic-principle.com/preprints/ali/alive.html
A more balanced take that says it is sometimes appropriate: http://brian.weatherson.org/doomsday.pdf

 

[Tycho- Taylor's Version]

http://arxiv.org/pdf/0806.3538v1.pdf

Qu to the rescue!

Although I'll admit that after quickly glancing at the front page I find it more intimidating than the guy in the "you gun get raped" meme.

Will attempt to understand though.

 

[ocean size]

Seems like Bayesian stats can either be very intuitive, or completely counterintuitive... awesome.

 

[Tycho- Taylor's Version]

Seems like Bayesian stats can either be very intuitive, or completely counterintuitive... awesome.

I want to laugh... I really do. I'm sure it's clever -- whatever it means.

So... Introductory books on Bayesian stats! More crap to add to the reading list lol

 

[PolishHeadlock]

Dividedbyzero.jpg

 

[ocean size]

I want to laugh... I really do. I'm sure it's clever -- whatever it means.

So... Introductory books on Bayesian stats! More crap to add to the reading list lol

No hidden pun, just referring to xkcd's take with the awesome. There is a website that gives a really good introductory explanation -

edit: ok here's is: http://oscarbonilla.com/2009/05/visualizing-bayes-theorem/ I think the cancer example is a really good one of where answers it gives are surprising compared to what back-of-the-envelope calculations you do in your head, yet strangely intuitive at the same time.

Here is a quote I read on a slightly unrelated topic that sort of sums up the difference between Bayesian and frequentist stats:

There are two ways to define statistics and both require data as well as hypotheses: (1) Frequentist statistics makes probabilistic statements about data, given the hypothesis. (2) Bayesian statistics works the other way round: it makes probabilistic statements about the hypothesis, given the data. Frequentist statistics prevailed as a major discourse as it used to be computationally simpler. However, it is also less consistent with the way we think &#8211; we are nearly always ultimately curious about the Bayesian probability of the hypothesis (i.e. &#8220;how probable it is that things work a certain way, given what we see&#8221 rather then in the frequentist pobability of the data (i.e. &#8220;how likely it is that we would see this if we repeated the experiment again and again and again&#8221.

 

[PolishHeadlock]

Seems like Bayesian stats can either be very intuitive, or completely counterintuitive... awesome.

I sent that to my sister who has a Phd in Econ and loves stat.

I'll let you know later if I loled.

 

[MortalWombat]

http://arxiv.org/pdf/0806.3538v1.pdf

I have read this paper end to end.


Here is my considered opinion on the matter, armed with the knowledge in that paper.

Spoiler