-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
-
We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.
Benfords Law: So this proves we live in a simulation right?
- Thread starter Tawpgun
- Start date
OP
OP
Yeah
An extension of Benford's Law predicts the distribution of first digits in other bases besides decimal; in fact, any base b ≥ 2. The general form is:
P(d) = log*b(d + 1) − logb(d) = logb*(1 + 1/d).
For b = 2 (the binary number system), Benford's Law is true but trivial: All binary numbers (except for 0) start with the digit 1. (On the other hand, the generalization of Benford's law to second and later digits is not trivial, even for binary numbers.) Also, Benford's Law does not apply to unary systems such as tally marks.
Well, that's just one example. Think about it this way. Is it more likely that I have 1 house or 12? How about 1 car or 367? Or one apple or 7? One and zero are the most common numbers in everything, and since zero doesn't "count" in this scenario, 1 counts as a number with 1 as the leading digit. I can't account for every single data set, because I didn't look into all of them, but I'd expect them all to have a similar, simple explanation.
Again, confounding variables. Not everything is some crazy numerology shit.
If the law holds (for whatever reason) for all numbers, a random sampling of numbers will likely pick up more of the more common numbers.Yep. It fits very perfectly for things that tend to grow exponentially. This video goes into it.
Benford's law (with Vi Hart, 2 of 2) (video) | Khan Academy
Vi Hart visits Khan Academy and talks about the mysteries of Benford's Law with Salwww.khanacademy.org
I do generally understand that.
Whats odd is it can also be applies to things that aren't based on exponential growth/decline. Like the guy that pulled random numbers from different data sets in a book. The numbers shouldn't have a relationship to each other, but they followed this sequence. Things in the natural world follow it. It all comes down to the logarithmic scale.
And I get that... a little. But it doesn't still mean its not a cool phenomenon that has amazing applications. Reading through explanations and then looking at more benford examples I seem to waver. I watched the netflix doc. This is magic. I watch the numberphile video. Oh no its actually intuitive. I read an example of benford in something that has nothing to do with growing series of numbers. Ok its back to magic. Someone explains maybe why. Oh ok. and so on.
There are some that do believe this, but its the kind of thing that we may never be able 100% prove.
For me when you think about stuff like this, and the fact that the universe is finite ( it has a beginning and an end in terms of time but also space) and also the fact that even matter itself is digital (you can't have 0.5 of an atom, its the smallest unit etc etc) it starts to make a lot of sense.
OP
OP
Well here is another thought. Do you think it's easier to see something that's 199 versus 999 light years. In that regard we are limited by our point in the universe/technology.
Another point is that the further you trend towards 10 of a unit the more likely you are to use a different unit. For example if I had 90% of an item in the 5th leading digit it might easier to do a different unit, a light year is a good example of that since it's 9.5 trillion kilometers.
Another point is that a light year is a dumb term. Most people in the field prefer parsecs so if we were to take that exact distribution and use it with parsecs it would be fairly different
The law should (roughly) hold for anything that grows faster as it gets bigger, not just exponential growth.
You'd only get the "expected" 11% for all numbers if the rate of growth never changes (linear growth). The thing is, linear growth is really rare in the real world. Not impossible, but really, really rare. Most things grow faster as they get bigger.
Lets say a population of 800 grows (for example) 8 times faster than a population of 100. It means it will take 8 times longer to go from 100 -> 200 than to go from 800 -> 900. That would mean it spend much less time in the 800s than 100s, naturally making it rarer.
Why is the number 1 so special? It's because it's where you "reset" your scale - suddenly you aren't talking about 100, but 1000. And now the same pattern happens again, where it takes 8 times longer to go from 1000 -> 2000 than to go from 8000 -> 9000.
The growth rate is still accelerating but now we're playing this little trick of moving the goalpost every time it crosses back into the 1s.
That's where scale-free behavior shows up, since every order of magnitude (10-99, 100-999, 1000-9999, etc) follows the same curve.
It's why units and bases don't matter - this is all about shifting the goalpost at some special number.
It doesn't literally show up in everything, it's just very common because exponential growth, power law distributions and other accelerating rates of change are very common while linear growth (what we implicitly expect for no particular reason) is actually super rare.
You'd only get the "expected" 11% for all numbers if the rate of growth never changes (linear growth). The thing is, linear growth is really rare in the real world. Not impossible, but really, really rare. Most things grow faster as they get bigger.
Lets say a population of 800 grows (for example) 8 times faster than a population of 100. It means it will take 8 times longer to go from 100 -> 200 than to go from 800 -> 900. That would mean it spend much less time in the 800s than 100s, naturally making it rarer.
Why is the number 1 so special? It's because it's where you "reset" your scale - suddenly you aren't talking about 100, but 1000. And now the same pattern happens again, where it takes 8 times longer to go from 1000 -> 2000 than to go from 8000 -> 9000.
The growth rate is still accelerating but now we're playing this little trick of moving the goalpost every time it crosses back into the 1s.
That's where scale-free behavior shows up, since every order of magnitude (10-99, 100-999, 1000-9999, etc) follows the same curve.
It's why units and bases don't matter - this is all about shifting the goalpost at some special number.
It doesn't literally show up in everything, it's just very common because exponential growth, power law distributions and other accelerating rates of change are very common while linear growth (what we implicitly expect for no particular reason) is actually super rare.
Last edited:
More like human are inherently pattern seeking creatures, and we tell ourselves stories to make sense of the world.
OP
OP
I dunno about this. Assuming you convert everything to parsecs it should still follow benford. They did show cases where they converted the units of measurement, which would change numbers around, and the law still held.
Exactly. You have to have 1 before you can get to 2. But not all 1s will proceed to 2. You have to have 2 to get to 3. But not all 2s will proceed to 3. Hence you will have more 2s than 3s and a more 1s than 2s.
The bigger the digit, the more difficult it is to increase it to the next digit. It's easier to get 1 million dollars than it is to get to 2 million dollars, and it's easier to get to 2 million dollars than it is to get to 3 million (and so on).
I mean, it seems pretty obvious why this distribution would occur? (Unless I'm dumb and missing something, which is very possible!)
Benford's Law is the same as Zipf's law right?
www.youtube.com
The Zipf Mystery
Support Vsauce, your brain, Alzheimer's research, and other YouTube educators by joining THE CURIOSITY BOX: a seasonal delivery of viral science toys made by...
www.youtube.com
Since when is the real world supposed to be random? Patterns exist everywhere in nature. I learned that in precare.
Well I would need to see the distribution but if you did it between 100-999 light years then yes as everything above 333ish would be 1000-3000, but if the distribution was between 100-399 then it wouldn't work. Likewise if you converted it to km it would probably shift the distribution slot
Probably because linear growth is more intuitive and what we teach in mandatory education but nature prefers to deal in exponents due to the fractal nature of.... nature.
This is mostly a case of people thinking randomness should be intuitive and predictable, which it rarely is.
if you really want to think about being in a simulation consider this:
imagine universe N.
assuming that
1. it is possible to create a simulation inside universe N (let's call this universe N^-1) where the inhabitants are capable of complex thought but can't tell they are n a simulated universe just by looking
2. the universe N exists over some massive time period where multiple civilizations occur
it stands to reason that at there would be, most likely, more than just one N^-1 universe created inside N. Even if we just go by the concept that there only one universes created in a given universe, and that universe N is the top - there could be no universe that simulates N, no N^2, that leaves us with two universes, and a 50/50 minimum chance that, if simulations are possible, the universe you are in is the simulated one.
The real minimum chance is of course 0 - if it turns out it's not possible to simulate abstract thought, then simulations of this sort aren't possible at all. You aren't part of a simulation because there are no simulations. But if there is at least one simulation, the best you can do is 50/50 if you pick a universe at random.
And if simulations can simulate simulations, well, stuff just balloons from there. You could be talking about a billion, trillion, whatever to one chance you're in the top order singular universe.
imagine universe N.
assuming that
1. it is possible to create a simulation inside universe N (let's call this universe N^-1) where the inhabitants are capable of complex thought but can't tell they are n a simulated universe just by looking
2. the universe N exists over some massive time period where multiple civilizations occur
it stands to reason that at there would be, most likely, more than just one N^-1 universe created inside N. Even if we just go by the concept that there only one universes created in a given universe, and that universe N is the top - there could be no universe that simulates N, no N^2, that leaves us with two universes, and a 50/50 minimum chance that, if simulations are possible, the universe you are in is the simulated one.
The real minimum chance is of course 0 - if it turns out it's not possible to simulate abstract thought, then simulations of this sort aren't possible at all. You aren't part of a simulation because there are no simulations. But if there is at least one simulation, the best you can do is 50/50 if you pick a universe at random.
And if simulations can simulate simulations, well, stuff just balloons from there. You could be talking about a billion, trillion, whatever to one chance you're in the top order singular universe.
Sounds like some case for design or something. Plus, we are (as a species) very good at noticing patterns. To what purpose though?
Nah, the speed of light is a ridiculously common constant in relativity so units based on it (light years/seconds/etc) are common.
Parsecs are really well suited for translating telescope measurements into distances so they've become standard in those cases, but that's not the entire field. For theoretical work, the speed of light is easier to work with (seriously, geometrized units are so much cleaner). Sometimes we also use astronomical units (the Earth-Sun distance) because it's a familiar reference point.
All have their benefits. Not everyone in the field is working with observational data.
Last edited:
OP
OP
I think in most of the examples I saw (this is mostly speculation, I dont know the actual data sets either) They didn't limit the data set to everything in this range. At least they didn't specify. Just that if you take these measurements between stars, no matter what unit, the law still applies.
OP
OP
I assume so, dunno how it translates though. Thats cool though. Cool to see it applied outside of numbers even if you do eventually "count" them. Wild it translates across all languagesBenford's Law is the same as Zipf's law right?
The Zipf Mystery
Support Vsauce, your brain, Alzheimer's research, and other YouTube educators by joining THE CURIOSITY BOX: a seasonal delivery of viral science toys made by...
But we can't see everything in the universe so logistically speaking there exists a range even if it's not intentional and since things that are father away are going to be further from default 1. So having a range limited by technology isn't really that different from having a range arbitrarily limited
As already explained in this post, this is because there will be stars 1 light-year distant, but not stars 999,999,999,999 light-years distant. Because the counting starts at a low limit and increases upward (to a finite amount), lower numbers will be over-represented. If you have 15 objects to count from lowest to highest, then the number 1 will appear 7 times in 15 numbers, as a very simple example.
It seems exactly as I would expect.
for a number to start with a leading 2, it already had to start with a 1 whilst it was increasing to 2.
just repeat thIs through to 9, as the number has already been a 1,2,3,4,5,6.7 and 8 before it finishes on 9.
A snapshot of a large amount of increasing numbers has to look like this.
for a number to start with a leading 2, it already had to start with a 1 whilst it was increasing to 2.
just repeat thIs through to 9, as the number has already been a 1,2,3,4,5,6.7 and 8 before it finishes on 9.
A snapshot of a large amount of increasing numbers has to look like this.
One of my math teacher told us about that (not un the curriculum, just for fun!). It's basically statistics. The 1/9 probability people assume for each digit would be true if every set of number were a power of ten data points. But let's say your data set Is 2000. Then +\- half the Numbers start by 1. Sorry if it's not clear. Statistics on english Is not easy!
It makes intuitive sense to me, as a natural consequence of picking random numbers between minimums and maximums that are also, for all intents and purposes, random themselves.
Let's say I generate random numbers, with the caveat that I first generate a random number to act as the cap. To make it simpler, the lower cap is always 1.
- If the cap is 10, the possible numbers are 1 to 10. The chance to have a leading 1 is 20% (two numbers, 1 and 10, over 10), with each other leading number each at 10% (one number, i.e. themselves, over 10).
- If the cap is 20, the possible numbers have a 55% chance of having a leading 1 (1, 10-19), 10% for 2 (2, 20), then 5% each for the remaining digits (3-9).
- If the cap is 30, the possible numbers have around 36.7% chance of having a leading 1 (1, 10-19), another 36.7% of having a lead 2 (2, 20-29), 6.7% for 3 (3, 30), then 3.3% each for the remaining digits (3-9), with 3.
- At 50, you have 22% each for 1, 2, 3 and 4, then 4% for 5, and 2% for 6-9.
- Then with a cap of 100 it sort of goes back to the beginning, only 1 doesn't get as much weight because the cap, 100, only gives it an extra 1/100 chance instead of 1/10. So it's 12% for 1, and 11% for 2-9.
If you add up the possibilities of each leading number over the different caps, I'm willing to bet the distribution looks a lot like the one in the OP.
Damn, that's so cool. Fucking brilliant.
Deus Ex: "You will soon have your God, and you will make it with your own hands."
It's because counting starts with 1 and then we utilize a number system which is base 10 which cycles back to leading number 1 first again as the sample grows.
Seems somewhat logical that smaller things will be statistically more common then larger things given that there are generally fewer larger things than smaller things. Mainly because larger things represent a greater deviation from an equilibrium state than smaller things.
that really doesn't make sense. Why should the amount of times 1 appears from 1 to 15 affect how far stars are from each other?
Stars aren't inherently closer to each other are they? And when you get further and further away, the bias of 1 appearing more often should go away.
Depends on the client and a whole slew of factors, but most importantly what risks you have.
It also depends on your testing methodology.
This example is based loosely on a not for profit audit I did with a really low materiality.
Say you have a client and you think there is a fraud risk b/c a lack of segregation of duties you may develop # of tests to satisfy yourself that there isn't an issue. One of those is Benford's. You could take the entire GL detail of let's say expenses and run a the test on it. You will get a distribution of numbers shown in the wiki ( https://en.wikipedia.org/wiki/Benford's_law ) with 1 being the most common and 9 being least. There are of course items which may break this. We had a client who had a large amount of proper bills that were in the range of $200-$300. 2 was like 28% instead of 17.6% (it doesn't have to be exact, just close to be proper). We looked at the detail and found that their internet, phone, and a few other monthly bills all started with 2. We also noted that each of these only had 12 payments (12 months). Now if I saw internet for $249/month and it happened 24 or 36 times and they only had 1 location, I would probably pull both June's and see if they had different addresses. Was the bookkeeper running their own internet/cable bill through the company?
Another thing you can do is even if it's not a fraud risk b/c they have good controls, this would qualify as a substantive analytical procedure and give you some confidence in expenses. It's so easy to do that I've added it to all of my audits.
edit 1: The reason that this ultimately works against bookkeepers is because when they steal money, they typically steal the same amount or close to it right under a threshold that would incur more review (or more eyes) of it. "Checks under $5000 don't need 2 signatures, only 1" or something like that. So they do a lot of checks to themselves for $4,800. Now if you run an analysis, you might see 4 is higher than it's supposed to be.
edit 2: I noticed your tag after writing all of this out, are you a CPA?
Ah, but what if what we consider abstract thought is just a poor simulation of actual abstract thought? We wouldn't know because we don't have a comparison and it fits within the bounds of this reality, but someone looking at this simulation would be like, "It's too bad your Sims can't think for real. Imagine what they'd do if they could."
If they're actually simulating the entire universe, they may not even know we're here.
This to me does not explain the law being true for random selections of numbers, eg asking people to name a number from 1 to infinity. It also doesn't seem to hold water for other random selections that aren't part of certain types datasets that would exhibit non-linear growth, like a bunch of 401(k) accounts. For instance, I doubt exponential/non-linear growth is at play in the amount of money in a bunch of kids' piggy banks.
That said, I haven't looked into this "law" to see if it really truly holds up in cases like I cited, though the OP and other posts seem to indicate it does.
OP
OP
Thats where I struggle.
I understand the exponential/logarithmic growth. Put picking random numbers off a newspaper? Financial numbers. A scientific study. graphs. A story about COVID. It still follows.
I really don't believe this would hold at all, if you told a bunch of people to pick a random number from 1-999, and you could somehow ensure they weren't biased towards lower numbers. That bias may be why that random number poll scenario could experimentally prove the law "true" quite often, as plenty of people will be biased toward saying "1" or "10" for instance.
Does it have to do with the fact that quantifications only have so much precision?
I don't get how this pattern makes things less random. If that were the case, any probability distribution other than the uniform distribution wouldn't be random.
This is probably the best intuitive explanation I've seen.
I think it also follows that a lot of numbering systems are going to put their base unit (1) as something that is relatively useful/common. And for the non-empirical data I'd just ascribe it human beings naturally gravitating towards rounded magnitudes for simplicity, thousands, hundreds, etc.
Stars are clustered into units, which means there are more close stars than far stars (due to gravity). So you have globular clusters, galaxies, galaxy arms, etc. At the core of the galaxy, you have thousands of stars within a few light-years, potentially, compared to where we are in the boonies. So overall, the distance between stars will be more clustered at the low-end, since most of the universe is empty space, and there is a limit to how far things are away (or at least how far we know things are away).
Lets say the entry point value (the start) of any system is "1". If we look arbitrarily at an infinite number of systems that begin at value "1" and have an equally weighted chance to close-system at any value thereafter (including 1, but 0 would mean the system never existed to begin with) then wouldn't this distrubution be very similar to the one in OP? Each stop along the way has a reduced probibility of being the point where the system is closed?
Real world values don't often have truly random points of entry, do they? You don't generally go from 0 of something to 99 of something in the Newtonian universe.
Real world values don't often have truly random points of entry, do they? You don't generally go from 0 of something to 99 of something in the Newtonian universe.
fwiw, I agree that "law" is not a good word here (it's really more of a pattern).
In any case, my point was not meant to explain every instance. Naturally some cases need to be looked at individually.
My goal was to describe why we should expect this to be so common. It's very likely that data sets that span several orders of magnitude exhibit some form of accelerating growth.
Regarding your examples, I would expect savings and pensions grow in a nonlinear way. People with more money are likely to be making more money. Salaries increase over time, typically with increasingly larger jumps. Even with children's savings, I was much easier to please with 25 cents when I had $5 saved than when I got to $50.
Asking people to choose arbitrary numbers is a different story. I would expect it has more to do with psychological biases than anything. An interesting coincidence but not representative of the vast majority of these cases.
I'm not saying accelerating growth is the only way to get this distribution, but it's so ridiculously common that it singlehandedly represents a huge number of real world data sets that span multiple orders of magnitude (enough to claim that it shows up "everywhere").
Last edited:
