Fast Inverse Square Root - A Quake III Algorithm

2020/11/28
951 313 再生回数

In this video we will take an in depth look at the fast inverse square root and see where the mysterious number 0x5f3759df comes from. This algorithm became famous after id Software open sourced the engine for Quake III. On the way we will also learn about floating point numbers and newton's method.
0:00 Introduction
1:23 Why Care?
3:21 The Code
4:18 IEEE 754
9:38 Bits and Numbers
12:09 1st Step: Evil Bit Hack
14:46 2nd Step: WTF
17:34 3rd Step: Newton
19:46 Summary
Picture of John Cramack is licensed under CC BY 2.0 from author Drew "Prognar" Campbell.
Source: flic.kr/p/6YxWYp

コメント数
  • This is great; I have way more experience with math than with coding, so it was very interesting seeing how people use math tricks to get around coding quirks! Like, I personally would never need to care about the speed of dividing something, but small bits of time add up really quickly in code! Super neat.

    MoggetslittlesisterMoggetslittlesister5 時間 前
  • Wait... what?

    Nine's Own GoalNine's Own Goal日 前
  • Cool stuff, I like it. Can you continue this work on describing different complicated things in code for us?

    Sergey VoronezhskiySergey Voronezhskiy2 日 前
    • Kinda, I want to do algorithms in general

      NemeanNemean2 日 前
  • Really Really well-made animations. Easy to understand.

    John YueJohn Yue2 日 前
  • Bold of you to assume I understood anything

    G LUONGG LUONG3 日 前
  • Bro, make more videos.

    Abobker ElaghelAbobker Elaghel3 日 前
  • And when you experience texture errors or texture blinks it often happens at steep angels which result in non smooth functions (or those with large derivatives), where Newtons methods becomes inaccurate. Or at very flat angels where devision with the derivative leads to problems...

    Andreas BeschornerAndreas Beschorner3 日 前
  • Ugh, it took me way too long to not confuse x and y. y is the output we want, so it's what's used in Newton's method, not x. f(y) = -x + 1/(y^2) diff(f(y)) = -2/(y^3) f(y) / diff(f(y)) = 0.5*(-y + x*y^3) y_new = y - f(y)/diff(f(y) = y - 0.5*(y - x*y^3) = y * (1.5 - y * y * x /2)

    1Maklak1Maklak3 日 前
  • but how much faster is it?

    MglistyMglisty3 日 前
  • 6:41 "Something definitely has gone wrong" i: *Let me introduce myself*

    謎の海月謎の海月3 日 前
  • Cool video! What did you use to make it? ^^

    Wassim BouazizWassim Bouaziz4 日 前
    • Adobe After Effects, Adobe Illustrator and LaTeXiT

      NemeanNemean4 日 前
  • Just wow , Thanks 😊 ✌️

    trashedlife1trashedlife14 日 前
  • COOL

    Tanim SkTanim Sk4 日 前
  • This algorithm will make a LOT more sense after watching *Zach Star's 'Approximations. The Engineering way'* on youtube: it is the Newton-Raphson Method for approximating square (/any) roots Also for those who have never used bit-manipulation that bit shifting by 1 to right ( >> 1) just means divided by 2, and is what compilers actually automatically replace divided by two during compilation, since it is such a fast operation. And the opposite bit shift of 1 to left (

    Santtu KähkönenSanttu Kähkönen4 日 前
  • Brilliant. Thanks for sharing!

    Antonio AstorinoAntonio Astorino5 日 前
  • And here I am thinking the exact comment in the video: WHAT THE *****?! Whoever came up with this thing was not human.

    Not TodayNot Today5 日 前
  • Awesome video!

    Alex Enrique CrispimAlex Enrique Crispim5 日 前
  • from where or how did you source the equation for representing binary32 bits (float) in decimal values before they did the log magic?

    Goliath StarkGoliath Stark6 日 前
  • Yes

    Anurag PandeyAnurag Pandey6 日 前
  • The thumbnail got me to click. It's my first time seeing a very random-looking hexadecimal be used in programming lol

    Carl Eric DoromalCarl Eric Doromal7 日 前
  • I've seen many attempts to explain this code, but you are the first one who explained where that magic constant came from. Thanks a lot!

    LuckLuck7 日 前
  • When limits, differentiation and C you learnt fall into one piece. Total denouement. That guy is genius.

    SohanSohan7 日 前
  • 👍👍👍 *_FANTASTIC_*

    Gabriel DibbleGabriel Dibble8 日 前
  • Bro the world needs more of your videos

    Alok TiwariAlok Tiwari8 日 前
  • This was cool to see and well explained. I would enjoy seeing an explanation of "Division by Invariant Integers using Multiplication" (should be a paper by that name)

    MatthewMatthew8 日 前
    • Thanks for recommending the paper, very appreciated. If I'm not mistaken, these algorithms are standard compiler optimisations, and since I'm likely going to cover compilers at some point in the future, you should see an explanation of that paper by then.

      NemeanNemean8 日 前
  • Quake teaching me that I’m just a fucking casual even before it was compiled.

    UnheartedUnhearted8 日 前
  • newton raphson method..I used this during one of my college project when one of the ATMEL microcontroller was not having sqrt function in it to calculate RMS value of Voltage.

    Sachin KiragiSachin Kiragi9 日 前
  • Fascinating... and over my head, lol. Really cool. 😎

    80s gamr80s gamr9 日 前
  • This is even faster: float fast_inverse_square_root(float x) { return x*x; }

    lethalsublethalsub9 日 前
  • "Inverse" is the wrong word, i think you mean "reciprocal". Inverse means the inverse function, so inverse square root is.. squaring. Reciprocal of "whatever" is 1/"whatever".

    deadmaydaydeadmayday9 日 前
  • 06:08 and 08:24 "0.0101 = 1.01 * 2^(-2)" instead of "2^(-3)"?

    Paulo Eduardo MorbeckPaulo Eduardo Morbeck9 日 前
  • Great video! Are you using 3b1b's animation library?

    Ofek ShilonOfek Shilon9 日 前
    • No, I use Adobe After Effects

      NemeanNemean9 日 前
  • It's even scarier than "only 1% error"; using the default magic number of 0x5F3759DF actually gives about 0.175% error maximally, dropping thereafter. It is now believed the original author of the function (not Carmack, who never claimed he was), found the constant through trial and error rather than derivation. If you use Chris Lomont's updated magic number of 0x5F375A86 (which he derived), the error drops to a staggering 0.09% after just 1 iteration. BTW, the original author is unknown, but speculated to be Canadian Velvel Kahan, Turing Award winner and mathematician, circa 1985.

    John DoeJohn Doe9 日 前
  • ¡Fast inverse square root! sounds like an anime technique

    Andrés PrietoAndrés Prieto11 日 前
  • Alright that's it, sub and notification bell are locked in, I'm waiting for another

    JoeleoJoeleo11 日 前
  • Jesus, I don't get the WTF Part.

    Neil ChouNeil Chou11 日 前
  • You didn't explain what happens to the bit of the exponent that gets shifted into the mantissa at the "evil bit hack" step.

    Laurent LÊ-HEBRARDLaurent LÊ-HEBRARD11 日 前
  • One nitpick: When you say "decimal number" you really mean "binary real number". First, it's a binary number, not a decimal number. Second, it's a real (floating-point) number, not an integer.

    David TribbleDavid Tribble12 日 前
  • Great vid. At the end, when applying Newton's method, you probably want y's inside f and f', and y_new not x_new. True?

    Quinn CulverQuinn Culver12 日 前
    • Yes. I honestly don't know why the code calls x "y". As in, y doesn't even refer to the y-axis or anything. I tried to hide that ugliness from the viewers, but some of you are just too observant :P

      NemeanNemean11 日 前
  • "I don't know what else to say, that's just how C works" An excellent explanation of all of C

    ThePuzzlemaker2ThePuzzlemaker212 日 前
  • i don't get that 1 in front of + at 10:24

    Neil ChouNeil Chou13 日 前
  • If you convert 0x5f3759df in python for example (print(int("0x5f3759df",0)) you get 1597463007 But if you calculate 3/2*2^23*(127-0.043) you get 1597488758.784 The mu should be 0,0450465679169398

    D o D oD o D o13 日 前
  • *Doom Eternal running at 500 fps go brrrrrrrr*

    KumaKuma13 日 前
  • I would like to see this outpot in pure hexadecimal format with 64bits...

    Satoshi NakamotoSatoshi Nakamoto14 日 前
  • sh**, take my subs and Thumbs-up

    juan sonjuan son14 日 前
  • I love C. Great video. Very clear.

    Noah SpurrierNoah Spurrier14 日 前
  • He will come back based on his comment in his community tab: *"I plan on covering algorithms in general. But if there's some obscure piece of code accompanying it, that's just more bonus points for the algorithm. - Nemean"*

    Kevin TanKevin Tan14 日 前
  • Very interesting solution to inverse root approximation. Good video.

    Alan MedinaAlan Medina15 日 前
  • Nice video and explanation. I didn't get it but me no smart

    Manu Barrio LinaresManu Barrio Linares15 日 前
  • As a millennial programmer, I thank the gods every day that these OGs paved the way with stuff like this. I probably couldn't even recreate pong from scratch...

    AdacadabraAdacadabra15 日 前
  • Important Fact: Standing Bodies Of Water are *always* level (level means no elevation or deviation from the starting point to the end). This is a scientific fact because this is observable, testable, repeatable, measurable, demonstrable by every single human being alive. This fact alone destroys the mathematical concept and religious idea known as the "heliocentric model". More specifically, this fact alone makes it impossible for us to live on an exterior of a pear-shaped sphere spinning at fantastical speeds going nowhere. Just before you start to attach straw man fallacies on to me, keep in mind these important things: There are three different sciences: Natural Science (which deals with the Objective World) Social Science (deals with societies and the relationship between people in societies) Formal Science (deals with languages such as mathematics which bares no connection to the Objective World) "What is the shape of the earth I stand upon?" this is a Natural Science question. Science does not belong to an institution or a group of people. It belongs to every single human being alive. We live in the present. Not in the past or the future. History can never be considered as a fact of reality in any way shape or form because of obvious reasons. We can't directly experience the past or the future. Just observing something is not a fact that something exists. We need observable, testable, repeatable, measurable, demonstrable practical proofs for something to be considered as a fact. This is also known as the Scientific Method. There is a difference between the corporeal world (the physical world) and the visible world. The reason why we can't conclude something as a fact based on our observations is because we know things get smaller based on how far they're from us when we see them through our eyes. If I see a railroad, the lines look like they're converging, but we know that's impossible because people measured the lines, and the lines are parallel. The lines don't actually meet in real life, it's just how we see things. Our eyes are spherical, we see euclidean (planar) world through spherical eyes. Without physically testing, repeating, measuring, and accessing something in full three dimensions, it's impossible to know exactly what it is that we're trying to quantify. Looking up at the sky does not give you measurable proof of the earth you stand upon. It's like looking up at the light in your room and then measuring the floor based on that light. It's absurd. Images and videos are never considered as scientific proofs because of obvious reasons. Images and videos can be manipulated, they're not tangible. We can't directly experience them. Mathematical equations bare no relation to the Objective World. Mathematics is just a language, like English. Just because something is mathematically correct does not make it real. It's like saying "I'm flying!", even though the sentence is grammatically correct, I'm obviously not flying right now. "Gravity" is a mathematical concept, it's pseudoscience. It does not have any practical proofs. Magical pulling forces don't exist in the Objective Reality. Motion only happens if something presses on something else (pressure variants). Pull is just a term for taking something closer to someone. Things falling down has got nothing to do with the shape of the earth. In the simplest sense things that weigh more than air fall, and things that weigh less than air float. Why do things even fall? No one has any practical proof for why things even have weight. The mathematical theorists are making assumptions about why things are falling, but understand that those are just assumptions, not facts. The sky, the Moon, the Sun, and the Stars are all intangibles. If every single human being can't observe, test, repeat, measure, demonstrate something in a practical fashion, it is considered a belief, or pseudoscience. Does every single human being has access to these lights in the sky? Obviously not. Not to mention stars look like flashing lights when looked through a Nikon COOLPIX P900 (It's still not a proof of anything because we can't move around them in full three dimensions). Admitting to yourself that you truly don't know something is the most honest realization you could have, it is how you progress further. For example, I do not know what the Sun is. The mathematical theorists are making assumptions of what the Sun is, and where it is located. They don't know what the Sun actually is, or where it is located. Unless I could move around the Sun in full three dimensions, the only possible stance I can have is "I don't know". Anything beyond that will be a belief. For example, if I say "The Sun is a cylinder-shaped object moving in the sky", it is just a belief because I can't move around the Sun in full three dimensions to know if it's a cylinder or a circle. The only thing I know is that it's a light in the sky moving in a straight line across the sky (because of how our eyes work it looks like the Sun is moving in an arc across the sky, but the Sun is actually moving across the sky in a straight line). Another example, "I know the full dimensions of earth" is also a belief because I never explored the whole world to know the full dimensions. Remember, something only becomes a fact if it's observable, testable, repeatable, measurable, demonstrable by every single human being alive. In this case, each and every individual should explore the Objective World to its fullest extent, listening to "authorities" is just a belief. The world "map" is also a concept, because every single human being didn't explored the whole world to verify or falsify the map. People are believing in whatever the "governments" say or show. People are literally believing in complete strangers and thinking the official "world map" is true and there is nothing more to explore because they see a blue sphere on their TV. Just because the majority of the population are believing something exists doesn't mean something actually exists in the Objective World. Once again, we need OTRMDPPs for something to be considered a fact of the Objective Reality. Letters before your name does not mean anything. Direct experience is the most important thing. If we can't experience something directly then most of the time it's useless for us. I can only represent myself. Personalities are OUT of the question. If I drop a brick on my head I know what's going to happen. The objective reality does not change based on your subjective opinions or beliefs. It is what it is. Why is this important? Well, the government and its associations are BLATANTLY lying about our existence, the shape of the world, and the dimensions of the world. What's more important than finding out how far the world extends? Full exploration is needed for further understanding of life's most important questions: where we are, why we're here, and what's it all about. Without knowing where we are everything we do is just a concept. For example, if I give you a board, and without giving you any instructions or rules, I want you to play a game, what game are you going to play? The government and its associations are telling you what you're suppose to do, what's expected of you, what you're not suppose to do. Who agreed to their "law" book? I certainly did not. How is it fair that the government and its associations (the police and the military) are imposing their subjective rules onto the human beings? Everyone has different rules about different things, subjective rules are personal, they're NOT objective. These governments, police, military are imposing their SUBJECTIVE rules onto others, this is nothing short of tyranny. Why can't we freely explore the objective world as much as we like? Why is there a physical, and mental restriction by the police, the military, and by psychopaths known as the governments? Remember, no one rules if no one obeys. "People are arguing and fighting over what game to play when they don't even know the board they're playing on" -Del (Beyond the imaginary curve youtube channel: jpworlds.info/lost/vswlgeHodOejVN21TWweLw) (People are arguing about what to do without knowing the full dimensions of the world) Another analogy: if I erase your memory and put you in a confinement, what is the first thing you're going to look for? Where's the exit. But if I put some actors to distract you by showing you books and telling you there is no exit, then you'll never try to even think about the exit, there will be other brainwashed people just like you who'll brainwash you even more. Then some other actors will tell you what to do, what's expected of you, what you're not suppose to do, and what's the punishment for breaking their god-given "law" book, some other brainwashed people will also have weapons so that common people like you will forcefully follow my law book, which makes my job incredibly easy by making you a slave if that's what I was ever up to. Why should we accept slavery? Why should we accept strangers imposing their subjective versions of what's good and what's bad onto us? If you have OTRMDPPs (Observable, Testable, Repeatable, Measurable, Demonstrable Practical Proofs) that large Standing Bodies Of Water can bend, (sounds absurd to even think about it) please feel free to email me at MysteriousPlane@pm.me, but understand that I should be able to demonstrate your claims in a practical fashion, because that's how the Objective World works. No offense, but If your claims does not have any practical proofs, then I'm afraid I have to conclude that you're either blatantly lying or you're really stupid (Or for some reason you want to defend your religious beliefs, because the Earth being a sphere is a religious belief, a mathematical concept, it has no practical proofs, it has no relation to the Objective World, that being real life).

    Santosh P.S.S.Santosh P.S.S.15 日 前
  • Thank you for this video. Really well explained.

    Say ASay A16 日 前
  • You only have one video? The YT community needs someone to break down more cool algorithms like this

    Shaun RoseShaun Rose16 日 前
  • what

    retardretard16 日 前
  • truely remarkable, but we compute sqrt with javascript now

    Tony NgTony Ng16 日 前
  • 👍

    Bart GillisBart Gillis16 日 前
  • why not use lookup-tables with an approximation : x= 3.2 -> int(x) = 3 , lookupTable(3) = 0.577 . Just grow the size of the lookupTables to the error margin you tolerate and there you go. Wikipedia states that this approximation routine is actually faster than a lookup table. I do not understand that. If you can accept an error of 5-10% lookup should definitely be faster. Did they just overengineer the normalization of a vector for the game?

    AlexanderAlexander16 日 前
  • ahh, love me some good maths breakdowns of code

    Dallin BackstromDallin Backstrom17 日 前
  • 6:09 wouldn't it be 1.01e-2 ????

    ElNico56ElNico5617 日 前
  • Oh gods. At the start of the video, just want to say I know where this is going without even having heard of this algorithm. Bitwise math is a hell of a drug, folks. :D

    Wayfarer ZenWayfarer Zen17 日 前
  • Plz make more

    Aleverette SAleverette S17 日 前
  • Oh man. Very useful 👌video

    RavikumarRavikumar17 日 前
  • quake devs be like: - i need a constant for three halfs - lets just write 0x5f3759df because i don't need a variable

    iamthewallrusiamthewallrus17 日 前
  • This came from a programmer versed in Assembly with vectors. Check out some of the old Amiga mega demos. Long before the first Quake was even made.

    Coma ToseComa Tose17 日 前
    • In fact Quake was made on an Amiga.

      Coma ToseComa Tose17 日 前
  • Overall, the breakdown of this made a lot of sense. I see a couple of problems with your step 2 explanation though that made it harder for me to understand. 1. Your color switching (at 17:00) reused two of the colors I had been associating with other parts of the equation. You also used the same color for the bit shift and the hex constant when it would have been more appropriate to use two separate colors. (Also, the bit shift color should probably be for the whole 1/2, but this is much more of a nitpick relative to other color problems.) 2. Most of the intuition to this process is explained before step 1 so details are lost in one watch (which, admittedly, is probably fine for most people watching casually). As a final nitpick, I kind of wish you had solved that hex constant to show off how it elegantly captures the correction term. Then you could have pointed out that they were using u = 0.045 instead of u = 0.043, suggesting they were expecting numbers closer to the middle of your graph on 11:10. I digress, thanks for sharing this cool algorithm and explaining it! Even though my understanding was about ~80% by the end, that was more than enough to get me the rest of the way on my own.

    Steven WolfeSteven Wolfe18 日 前
  • actually it is very easy for me, because i am a programmer

    Pancake RilaPancake Rila18 日 前
  • ..... Oh ! We call it " JUGAAD ". This is great and inspiring ! Thanks a lot for sharing this !

    Shylesh SrinivasanShylesh Srinivasan18 日 前
  • I think I might have partway cracked the code: The basic adjustment looks like (1−[([ln]([ln]2))+1]/([ln]2))/2~4.303′566′602′796′71…ē2 with [ln] the natural logarithm and ē a scientific notation marker with negative sign for exponent (Enh, so I'm a little bit of a calculus freak;-)

    Stephen DavisStephen Davis19 日 前
  • Wow this algorithm is genius

    Donald DuckDonald Duck19 日 前
  • i agree: what the fuck

    tomatotomato19 日 前
  • The adamant oil subsequently sail because iran congruently injure including a fine chicken. slippery, narrow butter

    Алесандр ИвановАлесандр Иванов19 日 前
  • Are there videos outlined like this that explain python code? I find it very entertaining and great source of knowledge

    Hot potatoHot potato19 日 前
  • wtf is wrong with youtube please i don't care

    NirreNirre19 日 前
  • We need more videos like this!

    Wolf FusagWolf Fusag19 日 前
  • what the fuck?

    Mike HawkMike Hawk19 日 前
  • Ah the newton thing is just the taylor series approximation

    gabrielwong1991gabrielwong199120 日 前
  • Fact that this has a million views is amazing

    aman vermaaman verma20 日 前
  • This video made me give up game programming.

    LanDi3000LanDi300020 日 前
  • would probably just make a large look up table.

    Bob BobsonBob Bobson21 日 前
  • My take away is that the Quake engine programmers were way smarter than I'll ever be.

    AnonymouseAnonymouse21 日 前
  • 6:09 not 0.0101 but rather 0.00101.

    brawagabrawaga21 日 前
  • 3:37 they aren't decimal, they are binary.

    brawagabrawaga21 日 前
  • I wonder what he used for creating the math animations, it seems a lot similar to the animations created using 3blue1brown's manim library.

    rustyelectronrustyelectron21 日 前
  • Just Newton's method, it has quadratic convergence and is more or less the best

    Tom KillwhangTom Killwhang22 日 前
  • 5:28 Thought that was kinda funny because it sounds like a play on words, but I think that's literally decimated/decimating lol

    Pharaoh on LFSPharaoh on LFS22 日 前
  • I must say that whoever came up with this algorithm is a genius. I do realise it is in fact a work of team, but someone from the team had an idea and the team worked on that idea that someone had. For an FPS game that needs to do fast calculations, this is pretty nifty thing to do to speed up the calculation process and in 1998 (which is actually the year Quake III Arena was in early development, and when first testing versions came out), this is brilliant. And that is why the game looks so good and runs so smoothly even on lower end machines (including those from that era). John Carmack is a god!

    Marek PoláčekMarek Poláček22 日 前
  • Tfw your documentation is so bad you need a whole JPworlds video to explain the code 😂

    seasongseasong22 日 前
  • 0:25 I would right it something like 1 / (x**0.5)

    Buster DafyddBuster Dafydd22 日 前
  • never thought i would actually use my math from school, damn. good vid :)

    CryztalCryztal22 日 前
  • I understood the first 50 seconds. :)

    HRobertSIHRobertSI22 日 前
  • Great video mate 👍 I love it

    Mateus SarmentoMateus Sarmento23 日 前
  • Very normal calculation if you ever sturdied digital electronics and computer architecture.

    h zedh zed23 日 前
  • This was awesome ! Thanks a lot for the explanation ! Those guys are crazy (in a crazy good way !) !

    Kadaj0666Kadaj066623 日 前
  • wait, isn't long 64 bits?

    Zareh GorjianZareh Gorjian23 日 前
    • By the C standard it is only required to be at least 32 bits and back when Quake III was released they actually were 32 bits. Only when computers had more memory available, did C compilers give you more bits for a long.

      NemeanNemean23 日 前
  • Is this guy on MasterClass yet?

    dilinatordilinator24 日 前
  • begging for more :(

    Eld Attack KrossaEld Attack Krossa24 日 前
    • Just kidding, I'm finally working on the next video

      NemeanNemean23 日 前
    • Beg

      NemeanNemean23 日 前
  • This is not _my_ job. But well explained. Thanks.

    popogastpopogast24 日 前
  • Professor at uni: Comment your code My code:

    NorhtherNorhther24 日 前
  • I enjoyed it a lot. So I did the math in the end. Apparently, the fact that we needed the answer for negative power (i.e. X^(-1/2)) did the whole trick in the end by turning the negative powers of Y into positive, hence avoiding the need for division. Therefore, the answer for X^(1/2) is just 1 division away.

    Aleksandr IvanovAleksandr Ivanov24 日 前
    • Russian..

      the wicaksonothe wicaksono19 日 前
  • This is such great quality, why is it the only video on your channel. I'm not trying to presure you or anything it's just weird that a first public video is already this great.

    jucomjucom24 日 前
JPworlds