Scary Robots, Grrrrrrr
We will never face a robot uprising because it's mathematically impossible for a computer program to determine even the most basic facts about other programs. Consider: a computer program comes in two flavors: one follows a list of instructions until completed (or forever because the instructions result in an "infinite loop"), the other sometimes follows a list of instructions and sometimes either waits to receive input (e.g. the user clicks on the mouse or types on the keyboard, or another bit of code sends it a number or some text) and then generally responds to that input according to other predefined instructions, or actively gathers input (to which it also generally responds).
The best a computer program can do to analyze another program is identify the points of entry through which the other program can receive input, identify the type of input accepted, and monitor the result. The inferences a computer can draw here are severely limited. Imagine a point of entry on a computer program takes in a number, let's call it x, and as a result, a window on the monitor displays the square root of (((x + 38) squared) - 438)/261.9 - 5 * x + 3 to the xth power, because this formula has some important scientific significance (as far as I know, it doesn't).
The only way to determine for certain that this is what the program does is to already have the formula in hand and to enter every possible number into the program and see if the result was what we'd expect it to be. Of course, there exist an infinity of positive numbers and an infinity of negative numbers, and if you add decimal places there are infinitely more numbers. To be fair, computer programs usually (but need not) use numbers whose range is limited, and even if they don't, for practical purposes the size of the number you can send into a program is limited by the amount of memory available for the operation (yes, you can have a number too large to fit on your hard drive and temporary memory combined. For example, pi); however, even if we get the correct result we want with every practical number, we can't be sure we'll get the same result every time for *any* number. We may only get this result the first time, or the first three times, or only for odd numbers, or only for odd numbers divisible by prime numbers, or only for odd numbers divisible by prime numbers when the previous number entered into the program was the square of a number divisible by 2 + the remainder of itself divided by 16.
Since the program's response to the number we give it might depend on the numbers we've already sent the program, we have to try every combination. In other words, we have to give the program one number, then restart it. We have to try every combination of one number, every possible sequence of entering two numbers (including repeats), every combination of three numbers... every combination of 9999 billion numbers... it never stops.
At least not with 100% certainty. We can always guess if we decide (arbitrarily) we've got enough data.
But we're still assuming we have the formula. If not, though we might try generating a formula at random and seeing if it fits, there exist an infinity of possible formulas we might generate, and we have to try every sequence of numbers on every formula, under every imaginable set of conditions. It would take a fast computer millenia to make a good guess. And even then, it doesn't know what the formula means.
And this is a rudimentary program. Programs with any kind of power have much more than one point of entry, and with even two points of entry, you explode the possible input combinations (one number into point one, then one into point two, then six numbers into point one, then 2 int point two..... etc.), and your guess at what the program does becomes much less likely to be correct.
Why does it matter that computer programs can't really read computer programs? To put it simply, you need to learn to read before you can write, and while computers can randomly generate programs if told to do so, those programs can only be composed out of preexisting instructions. So, the only way a robot's doing us harm is if we program it to do us harm. Cyborgs are a different story.
Even so, Military Researchers Develop Corpse-Eating Robots is not the most reassuring headline I've ever seen. Let's hope there's a good kill-switch tied into good heat sensors.
The best a computer program can do to analyze another program is identify the points of entry through which the other program can receive input, identify the type of input accepted, and monitor the result. The inferences a computer can draw here are severely limited. Imagine a point of entry on a computer program takes in a number, let's call it x, and as a result, a window on the monitor displays the square root of (((x + 38) squared) - 438)/261.9 - 5 * x + 3 to the xth power, because this formula has some important scientific significance (as far as I know, it doesn't).
The only way to determine for certain that this is what the program does is to already have the formula in hand and to enter every possible number into the program and see if the result was what we'd expect it to be. Of course, there exist an infinity of positive numbers and an infinity of negative numbers, and if you add decimal places there are infinitely more numbers. To be fair, computer programs usually (but need not) use numbers whose range is limited, and even if they don't, for practical purposes the size of the number you can send into a program is limited by the amount of memory available for the operation (yes, you can have a number too large to fit on your hard drive and temporary memory combined. For example, pi); however, even if we get the correct result we want with every practical number, we can't be sure we'll get the same result every time for *any* number. We may only get this result the first time, or the first three times, or only for odd numbers, or only for odd numbers divisible by prime numbers, or only for odd numbers divisible by prime numbers when the previous number entered into the program was the square of a number divisible by 2 + the remainder of itself divided by 16.
Since the program's response to the number we give it might depend on the numbers we've already sent the program, we have to try every combination. In other words, we have to give the program one number, then restart it. We have to try every combination of one number, every possible sequence of entering two numbers (including repeats), every combination of three numbers... every combination of 9999 billion numbers... it never stops.
At least not with 100% certainty. We can always guess if we decide (arbitrarily) we've got enough data.
But we're still assuming we have the formula. If not, though we might try generating a formula at random and seeing if it fits, there exist an infinity of possible formulas we might generate, and we have to try every sequence of numbers on every formula, under every imaginable set of conditions. It would take a fast computer millenia to make a good guess. And even then, it doesn't know what the formula means.
And this is a rudimentary program. Programs with any kind of power have much more than one point of entry, and with even two points of entry, you explode the possible input combinations (one number into point one, then one into point two, then six numbers into point one, then 2 int point two..... etc.), and your guess at what the program does becomes much less likely to be correct.
Why does it matter that computer programs can't really read computer programs? To put it simply, you need to learn to read before you can write, and while computers can randomly generate programs if told to do so, those programs can only be composed out of preexisting instructions. So, the only way a robot's doing us harm is if we program it to do us harm. Cyborgs are a different story.
Even so, Military Researchers Develop Corpse-Eating Robots is not the most reassuring headline I've ever seen. Let's hope there's a good kill-switch tied into good heat sensors.