>What I find particularly suspect is the idea that "intelligence" is like CPU speed, in that any sufficiently smart entity can emulate less intelligent beings (like its human creators) no matter how different their mental architecture.
>With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.
Glad to see this articulated here, this is one of the many problems I have with this topic.
It's possible that we find that the problem solving maximum that can be achieved on a CPU isn't enough better than human intelligence to be classified as a super intelligence.
There are plenty of reasons to think problem solving ability can be maximized (like go and chess, and image recognition, and individual differences in IQ). And no reasons to think we've somehow hit the universal maximum for proven solving ability. Especially since every year computers become better than us on some task.
But humans are the dumbest possible animal capable of creating a technological civilization. If we weren't, then our dumber ancestors would have done so.
I think this line of reasoning subtly assumes mind-body dualism[1]. Intelligence is necessary but not sufficient for the creation of technological civilization. Dolphins and other cetaceans are an example incredibly smart creatures that lack the necessary lifestyle and opposable thumbs to create technology. Octopi are likewise excellent problem solvers whose technological ambitions are thwarted by a very short lifespan and a generally solitary nature.
Human technology is at the crossroads of the right physical attributes mixed with the right kind of intelligence and subjected to the right kind of evolutionary pressures. There may be (or may have been) creatures with more "raw" intelligence that cannot be applied to the development of technology because one of the other factors is missing.
Sure, but we evolved to live on land, and opposable thumbs, far before we developed civilization. Meanwhile, cranial volume continued to expand long after e.g. thumbs.
There is more to it - the environment, and networking effects (so., the environment, which causes or prevents them). Only some of the human civilizations went all the way to the Internet, we still have people living as hunter/gatherers even today. So the advancement does not come from "inside" alone.
In any case, I like to think of humanity more like a brain, human = neuron. There are huge differences, for example, the human network also extends deep in time, not just in space (we manage to "network" with our ancestors, one-way from them to us, they gave us knowledge as well as all their tools). My point is "humanity" to me is a network effect. Place any modern human, adult or baby, in an prehistoric setting (20,000 years ago). Even if you gave them, let's say, an iPad-like device with all knowledge we have as of now, they would still not be able to do anything - because without the tools it would take quite some time to bootstrap. So "humanity" really is a lot - a lot - more than the sum of all humans, it's a giant ecosystem on its own.
I think depending on the environment you can have creatures with far higher intelligence than the average human and they still won't come up with modern technology. I think individual intelligence is clearly necessary and beneficial but overrated, that network effects are the main contributor, and that they don't appear automatically. We still have plenty of human societies right here and now that seem to be stuck.
Here's a Google Talk I once bookmarked: Joseph Henrich, professor in the Department of Human Evolutionary Biology of Harvard University, "The Secret of Our Success" https://youtu.be/jaoQh6BoH3c
An aside, arising from those thoughts, more of my own thoughts about something else: Each time you think about who "deserves" something, e.g. high pay, consider that most people who actually contributed so that the person even is in the position to achieve anything more than building a mud hut are dead, and another large amount of the contribution is "everybody else". That somebody can create a Dell or Microsoft, or invent theory of relativity, is because they are placed at the right points in the human network in time and space. What would they achieve by themselves in the forest? So much for "self-made" people. Self-made just like Newton who made the (much older) "standing on the shoulders of giants" famous. (https://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_g...)
This is entirely fair. I sometimes wonder if there might be some alternate or future timeline where octopi have evolved to be individually much smarter than humans but have no real technological accomplishments because their species is individualistic and territorial.
I still think it's roughly correct to state that humans are a pretty dumb animal that juuuuust manages to pass the threshold of traits required to develop technology.
> I still think it's roughly correct to state that humans are a pretty dumb animal that juuuuust manages to pass the threshold of traits required to develop technology.
I think what I wrote is pretty much in support, or at the very least not opposed. Individually, alone and without our "tools" and man-made environment, we are not much, as movies or games like to show us (e.g. when a guy with only a knife is dropped into the jungle).
In those environments we are actually a lot dumber than those living there, including many animals. So what does "general intelligence" even mean?
I think most of our intelligence is social: We manage our interactions, which enables the network effects. I think we spend most of our efforts managing our relations with one another, more than anything else we use our "intelligence" for.
Only if evolution is much slower than technological development for our dumber ancestors. If it would have taken them a million years, then they would have gotten smarter before they could do it.
Really O(N!) should nip that in the bud in itself alone. There aren't enough particles in the universe for even 1000 factorial. The scaling isn't trivial at all as the problem space grows and the interactions with itself would start eating up speed further.
While we may one day build computers able of out-competing us mentally in many fields but they would be many different machines and not just one uber-machine for the same reason that we don't try to combine the functionality of a wrench, scalpel, desktop computer, coffee mug, and an assault rifle into one tool. Even if there weren't dangers there would be major costs to it and seriously would you ever do that except as a joke?
OTOH, we have phones, which are telephone, TV, radio, notebook, phone book, calendar, calculator, computer, compass, GPS, camera, video recorder, sound recorder hybrids.
I'm not sure why do you think that algorithms for solving different problems will require different hardware.
What I am saying is that it won't be some magical omnitool that can go from curing death to undetectably forging nanotech instadeath rockets.
Even if hardware is shares configurations will not for the same reason Steven Hawkings wasn't drafted as a general during any of the brief Falkland Islands wars even when they were considering if MAD called for nuking Argentina if they lost.
There are very different specialties even if the hardware is the same. Having superhuman General and superhuman Surgeon skill level would be two seperate major projects to create and with lesser redundancies and in the worst case would call for two modules due to lack of shared synergies. You can't really combine processing and RAM hungry CAD programs with weather simulations and expect to see same memory footprint with the same performance.
>With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.
Glad to see this articulated here, this is one of the many problems I have with this topic.