"I actually took the time to spend a day at a university where I could gain access to IEEE papers without wasting my private money. I picked out twenty recent realtime related papers and did a quick survey. Twelve of the papers were a rehash of well-known and well-researched topics, and at least half of them were badly written as well. From the remaining eight papers, six were micro improvements based on previous papers where I had a hard time figuring out why the papers had been written at all. One of those was merely describing the effects of converting a constant which influences resource partitioning into a runtime configurable variable. So that left two papers which seemed actually worthwhile to read in detail. Funny enough, I had already read one of those papers as it was publicly accessible in a slightly modified form.
That survey really convinced me to stay away from IEEE forever and to consider the university ranking system even more suspicious."
That's funny. If you took a look at startups, you'd find that 99% of them are me-too crap. Badly designed websites for Yet Another Social Networking Gimmick, bad mobile apps, outlandish bad business ideas, etc.
1% of them ever truly go anywhere. 0.1% of them change the world.
My point is that anytime you randomly sample a field of work of any kind, you're going to get an exponential distribution of quality with a few hits and a long tail. All papers are not the Church-Turing thesis, and all startups are not Google or Apple.
A lot of academia-bashing amounts to criticism of academia for having the same sorts of failings that are present in equal measure in all other societal institutions public or private. Things like: elitism, know-nothing overpaid leadership, poor quality control, loss of focus, abusive labor practices, perverse incentives, poor success ratios for new ventures, etc.
You fail in academia too, but the floor is different. With a startup, you cease to exist or drop off into an abandonware project. In academia, failure means you get lackluster appointments, little grant money, etc. You kind of stagnate.
I've noticed that smart self-aware people in academia can right this by recognizing that kind of stagnation for what it is, namely failure, and responding by changing course, increasing effort, etc.
The dangerous thing about academia is that failure just looks like mediocrity... mediocrity is failure. This isn't the case in the real world, where mediocrity is just mediocrity and failure is... well... you see them sleeping on the sidewalk.
Your dopamine system is not calibrated this way. So to succeed in academia I think you have to realize that if you're not a star you're sleeping in a gutter and respond to your situation accordingly. Respond to mediocre results the way you would respond to sleeping in the gutter in the "real world."
Looking at journals isn't the only way to do it. You can take encyclopedias and biographical dictionaries, and check for citations and mentions there.
Funnily enough, one such attempt (http://en.wikipedia.org/wiki/Human_Accomplishment) finds that per capita scientific achievement seems to start declining around the 1920s or 1930s, which is also (with WWII and whatnot) the beginning of the modern university system.
I think per capita scientific achievement has declined, but I think it's because we did all the easy stuff already. We are starting to hit a wall where new discoveries increasingly require one or more of the following:
1) A lot of expensive tools. (accelerators, genomics cores, supercomputers, etc.)
2) A lot of labor. (The "armies of graduate students and postdocs" phenomenon is a product of this.)
3) Intelligence beyond the capacity of a single individual. (Supercomputers, large numbers of people over time, etc. are required to "grasp" something. Anything ending in "omics" is a perfect example of this.)
You can think of it as being loosely analogous to peak oil, though the underlying causes are different. With peak oil, it's because the resource itself is limited. With scientific discoveries, you might have an infinite ultimate supply but you have a finite horizon of vision. It's like if there actually was infinite oil deep in the earth, but we were strictly incapable of building drills longer than one mile.
Personally I think #3 is the limit we're really hitting. I sometimes doubt that we have the IQ power as a species to truly grasp the genome or to unify QM and gravity. We either don't have the IQ or we don't yet have the language or philosophical framework to think about those things.
We're not infinitely smart. Can your dog read Hacker News? Maybe aliens are watching us now and thinking "well, they're sort of clever but they're really just still animals... they only see two sides to every conflict, they can't think in more than three dimensions, and they can't hold more than seven or eight simultaneous ideas at once..."
I'm not saying academia is perfect. It needs to change. If we really are nearing our IQ limits, we need an environment that removes distractions and allows people to really focus to get that last bit of creativity and intelligence. Academia is too bureaucratic, noisy, interrupt driven, and stressful. All those are IQ-killers.
"Correlation doesn't imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouth 'look over there'" http://xkcd.com/552/
I might buy the End of Science argument, but this would be far from the first time that it has been made, and I'm not sure it really works. Look at computer science; how much of that was or could have been worked out in the absence of an actual computer? And as much computer development was aided by government funding, the commercial imperatives were just as compelling. (Look at IBM's pre-digital computer success.)
I agree... like an economic bubble, I don't think it's possible to ever "call the top" except in retrospect. But it does worry me. I don't think human beings like to think about it because it bruises our collective ego to think that there might be a bunch of stuff that we're too dumb to get.
But I do wonder if we've passed "peak innovation." The biggest reason is the 1960s. It really feels like virtually every piece of technology that we use outside of maybe biotech was invented in the 50s and 60s.
Looking back, I see two peaks. I see a peak in per capita innovation in the early 20th century, and then a second peak in absolute terms centered at approximately 1960.
A random sample of an entire field or industry should have an exponential distribution, yes. Journals, though, are quality-gated: there are supposed to be review processes in place so that only the interesting/important/hopefully-seminal stuff gets published. That they reflect the same exponential curve is a testament to the fact that the review processes don't work.
I agree that journals do not have the quality control they claim to have. I was responding to the larger idea that academia is somehow unique in having such a quality distribution.
Realtime Linux: academia v. reality http://lwn.net/SubscriberLink/397422/27eef125e03b8a2d/
"I actually took the time to spend a day at a university where I could gain access to IEEE papers without wasting my private money. I picked out twenty recent realtime related papers and did a quick survey. Twelve of the papers were a rehash of well-known and well-researched topics, and at least half of them were badly written as well. From the remaining eight papers, six were micro improvements based on previous papers where I had a hard time figuring out why the papers had been written at all. One of those was merely describing the effects of converting a constant which influences resource partitioning into a runtime configurable variable. So that left two papers which seemed actually worthwhile to read in detail. Funny enough, I had already read one of those papers as it was publicly accessible in a slightly modified form.
That survey really convinced me to stay away from IEEE forever and to consider the university ranking system even more suspicious."