If they're not limiting AlphaStar's TTFA, then it can respond instantly to problems all over the battlefield, which is superhuman in an uninteresting way.
In any real-world application (robotic manufacturing, self driving vehicles, precision guided munitions targeting...) extremely fast reaction times are an essential and expected advantage of machines. I see AI game playing as interesting mostly because it provides a way to pit software against people without building a bunch of expensive physical infrastructure. If anyone ever builds a "real" robotic army it will surely use all the APM and TTFA advantages it can possibly achieve. In that sense, allowing the same advantages in combat simulation games is closer to a proper man-machine contest than one where those advantages are taken away from the machine.
It comes down to what you are interested in. Right now DeepMind seems to be focused on beating humans on strategy and tactics, which is reasonable, since those are the contested areas and cpu winning on APM and TTFA has been solved for long ago. To not confuse one for the other, putting limits on the later (at some point maybe even stifling itself beyond what a pro player could reasonable to) seems like an excellent way to go about it.
It's disputable whether any such limitations currently applied to AlphaStar are really working as intended. There has been some controversy over it. I saw the matches and it did seem that some of the actions taken by AlphaStar would not be possible to do by a human. Human APM is limited not just by the mechanical speed of one's hands, but also by the mental models that exist in one's brain. Muscle memory can't do everything for you. But in AlphaStar's case, it can perform every type of action as if it were done with the speed of muscle memory.
The original commenter's remarks about TTFA seems to be equivalent to what I'm referring to.
They can train a discriminator network to classify actions as being generated by a human or bot. Then aim to generate actions that can't be distinguished.
> cpu winning on APM and TTFA has been solved for long ago
Has it? I'm not a RTS player but I thought that simple bots couldn't compete with top StarCraft players just with speed. It required strategy plus speed to first become competitve (though less strategy than required with slow APM and TTFA).
Apologies, I should have been more precise: A cpu will win when you start isolating and creating a scenario to test for APM and TTFA (obviously), so if you are interested in advancing strategic and tactical AI capabilities – basically out-thinking the human – it makes sense to cap those solved parts to make sure you are winning because of progress you make on your objective.
Brood War is much more dependent on micro than Starcraft 2. Sean "Day[9]" Plott has mentioned this a lot in his videos. You can only select 12 units at a time and cannot select more than one building in Brood War. And there is a bit of latency in Brood War that doesn't exist in SC2.
The idea is to see if the bot is playing as smart as a human. To measure this, you must factor out any advantages not available to humans. This is valuable feedback when developing an intelligent agent, but that doesn't mean you can't remove those artificial handicaps once going to production with something like self-driving cars.
Edit:
Alphastar does already factor out reaction time:
“We measured how quickly it reacts to things,” Silver said. “If you measure the time between when AlphaStar perceives the game. From when it observes what’s going on, then has to process it, and then communicate what it chooses back to the game. That time is actually closer to 350ms. That’s on the slow side of human players.” https://venturebeat.com/2019/01/24/alphastar-deepmind-beats-...
That may be true in some narrow technical sense, but it was heavily disputed by the StarCraft community. If you watch the exposition games, AlphaStar has superhuman ability to micro-control its units one-by-one during critical battles, which allowed it to beat its human opponents even when its army was far inferior on paper.
Not sure why you are getting downvoted. Upon closer inspection it was indeed perceived as contrary to what was expected from the StarCraft community. (And rightfully so)
I guess the thing people are interested in is a program that can autonomously play the intelligent parts of the game - the strategy and all that. I assume Starcraft is a poor choice for that, though, because of the massive importance of micro.
For instance, imagine you’re playing a chess game with a 100 ms timer and first to exhaust their timer loses. No human will win and I could create a program that could best Kasparov trivially by advancing each pawn. There’s the game and then the input layer problem.
Precisely. Focusing on APM is kind of a red herring anyways because during previous matches there were only a few milliseconds that it spiked above human levels, and it was usually when moving workers: https://deepmind.com/blog/alphastar-mastering-real-time-stra...
Even so, this latest version has max APM limits instated to appease pro-players. Since Alphastar is forced to perceive the state of the game through machine vision of the screen, it's reaction time is already on par with humans anyways (~350 ms for Alphastar vs. ~250 ms for humans).
Alphastar did some inhuman stalker micro that the APM debate seeks to cover, but doesn't.
Stalkers have a player operated ability to instantly move a short distance 'blink' once every 5 seconds. When in a fight, you optimally let the stalker(s) taking damage soak up as much damage as possible and then blink them backwards so they can recharge shields and continue firing from behind other stalkers. They don't stop firing, so all the work the opposing force did trying to kill a shooter, resulted in no outcome at all.
That functionality is balanced by the fact it is hard for a human to time activating the abilities of many stalkers at once in time with the damage they are taking and perform the many other actions the game requires at the same time.
Alphastar can perfectly blink back stalkers with limited apm because timing things is obviously not a problem for it, making stalkers way more value-for-money than they should be and can hold off high investment attacks more cost effectively. Ultimately sc2 is a game of economy and timing so this small change gives a massive advantage.
If I'm not mistaken, the blink micro was unrealistic because that version of Alphastar played a modified version of the game. It's "screen" covered the whole map and it had no need for a minimap. Human players can do some of the blink micro that Alphastar did, if restricted to just one screen of space.
What made its micro different was that it did it consistently from flanks on several sides of an army exceeding the boundaries of a human screen. It was also notable that it did that while macroing at home, but some macro actions during intense micro is done among better pros.
The blink micro advantage should be far reduced if Alphastar is playing the same StarCraft II installation as humans now.
> Focusing on APM is kind of a red herring anyways because during previous matches there were only a few milliseconds that it spiked above human levels, and it was usually when moving workers
If you believe Aleksi Pietikäinen -- and pretty much every one of the professional players who played against it -- the claim you're repeating here is so misleading as to be fairly considered an intentional lie on the part of the deepmind team.
For example, TLO's inflated APM are presented in that chart without comment. Specifically, without the comment that his high APM counts come from a particular game context in which holding the mouse button down (i.e. a single click with a duration) is counted by the game as thousands of APM.
AlphaStar APM spiked to ridiculous inhuman levels during stalkers micro. On top of that, it controlled units that were screens apart at the same time, which is not supposed to happen.
DeepMind's refusal to aknowledge it, on top of the sketchy
and misleading TLO chart didn't do them any favor.
> A. Like human players, AlphaStar perceives the game using a camera-like view. This means that AlphaStar doesn’t receive information about its opponent unless it is within the camera’s field of view, and it can only move units to locations within its view. All limits on AlphaStar’s performance were designed in consultation with pro players.
Your assumption us correct about the show matches against MaNa and TLO that many people are talking about in here. That was not the long term goal of the Alphastar team to keep it on that heavily modified version of the game. For one thing, it meant that Alphastar needed a customized version of the game that it couldn't play on the ladder. As far as an AI challenge goes, it's also really weak if the AI gets more direct access to game data than its human opponents.
I agree that trying to play video games really well is mainly about getting "cheap" experience for real-world applications, but that is exactly the reason for why they should limit APM and reaction times.
Because in the real world, you have to get a bunch of sensor data, potentially run it through it's own neural net to recognize objects, and then feed it into the decision making system. All that takes time - most likely much more time than the 1 frame it takes for the AI to make an API call.
If the AI actually played through the same interface as humans, i.e. it simply gets the rendered image as an input, and produces mouse/keyboard inputs as an output, then maybe we should disable artificial APM/reaction time limits. But as it stands now, the AI has an absolutely massive advantage simply due to using a much better interface, which it won't have in the real world.
Knowing speed runners and the like, I can imagine they will quickly find a way to determine if the player is in fact DeepMind via some sub milisecond method.
Depends what you mean by "instantly...all over the battlefield." It essentially uses the monitor/keyboard/mouse for its input and output, rather than some nice API of the game state. Thus, like a human player, it can see things on the minimap roughly instantly then use the keyboard or mouse to move the camera and start responding.
If you watch the first-person view of top human professionals it already looks pretty instant or "mechanical," and is sometimes hard to follow for me even as a semi-competent player and avid spectator. What's the TTFA for a pro when an enemy drop appears on the minimap? I would guess somewhere around 200ms at the quickest? That would be similar to the latency of the DeepMind neural network (supposedly 350ms). And, of course, Starcraft 2 already has an approximate input latency of 200ms (so that all players can receive all other players' inputs and run them against their game state).
At the end of the day I don't see pure reaction speed as being a huge issue in Starcraft 2. Perhaps it would give a computer an "unfair" advantage in some rare cases like two cloaked ghosts running into each other and trying to snipe each other.
That's how I feel about DeepMind for sc2, it doesn't sounds like a smart AI but just an AI insanely fast ( can micro everything instantly ) which is not realistic. Also APM is a usless metric especially for a computer since a computer only do useful actions, so you have a pro player doing 400 apm and DeepMind just 120 but 100% useful.
I don't understand your criticism. DeepMind's APM is limited, and you're saying there's still a problem because DeepMind will use its APM more effectively? Well...yeah, that's a big part of what it means to play Starcraft 2 better than your opponent, whether it's a human or a computer.
This is how APM is treated in the professional scene as well. Everyone knows that bursts of 500 APM when you're spamming at the beginning of the game aren't some incredible display of skill. But sustaining a good number of useful actions per minute is incredibly important, and a huge part of the mental process in Starcraft 2 is constantly deciding where and how to invest your actions.
It's really uninteresting that the machine wins simply because it doesn't have to deal with the inaccuracy and delays of mouse clicks and selection boxes. You can say that it's a limitation of humans, but aren't we searching for evidence of high-level strategic thinking in the machine, instead of purely mechanical advantages of the machine executing copy-pasted human strategies with inhuman accuracy?
This video seems to imply that latency of the neural networks is around 350ms, which is fast but within human ranges: https://youtu.be/cUTMhmVh1qs?t=1488
In terms of APM, yes, which is what this FAQ addressed:
> AlphaStar has built-in restrictions, which cap its effective actions per minute and per second. These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.
So yeah, they've tweaked this specifically, although not much details as to how.
If they're not limiting AlphaStar's TTFA, then it can respond instantly to problems all over the battlefield, which is superhuman in an uninteresting way.