Hey folks who understand deep learning and games. This is the famous method that has allowed computers to beat human players reliably at chess and go. Many intricate games have semi-open information. For instance, in Catan you see the other players draw resource cards to their hidden hand and then play them. But only an idiot savant or someone who keeps written notes (against all social convention) can keep track of what’s on anyone’s hand at a given moment.
In other games such as poker you can in principle assign probabilities to what card another player just drew by looking at what cards the full deck contained, what cards have been played so far, and what cards you are currently holding. Few human players outside of the movie Rain Main can do this.
Let’s call these methods card counting. Here’s my question. If you teach a computer to play Catan or poker with deep learning, should you introduce rules against card counting? If you don’t, then the computer is effectively not playing the same game as humans do. This would make deep learning Catan different from when deep learning algorithms play chess and go with exactly the same rules as humans.
Aardvarchaeology - by Dr. Martin Rundkvist 
My guess is that even if card counting were not allowed, the computer would not play it in the same way as a human would.
Many chess computers play by brute force, calculating many more possibilities than a master chess player could. I believe that the Go computer which beat a human expert, though, learned by deep learning. Still, that doesn’t mean that the resulting algorithm is anything like that humans use. And maybe different humans use different ones.
LikeLiked by 1 person
“If you teach a computer to play Catan or poker with deep learning, should you introduce rules against card counting?”
You just summarized a very important issue in any computer game, which TVtropes discusses in their page “the computer is a cheating bastard”.
Actually, that seems like a new take of an old issue. Or a new complication, as tech is marching on.
As the computer is all-knowing – it knows all your cards and token positions, and it usually also tracks all your past decisions – it has potentially an unfair superhuman advantage over the player.
(some game developers actually deliberately/lazily exploit this to increase the game challenge – notably some sniper games where NPC foes with a pea-shouter have no trouble spotting and hitting you from across the map)
Segregating what the computer should know as the game manager, versus that it could know as a player is a first step to balance the game.
Now the tricky part is defining that it could know. Letting the computer-player analyzes the human player behavior seems fair, as humans players are also into spying their fellow players and metagaming. As I understand it, chess and to some extend poker are all about metagaming. Your opponent is doing this, that means they may be planning of doing that…
But if the computer is reaching inhuman levels of metagaming, by example by a highly accurate prediction of the cards in your hands through perfect card counting, we are in effect back to square one, the all-knowing computer.
So some level of limitation in card-counting (or other ways of analysis) seems to right way to go. But where to set the limit?
It doesn’t help that I have fellow gamers who are very good at card-counting/resource-counting and metagaming. I gave up playing Agricola with them.
LikeLiked by 1 person
It would be hard to exclude some level of “card counting” from a deep-learning-based Catan AI, as you would at that point have to explicitly exclude the knowledge of cards played by opponents.
I am merely an egg when it comes to “deep learning”, but my layman’s understanding[*] is that instead of feeding in “current state of the observable system” (that is, the board, and own hand), you feed in “current state of the observable system, plus a possibly narrowed view of the previous internal state”. This allows an AI to react differently on identical observable external state, depending on what happened in the past (say, if you failed to get trades for sheep last turn, you should probably not bank on being able to trade for sheep this turn; no matter what’s in your hand and on the board).
Since the specific “this is what other players have on their hand” is not explicitly held anywhere (it is a neural network…), it is somewhere between difficult and impossible to excise it from the network. It MAY be possible to guide the AI away from it if it is not in “learning mode” when playing for reals (that is, updating weights in the network) and have extremely guided learning that identifies cases of “this could only have been made using card counting” and explicitly excludes those during training.
But, then, that would mean generating a LOT of games where card-counting would have an advantage, then explicitly punish them out, and that that’s really a lot of person-power spent.
[*] Actually, no, “deep learning” is explicitly havig multiple layers and the ability to form the final decision/classification from a combination of one or more layers, what I am talking about Up There is recurrent neural networks, which you may also want. Nonetheless, the main point that there’s no localised “card-counting cell” in the NN model, it is (if card counting is present) otherwise distributed across the entire model and can only really exist in a recurrent system; otherwise the decision is memory-less and blindly does the same action when given the same inputs (which, of course, can still be eerily good).
LikeLiked by 1 person
Yeah, I guess no computer can ever really play Catan & Co under the same conditions as a human.
LikeLike
I think the closest you can get is “fed information about each play, plus information about what player made it”, followed by “this is the current board, this is your current hand, please indicate action(s)”.
But in order to make a computer “play Catan & Co under the same conditions as a human” would most probably require breaching the “general artificial intelligence” barrier, including but not limited “general vision”, “general speech recognition”, and “fine-motor capabilities” and is not at ALL easy to say if it falls in one if “impossible, in principle”, “impossible, in practice”, “really hard”, or “we just haven’t figured it out yet” (I consider the fact that we haven’t seen it, 60+ years later, as a sign taht it was probably not just merely “hard”).
LikeLiked by 1 person
Not keeping track of what cards have been drawn seems to be a convention, not a rule of the game. Most card games I’ve played assume people are going to take into account all available information. I suppose you could avoid letting the computer see what cards other players have drawn, but that puts the computer at an unfair disadvantage. What happens if a player happens to remember a card another player has drawn and unconsciously takes it into account? What if they just happen to consciously remember?
I suppose it’s like bodyline bowling in cricket. In England, traditionally, bodyline bowling was “not cricket”, but nothing in the rules said anything against it. Once cricket went international, it became common practice outside of England.
LikeLiked by 1 person