language_to_ib_point incorrectly converts language to binary matrix for encoder weights

[nathimel]

I suspect that ib_encoder_to_point is correct and that if there is a bug in language_to_ib_point then it comes from converting the language to the encoder, i.e. the Speaker weights, in language_to_ib_encoder_decoder.

I’m pretty sure the LiteralSpeaker has to be the problem. Because if we look at line 230 of agent.py we’ll see it initializes a binary matrix, which is not right.

• I should consider refactoring these agents to be more flexible at initialization and have this be default behavior if a weight matrix isn’t passed in.
• What we probably want to do is use the regular Speaker class and pass in weights as one of the kwargs, since CommunicativeAgent will then initialize correctly.
• This will require some rewriting of the relevant functions in rate_distortion