"Mutual Information in Machine Learning"Koliander, GüntherMutual information appears to be convenient as a measure of information between two random quantities and has thus been proposed as part of a cost function in the optimization of neural networks. Unfortunately, there are some fundamental problems with this approach. In the talk, we will first learn some basic definitions and properties of information theoretic quantities and discuss why mutual information should not be used in the context of neural networks. Finally, we will adapt the approach to circumvent some of the major issues and see how an information-theroretic inspired training could work. |
| https://arxiv.org/abs/1905.07822 |
« back