Learning How To Listen In