In this talk I will show how the characteristic polynomial of a random unitary matrix has been successfully used in number theory to model the Riemann zeta function, and then I will present some new work re-interpreting and generalizing the previous random matrix results in a probabilistic setting. This is joint work with Paul Bourgade, Ashkan Nikeghbali and Marc Yor.