Recent developments on Feynman integrals and string amplitudes greatly benefitted from multiple polylogarithms and their elliptic analogues — iterated integrals on the sphere and the torus, respectively. In this talk, I will review the Brown-Levin construction of elliptic polylogarithms and propose a generalization to Riemann surfaces of arbitrary genus. In particular, iterated integrals on a higher-genus surface will be derived from a flat connection. The integration kernels of our flat connection consist of modular tensors, built from convolutions of Arakelov Green functions and their derivatives with holomorphic Abelian differentials. At genus one, these convolutions reproduce the Kronecker-Eisenstein kernels of elliptic polylogarithms and modular graph forms. I will conclude with an outlook on open problems and work in progress.