It has been conjectured that the fundamental theory of strings and branes has an $E_{11}$ symmetry. I will explain how this conjecture leads to a generalized space-time, which is automatically equipped with its own geometry, as well as equations of motion for the fields that live on this generalized space-time.