Langlands conjectured a very close relation between the arithmetic of number fields on one side, and modular forms on the upper half plane (and functions on more general locally symmetric spaces) on the other side. Using an analogy between number fields and fields of meromorphic functions on Riemann surfaces, these conjectures admit a natural counterpart in which quotients of the upper half plane are replaced by moduli spaces of bundles. I will try to motivate this geometric analogy and explain some basic aspects of the geometric Langlands correpondence. In particular I will try to give a series of explicit examples of modular forms in this context that were found in a joint work with B.C. Ngô and Z. Yun. These turn out to be closely related to some classical exponential sums and give rise to local systems with surprising properties.