Abstract: Spin glasses are models in statistical physics which have disordered (i.e. random) interactions. In the most interesting cases, the interactions can have both signs which leads to frustration effects which are very difficult to analyze. For standard short range spin glasses, there is, mathematically, essentially nothing known, and there are ungoing controversies in the theoretical physics community. As mean-field models are much easier to analyze, Sherrington and Kirkpatrick proposed in 1972 a model, now called the SK-model, which was published in a paper with the title "Solvable model of a spin glass", but they realized shortly afterwards, that their solution cannot be correct at low temperature, as it leads to a negative entropy. In 1977, Thouless, Anderson and Palmer proposed an approach based on a refinement of standard mean-field arguments, but it also didn't lead to a complete understanding of the model. Finally, Parisi in 1979 found his famous solution, obtained by mathematically non-rigorous replica methods, which however was hinting at a spectacularly rich mathematical structure behind the model, with an infinite dimensional symmetry breaking. In mathematics, the development started slowly, but in the last 20 years, it lead to important results. In particular, the Parisi formula was proved by Talagrand in 2006, using a breakthrough work by Guerra obtained before, and the important ultrametricity property was more recently proved by Panchenko. Despite all this success, the situation is however still quite unsatisfactory, as most of the spectacular results obtained recently use a very special property the SK and related models have, but which is lacking in many models which are important in applications, for instance in coding theory, theoretical computer science, compressed sensing, or neural networks. We will give a mainly non-technical overview, and in particular present some recent developments connected with the old TAP approach.