The automorphism tower of a group is obtained by computing its automorphism group, the automorphism group of that group, and so on, iterating transfinitely. The question, known as the automorphism tower problem, is whether the tower ever terminates, whether there is eventually a fixed point, a group that is isomorphic to its automorphism group by the natural map. Wielandt (1939) proved the classical result that the automorphism tower of any finite centerless group terminates in finitely many steps. This was successively generalized to larger and larger collections of groups until Thomas (1985) proved that every centerless group has a terminating automorphism tower. Building on this, I proved (1997) that every group has a terminating automorphism tower. After giving an account of this theorem, I will give an overview of work with Simon Thomas and newer work with Gunter Fuchs and work of Philipp Lücke, which reveal a set-theoretic essence for the automorphism tower of a group: the very same group can have wildly different towers in different models of set theory.