This talk traces a genealogy of affect theory from the early modern era through to the present day, establishing the central significance of music for this history. It demonstrates that the theory of affect we have inherited today has its origins in eighteenth-century aesthetic debates concerning music’s capacity to function as a sign and to move its listeners. In the early modern era, the affects were important components of an elaborate semiotic system that sought to explain the impact of art. Today, by stark contrast, affect is often explicitly opposed to theories of the sign and of representation; theorists describe affect as corporeal and immediate, working on our autonomic systems. The genealogy elaborated in this paper shows how affect theories became separated from theories of representation, and it illustrates the central and surprising role that music played in this separation.