In early modern England, architecture was not a visual art. Architects and architectural historians still speak about the process of “reading” architecture, which encompasses both the way the building is visually perceived and what it expresses to the viewer. “Reading,” in this formulation, functions as a metaphor or analogy; one could not read a building by reading a text, and these processes require very different sorts of literacy. In early modern England, however, the relationship between reading a book and reading a building is often not one of metaphor or analogy; buildings can be read as texts and through texts, and what they express is not necessarily dependent upon visual experience. Modern scholars frequently replace the term “visual arts” with the more inclusive categories of “visual culture” or “visual discourse.” I argue that in the case of early modern English architecture, it is the word “visual” which is often misleading. This dissertation considers a series of authors who write fluently about architecture, but not in the visual, aesthetic, and material terms we might expect. Instead, architecture becomes an occasion for narrative, a way of recording the detailed and specific histories of individuals, families, or communities. I begin by reading Sir Henry Wotton's Elements of Architecture, not as England's first Vitruvian architectural treatise, but as an English country house poem. My second chapter pairs two very different texts which use architecture to narrate the instabilities of post-Reformation London: The Alchemist by Ben Jonson and John Stow's Survey of London. Next, I read George Herbert's critically neglected poem “The Church-Porch” by considering the ways in which its language and content relate to the historical, religious, and secular functions of a parish church porch. Finally, I turn to the late diaries and architectural projects of Lady Anne Clifford to argue that she perceived her books and her buildings as interdependent records of her own identity and ancestral history. These authors teach us to read architecture, not as an imperfectly articulated visual language, but as a skillfully executed, literary, historical, and narrative art.