He commissioned Peabody & Stearns, to build 306 Dartmouth Street in Boston but died before it was completed and in 1873 his estate sold the finished mansion to Charles Whitney. Thayer was survived by his widow and their son who moved to 191 Beacon Street.