The Woodruffs commissioned architect Otis Wheelock to design their new structure. The building was occupied by many businesses, notably the Merchant Exchange. It was also the first home to the Agricultural Insurance Company. The structure housed a large auditorium on it's third floor. Many 19th century orators spoke at Washington Hall, the most famous being abolitionist Frederick Douglass.
The building became the home to the Young Men's Christian Association (YMCA) in 1870. Washington Hall was demolished in 1913 to make way for the current YMCA Building.