The mansion is a twenty-two room Georgian-style mansion hand-cut and made of blue limestone. General Jacob Jennings Brown planned on using it as a temporary jail for War of 1812 prisoners but the war ended before the mansion was fully finished. It remained in the Brown family but the family never lived there.
On April 1st, 1946, Samuel Upham, Watertown philanthropist, bought the Brown Mansion and gave it to the village of Brownville. Since then it has been used as a museum, school, library, civic center and village meeting place.
More info needed.