Orr moved to Topsham, in 1801 and continued the practice of law; was overseer of Bowdoin College in Brunswick, and served as trustee from 1814 to 1828 and as treasurer in 1815 and 1816.
Orr was elected as a Federalist to the Fifteenth Congress (March 4, 1817 – March 3, 1819) but was not a candidate for renomination in 1818.
He resumed the practice of law in Topsham and, in 1822, returned to Brunswick to continue the practice of law.
He died in Brunswick, Maine on September 3, 1828, and he was interred in Pine Grove Cemetery.