Historical records first mention Andrew Bay, "a broad Scotchman," in 1748 as a member of the Presbytery in New Castle. He was pastor at March Creek, Pennsylvania and Deer Creek Maryland. In 1768, the synod asked him to settle near Albany to service settlements in Montgomery and Washington counties. Bay came to FPCN in 1773.

Several authorities agree that he was a highly talented and eloquent preacher, but some people suspected that he drank too much. Some people asked the Presbytery of New York to dismiss Bay, claiming that "the congregation generally seemed disaffected with Mr. Bay." Although the Elders declined to make a formal complaint, "many charges were implied against Mr. Bey's [sic] prudential and moral character." The Synod held an inquiry at Newtown on June 20, 1775. The following day, the congregation voted by ballot. Several members voted to dismiss Bay. When the presbytery dissolved his pastoral relation, he appeared before the synod on May 28, 1776. The synod sustained the action.

Bay is said to have died at the parsonage where his family lived during the American Revolution.