The early history of the Greenleaf Houlton Residence is somewhat uncertain. It is speculated to have been erected in the early 19th century, and may have been a stable with a straight uninterrupted gambrel roof, rather than the residence with two front gables it currently is.

However, it is known that the residence was acquired by Greenleaf Houlton in 1858. Greenleaf had been born in 1825 in Houlton, Maine, which had been founded by his father, James Houlton. As a carpenter, Greenleaf had moved to St. Andrews in circa 1854 to find work on the railway. During his career here, he patented a new type of car axle box and personally modified the roof of this building to have its two characteristic asymmetrical gables. He was also a member of the American Consul. The relocation of the railway from St. Andrews to McAdam in 1884 motivated Greenleaf to move to San Francisco, where he died in 1891.

The history of the Greenleaf Houlton Residence is intertwined with that of the famous Algonquin Hotel, which sits across the street. During the construction of the hotel, there were four rental cottages built around the property, with this building sometimes being called Cottage Number 5. This building served as both the residence of the General Manager of the Algonquin Hotel and the hotel's office space.

In terms of architecture, the Greenleaf Houlton Residence combines Gothic Revival and Dutch Colonial features. The building is topped with a gambrel roof with two cross gables, with one being noticeably bigger than the other. The fenestration is asymmetrical and varied, with a single Gothic window under the smaller gable.