See also: Rosemary Hall for its history before the 1971 re-affiliation with Choate.

The schools that would eventually become Choate Rosemary Hall were begun by members of two prominent New England families, the Choates and Atwaters.

Rosemary Hall was founded in 1890 by Mary Atwater Choate at Rosemary Farm in Wallingford, her girlhood home and the summer residence of Mary and her husband,William Gardner Choate. Mary, an alumna of Miss Porter's School, was the great-granddaughter of Caleb Atwater (1741–1832), a Connecticut merchant magnate who supplied the American forces during the Revolutionary War. In 1775 General George Washington visited the Atwater store in Wallingford en route to assuming command of the Continental Army. On that occasion, Washington took tea with judge Oliver Stanley at the "Red House" (built 1690–1750), now Squire Stanley House on the Choate campus.

In 1878 Mary Atwater Choate had co-founded a vocational organization for Civil War widows, the New York Exchange for Women's Work, prototype of many such exchanges across the country (it survived until 2003). In 1889 Mary planned a new institution on the same principle of female self-sufficiency and she advertised in The New York Times for a headmistress to run a school that would train girls in the "domestic arts." The advertisement was answered by Caroline Ruutz-Rees (1865–1954), a 25-year-old Briton teaching in New Jersey.

On October 3, 1890, the New Haven Morning News reported: "The opening of Rosemary Hall took place at Wallingford yesterday ... at the beautiful Rosemary Farms, which have been the property of Mrs. Choate's family for five generations. ... Rev. Edward Everett Hale addressed the school girls in his inimitable way, at once attractive and helpful. 'Never forget,' said he, 'that it is a great art to do what you do well. If you limp, limp well, and if you dance, dance well'."

The eight arriving girls occupied the original school building, "old Atwater House" (built 1758), at the northwest corner of Christian and Elm streets, where new Atwater House now stands. They also had the use of "Atwater homestead" (built 1774, now known as Homestead), which stands at the center of the present day campus, on the northeast corner of Christian and Elm streets.
Caroline Ruutz-Rees (pronounced "R-Treece"), headmistress of Rosemary Hall until 1938, was a figure of extraordinary personality and influence, a militant feminist and suffragist of national prominence. On the Wallingford golf course she wore bloomers, which shocked the locals, and on buggy rides to Wallingford station she carried a pistol. She quickly changed Rosemary Hall's mission from "domestic arts" to that of a contemporary boys school.

The Choate School was founded by William and Mary Choate in 1896. William Gardner Choate (1830–1921), Harvardclass of 1852, was U.S. District Judge for the Southern Circuit of New York from 1878 to 1881, and afterward a partner of Shipman, Barlow, Laroque, and Choate. He was a national authority on admiralty, railroad, bankruptcy, and corporation law. Like his younger, more famous brother, he was a prominent clubman (Harvard and The Century). That brother was Joseph Hodges Choate, lawyer, prosecutor of the Tweed Ring, andAmbassador to the Court of St. James's.

William and Mary Choate invited Mark Pitman (1830–1905), their tenant in the aforementioned Red House, to start a boys school under their sponsorship. Pitman,Bowdoin class of 1859, was sixty-six years old, a widower, and had been principal of Woolsey School in New Haven, Connecticut, since 1872. He accepted the Choates' offer, not least perhaps because it would provide employment for his unmarried adult daughters, Leila, Elizabeth, and Helen. Six boys entered the new school in fall term 1896, their average age about ten. Four of the six lived in Red House with the Pitmans, and Red House (now Squire Stanley House) has remained a dormitory to the present day.

Pitman taught Latin, English, history, and science; Elizabeth taught art, Helen piano, and Leila was writing teacher and school nurse. Mary Choate's physician brother, Dr. Huntington Atwater, taught crafts and was school doctor. There was no formal relationship with the Choates' other foundation, Rosemary Hall, a hundred yards to the east on Christian Street, but there were coeducational audiences for plays and recitals and Mary Choate hosted dances at the Homestead.

In 1897 the boys school erected Choate House across the street from Red House, the first purpose-built institutional building (and John F. Kennedy's dormitory in 1931-2). It contained recitation rooms, an infirmary, a dining room, and housing for fifteen boys. In 1899 Choate House was venue for the first "Junior Dance," but a year later the Rosemary girls would depart for a seventy-one year absence.
The official history of Choate Rosemary Hall, written by Tom Generous, says that the rift between Caroline Ruutz-Rees and Mary Choate, proponents of two very different sorts of feminism, was public knowledge as early as 1896. In that year the two women did not share the lectern at Prize Day, and local newspapers published "denials" of a rumor that Ruutz-Rees would leave the school. But by 1900 the headmistress and her educational style had acquired influential champions among the students' parents and two of them, residents of Greenwich, Connecticut, joined forces to effect the removal of the school to their town.

Shipping magnate Nathaniel Witherell donated 5 acres (20,000 m2) of land in the Rock Ridge section of Greenwich. Julian Curtiss gathered a group of investors and established a joint stock corporation funded through the sale of six-percent bonds. Ruutz-Rees was the chief shareholder. The Greenwich residence of Rosemary Hall began in fall term 1900, when 57 girl students moved into the Main Building, known as "The School," a U-shaped shingled house on Zaccheus Mead Lane.

In Wallingford, Mark Pitman died on December 3, 1905. Until 1908 Sumner Blakemore was titular headmaster, but the school was effectively the domain of the three Pitman sisters. At the 1908 graduation ceremony the Japanese Consul General watched his countryman Noyobu Masuda give the valedictory address. Then Judge Choate introduced the man who would assume the headmastership in the fall, George St. John, and his wife, Clara Seymour St. John. She was a Bryn Mawr alumna, member of a well-connected Connecticut family, sister of future (1937–1951) Yale president Charles Seymour, and descendant of Yale president (1740–1766) Thomas Clap.

George Clair St. John (1877–1966), Harvard class of 1902, aged 31 in the fall of 1908, had grown up on a farm in Hoskins Station, Connecticut. He was an ordained Episcopal priest. He had taught at Hill School in Pennsylvania and Adirondack-Florida School, and was teaching at Hackley School in Tarrytown, N.Y., when Samuel Dutton of Columbia Teachers College recruited him for the headmastership. St. John "knew, long before I read Mr. Dutton's letter, that I wanted some day to have a school of my own. In my thought about it, Dean Briggs was my first text." This was LeBaron Russell Briggs, dean of men at Harvard when St. John was there and afterward dean of the faculty until 1925. Briggs, St. John wrote, "fathered the whole college," and the St. Johns too would serve in loco parentis.
Their first move to secure parental powers was decisive. In September 1909, as the official history tells it, the St. Johns "signed with the Choates an 'Agreement to Lease and Purchase.' Under its terms, the younger couple rented the school, its property and reputation, for five years at a sum equal to 11 percent of The Choate School's net income per year. ... Less than twenty months later, on May 12, 1911, St. John reported to the trustees that he had purchased the title to the school by acquiring mortgages of roughly $41,000. He resold the title in turn to The Choate School, Incorporated, for $23,000 cash and $38,000 in stock ... For all intents and purposes, The Choate School belonged to George St. John."

In his first quarter-century as headmaster, St. John created much of Choate as it is regarded today. Of the Georgian brick and stone campus, he had built by 1932 Hill House, West Wing, the Gymnasium, Memorial House, the Chapel, the Library, the Winter Exercise Building, and Archbold Infirmary, which was the largest school infirmary in the country. He grew the enrollment from 35 to 505 boys and the faculty from 5 to 64 masters. In the decade following the First World War (classes of 1918 to 1928) Choate sent 412 of its 618 graduates to Yale, Princeton, and Harvard, according to a table published in The Choate News in fall term 1928.

George St. John belonged to the generation of legendary, long-serving headmasters who shaped the New England prep school, chief among whom wereEndicott Peabody of Groton, Frank Boyden of Deerfield, Horace Dutton Taft of Taft, Frederick Sill of Kent, Samuel Drury of St. Paul's, Alfred Stearns of Andover, Lewis Perry of Exeter, and George Van Santvoord of Hotchkiss.

The Rev. George St. John of Choate was succeeded in 1947 by his son, the Rev. Seymour St. John '31 (1912–2006), and the "St. John dynasty" was continued to 1973. Seymour was Yale class of 1935 and was ordained at Episcopal Theological Seminary (now Virginia Theological Seminary) in 1942. During his time as head he built as many buildings as his father had built, greatly broadened the curriculum, raised the national profile of the school, and made it more progressive (Eleanor Roosevelt, Norman Thomas, and William Sloane Coffin were regular speakers) and cosmopolitan (Russian, Near Eastern, and Afro-American studies centers were founded, and Russian, Chinese, and Arabic courses were begun). St. John was a longtime advocate of coeducation and initiated the Choate-Rosemary contacts. At his death, headmaster Edward Shanahan told the New York Times, "The merger demanded an enormous expenditure of resources by Choate because the building of a new campus was to occur within the footprints of its property. Seymour was central to the decision to expand those resources."