The earliest antecedent of the Johns Hopkins Club, Inc. was an informal "literary and social" group that began meeting in the Autumn of 1876. Composed of instructors and fellows, and modeled after the German student "Kneipe", the group gathered on Saturday evenings to converse and dine. Lacking a club house or meeting hall, the group convened in the private rooms of various Baltimore restaurants. Meetings often featured German conversation and singing, as many of the University's original instructors and fellows had studied in Germany.

By the beginning of 1879, several students and faculty members, including participants from the original "Kneipe" group, felt the need for a more formal association and a permanent location. An organizing committee that included Herbert Baxter Adams, then Fellow in History, drew up plans for what became in 1880 the Johns Hopkins University Club. The Club offered membership to faculty and graduate students for an annual fee of five dollars. Two rooms on Garden Street were rented and furnished by the Club, and were stocked with a wide assortment of magazines and newspapers. The Club quarters were open to members from 9:00 A.M. until midnight each day, with a special gathering every Saturday night. Activities ranged from quiet reading on weekdays to boisterous "musical evenings." Since the rooms did not have a kitchen, a steward was employed to bring in refreshments from nearby restaurants.

In 1887 the Johns Hopkins University Club "dissolved in favor of" the University Club of Baltimore. Although a number of Hopkins students and professors were among the original members of the University Club, the Club was not officially associated with Hopkins. Membership was consequently not limited to Hopkins people but included non-academic professionals, graduates of other colleges, and Baltimore residents "in close sympathy with the university spirit." Club members had to be over the age of twenty five and were required to pay annual dues of thirty dollars. The University Club secured a house at 1005 N. Charles Street and opened it to members and guests in November of 1887. Basil C. Gildersleeve, Hopkins Professor of Greek, served as the Club's first president. In 1902 the Club purchased new quarters at 801 N. Charles. Although the University Club fulfilled some of the functions as the earlier JHU Club, there were those who felt that it failed to serve a sizable portion of the Hopkins community. In an essay written for the 1892 "Hullabaloo", Herbert Baxter Adams noted that, due to the age requirement and high membership fee, "the younger academic element is practically shut out."

Seven years later, Adams still felt the need for an organization specifically designed for the Hopkins community. At an alumni association in February, 1899, he suggested the formation of a Johns Hopkins Club. The new Club was limited to faculty, alumni, and graduate students, and was to charge a moderate membership fee. An organizing committee sent out a circular letter to prospective members and quickly received 250 acceptances. The Club was formally organized in October 1899, with Joseph S. Ames, then Professor of Physics, as its first President. Annual dues were set at ten dollars for resident members and five dollars for graduate students and non residents. At the time of its formation, the Johns Hopkins Club was unique among Baltimore clubs for limiting its membership to college graduates.

The Johns Hopkins Club rented a house at 706 St. Paul Street and opened in January 1900. Unlike the facilities of the earlier Johns Hopkins University Club, this house had a kitchen and dining area, along with several "sleeping rooms" that were rented to club members (usually instructors) on a monthly basis. In 1902 the Club moved to another house at 516 Park Avenue. The Club wished to acquire yet more spacious accommodations, and in 1905 it purchased a large house at 227 N. Monument St. (corner of Howard Street). The Johns Hopkins Club Building Co. was created at this time to facilitate the financing and improvement of the Monument Street house. By 1910 the Club had 600 members. Recreational and educational activities were given equal emphasis; monthly field nights included a lecturer, a dinner, and a late night musicale. When the University moved to the Homewood campus in 1916, the Club was installed in the Carroll mansion. Dining and rooming facilities continued to be provided, along with a variety of social activities. In 1929 the University approved a proposal to restore the mansion as a museum. The Club, forced to vacate its premises, was offered the use of a dining room in the newly completed Levering Hall, but found that facility unsuitable for the full range of its functions. The Board of Governors considered dissolving the organization but decided instead to merely suspend its activities "until such future time as the Board may determine that their resumption is practicable." During the next few years, Club meetings were infrequent and membership declined sharply. Then, in 1936, the Club received heartening news.

In 1907 members of the Marburg family (Amelia, William, and Theodore) had donated $50,000 to be used for the construction of a building at Homewood as a memorial to their brother, Charles L. Marburg. By 1936, William had died and the money was as yet unspent. In that year, Theodore and Amelia added $30,000 to the original gift and announced that they wished to have the fund used for the construction of a faculty club. The Trustees provided a site on the north side of the Botanical Gardens. Work was begun in the summer, in accordance with the specifications of Theodore Marburg, who personally approved much of the design. The architectural firm of Wrenn, Lewis, and Jencks had designed a "modern counterpart to the Homewood House" that featured a large kitchen, several dining rooms, lounges, two libraries, and a tap room. There were, however, no provisions for lodgers. By agreement with the Trustees, the University was to own the building and keep the grounds, and the Club was to be financially independent and reimburse the University for electricity and insurance.

The Johns Hopkins Club entered its new facility in April 1937, and was incorporated as the Johns Hopkins Club, Inc. on November 17, 1937. Membership was initially limited to Hopkins faculty members, including, for the first time, women faculty members. According to the wishes of Theodore Marburg, however, women were excluded from the main dining room, so a separate dining room for women was provided at the east end of the building, known then as the Ladies' Dining Room. It was soon found that the faculty alone could not support the Club, so alumni, and later graduate students and administrative officers of the University, were invited to become members. Membership soared, and by 1940 the Club had almost 1000 members and served about 140 meals a day. The alumni rapidly became the most frequent users of the Club in the evenings, and the faculty at lunch. Meetings of private organizations were generally not allowed, although the Club was used extensively for departmental meetings during the day.

By 1960, the facilities of the Club had become severely overtaxed. Although the building had been renovated and expanded, membership had more than tripled to 2700 resident and 400 non resident members, with an average of 360 meals served per day. Several options to relieve this situation were considered by the Board of Governors, including placing ceilings on graduate student and alumni membership, and encouraging the East Baltimore campus to form its own club. Ultimately, however, none of these nor any other major action was taken, though the Board of Governors did begin to reject more applications than before. The membership problem largely solved itself as membership leveled off to about 4000 (including non resident members) in the 1970s, with applications for membership roughly balancing those resigning or dropped from the rolls for non payment of dues or delinquent house bills.

In 1981, a major renovation of the building was undertaken, including a new kitchen, but the improvements were mainly in quality, rather than for expansion.