Originally named Woolloomooloo Hill (as well as Paddys Point, Kurrajeen or Currajeen, and Yarrandabbi), Potts Point was later named after Joseph Hyde Potts, an accountant who later became the Secretary of the Bank of New South Wales (now Westpac), and who had purchased 6.5 acres from Judge John Wylde (commemorated by Wylde Street, Potts Point) and named his property Potts Point.
In 1822 Judge Wylde had been given a land grant of 11 acres at the end of the point in what is present-day Potts Point. In 1831, other grants were made, south of Wyldes grant, to Alexander Baxter, John Busby, H. C. Douglas, Edward Hallen, A. B. Spark and Justice John Stephens. John Busby (1800-71) was the engineer who constructed the scheme that piped water from Centennial Park to Hyde Park, thus ensuring a supply of fresh water for the young colony.
In the early 19th century a number of stately mansions were constructed in Potts Point. These included:
--- Tusculum (1831-35), which was designed by architect John Verge (1771-1861)
--- Rockwall (1831-37), which was also designed by John Verge and built on a grant of land made to John Busby
--- Tarana (c. 1889)
In the years just after World War I, a number of apartment buildings were constructed in Potts Point, to accommodate the numerous single and widowed women living alone after the War.
Many Art Deco buildings were built, particularly in Macleay Street.
The Minerva (1938-39) in Orwell Street was a magnificent Art Deco building which first operated as a theater and then later as a cinema and now as film studios.
In the 1930s, 1940s and 1950s Potts Point (and particularly the area around Darlinghurst Road, commonly known as Kings Cross) was home for many bohemians and artists. In the 1960s American service personnel would regularly visit Potts Point and Kings Cross on "R and R" (rest and recreation) leave from the Vietnam War.
Potts Point (and its alter ego Kings Cross) has long been a tourist mecca with many hotels. However, in the years after the 2000 Sydney Olympics, most of the hotels were converted to strata units. Some hotels were demolished and blocks of strata units constructed in their place. Potts Point had been discovered by the yuppies (rich young urban professionals), who appreciated the convenient location of Potts Point and its lively inner city ambience. At the same time many traditional, less affluent residents were forced out by higher rents and some beloved businesses (such as Barons and New York Café) closed down.
Potts Point is actually no longer a point. In 1942, Garden Island (just off the end of Potts Point) was joined to Potts Point by the construction of the Captain Cook Dock by the Royal Australian Navy. Garden Island has been used as a naval depot ever since 1866 and the whole area is now a defence complex with restricted access.