The name "Fitler Square" is also used to describe the neighborhood surrounding the square, bounded roughly by 21st Street on the east, the Schuylkill River on the west, Locust Street on the north, and South Street on the south. To the east of this neighborhood is the Rittenhouse Square neighborhood; to the west is the neighborhood of University City, home to the University of Pennsylvania and Drexel University; to the south is Southwest Center City (also known as "South of South" or the "Graduate Hospital area"). The portion of Center City surrounding Fitler Square and nearby Rittenhouse Square is sometimes referred to as "Rit-Fit" after the two parks.

Before the 1950s the neighborhood was a prime example of the urban blight that had overcome much the city. The park itself was described as a "mudhole inhabited by drunks and empty bottles".[1] In the mid 1950s, The Center City Residents' Association successfully petitioned Mayor Clark to do something about the decline of the neighborhood. Working together, they freed up mortgage money for the construction of new homes and rehabilitation of the neighborhood.

Also threatening the neighborhood was the proposed Crosstown Expressway. The threat of its construction, which would demolish much of the neighborhood, was enough to reduce property values and add to the neighborhoods blight. The Residents' Association was successful in changing the plans and following years saw drastic neighborhood improvements this.

Today the neighborhood is mostly residential and composed of single-family homes, and it is within a short walk of the commercial areas of Center City. On the television show Philly, Kim Delaney's character "Kathleen" was portrayed as living in a small apartment building overlooking the park.