The New York Society for Acoustic Ecology have created an on-line soundmap of the city. The map includes recordings from a range of locations: Fulton Street Fish Market, Ludlow Street, Broadway from 26th to 28th Streets, Staten Island, Carroll Park in Brooklyn... The map could do with some further development as it is not yet very detailed, but it will be worth seeing how it progresses (it launched in August 2005). This soundmap is part of a wider group of related projects – see: NYSoundmap.
Postscript September 2015
This looked cutting edge a decade ago... However, only the last of those links still works. The New York site includes a page with links out to other soundmaps but sadly none of the original links work. Here is one that does seem to exist in a different guise: Soundmapping of Cologne.