Living in NYC, I've come to see the rivers as being more psychological barriers than physical ones, at least in the time of modern transit. It's not hard for anyone in the 4 main boroughs of NYC to get around, and provided one isn't in the outer bits of any of them (maybe Jamaica in Queens, or Coney Island in Brooklyn), the whole city is pretty accessible. The rivers provide a minor hassle and a line in the sand, and that's enough for us to use as a start for larger groupings of neighbourhoods. It perpetuates as people choose where to live or where to place businesses on these perceptions. Maybe sometimes a bit like race, but with sharper differences.
The fascinating thing is how much stronger the perceived and created differences are than is justified given the ease of communication; Manhattanites generally don't go into other Boroughs without a lot of arm-twisting, people don't generally like to move from Queens to Brooklyn or vice-versa despite similarities of many communities, and so on. There's little good reason not to build tech businesses in Brooklyn or Queens, but only recently have select parts of those Boroughs seen significant growth of those industries.
I've seen parallels of this in other cities I've lived in, although generally without the level of connectivity that NYC has. My constant thought experiment is to wonder, if the rivers were to be erased, how long would the current barriers exist? Would there remain a scar like the path of the Berlin Wall, years later?