Another big issue is the loss of marshland. Everyone wants a house on the water, filling in marshland and building bulkheads has taken away the vast majority of marshland and tributary access. These are the breeding grounds of a lot of marine life. They either breed there, or rely on a food scource that does.
For example, herring. There are so many species that rely heavily on the abundance and migrations of herring, that if they were to be taken out of the oceans ecosystem. Most every other species would die off, either directly or indirectly.
Crabs and bivalves (clams...) are the warning species, first to succumb, their numbers in the last few years have been in steady decline. There was always years of feast and famine, and numbers fluctuate, but the sustained periods of drop are becoming much more severe, and longer lasting.
Most of the species are not in jeapordy, they will just not be present in certain areas, typically due to pollution. If the water returns to habitable, so will the marine life return. Here during the '70s and '80s stiped bass was all but non existant. They were protected and illegal to harvest for a while. (I cant say why, but most evidence points to elevated levels of mercury and several other toxins at the time that were mostly regulated down. Now we have a world class striped bass fishery year round.