Once a mixed-race fishing community, the island is now empty, showing the gap between the state’s history and what it professes to be.