Racism is ever-present in Harlem Shuffle, to the point where characters struggle to envision a world with equal rights. Despite the civil rights protests which are scattered throughout the book and the awareness of inequality in society, characters like Ray take a pessimistic view of the issue of racism. Ray tends to fantasize about a more successful future, one in which he lives in lavish buildings and provides everything his family could desire. However, these reveries are notable in that they never eliminate racism. He is always living in buildings owned by White landlords, and he is always forced to explain himself to White authority figures. Racism is so prevalent that even the most ambitious characters cannot see a future in which they are not marginalized.

In the novel’s depiction of Harlem, racism is a more nuanced issue than simply Black versus White. While the African American characters are oppressed by White society, there are divisions and fractures within the Black community based on skin color. The Dumas Club is renowned for only admitting light-skinned African American men, meaning that Ray is doubtful about his chances of becoming a member. Likewise, the dark color of Ray’s skin is an issue for his in-laws, who hope that his children inherit Elizabeth’s lighter skin.