When Atticus speaks of people being "trash," he's referring to the quality of their character.
Atticus talks about how some men carry their resentments into a jury box. Also, he mentions that white men cheat black men but "whenever a white man does that to a black man, no matter who he is, how rich he is, or how fine a family he comes from, that white man is trash.” Therefore, Atticus is describing that man's character and identity. He is not criticizing his manners, appearance or lineage but the essence of who he is.
