They're the epitome of wickedness. The mere fact that they gather frequently in the local tavern is enough for the Pardoner to link them to a host of sins, including lechery, gluttony, drunkenness, and blasphemous oath swearing.
In the Pardoner's Tale sin runs much deeper than that: it's the weakness in your armor that allows even worse sins to dig their way into your soul. So when the Rioters discover those eight bushels of gold, the manner of "lyvynge" in which they've already engaged allows the devil to easily convince them to kill each other rather than share their gold.