Stealing clothes is the name of a kind of weed. It is said that the fruits of that kind of grass are prickly and often stick to the fruit when visitors are not paying attention.
The clothes are the clothes of the visitor, so they are called stolen clothes.