Originally Posted By MetroBrah⏩

Swarthy comes from 16th century English swarty and swart which mean "black" (strictly the color). This really didn't mean much more than "dark-skinned". In point of fact, it was never attached to a particular group as it was never intended to be a specifically racial term. Generally through my own learned context and from what I've read, I've always gotten the impression of a dark-skinned person because that's the way it was used. With yet more context, it came from the language of peoples who were generally light-skinned. Therefor, anyone with skin darker than their's could be considered swarty or with it's more modern form, swarthy.