Well, it is a neutral trait. Ultimately it just refers to sticking by a person or group when it's not in your best interest. Historically there have been a lot of examples of bad loyalty (tribalism, nationalism, racism, etc.), so I can see your point.
Then again, magic isn't a "good" element either...
Laughter is pretty positive. The Joker is an extremely unique subversion of laughter as someone who takes extreme joy in cruelty and criminality. Which is why he's one of the most famous and popular comic book villains of all time.
And being generally steadfast in convictions is usually described as stubbornness. The defining characteristic of loyalty is that the conviction is in a group or person (besides yourself) that leads you to stick with them against your personal interests.
The issue is that it also means sticking with them against anybody ELSE'S personal interests as well. Such as, say, an entire population. It does indeed depend on the character.