Most of my Swedish language students know the theoretical difference between “de” (=they) and “dem” (=them). It is when we go into sounding like a Swede that it gets a bit more complicated :).
Most Swedes pronounce the two different words the same way – “dom” – which sometimes lead to confusion and mistakes by Swedes when writing.
Here is an excellent video explaining the difference between “de” and “dem”. Detta med de och dem; Video by Maia Andréasson.