According to my limited knowledge it's a christian trait where they cleanse the soul. In my old church it was believed that the soul is born a sinner and this is to cleanse them of all sins so that they may start afresh. But I see other religions practice it too.
It has become a traditional thing too. I was baptised and back then my parents weren't very religious. It seems to be more popular in England than South Africa though. Only church-going christians baptise their children here.
(I'm English, moved over to SA when I was a child

)