Do you feel that learning Portuguese ruins your ability to speak Spanish?

I have been studying Spanish on my own now for about four years and it is by far the language that I use most often. However, I am very interested in South American cultures, including that of Brazil, and I would like to learn Portuguese. However, I have heard that learning Portuguese can ruin your ability to speak Spanish beyond repair, like that you will ever be confused by grammar and pronunciation. Has anyone here attempted this or experienced it? Thank you!