Natalie Anderson

Natalie Anderson’s sudden departure fromThe Challenge: Double Agentsshocked viewers andher castmatesalike. TheSurvivorwinnerandChallengerookie, 34, already won her gold skull, meaning she secured a spot in the final. So when hostT.J. Lavinannounced that she’d been compromised and would be leaving the game,Double Agentslost one of its strongest competitors.

“Wes [Bergmann]reached out because he heard I left unexpectedly and he was like, ‘I’m just letting you know, if you stayed, you would have won,’ " Anderson tells PEOPLE of her firstDouble Agentspartner. “I was like, ‘Dude, I know.'”

On the show, Anderson simply cited “a personal matter” as her reason for exiting. Now, Anderson feels comfortable sharing that she left the show because she found out she was unexpectedly pregnant.

“It would be really unfair to say I left because it was difficult. My fans know me. I embrace difficult. They’re not going to buy, ‘The house was too stressful for me,'” Anderson says ofThe Challenge. “I want to own my story.”

Anderson and her boyfriend of over one year Devin Perez hadn’t been trying to conceive — production only asked her to take a pregnancy test after she told her twin sister andAmazing Racepartner Nadiya over FaceTime that she had missed her period. Anderson didn’t think anything of her missed period though because of the stressful circumstances.

Never miss a story — sign up forPEOPLE’s free weekly newsletterto get the biggest news of the week delivered to your inbox every Friday.

Still, production wanted to “take the precautionary steps,” Anderson says, “and so they asked me to take a pregnancy test. I literally did not think it was going to be positive.”

When it turned out that way, Anderson knew she couldn’t continue to compete, as much as she wanted to. “I literally said, ‘I think I can do this pregnant. Let me stay and compete.’ And production just looked at me like, ‘This girl is crazy,'” she recalls.

“I know my body and I would’ve put myself in that. If I could have signed some waiver and competed pregnant, trust me, I would have done it. But there was really no way from a liability perspective they would ever allow me to compete pregnant. I knew that as soon as I found out,” says Anderson.

Anderson’s pregnancy news came as she found herself in a good spot in the game, especially for a newcomer.

“It seemed like this really unfortunate thing at a really bad time,” the formerAmazing Racecontestant says. “I was sure that none of the girls would try and challenge me for my gold skull. I was sitting pretty for the final. The check was already written for me. I was ready to win. I couldn’t feel happy about it because I was out there for one reason and that was winning.”

Reluctantly, Anderson left Iceland and once she returned to the U.S., the reality star learned she was eight weeks pregnant. “Watching back the season, I look at myself and I’m so proud of what my body was able to do pregnant,” Anderson says. “It’s crazy. I never once got sick. I never had any morning sickness. I was just stuck in this crazy house sleeping on a bunk bed with a bunch of lunatics.”

Anderson soon started looking forward to motherhood. She adjusted her exercise routine accordingly and anticipated starting a family with her boyfriend.

“I said, ‘You know what, maybe I didn’t get to win half a million dollars, but everything happens for a reason,'” she continues. “My boyfriend is already a father and he’s somebody I could see myself having a family with. We were like, ‘let’s do this.’ I was ready for this new chapter.”

One week after coming home in September, Anderson’s story changed once more when she suffered amiscarriage. “I had to digest losing the pregnancy and being like, ‘Now I have noChallengewin under my belt and I don’t have this new journey I was looking forward to,'” she says. “It was really difficult.”

Anderson relied on working out for a sense of normalcy and leaned on her boyfriend for support. “I had to really go through the healing process with myself and my boyfriend,” she says. “I focused on what always makes me feel good, which is the routine of training.”

She also found strength in having missed out on the$2 million prize inSurvivor: Winners at War.

“Losing $2 million, it seemed like the hardest thing I would ever have to do until this happened to me,” Anderson says. “And this was even harder because the $2 million, I had to take responsibility for it playing out the way it did. I never thought I would lose something that would be as hard as losing $2 million. But this was even harder.”

Natalie Anderson onSurvivor: Winners at War.CBS via Getty

Natalie Anderson

Though none of herChallengecastmates knew about her situation at the time, Anderson has since confided inTori Dealand Aneesa Ferreira, the latter of whom she says “really helped me embrace this and use this in a way that can be positive.”

Now Anderson feels “excited to move forward and figure out the next steps in my life.”

“I really did do the work necessary to get to this space that I’m in now,” she says. “After any kind of loss, it’s up to us to create that safe environment to come back to a healthy head space. And if I can help anybody, that was my goal. I want anybody who is going through something hard to remember: the last thing you need to do is be hard on yourself. Be kind to yourself and embrace all the difficult stuff that happens in a way that you can grow and become a stronger person on the other side.”

Anderson hopes to return toThe Challengeand jokes that after this whirlwind experience, “I feel like the universe owes me $2 million fromSurvivorand half a million fromThe Challenge.”

source: people.com