His departure came two days after they were held to a 1-1 draw in the FA Cup by non-league side Concord Rangers.
"I've achieved loads at the club and it will always have a special place in my heart but I need a fresh challenge now," Cox told BBC Radio Nottingham.
"We've had three amazing seasons and achieved every goal we set out to do.
"We have put some great infrastructure in place at the football club. We have the four teams through to the youth team. I feel now is the time to move on and take on another challenge."
Cox led the Stags to promotion back into the Football League in 2013 after a five-year absence.
They finished 11th in League Two last season, but have not won for nine games in all competitions and are currently 19th in the table.
Cox also paid tribute to chairman John Radford, saying: "He has been fabulous with me. He has been a pleasure to work with and I've enjoyed every minute."
He added: "This is probably why I have stayed as long as I have. There is no secret there have been one or two approaches for myself but I stayed loyal to the football club and to the chairman."
In a club statement, Radford said: "Paul and I have regrettably reached the conclusion that it's in the best interests of Mansfield Town Football Club if he stepped down from his current role with immediate effect."
The club have put assistant manager Adam Murray in temporary charge for Saturday's home League Two game against Plymouth.
Cox was appointed Stags boss in the summer of 2011 after Mansfield had finished 12th in the Conference, and led them to promotion in his second year in charge.