Among the new features was an increase in recommended video content, in a similar style to rival app TikTok.
The display and feeds were criticised as people felt they were not seeing photos from friends and family as much.
Meta, Instagram's parent company, said it wanted to "take the time" to get the changes right.
The revamp of Instagram towards more video content is believed to have been ignited by the soaring popularity of competitor TikTok - where users post and watch more video as opposed to static photographs.
According to data from digital analytics company Sensor Tower, the TikTok app has now been downloaded more than three billion times around the world - the first app not owned by Meta to reach this landmark.
Instagram boss Adam Mosseri told The Verge that the test version of the revamped app would be phased out within weeks.
"I'm glad we took a risk - if we're not failing every once in a while, we're not thinking big enough or bold enough," he said.
"But we definitely need to take a big step back and regroup. [When] we've learned a lot, then we come back with some sort of new idea or iteration. So we're going to work through that."
Mr Mosseri had previously posted a video explaining the move towards video, saying that full-screen videos would be promoted over photographs.
But after some backlash, he followed it up with a series of tweets saying he wanted to "continue supporting photos".
In response to Mr Mosseri's Twitter video, US model Chrissy Teigen said that users "don't wanna make videos" and are seeing lower engagement for photos.
Reality TV stars Kim Kardashian and Kylie Jenner also shared an online "petition" to "Make Instagram Instagram again" on their stories.
Toni Tone, author and content creator, posted how frustrated she was with the changes and how Instagram was so much better when she received content from the people she chose to follow.
A Meta spokesperson told the BBC: "Based on our findings and community feedback, we're pausing the full-screen test on Instagram so we can explore other options, and we're temporarily decreasing the number of recommendations you see in your feed so we can improve the quality of your experience.
"We recognise that changes to the app can be an adjustment, and while we believe that Instagram needs to evolve as the world changes, we want to take the time to make sure we get this right."