French Open champion Halep was also knocked out, losing to Taiwan's Hsieh Su-Wei 3-6, 6-4, 7-5 as the seeds' carnage continued.
She's now the 48th-ranked singles player in the world, and on Monday, she will play in the second week of singles at Wimbledon for the first time in her career.
Del Potro pushed Novak Djokovic all the way in one of Wimbledon's most memorable semi-finals five years ago before his career was almost ruined by wrist problems, but he appears back to his best.
Halep broke back with a pummeling forehand for 3-3 but dropped serve again at 4-4 when she netted at the end of a baseline duel and Hsieh then calmly held serve to take the contest into a decider.
The 138-ranked Gulbis will challenge Kei Nishikori, who won three straight sets to past Australian star Nick Kyrgios.
His next round rival will be Jiri Vesely of Czech Republic.
Rafael Nadal (2), Spain, def. Alex De Minaur, Australia, 6-1, 6-2, 6-4.
Hsieh in the 4th round will meet Dominika Cibulkova, who dispatched No. 15-seeded Elise Mertens 6-2, 6-2.
Bencic, who staged a strong return to competitive play at Wimbledon after being sidelined for several months due to injury, will next face off against former world No. 1 Angelique Kerber of Germany, now world number 10.
"For me, I just try to enjoy the tennis, enjoy the city, enjoy the food. I'll go and watch some football - England, it's coming home or not?" She stayed there for every point. But I am too exhausted. "My muscles are gone", she said.
A rule of thumb, while serving, is to avoid taking much time between your first and second serves. Only one of the top 10 seeds will be in action in Week 2: No. 7 Karolina Pliskova. Seven-time champion Serena Williams, the 25th seed, also remains in contention.
The 20-year-old Tiafoe said he started feeling badly late in the third set after taking the opening two against Karen Khachanov of Russian Federation before fading to a 4-6, 4-6, 7-6 (3), 6-2, 6-1 loss. "Because every day, something odd is happening in the draw".
Halep was leading 5-2 in the final set and had a match point, only to crumble to a 3-6 6-4 7-5 loss on Court One.
In this case, though, Gulbis already has been a major semifinalist - he made it that far at the 2014 French Open, but his ranking slid because of a series of injuries - while the up-and-coming, 21-year-old Zverev has yet to have that sort of breakthrough.
"It's unbelievable. I was 2-5 down in the final set but the crowd pushed me to fight". She wouldn't win another game. "I was extremely negative yesterday and just needed to come out with a very different state of mind and different approach today", he said.
"It was a positive match against a player with a lot of energy", said Nadal, the 2008 and 2010 champion.
Hsieh had 26 winners against 36 unforced errors. "I had match point", Halep said.
Hsieh Su-wei's game is so unorthodox she admits that finding practice partners has proved challenging in the past.