Captain Igor Akinfeev saved two penalties as Russian Federation triumphed 4-3 in a dramatic shootout in Moscow to reach a first quarter-final since the Soviet era, in 1970.
Spain came close to grabbing a victor five minutes from time as a low strike from Iniesta was pushed away by Akinfeev, with the follow up shot dribbling across the face of goal with nobody on hand to apply the finish touch.
Once Spain took the lead in the 11th minute, however, Sergei Ignashevich turning a Marco Asensio free-kick into his own net under heavy pressure from Sergio Ramos, it was hard to see a way back into the game for the hosts.
Spain dominated possession but were unable to make it count and were punished when Russian Federation were awarded a penalty.
Forward Artem Dzyuba buried it for his third goal of the tournament to awaken the subdued crowd.
Iniesta said he had no complaints about being left on the bench by coach Fernando Hierro, before being brought on during the second half.
Meanwhile, Mark Geiger has been assigned to referee Tuesday's World Cup last-16 match between England and Colombia, four years after he became the first American to officiate in the knockout stage of soccer's showcase.
Despite 75 percent possession, Spain's famed tiki-taka passing game rarely carried them into the box and they had to wait until the 85 minute to test Akinfeev with a low drive from substitute Andres Iniesta.
But for all the talk of tactics and generational change, Spain's 2018 World Cup will be remembered for a simple act of self-sabotage.
Andres Iniesta confirms retirement from Spain squad after World Cup exit
Spain dominated possession but were unable to make it count and were punished when Russian Federation were awarded a penalty. But Schmeichel made a smart save to keep the 1-1 stalemate intact through two hours of at times pedestrian football.
For the second time Russian Federation advance to the World Cup quarter-finals.
But Stanislav Cherchesov's Russian Federation, gathering strength from its fans in the capital city, ensured a gladiatorial battle to wrest an unbelievable win against one of the tournament favourites Spain.
"In 2008, '10, '12, well, we had the players we had, and we played at a level, with a style had nobody had done before, " said Hierro, who took over as coach when Julen Lopetegui was sacked two days before Spain's opener.
More than 74 per cent possession, an astonishing 1,137 passes and utter territorial domination, yet the only statistic that counted in the end for Spain was the two penalties they missed after they strangled themselves out of the World Cup.
Hierro's choice was risky given his lack of coaching experience - just one season in charge of second division Oviedo - but he was reluctant to reflect on Spain's chaotic start to the tournament.
Spain was more urgent in extra time after being too passive for 90 minutes. We don't like this type of structure with three central defenders but they trusted me and I spoke with every player individually. The 34-year-old Barcelona midfielder came off the bench and nearly won the game with an 85th-minute shot well saved by Akinfeev.
Spain join Germany, Argentina and Portugal as the fancied teams eliminated.
Before Russia, South Korea were the last World Cup hosts to send Spain packing, following a 5-3 shootout victory in 2002. "We have our personality, and this is a good thing". "We've got to cut down on the defensive mistakes", said Carvajal.