The English champions, who were beaten 40-26 in France, raced into an early lead through Huw Jones.
Alex Dombrandt, Joe Marchant and Louis Lynagh also crossed in the first half.
Yvan Reilhac and Julien Tisseron scored for Montpellier and Smith failed to convert Lynagh's second try as Quins bowed out despite a 33-20 win.
It was a top effort by the Quins, who with a raucous home support backing them, raced into a early lead with the hosts going into the break with an 11 point lead.
The second half was a lot less open with Montpellier pegging back Harlequins at every opportunity.
Harlequins were camped on the Montpellier line for a long spell in the second half with multiple scrum resets eventually resulting in a Louis Lynagh second try.
It was all down to Marcus Smith, who is the England starting fly half, to nail a relatively simple conversion. However, he hooked it wide breaking the hearts of the home supporters and resulting in Montpellier reaching the last eight of the competition.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here