Inter started life under new coach Stefano Pioli with a credible 2-2 draw with city rivals AC Milan last week.
And, despite a few shaky moments on Monday, the former Lazio handler won more plaudits for another solid display that closed the gap on leaders Juventus to 12 points.
Inter, rocked 4-1 at home by Fiorentina in the corresponding fixture last season, were 3-0 up after a series of defensive errors allowed Marcelo Brozovic, Antonio Candreva and Icardi to beat Ciprian Tatarusanu before the 20-minute mark.
Despite reducing arrears before the interval thanks to Nikola Kalinic, hopes of a fightback dipped when Fiorentina skipper Gonzalo Rodriguez was shown a straight red card for an arm-up foul on Icardi.
But Inter were shook up when Josip Ilicic struck on the hour to set up a nervous finale, and Icardi told Sky Sport: "We went off the boil in the second half ... and that's something we have to manage better.
"I don't know if we're afraid to win, but we have to kill off games better when we get in front."
After an Ilicic free kick and a tame Kalinic strike kept Handanovic on his toes, Kalinic reduced arrears in style after holding Danilo D'Ambrosio at bay to beat the Inter 'keeper down low on 37 minutes.
It was his sixth goal of the campaign, all of which have come away from home, but when Rodriguez saw red for stopping an Icardi offensive with his forearm Fiorentina's hopes dimmed.
Inter could have gone in with a 4-1 half-time lead, but Tatarusanu produced a great stop at his far post to thwart Ever Banega's well-struck free kick.
Fiorentina coach Paulo Sousa deployed a 3-4-2 designed to optimise their attacking chances after the restart, and an Ilicic drive on 52 minutes tested Handanovic.
But Inter threatened through Banega, Icardi and then Perisic, who hit the post, in quick succession before Candreva forced Tatarusanu into a two-handed save.
Those misses returned to haunt Inter when Ilicic was allowed space on the left to beat Handanovic with a firm angled strike at his far post on the hour.
Pioli replaced the creative Banega with defensive midfielder Felipe Melo, who earned a caution minutes later for a foul on Federico Chiesa after he had replaced Bernardeschi.
Mario should have put the result beyond reach eight minutes from the end, only to fire over from three metres out on a one-on-one with Tatarusanu.
Thankfully for Inter's fans, Fiorentina's three-man defence capitulated on a late counter that saw Icardi fire past Tatarusanu after the 'keeper had spilled a Perisic drive.
Earlier, Gregoire Defrel came off the bench to hit a sensational late leveller as Sassuolo dented Napoli's Serie A title hopes in a 1-1 draw at the San Paolo.
A dominant Napoli looked on their way to a 1-0 win that would have moved them to within six points of Juventus after Lorenzo Insigne put the hosts ahead in the 42nd minute with a sensational strike.
But Sassuolo, who became the first Serie A side this season to field an all-Italian starting XI, had Frenchman Defrel to thank for a share of the spoils when he came off the bench to beat Pepe Reina with an equally impressive, 82nd-minute volley.
It left Napoli seventh, eight points off the pace.
Although Napoli coach Sarri admitted the absence of Polish striker Arkaduisz Milik, ruled out till January with a knee ligament injury, has limited their attacking options, he said the real problem was a lack of finishing.
"We're getting into the penalty area easily, but at 1-0 up I saw a few too many fancy moves and that angers me because it shows we're not focused on killing off games."