East Fife manager Jim Moffat was furious: "It was daylight robbery," he said. "But we've got to be more ruthless and take our chances. We dominated the game but still lost."
East Fife started the sharper side with Paul Hampshire and John Martin both causing problems for an Arbroath defence which was stuttering and looking jaded.
The Fifers, who hadn't scored in their three previous competitive games this season, weren't able to take advantage, however, and as the first half wore on Arbroath began to come more into the game.
Greg Miller twice created problems for the visitors, firstly with a dangerous cross which John Dodds did well to palm away, then with a 15-yard shot which was saved.
Half-time was reached, however, with neither side looking capable of getting on the score-sheet.
Seven minutes after the restart Marco Pelosi set up a great chance for Martin with a defence-splitting pass, but the East Fife striker shot weakly past the post with only Neil Inglis to beat.
Arbroath came storming back, and Alan Brazil was unlucky when his header hit the post.
Jamie Bishop then struck the crossbar with another powerful header which had Dodds beaten.
The home side's pressure finally paid off after 65 minutes when Hugh Davidson hooked home a Marc McCulloch cross at the far post.
Stein came close to adding a second after 83 minutes but his shot went narrowly wide.
Arbroath manager Harry Cairney was happy with the result: "We didn't play well but at least we got the three points. That's four points out of six so far so. It''s a solid if not spectacular start to the season. If we continue to win at home and draw away we'll do okay."