The equaliser came with less than 15 minutes remaining on the clock as Craig Johnstone converted a cross from Scott Durie cross.
Robert Sloan sealed the comeback as he scored a left-footed free-kick from the edge of the box.
East Fife: Michael Brown, Durie, Young, Smart, Deland (Sloan 75), Linn, Murdoch, Muir, Johnstone (Ovenstone 88), Hislop, Byrne (Crawford 63). Subs Not Used: Baillie, Cargill.
Booked: Byrne.
Goals: Johnstone 76, Sloan 82.
Peterhead: Jarvie, Donald, Stuart Smith, MacDonald, Moore, McVitie, Emslie, Anderson (Gethans 87), Sharp, Wyness, Bavidge. Subs Not Used: Bateman, Clark, Strachan, Scott Ross. Booked: MacDonald, Emslie, Anderson.
Goals: Bavidge 43.
Att: 565
Source: BBC Sport
Source: BBC Sport