She should be a frog when you see her in QFG4. Am I missing something?

I don't think it's explicitly ever mentioned how she regained her form. Probably knowing the spell to begin with gave her insight on how to break it. Her page on the Quest for Glory Omnipedia has a summary of the information known about her: https://questforglory.wikia.com/wiki/Baba_Yaga

I'm going totally hypothetical here, but at the end of Qfg1 Baba is able to utter an incantation while in frog form. Given her ability to still practice the arcane arts I believe it's possible she fixed herself. She also could have had Erasmus reverse the curse. They did play cards together. Finally don't forget about the dispel potion.

I was mostly under the impression that she pretty much just worked up enough magical power to undo the curse.

It just took some time since she was a frog and probably needed to sleep off a lot of adrenaline from being changed into a frog in the first place.

Or something along those lines.