That's odd... the only thing I can think of is to make sure that your party leader (the character in spot 1) is the one carrying the vellum scroll. If that doesn't work, something you could try (make sure you save your game first) is to drop the scroll, then go back to the caves under Sorpigal and meet the wizard again. He should give you another, and then you can try delivering it to Agar.
But to answer the second part of your question, it's totally possible to continue the game without doing this quest, and you can always come back and try to do it later. MM1 doesn't really have a traditional quest line... there are certain things you must do to win the game, but in general it's very open and almost all quests are completely optional. Many can be repeated to get more experience, including this one. Doing this quest will eventually give you clues about how to finish the game, but you can finish without those clues if you figure it out yourself.
Still, it's best if you can get the quest to work. Hopefully giving the vellum scroll to your leader will work. Anyone else have any ideas?