Edit: I suppose I should mention this is Ultima 7. I just checked and Serpent Isle does it as well.

When I start the game up, my monitor goes all black. My monitor gives me a message saying "Out of Range" and some numbers listed as V: and H:. If I alt-tab, Windows appears again and the game shrinks down to a Window and works fine, but is there any way to get it to work in full screen?

Did you launch the game from the Start menu shortcut? There is some special parameters needed which were not getting right if launched from other means.

Other than that, you could modify dosboxULTIMA7.conf and change fullresolution=
Resolutions you might try would be 0, or your desktop screen resolution. Also, 800x600 (or 960x600) could be good options.

Have you tried changing the graphic mode with the setup program?

I've launched from the start menu, both the setup option and the game itself, I get the issue either way. I've tried every option on the graphical setup thing. How would I edit the dosbox config, just it notepad?

Yes.
As GoingDown wrote, it's probably related to the resolution, so try setting it to your monitor native res.

New issue: I opened it and tried to change the full resolution field, but when I go to save it I get an access denied message.

Running Windows 7/Vista? It sounds like file access protection error. You cannot modify files in protected folders (Program Files etc.)

You need to start editor as administrator first, then open the file. (search notepad icon in startmenu, click it with right mouse button, select "run as administrator"