I’ve been using GOG for various older games for several years now. Finding the location to switch a game from full screen to windowed was a challenge initially. I run various distros of Linux and normally install the games under ~/Programs/<game name>. If you want to change settings, there should be a file in that directory named:

dosboxXXXX.conf

In my case for Master of Orion 2, the file is named dosboxMOO2.conf. There are other conf/settings files but for my purposes this is the one I normally change.

For windowed mode instead of full screen, set fullscreen to false. Also, go ahead and make an entry for windowresolution such as 800×600. One other setting I often change is setting autolock to off (this prevents the mouse from becoming trapped when you click in the DosBox window). Some people like the autolock setting, especially handy for some games, but with dual displays and other work going on it’s a pain because I can never remember the key shortcut to unlock the mouse.