I'm on the fence on whether or not it's MAME or the monitor. I know that correctly modifying my nvidia drivers will get me the output I want on a CRT but the Wii Virtual Console games that are straight emulations are all scaled to 480p and look perfect on my 50" LCD TV. That's NES, SNES, Genesis, PC Engine, and NEOGEO games that all look to be scaled perfectly to 480p, have no ugly filtered look and the pixels are perfectly square. On the other hand, emulated games I've downloded from Xbox Live are all windowed, usually have filtering, and the aspect looks like crap because they let you stretch it vertically and horizontally out of proportion. I'm not sure how Nintendo does it but their video emulation is pretty darn clean.
I tried to mess with my nvidia driver but my CRT (I have an LCD as primay and CRT as secondary) keeps saying 'out of sync'. In older nvidia drivers I could get it to display 384×224 easily but setting the frequency in the new nv_disp.inf file has changed and I can't get it right so I get the sync error. :(
> Thats not how lcd's work. They always and only do native resolution. Best
> output is to have mame do the stretching and have the output be the lcd's native
> resolution. Why? otherwise you have some software stretching, then some
> hardware stretching... Stretching a stretched image sucks.
> Anyway, it isn't mame... its the monitor