Apple service manual agrees with you and technical specifications list 2GB (x2) is maximum supported size. Either Apple stock part number MB413G/B or MC322G/A.
Yeah, there are some real weird quirks with some of the iMacs of that era. There is one from 2007 that will take a maximum of 2GB of RAM - either as one single 2GB stick, or two 1GB sticks, but for whatever reason it will not operate with two 2GB sticks no matter what you do. EveryMac might say it will work, but it just won't.
·
·