External AppSettings File NOT merging with web.config

In my app, I have a web.config file with an appSettings section. The appSettings section contains a number of keys that the app uses. The appSettings section also contains the file="AppSettings.config" attribute. The AppSettings.config file then contains a subset of the values from the main web.config. The idea is for the web.config to contain all the base/default settings, and then to provide overrides of the defaults in the AppSettings.config file.