You know this pain when some random keyboard layout pops out in languages and spoils your day. So here are some tricks on how to fix it.
Use Regedit to navigate to the following registry keys, where you will find the list of keyboards preloaded at boot.
HKEY_USERS\.DEFAULT\Control Panel\International\User Profile
HKEY_USERS\.DEFAULT\Control Panel\International\User Profile System Backup
Find the keyboard identifier among the list of Keyboard Identifiers
Delete the key.
As an additional step, change permissions to prevent the system from adding new keys to that registry branch. For this, remove inheritance and delete the system principal.
You can also forbid remote layout by adding this key:
Disable Auto Detect Keyboard Layout: