Microsoft is experimenting with integrating Copilot AI functionality across various platforms. However, this time the company is doing something that many users might not be aware of. The Gaming Copilot AI is being trained by default using users' screengrabs, performing OCR on the screenshots, and sending the extracted text back to Microsoft servers to train its large language models—all without users being informed of this process. A Resetera forum member, "RedbullCola," discovered this through network traffic analysis, noticing that the Gaming Copilot AI app was transmitting data to Microsoft servers without his knowledge. To make matters worse, this occurred while he was testing a game under an NDA. This situation raises multiple security concerns, as it could potentially breach the NDA agreement between the user and the game company, risking the leakage of text from the unreleased game.
We verified this ourselves and can confirm that the feature is indeed enabled by default. The only option you can opt-in for is model training on your voice chats, which is thankfully disabled by default. To check whether your setting for training text models from your screenshots is being used, go to Game Bar, open Gaming Copilot, head to Settings, and click on Privacy. There, you can uncheck the training slider to prevent your screen records from being sent to Microsoft for LLM training. For example, under the GDPR, using EU users' personal data to train AI requires a transparent notice. Automatically enabling screenshot collection for model training without an appropriate legal basis or explicit informed consent could breach the law, and invoke a lawsuit from EU's governing body.
While setting up the Gaming Copilot, there were no options to inform a user whether this feature would be enabled by default. There should at least be a screen that prompts users before enrolling them in a data collection program aimed at assisting gamers in the future. This is particularly important for those working in the game industry, as this feature could potentially leak game content that is protected by years of work and non-disclosure agreements. We are awaiting to see if Microsoft will make a statement regarding collecting text from gamer's screenshots, or the case will remain in point.