Microsoft Copilot Sends Users' Photos and Sexual Prompts to Human Contractors Unfiltered

People who upload a photo to Microsoft Copilot and ask it to change something tend to assume a machine is the only thing looking. For some of those requests, a stranger is looking too. Human contractors hired to improve Copilot are reviewing users' image-editing prompts along with the photos attached to them, according to internal documents obtained by 404 Media.

The work involves hundreds of people. Their task is to score whether the image Copilot generated actually delivered what the user asked for. Faces in the uploaded photos are never blurred or censored.

Grading the results

What users ask for is often sexual. One example of the judgment call contractors face is deciding whether breasts enlarged by the AI came out big enough to satisfy the prompt. Microsoft's guidance for calls like that is to use "intuition."

The contractors describe a steady flood of sexual and nonconsensual requests. They have seen upskirt photos, real women placed into sexual positions, and a request to shorten a real woman's skirt. They have seen children's cartoon characters sexualized, and at least one prompt that appeared to be aimed at producing a lewd image of young girls.

It goes beyond sex. One worker said pro-anorexia content made them "recoil." Others encountered imagery of animal sacrifice.

Not moderators, and not warned

None of these workers are there to police what Copilot users do. The job is quality scoring, which means rating how well the model followed instructions, not deciding whether the instructions should have been followed at all. Many said they got no warning before disturbing material landed in front of them.

That leaves them in a strange position: evaluating how faithfully an AI carried out a request to undress or reposition a woman who never agreed to any of it. One contractor asked who had decided that generating porn was what Copilot was focused on.

The documents do not show that every Copilot conversation gets a human reader, and the review described covers image-editing requests. But the gap between expectation and reality is wide. As Futurism notes, the average user assumes chatbot sessions are not read by people at all, except perhaps for moderation.

An industry habit

Microsoft is not alone in this. OpenAI has used a similar practice of having humans review prompts, and its handling of user uploads has had its own failures, including the episode in which OpenAI agents leaked 53 ChatGPT user images onto third-party hosts.

The people in those uploaded photos are the ones with the least say. A woman whose picture is fed into Copilot by someone else did not consent to the edit, and she did not consent to a contractor studying her uncensored face to decide whether the result was good enough.