Microsoft agrees to new AI privacy rules for schools

The debate over AI in schools is moving beyond classroom experimentation and into procurement contracts. Microsoft has agreed with the American Federation of Teachers (AFT) and its New York City affiliate, the United Federation of Teachers (UFT), on ten safety and privacy principles for educational AI. School districts will be able to incorporate the terms into contracts, giving the commitments a stronger enforcement mechanism than a general corporate pledge.

Key points

• No training on student data: Microsoft says it will not use student or educator data to train AI models and will reduce the amount of data collected in the first place.
• Plain-language disclosure: Families should be told how the tools work and how information is handled in language they can understand.
• No AI companions: The agreement prohibits AI companions, a category associated with ongoing, relationship-like interaction with students.
• Human review for high-risk decisions: Important decisions classified as high risk must include human oversight rather than relying entirely on automation.
• Contractual implementation: Beginning in November, districts can add the principles to new or existing agreements without renegotiating the entire contract.

The agreement comes as concern about student-facing AI and screen use is growing. New York City and Los Angeles have both introduced one-year restrictions on AI tools for many students, giving their systems time to consider where these products belong and what safeguards are needed. For Microsoft, putting privacy and safety commitments into school agreements may help address concerns about data use, limited transparency, and automated decisions. It may also make districts less likely to respond with broad restrictions on the company’s education products.

The AFT’s position is more nuanced than simple support for or opposition to AI. Its president has called for fewer screens before third grade and for a ban on student-facing AI before middle school. At the same time, the union has worked with technology companies, including Microsoft, on educator AI training. The new agreement reflects a middle path: schools may use AI, but vendors must accept clearer and more accountable boundaries.

The most important aspect of the deal may be its contractual form. If more districts adopt similar language, school AI procurement could be judged not only on features, price, and deployment speed, but also on data minimization, family disclosure, human supervision, and restrictions on product design. The principles alone will not guarantee protection. Their value will depend on enforcement, district auditing capacity, and whether teachers and families have meaningful ways to question or challenge implementation. Even so, the agreement signals that the next phase of education AI will be as much about governance as it is about product capability.

Source: The Verge AI