The privacy questions your school should be asking about AI tools — before the next staff meeting

Why this conversation matters now

New Zealand schools are adopting AI tools faster than policy is being written. Teachers are already using ChatGPT, Copilot, Gemini, and a growing list of education-specific AI platforms — often before anyone has had a proper conversation about what data those tools collect, where it goes, and who owns it.

The Privacy Act 2020 applies to schools. The fact that an AI tool is free doesn’t exempt it from privacy obligations. And the Office of the Privacy Commissioner has been clear: schools cannot simply transfer risk to a vendor by clicking “I agree” on a terms-of-service page.

The questions your school should be asking before using any AI tool

1. What data does this tool collect about students or staff?

Most free AI tools are not designed for children and collect broad usage data as part of their commercial model. If students are using a tool, check whether it is compliant with the Children’s Online Privacy Protection Act (COPPA) or equivalent NZ obligations. For students under 16, parental consent may be required.

2. Where does that data actually go?

AI companies training on user data is standard practice in free tiers. Before students type anything into a free AI tool, understand whether that content could be used to train future models — and whether that content could theoretically be accessed by humans at the company.

3. Who owns what students submit?

Some AI platforms claim broad rights over submitted content. If a student submits their writing to a free tool to get feedback, that content may become part of a training dataset. Read the privacy policy — or ask the vendor directly.

4. What happens when someone asks to have data deleted?

Under the Privacy Act, individuals have the right to request deletion of their personal information. If an AI tool is holding data about students, can you actually get it deleted if needed? Get this in writing.

5. Does this tool meet the minimum standard for NZ government IT procurement?

If your school is state or state-integrated, the NZ Government ICT Security Standards may apply to tools used for administrative or student management functions. AI tools used for curriculum support occupy a greyer area — but it’s worth checking.

What to do with this information

Share this with your school’s leadership team or IT coordinator. The goal is not to block AI tools — it’s to make sure the tools being used have been checked. [1]

If you are evaluating a specific AI tool, the Office of the Privacy Commissioner has a simple privacy impact assessment template that schools can use. It takes less than an hour to complete for a new tool.

---

What to take away from this

• The Privacy Act 2020 applies to schools using AI tools — free tools are not exempt from privacy obligations

• Before any student uses an AI tool, confirm what data it collects, where it goes, and who can access it — and get answers in writing from the vendor

• AI companies training on user data is standard practice in free tiers — assume anything students type could be used to train future models

• Tools that don’t have a clear data deletion process under the Privacy Act should not be used with student data until that process is documented

• State and state-integrated schools should check whether AI tools meet NZ Government ICT Security Standards before adopting them for administrative or student management functions

• When in doubt: before adopting any AI tool with students or staff, ask the vendor in writing: “What data does this collect, where does it go, and can we have it deleted?” If the answer is unclear, don’t use it until it is. For a practical starting point on passwords and authentication, see our Passwords, Passphrases and MFA guide.

Sources and references

[1] New Zealand. Privacy Commissioner. (n.d.). Privacy impact assessment. https://www.privacy.org.nz/responsibilities/privacy-impact-assessments/

[2] New Zealand Government Chief Digital Officer. (n.d.). New Zealand information security manual. https://www.nzism.gcsb.govt.nz/