Using AI Tools Safely for School and Teaching

AI tools are not going away and they are not automatically unsafe. But they do come with a specific set of privacy and security risks that most people do not think about until something goes wrong.

The useful version of AI safety in a school context is not about banning tools. It is about understanding what you are actually handing over when you paste something into a prompt, upload a document, or sign in with a school account.

This guide covers the practical side of that. It is written for educators and students who want to use AI usefully without creating new problems in the process.

Why AI tool safety needs its own attention in schools

A lot of school technology risk comes from things being used in a way that was not intended. AI tools are particularly prone to this because:

• they feel casual and conversational
• the interface does not warn you about what gets stored or used
• signing in with a school or personal account can look the same on screen
• free tools often have commercial data-use arrangements that are not obvious
• output can feel authoritative even when it is wrong or fabricated

That means the risk is not only about malicious use. It is also about ordinary use becoming risky because the privacy and data implications were not checked first.

What you should think twice about pasting in

This is the practical core of AI tool safety. The question to ask before any upload or prompt is: what would happen if this content ended up somewhere I did not intend?

That means pausing before pasting:

• student names or identifying details
• class materials that include personal information
• feedback or assessment drafts with student work attached
• internal notes, planning documents, or communications
• anything that would normally require consent or careful handling

The Office of the Privacy Commissioner’s Children’s Privacy Project notes that children and young people deserve a higher standard of privacy protection, not a lower one. That applies to AI tools just as it applies to any other platform processing student data (1).

If the content includes information about someone who is not you, treat it as sensitive by default.

Verifying AI outputs before acting on them

AI tools generate plausible-sounding content that can be confidently wrong. That matters especially in a school context where information accuracy has real consequences.

Safer habits include:

• treating AI-generated text as a draft, not a finished fact
• checking claims against a reliable source before relying on them for planning, policy, or student-facing material
• not using AI output as a substitute for reading the underlying policy, guidance, or documentation where it exists
• noting that AI may produce outdated information, invented citations, or confident errors that look legitimate

This does not mean AI is useless. It means the output should be reviewed rather than forwarded, posted, or printed without a check.

Account and sign-in implications

When you sign into an AI tool, you are often creating an account that the provider manages. That account may:

• store prompts and uploads
• retain conversation history
• use submitted content for model training or improvement in free or tiered plans
• share data with third parties as part of the service agreement

Before using a tool with students, understand:

• what the tool’s privacy notice actually says about data retention and training use
• whether students are using personal accounts, school accounts, or a class-managed setup
• whether the tool is designed for or marketed toward under-18 users
• whether a privacy impact assessment has been done for the specific tool being used

The Office of the Privacy Commissioner provides privacy tools for agencies and a Privacy Impact Assessment template that schools can adapt for AI tool evaluation (2)(3).

What educators specifically need to check

If you are introducing or recommending an AI tool for classroom use:

• does your school or institution have a process for evaluating AI tools before they are adopted?
• are students using individual accounts or a shared class account with appropriate privacy settings?
• have you tested the tool’s output for accuracy before assigning it to students?
• are you clear about what data the tool collects and who can see it?
• does the tool retain submitted content for training purposes, and is that acceptable for your context?

If you do not know the answers to those questions, treat the tool as one to use cautiously until the school or IT team has evaluated it properly.

What students specifically need to know

If you are using AI tools for study or classwork:

• the same caution applies to personal information, assessment drafts, and anything you would not want shared
• AI output is not always accurate — do not submit AI-generated content as your own work without understanding what your school or course expects
• the account you use matters: a free personal account may have different data terms than a school-provided account
• if a tool asks you to upload something with your real name, school, or class details attached, think before you do it

For the student-focused safety pathway, start with Digital safety basics for students in New Zealand and continue with Passwords, passphrases, and MFA: a simple guide.

What to do if something feels wrong with an AI tool

Do not ignore the uncomfortable feeling.

1. Stop using the tool and do not paste further content into it.
2. Check the privacy settings and account activity if you have an account there.
3. Do not share AI output that you have not verified or that makes you uncomfortable.
4. Tell a teacher or school support contact if you have shared something that now feels like a mistake.
5. If student data may have been exposed, follow the school’s normal privacy breach process.

If the incident involved a suspicious prompt or account access, use How to Spot Phishing Emails, Scams, and Fake Messages and Passwords, passphrases, and MFA: a simple guide as the follow-on safety guides.

A practical AI safety checklist

Before using an AI tool with school-related content, run through this:

• Have I checked what data this tool collects and where it goes?
• Am I signing in with a school account, personal account, or class account — and do I know the difference in data terms?
• Does the tool retain submitted content for training purposes?
• Am I about to paste student names, assessment content, or personal information?
• Have I verified the AI output for accuracy before using it?
• Is this tool approved or evaluated by my school or course?
• If something goes wrong, do I know who to tell?

If you are still uncertain, slow down. A useful tool used unsafely can create more problems than no tool at all.

Knowledge check

Test your understanding of safe AI tool use in school contexts with these questions. Remember to click on each question to reveal the answer.

Q1: A classmate pastes their full name, school, and the draft of their assessment into a free AI tool to "check their writing." What are the privacy risks of doing this?

Answer: Several concerns apply. The free AI tool may store the submitted content and use it to train future models — meaning the assessment work, including personal identifying information, could appear in responses to other users. The classmate has also handed their personal information to a third-party service without knowing how it will be used or stored. If the tool has data-sharing arrangements with advertisers or other partners, there is no guarantee the content stays private. A safer approach would be to use school-approved tools with clear data-handling policies, or to check with a teacher before using any AI tool with personal or assessment content (1).

Q2: An educator wants to use a new AI tool to generate discussion prompts for a class. Before introducing it to students, what checks should they do first?

Answer: The educator should: check whether the tool’s privacy notice explains how submitted content is used and stored; confirm whether student data can be accessed by the tool provider or used for model training; test whether the tool’s outputs are accurate and appropriate before assigning them; and ask whether the school has a process for evaluating new digital tools. If any of these questions cannot be answered clearly, the tool should not be used with students until a proper evaluation has been done. Under the Privacy Act 2020, schools need a lawful basis for processing student data — using an unevaluated AI tool does not meet that standard (2)(3).

Q3: You are using an AI tool to help plan a research project and it generates a citation for a study. How do you check whether the citation is real?

Answer: Do not assume the citation is real just because the AI presented it confidently. AI tools commonly generate plausible but invented citations — this is called “hallucination.” To verify: search for the title or author name in a scholarly database like Google Scholar or your school library’s database; check whether the publication venue or journal actually exists; if the study claims to be from a specific year or institution, search for it by those details. If you cannot confirm the citation exists, do not use it — a wrong citation can undermine the credibility of your entire project and may be considered academic dishonesty by your school.

Sources and references

[1] New Zealand. Office of the Privacy Commissioner. (2025). Children’s Privacy Project. https://www.privacy.org.nz/focus-areas/children-and-young-people-policy-project/

[2] New Zealand. Office of the Privacy Commissioner. (2025). Privacy tools for agencies. https://www.privacy.org.nz/responsibilities/privacy-tools-for-agencies/

[3] New Zealand. Office of the Privacy Commissioner. (2025). Privacy Impact Assessments. https://www.privacy.org.nz/responsibilities/privacy-impact-assessments/

What to do next

• Build your sign-in and account safety habits with Passwords, passphrases, and MFA: a simple guide.
• Check Privacy checks for school tools and student data before adopting a new tool.
• Follow the educator pathway at For Educators and the student pathway at For Students.
• Use How to Spot Phishing Emails, Scams, and Fake Messages if an AI tool interaction has involved a suspicious message or sign-in prompt.