New research shows many Kiwis are confident they can spot an AI-generated fake but suggests that confidence may not backed up by their online behaviour.
Cybersecurity company Avast's 2026 Safe Tech Report has found 66 percent of New Zealand adults believe they would recognise an AI fake.
But another 40 percent also admitted to sharing, or considering sharing, sensitive information with an AI tool - with 21 percent having shared or considered sharing a CV and 21 percent sharing or considering sharing a work document.
The report describes that as an "AI confidence trap".
It also says while people might know the classic red flags of a good scam - things like pressure to act quickly or an offer that seems too good to be true - they can talk themselves past them. So what does this say about how we're going with our online safety in the AI era?
Stephen Kho from Avast told Nine to Noon despite the fact people were aware of AI's potential to generate fakes, the AI confidence trap came into play.
"But then we go, because it's so convenient, we are using these AI tools, these publicly hosted AI tools that are potentially hosted overseas mainly America and China and we're uploading our sensitive data ... including medical records, financial records because they're so helpful in helping us plan, identify and do diagnosis."
Uploading documents like this to AI was very similar to uploading them to the internet and they could be used for model training, he said.
People should be very careful and to share less with AI, he said.
"All these tools, in fact if you look at the fine print it would say you know the information you give there can be used and probably eventually owned by these companies."
He advised that if people did want to share something like financial information with AI they should ensure that the information has been made anonymous by removing any private or personal information to ensure it could not be traced back them.
"The idea is that if you're not confident about handing it to someone on the high street, then you shouldn't do it to an AI chatbot as well because they are not your confidante, they're not secret companions or personal experts that have signed legal obligations - these are just in fact AI bots on the internet."
Kho said that insecure configuration of AI platforms means information can be dumped into publicly accessible hacked data which is then traded on the internet and potentially leveraged for scams.
AI tools making it harder to identify scams
Kho said AI has enabled deep fakes which include video or audio.
"We've had a really significant hack recently where a person got called up and middle of the night, you know he'd just done a money transfer for the company and got pulled into a conference call, a zoom call with the CEO and CIO," he said.
They said it was a secret last minute transaction and the person needed to transfer $20 million to an account for what was a secret merger acquisition.
"On all accounts it's the people he's recognised on this board meeting, this zoom call, so he did what he was asked to do but it turned out it was a pretty sophisticated fake video of the CIO and CEO ... asking to transfer this money."
In the past it was easy to identify fake SMS or Facebook messages or emails because they often had incorrect spelling or grammar and the context was often not right, he said.
"These days those are all gone, so the quality, the speed, how real the fake website is that mimics the shop that you like, they're all so real now that it's really really difficult to pick out."
AI had made it quicker to produce these scams and they were much more believable, he said.