Artificial intelligence could soon transform how Deaf and hard‑of‑hearing New Zealanders access information, with a major initiative to bring New Zealand Sign Language (NZSL) further into the digital age nearing completion.
The partnership between the Deaf and Hard of Hearing Foundation and Kiwi start‑up Kara Technologies aimed to deliver real‑time NZSL translations through lifelike digital avatars.
The avatars were designed to provide additional support in situations where the community may face delays or lack access to NZSL interpreters, such as on public transport or during emergencies.
Kara Technologies' NZSL expert Jon Tai‑Rakena said it would help address a shortage of interpreters.
"We don’t have enough interpreters. We miss a lot of information," he said through a NZSL interpreter.
"It’s really important that we can have an avatar that can provide information visibly in sign language. It’s extremely valuable to us."
NZSL was used by more than 24,500 people nationwide, including around 4500 Deaf New Zealanders who rely on it as their first language.
Capturing every movement
Kara Technologies was building a library of more than 10,000 individual signs. The process began with motion‑capture technology, similar to that used in blockbuster films like Avatar and The Lord of the Rings. Signers wore specialised suits while multiple cameras recorded every gesture, expression, and hand movement in detail.
Artificial intelligence then assembled the signs into complete sentences, producing a seamless translation in real time.
Company co‑founder Farmehr Farhour said the goal was to complement, not replace, human interpreters.
"Our focus is to be there where human interpreters cannot be – especially in emergencies where you need information instantly in an accessible format," he said.
Te reo Māori
The initiative was also incorporating te reo Māori concepts used in New Zealand, particularly among the Deaf Māori community, known as Turi Māori.
"Turi Māori have concept signs within the communities. And sign language also has some of those signs," Tai‑Rakena said.
"We’re focusing on making sure that we capture all those everyday use of signs like kai [and] hāngī."
Gallardo said the response from the Deaf community had been "fabulous".
"We are ensuring that we are listening to them and that they have a voice in how this technology is brought to life."
The project team carried out three rounds of consultation with Deaf communities in Auckland, Wellington, and Christchurch. Māori consultation was also underway to ensure Deaf Māori concepts were included.
Live public transport updates a world first
Auckland Transport had signed on to use the technology for live updates on its train, bus, and ferry services.
Kara Technologies said it would be the first time in the world avatar translations were delivered in real-time rather than pre‑recorded.
Deaf and Hard of Hearing Foundation chief executive Natasha Gallardo said the Deaf community was excited about the rollout.
"If there’s a delay or a train isn’t coming, it might go over the PA system – but as a Deaf person you can’t obviously hear what that message is," she said.
"So you can be still waiting on the platform wondering where the train is."
Auckland Transport would roll out the technology in due course.
SHARE ME