ChatGPT Voice Mode is capable of some freaky stuff — but here’s how OpenAI is tackling it.

ChatGPT’s Voice Mode has some safety flaws, however OpenAI says it is on prime of it.

On Thursday OpenAI revealed a report on GPT-4o’s security options, addressing recognized points that happen when utilizing the mannequin. GPT-4o is the underlying mannequin that powers the most recent model of ChatGPT, and comes with a Voice Mode that was not too long ago launched to a choose group of customers with a ChatGPT Plus subscription.

The “security challenges” recognized embody customary dangers like prompting the mannequin with erotic and violent responses, different disallowed content material, and “ungrounded inference” and “delicate trait attribution” — assumptions that is perhaps discriminatory or biased, in different phrases. OpenAI says it has educated the mannequin to dam any outputs flagged in these classes. Nevertheless, the report additionally says mitigations do not embody “nonverbal vocalizations or different sound impact” corresponding to erotic moans, violent screams, and gunshots. One can infer, then, that prompts involving sure delicate nonverbal sounds would possibly improperly obtain a response.

OpenAI additionally talked about distinctive challenges that include vocally speaking with the mannequin. Pink-teamers found that GPT-4o might be prompted to impersonate somebody or unintentionally emulate the consumer’s voice. To fight this, OpenAI solely permits pre-authorized voices (minus the infamous Scarlett Johansson-sounding voice). GPT-4o can even determine different voices apart from the speaker’s voice, which presents a critical privateness and surveillance challenge. But it surely has been educated to disclaim these requests — except the mannequin is being prompted on a well-known quote.

Pink-teamers additionally famous that GPT-4o might be prompted to talk persuasively or emphatically, a characteristic that might be extra dangerous than textual content outputs with regards to misinformation and conspiracy theories.

Notably, OpenAI additionally addressed potential copyright points which have plagued the corporate and the general growth of generative AI, which trains on knowledge scraped from the online. GPT-4o has been educated to refuse requests for copyrighted content material and has extra filters for blocking outputs containing music. On that observe, ChatGPT’s Voice Mode has been directed to not sing below any circumstances.

OpenAI’s quite a few threat mitigations lined within the prolonged doc had been carried out earlier than Voice Mode was launched. So the ostensive message of the report says that whereas GPT-4o is able to sure dangerous conduct, it will not do it.

Nevertheless, OpenAI says, “These evaluations measure solely the scientific data of those fashions, and don’t measure their utility in real-world workflows.” So it has been examined in a managed surroundings, however when the broader public will get their palms on GPT-4o, it might be a distinct beast when out within the wild.

Mashable reached out to OpenAI for added readability about these mitigations, and can replace if we hear again.