David Bridges
OpenAI Sora is implementing strict measures to limit the representation of real individuals within its platform, significantly enhancing safety protocols to mitigate potential misuse. This initiative reflects a growing awareness of the ethical implications surrounding AI-generated media. Users can expect a robust set of features that enable them to create and customize AI-generated videos, but they will also encounter certain restrictions designed to prevent harmful applications of this technology.
Recently unveiled during OpenAI’s 12 Days of OpenAI event, the video generation tool boasts an array of editing functionalities that enhance user creativity. However, as users are discovering, there are clear boundaries to what can be achieved with Sora. The tool’s capabilities are designed to foster innovation while prioritizing ethical considerations and user safety.
According to the system card associated with Sora, the option to upload images of individuals will be limited to a select group of users participating in a “Likeness pilot.” This means that the majority of users will not have the capability to generate videos featuring real people based on uploaded images. This pilot program aims to carefully evaluate the implications and effectiveness of these features. An OpenAI representative emphasized that the restrictions are in place to mitigate risks associated with the misuse of likeness and the creation of deepfake content. OpenAI is committed to actively monitoring for any misuse, taking necessary actions against offenders, and using insights gained from early experiences to refine their safety strategies.
Mashable Light Speed
OpenAI’s Sora first look: YouTuber Marques Brownlee breaks down the problems with the AI video model
The decision to restrict the portrayal of individuals in Sora videos is a prudent approach from a legal and ethical standpoint. The potential for misuse is significant, encompassing risks such as non-consensual deepfakes, inappropriate representations of minors, fraudulent schemes, and the spread of misinformation. To effectively combat these dangers, Sora has been specifically trained to reject certain requests stemming from text prompts or image uploads, ensuring that harmful content is not generated.
Moreover, Sora is programmed to decline prompts related to NSFW (Not Safe for Work) material and NCII (Non-Consensual Intimate Imagery), along with the creation of lifelike representations of children, though it does permit fictitious portrayals. To further enhance transparency and accountability, OpenAI has integrated C2PA metadata into all Sora-generated videos and made a visible watermark the default setting, even though users have the option to remove it. Additionally, an internal reverse image search feature is employed to verify the authenticity of the videos produced, reinforcing the platform’s commitment to safety.
While numerous safeguards have been established to prevent misuse of the Sora platform, there remains a pressing question regarding how it will handle the challenges posed by extensive stress-testing. Currently, access to Sora is restricted due to overwhelming demand, highlighting the public’s interest in this innovative technology and its potential applications.
Topics
Artificial Intelligence
OpenAI
var facebookPixelLoaded = false;
window.addEventListener(‘load’, function(){
document.addEventListener(‘scroll’, facebookPixelScript);
document.addEventListener(‘mousemove’, facebookPixelScript);
})
function facebookPixelScript() {
if (!facebookPixelLoaded) {
facebookPixelLoaded = true;
document.removeEventListener(‘scroll’, facebookPixelScript);
document.removeEventListener(‘mousemove’, facebookPixelScript);
!function(f,b,e,v,n,t,s){if(f.fbq)return;n=f.fbq=function(){n.callMethod?
n.callMethod.apply(n,arguments):n.queue.push(arguments)};if(!f._fbq)f._fbq=n;
n.push=n;n.loaded=!0;n.version=’2.0′;n.queue=[];t=b.createElement(e);t.async=!0;
t.src=v;s=b.getElementsByTagName(e)[0];s.parentNode.insertBefore(t,s)}(window,
document,’script’,’//connect.facebook.net/en_US/fbevents.js’);
fbq(‘init’, ‘1453039084979896’);
fbq(‘track’, “PageView”);
}
}
Daniel Mercer
