The creator of ChatGPT has introduced a novel AI tool capable of generating photorealistic videos solely based on text prompts.
The latest creation from the Silicon Valley startup OpenAI, named Sora (which translates to “Sky” in Japanese), can produce synthetic video clips of up to one minute in length that exhibit an almost lifelike quality.
According to OpenAI, this AI tool can accommodate detailed instructions and craft “intricate scenes featuring multiple characters, specific movements, and precise details.”
Sam Altman, the head of OpenAI, promoted the tool on X (formerly Twitter) by encouraging users to submit prompts for videos, which Sora would then bring to life.
One example of a remarkably realistic clip was prompted by: “An instructional cooking session for homemade gnocchi conducted by a grandmother who is a social media influencer, set in a charming Tuscan country kitchen with cinematic lighting.”
Despite its initial hyperrealistic appearance, the video clip is not without flaws.
In the video, inconsistencies are noticeable, such as the sudden appearance of a spoon wielded by the grandmother, the disappearance of a ring on her left hand, an oddly balanced rolling pin on a shelf, and what seems to be a wooden wine bottle in the background.
OpenAI acknowledged in a blog post that their model encounters challenges with physics simulations and scenarios involving multiple characters, sometimes resulting in the spontaneous appearance of animals or individuals.
In another scene depicting a grandmother celebrating her birthday, the characters’ hands exhibit unnatural contortions, and the flames on the birthday cake remain static despite her attempts to blow them out. Additionally, in a different clip, a basketball explodes upon passing through a hoop, leading OpenAI to describe such anomalies as potentially generating humorous outcomes.
Sora can interpret requests for specific camera angles or video styles, producing either eerily realistic footage or cartoonish animations reminiscent of Pixar films.
For instance, a prompt that yielded a realistic video requested: “An extreme close-up of a 24-year-old woman’s blinking eye, set in Marrakech during the magical hour, captured in cinematic 70mm film style.”
The introduction of Sora comes at a time of heightened interest in AI tools following the debut of ChatGPT just over a year ago.
However, the emergence of this video generation tool raises concerns among artists, animators, and filmmakers regarding potential job displacement and the proliferation of highly convincing “deepfake” videos online.
OpenAI assured that Sora would decline requests involving “extreme violence, explicit content, offensive imagery, celebrity likenesses, or others’ intellectual property.”
Currently, Sora is available only to a limited group of technical testers, artists, and filmmakers, allowing OpenAI to identify any issues and opportunities associated with the technology.
The company plans to engage external security experts to assess the tool for vulnerabilities, recognizing that it is impossible to anticipate all potential misuses.
While OpenAI is not the sole entity launching AI models for video creation, Sora stands out due to the success of ChatGPT and DALL-E, an image generation tool, both of which have garnered millions of users.
Criticism has been directed at OpenAI in the past for allegedly disregarding copyright laws by extensively utilizing data to train its AI models. The company faces multiple lawsuits from artists and the New York Times on these grounds, with OpenAI defending its practices under fair use regulations.
OpenAI has not disclosed the specific videos used to train Sora, except to mention that the dataset comprises publicly available and licensed footage.
