Not to be surpassed by rivals such as Google, which recently unveiled a text-to-video tool, the AI startup OpenAI introduced its own text-to-video model, named Sora, on Thursday. Sora, akin to Google’s Lumiere, comes with limited availability but boasts the capability to create videos of up to 1 minute in length.
The realm of generative AI has witnessed a surge in the text-to-video domain, with OpenAI, Google, Microsoft, and others expanding their focus beyond text and image generation. This strategic move aims to solidify their positions in a sector expected to generate $1.3 trillion in revenue by 2032. The objective is not only to capture the interest of consumers who have been captivated by generative AI since the advent of ChatGPT over a year ago but also to explore new horizons in AI technology.
OpenAI, renowned for innovations like ChatGPT and Dall-E, announced that Sora will be accessible to a diverse group of individuals, including “red teamers” specializing in combating misinformation, hateful content, and bias. Additionally, visual artists, designers, and filmmakers will have the opportunity to provide feedback from a creative perspective. The adversarial testing by experts is crucial to address concerns related to the creation of convincing deepfakes using AI for generating images and videos.
Apart from seeking external feedback, OpenAI aims to showcase its advancements to the public to offer insights into the upcoming AI capabilities. Sora’s unique strength lies in its ability to comprehend lengthy prompts, as demonstrated by a 135-word example. The sample video shared by OpenAI illustrates Sora’s versatility in creating diverse characters and scenes, ranging from people, animals, and whimsical creatures to urban and natural landscapes, serene gardens, and even a submerged New York City.
This proficiency is attributed to OpenAI’s prior work with models like Dall-E and GPT. Leveraging Dall-E 3’s recaptioning technique, Sora can provide detailed descriptions for visual training data, enabling the generation of intricate scenes with multiple elements and accurate details. The model not only interprets user prompts but also understands the physical attributes of the requested elements in the real world.
While the sample videos exhibit remarkable realism, especially in scenes featuring intricate details, there are instances where inaccuracies are noticeable, such as in close-up human faces or swimming sea creatures. Sora can also generate videos from static images, extend existing video sequences, and fill in missing frames, akin to Lumiere’s capabilities.
OpenAI views Sora as a foundational model for understanding and simulating real-world scenarios, a crucial step towards achieving Artificial General Intelligence (AGI) – a form of AI closer to human-like intelligence. Despite its strengths, Sora has acknowledged weaknesses, such as challenges in accurately depicting complex physics and understanding cause-and-effect relationships. For instance, it may struggle to show a bite mark on a cookie after a person takes a bite.
Furthermore, Sora’s occasional confusion between left and right showcases its limitations. OpenAI has not specified a widespread release date for Sora but emphasized the importance of implementing safety measures before making it more broadly available. These safety protocols include adhering to standards that prohibit extreme violence, sexual content, hateful imagery, celebrity impersonations, and intellectual property violations. The organization recognizes the dynamic nature of AI applications and the necessity of continuous learning from real-world interactions to enhance the safety and efficacy of AI systems over time.
