It appears that AI startups, aside from OpenAI, are actively pursuing their goals this week despite the ongoing concerns surrounding the conflict at Open AI. One such development is the introduction of Stable Video Diffusion by Stability AI, a type of AI that generates movies by animating existing images. This innovative model, unveiled recently, stands out as one of the few open-source video-generating models currently available. Built upon Stability’s existing StABLE Diffusion text-to-image model, Stable Video Diffusion is being positioned as a “research preview” by Stability.
To utilize Stable Video Diffusion, users must agree on the intended applications, such as educational or creative tools, design processes, and more. However, there are concerns about potential misuse of the model, particularly in the absence of a built-in content filter. Past experiences with similar AI research previews, including Stability’s previous releases, raise apprehensions about unauthorized use, such as the creation of nonconsensual deepfake content.
Stable Video Diffusion comes in two variations: SVD and SVD-XT. SVD transforms images into 576×1024 movies with 14 frames, while SVD-XT maintains the same infrastructure but increases the frame count to 24. Both versions can produce films ranging from three to thirty frames per second.
The models underwent training on extensive datasets comprising millions of videos before fine-tuning on smaller sets of hundreds of thousands to a million recordings. The origin of these videos remains unclear, raising potential concerns about copyright infringement. Despite these challenges, the models, particularly SVD-XT, excel in producing high-quality four-second clips comparable to leading models in the industry.
However, limitations exist, as highlighted by Stability, including the models’ inability to consistently render faces and people accurately, respond to textual prompts, or generate videos without motion. Nevertheless, Stability envisions further development, with plans to expand the capabilities of Stable Video Diffusion, potentially commercializing it for various applications in advertising, entertainment, and beyond.
In a bid to secure its financial standing, Stability AI recently secured \(25 million through a convertible note, bringing its total funding to over \)125 million. Despite facing financial challenges and leadership changes, including the departure of Ed Newton-Rex, Stability AI remains focused on advancing its technologies and expanding its market presence.
