Stability AI is expanding its range of generative AI models today with the introduction of Stable Video 3D (SV3D).
The latest addition, SV3D, is a generative AI video tool tailored for rendering 3D videos. Stability AI has been actively enhancing its video capabilities through the development of Stable Video technology, enabling users to create short videos from either an image or text prompt. SV3D represents a progression from Stability AI’s prior Stable Video Diffusion model, now geared towards novel view synthesis and 3D generation tasks.
With SV3D, Stability AI introduces a new dimension to its video generation model by empowering users to construct and transform multi-view 3D meshes from a single input image.
SV3D is now accessible for commercial utilization under a Stability AI Professional Membership priced at \(20 per month for creators and developers with an annual revenue of less than \)1 million. For non-commercial applications, users have the option to download the model weights from Hugging Face.
Here’s a sample video swiftly generated using the SV3D model. Despite some minor distortions, the objects in the video maintain their forms consistently solid as the camera revolves around them.
Target Applications: Game Development and E-commerce
“Incorporating camera path conditioning into our Stable Video Diffusion model for image-to-video diffusion, Stable Video 3D can produce multi-view videos of an object,” as stated in a blog post by the company detailing the new model.
Varun Jampani, lead researcher at Stability AI, highlighted the significance of SV3D in generating 3D assets, particularly within the gaming industry. Furthermore, he emphasized its role in crafting 360-degree orbital videos, which prove beneficial in e-commerce by offering a more immersive and interactive shopping experience.
Evolution from Stable Zero123 to SV3D
Stability AI is recognized for its Stable Diffusion gen AI models for text-to-image conversion, including SDXL and Stable Diffusion 3.0, the latter still in its early research phase. Stable Diffusion 1.5, an open-source image generation model, serves as the foundation for numerous other AI image and video products, such as Runway and Leonardo AI.
In December 2023, Stability AI unveiled the Stable Zero123 model, introducing new capabilities for generating 3D images. Emad Mostaque, founder and CEO of Stability AI, indicated that Stable Zero123 marked the initial release in a series of 3D models.
SV3D technology adopts a distinct approach to 3D generation compared to Stable Zero123.
“Considered as an advancement over our previous offering, Stable Zero123, Stable Video 3D functions as a novel view synthesis network, taking a single image as input and producing novel view images,” explained Jampani. While Stable Zero123 generates one image at a time based on Stable Diffusion, Stable Video 3D, built on Stable Video Diffusion models, simultaneously produces multiple high-quality novel views, enhancing the creation of superior 3D meshes from a single image.
Consistent Views from Various Angles
In a research paper, Stability AI researchers elaborate on the methodologies employed to achieve 3D generation from a single image using latent video diffusion.
The report highlights challenges faced by previous approaches in 3D generation, emphasizing the limitations in views and inconsistencies in novel view synthesis (NVS) that impact 3D object generation outcomes.
SV3D excels in generating coherent multi-view images of an object from various angles, ensuring consistency and quality. Stability AI asserts that SV3D delivers coherent views from any perspective, a key strength of the model.
The research paper underscores this advancement by stating that, “Unlike prior methods struggling with limited perspectives and output inconsistencies, Stable Video 3D excels in providing coherent views from any angle with effective generalization.”
Apart from its novel view synthesis capabilities, SV3D focuses on optimizing 3D meshes. Leveraging its multi-view consistency, SV3D enhances the quality of 3D meshes by directly generating them from the novel views it creates.
Two Specialized Variants: SV3D_u and SV3D_p
SV3D is available in two distinct variants tailored for specific use cases.
SV3D_u generates orbital videos based on single image inputs without requiring camera conditioning. On the other hand, SV3D_p extends this functionality to accommodate both single images and orbital views, enabling users to craft 3D videos along designated camera paths.
VentureBeat’s mission is to serve as a digital hub for technical decision-makers seeking insights into transformative enterprise technology and facilitating transactions. Explore our Briefings for more information.
