Journal of Artificial Intelligence Advancements
Over the recent weeks, I’ve been engaged in creating a personal video on my mobile device using Apple’s iMovie application. The concept revolves around compiling various clips of my family captured throughout the month of February, with the intention to continue this project until March. The current footage showcases moments like my five-month-old daughter babbling and gesticulating, my five-year-old son playfully chasing me with a snowball, and a visit to the eerie, dilapidated amusement park in our neighborhood, among other snippets.
While immersed in the creation of my home video, thoughts of Sora, OpenAI’s groundbreaking text-to-video system, crossed my mind following its recent unveiling. Sora has the remarkable ability to interpret prompts from users and generate intricate, imaginative, and lifelike one-minute videos. The promotional material from OpenAI exhibited a range of fantastical video sequences: an astronaut stranded on a wintry planet, two pirate ships engaged in a duel within a coffee cup, and even “historical footage of California during the gold rush.” However, two particular clips stood out for their intimacy, reminiscent of footage one might capture on a personal device. The first clip, prompted as “a beautiful homemade video showcasing the people of Lagos, Nigeria in the year 2056,” portrays what appears to be a gathering of friends or family seated at an outdoor eatery; the camera pans from a nearby open-air market to a cityscape, illuminated by the shimmering lights of cars on bustling highways at twilight. The second clip captures “reflections in the window of a train traveling through the Tokyo suburbs,” offering a view akin to what any commuter might witness; the window’s glass reflects the silhouettes of passengers against the backdrop of passing buildings. Interestingly, none of the passengers seem to be filming the scene.
While these videos exhibit some imperfections, such as an overly polished, slightly cartoonish quality in some instances, others manage to encapsulate the essence of real-life moments. The intricacies behind this technology are complex to articulate simply; in essence, Sora revolutionizes video creation akin to how ChatGPT revolutionized text generation. OpenAI asserts that Sora not only comprehends the user’s prompt but also grasps how these elements manifest in the physical realm. Through its statistical and quasi-unconscious processes, Sora discerns the movement and interaction of various objects in space and time. While it may not fully grasp specific cause-and-effect scenarios—like a person taking a bite of a cookie without leaving a mark—it excels in conceptualizing three-dimensional scenes that unfold dynamically. This advancement signifies a leap towards constructing versatile simulators of the physical world. Sora’s functionality extends beyond mere pixel manipulation; it envisions and materializes scenes that evolve in a spatial and temporal continuum, akin to the cognitive processes in our minds when we visualize scenarios and settings.
Presently, Sora is accessible to a limited group of experts for testing purposes and not yet available to the general public; OpenAI introduces it as a preview to showcase the forthcoming capabilities in AI technology. The demonstration videos prompted me to ponder the possibilities when I could utilize such a system personally. Could a future iteration of Sora be tasked with generating clips for my home video? Would it be capable of creating a scene like “a phone video capturing a five-month-old girl in a red sweater, waving her arms and mimicking her brother saying, ‘Lego’”? What if the AI could access my archived home videos or photo library, offering diverse perspectives of my family and residence? The notion of AI drawing from not just visual data but also abstract concepts—the essence of Lagos or Tokyo, the concept of familial bonds, the essence of a “beautiful homemade video”—is both intriguing and uncanny. Sora transcends mere image manipulation; it encapsulates a profound understanding of the content it produces.
The advent of “synthetic” videos, as exemplified by Sora and its progeny, raises questions on their varied applications. Malevolent entities may misuse such technology to create deepfakes, potentially propagating misinformation or leveraging them for malicious intents. Businesses might incorporate synthetic clips in presentations, while creatives in the film and advertising industries could utilize them for storyboarding or even complete production, subject to the permissions of film industry unions. Novel creative ventures, currently inconceivable, are poised to emerge—offering novel avenues for entertainment, education, and engagement. The potential unrestricted access of these systems to copyrighted materials may evoke visual styles pioneered by renowned artists, eventually leading to their overutilization and dilution. Scenes that are presently costly and time-intensive to film could become readily accessible and affordable. A screenplay commencing with “EXT. A market in Lagos” might pose no logistical challenges.
The evolution of synthetic video will inevitably redefine the essence of the medium itself. There’s a possibility that skepticism towards video authenticity may rise, eroding trust in visual representations. We might reach a juncture where discerning between synthetic and authentic content becomes blurred. In 2018, Peter Jackson’s documentary “They Shall Not Grow Old” reconstructed archival footage of the First World War in color, arguably bringing it closer to reality than the original black-and-white recordings. Just as the colorists aimed to maintain authenticity, AI endeavors to emulate reality; if synthetic video is rooted in robust statistical inference, it could eventually be perceived as sufficiently realistic.
The act of filming may lose its significance over time. Platforms like YouTube host compilations of footage capturing events like the Beirut explosion, offering myriad perspectives filmed on mobile devices. With synthetic footage capable of rendering scenes based on extensive datasets, the need for physical filming could diminish. When prompted to generate an “aerial view of Santorini during the blue hour, showcasing the stunning architecture of white Cycladic buildings with blue domes,” Sora produced footage akin to what a tourist might capture using a drone. In such scenarios, the necessity for physical filming apparatuses becomes redundant as the AI comprehends and visualizes the essence of locations like Santorini, akin to human perception.
Reflecting on a personal moment with my children, where my son entertained his sister and elicited a smile, I contemplated capturing the scene on video before recalling my deliberate choice to keep my phone aside to minimize distractions. While I vividly remember the moment, envisioning it as a valuable addition to my home video, it raises the question—why not prompt an AI to recreate such moments? Would there be any qualms regarding the authenticity of a synthetic video depicting a real occurrence?
Perhaps, there lies no inherent issue. However, the synthetic video would inherently differ from any footage I’ve captured, as it transcends being a mere recording to become an interpretation of an idea.
Ideas serve as the cornerstone of contemporary AI capabilities. The potency of artificial intelligence stems from the fundamental premise that everything essentially comprises information. Whether it’s the arrangement of chess pieces on a board, the literary style of a revered author, or the ambiance of Lagos at dusk—these facets can be articulated through text, images, videos, or audio, as they are inherently idea-centric. Ideas are versatile and transcend specific mediums; they embody fluidity. While a book, a photograph, or a film may appear static, they inherently possess malleability. There’s always an alternate sentence structure, a different camera angle, or a cinematic adaptation. A singular prompt, with minor tweaks, can elicit diverse responses from AI models like ChatGPT, DALL-E, or Sora. When interacting with AI systems, specificity is not paramount; they inherently comprehend the essence of the prompt.
In the past decade, Karl Ove Knausgaard’s literary masterpiece “My Struggle” blurred the lines between fiction and memoir, challenging traditional genre classifications. Texts, by their nature, are interpretive renderings of ideas, characterized by fluidity and subjectivity. While we instinctively acknowledge the interpretive nature of textual content, a similar intuition must evolve for other media forms, including audio and video. The veracity of a book necessitates external validation beyond its textual confines, transcending its inherent form. Conversely, books possess the unique ability to transport readers beyond mere representation into realms of imagination—a trajectory that seems to define the future of all forms of media.
In this evolving landscape, the essence of authenticity and representation in audiovisual media is undergoing a transformative shift. As with textual content, the veracity of audio, video, and other representational forms will necessitate critical evaluation beyond their intrinsic medium-specific attributes. The trajectory seems to veer towards a realm where imagination supersedes mere representation—a trajectory that encapsulates the future trajectory of all media forms.
