It seems from a cursory look at the headlines that conceptual synthetic intelligence is prevalent today. In reality, some of those headlines may have been generated by conceptual AI, such as OpenAI’s ChatGPT, a bot that has demonstrated an impressive ability to produce text that mimics human writing.
But what exactly is meant by generative AI?
In the past, when people referred to AI before the recent surge in conceptual AI, they typically meant machine learning models that were trained to make predictions based on data. For instance, these models were designed to determine if a specific X-ray indicates the presence of an underlying tumor or if a borrower is likely to default on a loan, by analyzing vast amounts of data.
A machine learning model that is focused on generating new information rather than just making predictions about specific data is known as relational AI. A system utilizing relational AI is trained to generate more items that resemble the data it was trained on.
The distinction between conceptual AI and other forms of AI can sometimes be blurry. According to Phillip Isola, an associate professor at MIT, the same techniques can often apply to both.
The technology itself is not entirely new, despite the hype surrounding the emergence of ChatGPT and similar models. These powerful machine learning models leverage mathematical and research advancements that have roots dating back over 50 years.
Increasing Complexity
An early example of conceptual AI is the Markov chain, a simpler model. This method, named after Russian scientist Andrey Markov, was developed in 1906 to model the behavior of random processes. Markov models have been widely used in machine learning for tasks like next-word prediction, such as in predictive text systems.
A Markov model for word prediction predicts the next word in a sentence based on the preceding words. Tommi Jaakkola, a professor at MIT, notes that these simple models, limited by their short memory, struggle to generate coherent text.
The key difference, he explains, lies in the complexity and scale at which these models can operate. The ability to create sophisticated outputs has been around for much longer than the past decade.
In the past, the focus was primarily on optimizing machine learning models for specific datasets. However, the emphasis has shifted towards training models on larger datasets with remarkable results, sometimes containing hundreds of millions or even billions of data points.
Similar to a Markov model, the fundamental principles underlying ChatGPT and similar models are alike. The significant difference lies in the scale and complexity of models like ChatGPT, which have been trained on massive datasets, including a substantial portion of the available text on the internet.
Advancements in Model Complexity
While larger datasets have contributed to the rise of conceptual AI, significant research breakthroughs have also led to more sophisticated deep-learning architectures.
Introduced in 2014 by researchers from the University of Montreal, generative adversarial networks (GANs) utilize two competing models: one generates output (e.g., images) and the other discriminates between real and generated data. Through this adversarial process, the generator improves its output quality. Models like StyleGAN, a photo generator, are built on this framework.
Propagation models, introduced by researchers from Stanford University and the University of California at Berkeley, learn to generate new data samples resembling the training data, such as in image generation systems like Firm Diffusion.
The transformer architecture, developed by Google researchers in 2017, has been instrumental in creating large-scale language models like ChatGPT. In natural language processing, transformers create an attention map that captures relationships between each token in the text, aiding in contextual understanding and generation of new text.
These are just a few examples of the diverse methodologies within relational AI.
Diverse Applications
These methods share a common trait of converting input into tokenized representations, enabling the generation of new data resembling the original. This versatility allows for a wide range of conceptual AI applications.
For example, Isola’s team is exploring the use of conceptual AI to generate synthetic image data for training computer vision models, enhancing object recognition capabilities.
Jaakkola’s team is leveraging conceptual AI to design novel protein or crystal structures for advanced materials. By understanding the dependencies within structures, similar to language dependencies, a relational model can create stable and feasible structures.
While generative models can achieve remarkable results, they may not be optimal for all data types. Shah highlights that for structured data like spreadsheets, traditional machine-learning approaches may outperform conceptual AI models in making predictions.
The potential lies in creating user-friendly AI systems that can interact seamlessly with humans and machines, offering a new paradigm in human-machine communication.
Challenges and Future Prospects
Despite the benefits of conceptual AI, there are concerns about potential issues like job displacement, amplification of biases, and copyright challenges. However, Shah envisions empowering artists to create innovative content and foresees transformative impacts on economics.
Isola sees promise in using relational AI for creative problem-solving and anticipates the development of AI agents with broader intelligence capabilities in the future.
While the operations of these models differ from human brain functions, they share similarities in fostering creativity and generating novel ideas. Conceptual AI, as Isola suggests, has the potential to enable agents to think creatively and devise intriguing concepts or plans.
