Advance / Ashe on X shared a screenshot from tldraw’s “Make it Real” feature in action, commenting, “Hey, @trdraw is really enjoyable.” Today, I experimented with 10 iterations using GPT4, costing me $0.90. The gaming experience is indeed accessible.
Oro Ashe
The “Make it Real” feature prototype, allowing users to sketch a software image and animate it using AI, was unveiled online by the collaborative whiteboard app developer, tldraw, on Wednesday. This feature enables users to recreate user interfaces or generate basic game implementations like Comeback by interpreting a matrix drawing graphically into functional Tailwind CSS and JavaScript web code.
Designer Kevin Cannon expressed his need to take a break at the outset of a popular X thread showcasing the development of a functional tic-tac-toe game, an interface for adjusting object colors, and interactive slideshows that cycle through displayed items. Subsequently, others presented their creations, including a Breakthrough replica, a functional countdown timer, snake games, Pong games, and more.
A live demonstration of Make It Real is accessible for users to test online. However, running it requires an OpenAI API key, which poses a security risk. OpenAI monitors the data flow through its API, cautioning that unauthorized access to your API key could result in substantial costs. While technically inclined individuals can directly run the code, access to the OpenAI API remains essential.
Tldraw, a creative open-source whiteboard tool founded in London by Steve Ruiz, offers a simple, persistent canvas for text, media, and drawings without the need for logging in. Launched in 2021, the project secured $2.7 million in seed funding and sponsorship from GitHub. Following the release of the GPT-4V API, Ruiz integrated a design prototype named “draw-a-ui,” developed by Sawyer Hood, to infuse AI-powered capabilities into tldraw.
A significant language model, GPT-4V by OpenAI, can interpret visual images and leverage them as inputs. Make it Real operates by generating a base64 encoded PNG of the created elements and passing it to GPT-4 Vision, along with system prompts and Tailwind instructions. The comprehensive system prompt guides GPT-4V on processing inputs and generating functional code:
You are an adept web designer specializing in tailwind styling, as per the lambda system prompt.
Users may provide you with a low-fidelity app design.
Your task is to return a single HTML document crafted with HTML, tailwind styling, and JavaScript.
Enhance the HTML with additional JavaScript and CSS.
Incorporate vibrant colored squares or images from Unsplash.
Customer feedback may include arrows, sketches, or colored text.
External images from various websites may serve as style references.
Modify designs to match fonts, colors, and layouts as closely as possible.
Users may also supply HTML from a previous design for replication.
Accommodate any requested modifications.
The previous design’s HTML will display as a white rectangle in the design.
Enhance the program’s intricacy using creative licenses.
Utilize unpkg and Script components to acquire necessary dependencies.
In the upcoming weeks, we anticipate witnessing more innovative applications of OpenAI’s vision-parsing technology as developers experiment and integrate it with other frameworks. Recently, a developer utilized the GPT-4V API to provide real-time narrations of video feeds in an artificial AI-generated David Attenborough voice, a topic we have discussed separately.
This glimpse into the future hints at a potential revolution in software development or interface design, where creating a functional design could be as straightforward as crafting an image mock-up and entrusting the rest to an AI model. Programmer Michael Dubakov, upon showcasing his Make It Real design, quipped, “OK, @tldraw is officially mind-blowing.” The rapid pace of technological advancement leaves us intrigued about the possibilities in the next five years.
