Simplifying the Usage of an Advanced Language Model on Your Windows Computer by Nvidia
By Tom Warren, a seasoned editor specializing in Microsoft, PC gaming, consoles, and technology. He founded WinRumors, a platform dedicated to Microsoft news, before joining The Verge in 2012.
Nvidia has introduced an initial version of Chat with RTX, a user-friendly application that allows individuals to deploy a personalized AI chatbot on their PC. This innovative tool enables users to input YouTube videos and personal documents to generate summaries and receive tailored responses based on the data provided. The entire process operates locally on your PC, requiring only an RTX 30- or 40-series GPU with a minimum of 8GB of VRAM.
After exploring Chat with RTX briefly, I have identified its potential, despite encountering some minor flaws in the application. I anticipate its significant value in data research for journalists or individuals analyzing document collections.
The tool excels in processing YouTube videos, allowing users to input a URL and search through transcripts for specific references or summarize the entire video content. This feature is particularly beneficial for navigating video podcasts and pinpointing specific segments within discussions, such as the recent discussions on Microsoft’s revised Xbox strategy.
However, there were some glitches in the functionality when searching through YouTube videos. An attempt to search the transcript of a Verge video led to Chat with RTX downloading the transcript of an unrelated video, highlighting existing bugs in this early phase of demonstration.
Efficient Document Search within Local Files with Chat with RTX
When operating smoothly, this tool swiftly retrieves references within videos. I created a dataset of FTC v. Microsoft documents for analysis using Chat with RTX. During my coverage of the court case last year, sifting through documents swiftly was often challenging, but Chat with RTX enabled near-instant queries on my PC.
For example, the chatbot effectively summarized Microsoft’s detailed strategy for Xbox Game Pass based on internal documents disclosed during the trial:
Xbox Game Pass is defined as a gaming content subscription service that provides access to a game library for a fixed monthly fee. It aims to be a platform that allows gamers to play across various devices, enabling publishers to reach a wide audience. With support from Project xCloud and Xbox Series X, the service aims to offer a profitable platform for publishers to monetize their games. The primary strategic objective for the Gaming CSA is to expand Xbox Game Pass, targeting an estimated global subscriber base of 750 million. The service faces three key content dynamics: the need for unique content, expansion beyond consoles, and limited content availability.
This tool is also useful for scanning through PDFs and verifying data accuracy. While Microsoft’s Copilot system struggles with PDF integration in Word, Nvidia’s Chat with RTX efficiently extracts essential information without noticeable delays, unlike the lag often experienced with cloud-based ChatGPT or Copilot chatbots.
Despite its strengths, Chat with RTX may exhibit inaccuracies at times.
The installation of Chat with RTX on my PC, equipped with an Intel Core i9-14900K processor and an RTX 4090 GPU, took approximately 30 minutes. The application size is close to 40GB, with the Python instance utilizing around 3GB of RAM out of the available 64GB on my system. Once operational, Chat with RTX is accessible via a browser, with a command prompt running in the background to display ongoing processes and any encountered error codes.
Nvidia does not position this as a polished application for all RTX users to immediately adopt. Several known issues and limitations exist, including occasional inaccuracies in source attribution. Additionally, an attempt to index 25,000 documents initially led to app crashes, requiring a reset of preferences to resume operations.
Furthermore, Chat with RTX lacks contextual memory, which hinders the ability to formulate follow-up questions based on previous interactions. The tool generates JSON files within specified folders for indexing, discouraging the use of this feature on entire Windows Documents folders.
Nevertheless, this offering from Nvidia serves as a compelling technological demonstration, highlighting the potential of locally operated AI chatbots on personal PCs. It showcases the capabilities for analyzing personal files without the need for subscriptions to services like Copilot Pro or ChatGPT Plus.
