A few weeks back, Mark Zuckerberg, the CEO of Meta, made an announcement on Facebook regarding the open-sourcing of the large language model (LLM) Code Llama by his company. This LLM, similar to GPT-3.5 and GPT-4 in ChatGPT, is an artificial intelligence (AI) engine designed to assist in coding tasks.
Zuckerberg highlighted three key aspects of this LLM: its open-source nature, its focus on aiding in code writing and editing, and the fact that it boasts a model with 70 billion parameters. The objective behind this initiative is to enable developers to present more complex challenges to the model, thereby enhancing its accuracy in providing solutions.
The decision to open-source the Code Llama model is intriguing as it offers the possibility of downloading and installing the model on a personal server, eliminating concerns about Facebook potentially utilizing code for training or other undisclosed purposes.
While setting up a Linux server and navigating various technical processes may seem daunting, it’s worth noting that Hugging Face has already integrated the Code Llama 70B LLM into their HuggingChat interface, simplifying the accessibility of this advanced AI tool.
Initiating with Code Llama
To commence using Code Llama, creating a free account on Hugging Face is necessary. Existing users can leverage the 70B Code Llama LLM with their current account credentials.
A crucial distinction to be aware of is that unlike the option to self-host Code Llama and keep code private, Hugging Face’s service indicates that input may be shared with model authors unless this setting is disabled in the account preferences.
Upon logging into HuggingChat, users are greeted with a clean chat interface. By following a few simple steps, such as changing the current model in the settings to Code Llama, users can seamlessly transition to utilizing the advanced capabilities of this model within the chat interface.
Test Scenario Evaluation
- Test 1: Crafting a WordPress Plugin
The attempt to create a WordPress plugin using Code Llama resulted in inadequacies. The generated code lacked essential elements, such as the plugin header, and exhibited formatting issues that hindered interpretation by programming editors, indicating incomplete code generation. Consequently, this test was deemed unsuccessful.
- Test 2: Refining a String Function
In this test, the objective was to rectify a bug related to string manipulation. While ChatGPT swiftly addressed the issue, both Bard and Code Llama fell short, failing to provide effective solutions. This highlights a gap in the AI’s ability to troubleshoot and resolve complex coding challenges.
- Test 3: Identifying an Elusive Bug
Revisiting a challenging coding problem, the AI responses from Code Llama, akin to Bard, focused on superficial aspects rather than delving into the core issues of the code. This approach led to recommendations that did not contribute to resolving the underlying bug, indicating a limitation in the AI’s problem-solving capabilities.
Conclusion
The comparison of AI performance across the tests underscores the current shortcomings of Code Llama in addressing intricate coding tasks effectively, especially when compared to ChatGPT’s proficiency in similar scenarios. While the privacy aspect of self-hosting Code Llama may be appealing, the priority lies in the AI’s ability to deliver accurate solutions.
Despite initial expectations for Code Llama’s capabilities, the outcomes of the test scenarios have revealed room for improvement. As technology enthusiasts, we often anticipate groundbreaking innovations; however, Code Llama, in its current state, falls short of meeting these expectations.
If you’ve experimented with AI tools for coding assistance, share your experiences and insights in the comments section below.
Follow my latest project updates on social media. Subscribe to my weekly newsletter on Substack and connect with me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.
