A few weeks back, Mark Zuckerberg, the CEO of Meta, made an announcement on Facebook regarding the open-sourcing of Code Llama, a large language model (LLM) developed by his company. This AI engine, akin to GPT-3.5 and GPT-4 in ChatGPT, is designed to assist in coding tasks.
Zuckerberg highlighted three key aspects of this LLM: its open-source nature, its focus on aiding in code writing and editing, and the staggering 70 billion parameters within its model. The objective is to enable developers to present more complex challenges to the model, thereby enhancing the accuracy of its responses.
The decision to open-source Code Llama is intriguing as it allows users to download and install the model on their servers, eliminating concerns about Facebook potentially harvesting their code for training purposes or other questionable uses.
While setting up Code Llama on a Linux server involves technical complexities, Hugging Face has already integrated the 70B Code Llama LLM into their HuggingChat interface, simplifying the process for users.
Getting Started with Code Llama
To begin using Code Llama, users are required to create a Hugging Face account. Existing users can leverage the 70B Code Llama LLM with their current accounts.
It’s important to note that while installing Code Llama on a private server ensures code privacy, using it through Hugging Face may involve data sharing with the model’s creators unless specific settings are adjusted.
Upon logging into HuggingChat, users can switch to the Code Llama model by accessing the settings and selecting the codellama/CodeLlama-70b-Instruct-hf option. Once activated, the chat interface will indicate the Code Llama model is in use.
For testing purposes, the author attempted various coding tasks using Code Llama within HuggingChat.
Test Results
Test 1: WordPress Plugin Development
The prompt involved creating a PHP 8 compatible WordPress plugin with specific functionalities. However, Code Llama’s response fell short by omitting essential elements and generating code that was incompatible with standard code formatting tools, leading to an unsuccessful outcome.
Test 2: String Function Rewrite
In this test, Code Llama successfully provided a solution to fix a bug related to string manipulation, delivering a satisfactory result.
Test 3: Bug Identification
Code Llama, similar to Bard, struggled in addressing a complex coding issue by focusing on superficial aspects rather than the core problem, leading to ineffective recommendations.
Conclusion
While the testing scope was limited, Code Llama’s performance, especially in comparison to ChatGPT, was underwhelming. The AI’s inability to excel in critical coding tasks indicates room for improvement before it can match the capabilities of existing models like ChatGPT.
Despite the potential data privacy advantage of using Code Llama on a private server, its current limitations in providing accurate solutions raise concerns about its readiness for widespread adoption.
In the realm of AI coding assistance, the author’s experience suggests that while Code Llama has potential, it falls short of the performance benchmark set by ChatGPT. As the AI landscape evolves, advancements and refinements are essential for enhancing the efficacy of such tools.
If you have explored AI coding assistants, share your experiences and insights in the comments section.
For more updates on my projects, connect with me on social media. Don’t forget to subscribe to my weekly newsletter on Substack and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.
