This is a guide to help you get the most out of the Context Module while using Codeium Chat.
Background: the Value of Context
Before we get started, a very brief discussion of what "context" is and why it matters.
Say you wanted to ask a question about a specialized set of documents (eg. a private codebase) that the AI was never trained on. One option is to "fine tune" the AI model on that set of documents. With the latest AI models, though, this is no longer strictly necessary. Instead, you can just include the documents as "context" to the original in the model input, and the model will use that information and respond as if it has known about that document all along.
In a nutshell, that is what the Context Module does: find the right pieces of content and feeds it to the model to help it generate a better response.
Note: Codeium does offer full fine tuning for enterprise, and the best solution combines the two features together, but the Context Module is highly effective at personalization.
How to use context-enabled chat
To get started using Context-Enabled chat, just ask a question that would require knowledge of your local codebase. Below are a few ideas for ways that context chat can be useful.
Exploring Your Codebase
If you're just getting familiar with a new codebase, Context Chat can be helpful to find if and where certain functionality is implemented. Besides the response itself, you will also see a list of the items that were included as context for the model.
Clicking on those items will take you to the original source code in your repo, so you can do further investigation.
Understanding existing functionality
Even if you know certain functionality exists, it may be helpful to ask chat how it works, or why it's implemented the way it is.
Making multi-file suggestions
Codeium Chat already has Code Actions like "Generate Docstring" or "Refactor Code" that can help you edit existing code. However, these currently only apply to code you select.
If you don't even know what function you need to modify, or if you think the modification would involve more than one function, you can just ask chat.
Explain an Error Message
Even the best programmer gets plenty of errors! Debugging a stacktrace can often require understanding of multiple files, not just the file that actually triggered the error.
Context Chat can be helpful for this this, pulling context automatically cross multiple files before answering. Just paste in the error message and ask Codeium Chat to explain. No need to copy paste in the source code which might have to come from many files.
Tips and Tricks
Our work on the Context Module is just beginning and there are several known limitations. We're working on all of these, but in the meantime, here are some tips for getting the most out of chat.
Codeium Search is an embedding-based semantic search capability that helps to the Context Module retrieve relevant context for users. Search makes the context retrieved much more relevant. If search is not enabled, then you should only expect answers to questions about code you have recently edited.
Edit the file you care about
Due to how we currently gather context, sometimes the model will not correctly identify that you are asking a question about a particular file.
If the model seems to persistently not understand what you are asking, try to make any edit to the file, or any file that is in a nearby directory, then ask again.
Ask follow ups
Often times the model won't get it exactly right the first time. It can help to treat it a bit like a conversation, asking follow up questions to get closer to the right answer.
For example, in the conversation below, I simply clarified that I meant the current codebase, and it was able to correctly answer my question.
Ask more focused questions
What does our API do?
Is a difficult question for two reasons:
- It is ambiguous. In all but the simplest codebase, there are probably many APIs and it would be hard to know which you mean. As a result Context Module will fetch a few snippets from all over the codebase, but not have enough from any one part to provide a coherent answer.
- Context is limited. The model can only consider so much context at once. If your question encompasses too broad a scope, it will simply not be able to capture enough of the contex to provide a holistic answer.
We are actively working to fix these limitations, but for now, a good rule of thumb: If your question would require looking at more than 5 files to answer correctly, try to narrow it down.
"How does our user registration endpoint work?"
"What does the package in src/app/foo` do?"
Chat in the same language as the codebase
The models are multilingual, but the Context Module is not great at matching natural language context if the question is asked in a different language than the content. This will be impossible if search is not enabled.
For the raw code itself, this is generally not a problem. However, but if you the documentation, README, HTML, Markdown are in English, it's more effective to ask the chat message in English as well.
There you have it! We're thrilled to launch the Context Module and we'll be making massive investment into it in the months to come. We'll be keeping this article updated with more guidance and feature improvements as we go along.
We'd love to hear from you. If you've tried it already, feel free to comment below on other tips, tricks that have helped you get the most out of the model