Note: This feature is available to Codeium Teams and Enterprise Users.
About
Codeium Analytics offers detailed insights into your team's adoption and usage of Codeium. By breaking down metrics by subgroups within your organization, you can compare and contrast tool usage across different teams. This provides deeper insights into how Codeium is utilized and its impact on productivity.
Accessing Analytics Data
Web Portal to view the Dashboard:
- Codeium Saas: codeium.com/team/analytics
- Self-Hosted Codeium: <enterprise_portal_url>/team/analytics
Permissions Required:
- Admin or Observer permissions in the Codeium instance are required to view these metrics.
API access:
- Users can also access these metrics via the Analytics API, allowing seamless integration into existing dashboards.
A sample snapshot of the analytics dashboard is shown below.
Understanding the Usage Metrics
We understand that coding metrics alone don't capture the full scope of what makes a great developer. We don't claim that our metrics provide a complete picture of a developer's productivity. Instead, we offer a few key metrics to help connect the dots. It's then up to the organization to link these metrics to PR review cycle times, DORA metrics, or other internal measures.
One of the key metrics that we track is Percentage of Code Written (PCW). We believe that PCW should be the definitive measure of the value a code assistant provides to developers as it reflects the actual results of using the code assistant and accounts for varying coding durations among developers.
PCW measures the percentage of code committed to the codebase that was generated by Codeium. For example, if a developer accepts five lines of code but deletes three, only the two lines that persist are counted. This metric offers a clear-cut measure of AI effectiveness, providing a genuine measure of value. Unlike completion rates, which can be gamified by providing shorter, more frequent suggestions to inflate metrics, PCW reflects the actual, sustained contribution of the AI to the codebase. This ensures that the focus remains on meaningful code generation and overall value rather than inflated acceptance rates.
In addition to PCW, we track a variety of other metrics to provide a comprehensive view of Codeium's impact and usage. These include:
- Completions by Date
- Chats by Date
- Total Code Blocks Explained
- Docstrings Generated
- Total Code Blocks Refactored
- Language Distribution
- IDE Distribution
- And more…
We also measure Time Saved. This is calculated as: 3 * (number of characters saved / 200). This formula considers the number of characters saved by autocomplete, an average typing speed of 200 characters per minute (CPM), and a scaling factor of 3. The scaling factor of 3 includes time saved from typing, searching for answers, and thinking.
Additionally, we provide a metric for Total Estimated Value Saved, which is adjustable via a slider based on the average compensation for developers in your organization.