All Posts

Published on
November 4, 2023
On Coding Benchmarks: Thoughts on SWE-Bench & Why Evals are Hard
finetuning general evals
Our thoughts on SWE-Bench after reading a recent paper that found issues with SWE-Bench resulting in SWE-Agent + GPT 4's performance dropping 3x.
Published on
October 13, 2023
Improving Code Completion LLMs with Repo-Specific Finetuning
finetuning general
The inner workings of our repo-specific finetuning pipeline that yields 50% more accurate code completions.
Published on
October 12, 2023
Why Finetune Code LLMs for your Codebase?
finetuning general
Our thesis on why finetuning with internal engineering data sources is crucial for complex, enterprise-grade codebases with unique patterns & frameworks.

On Coding Benchmarks: Thoughts on SWE-Bench & Why Evals are Hard