#evals

Articles tagged with #evals

Your Evals Have a Rotten Tomatoes Problem
You push a change to a prompt and your eval score drops from 0.91 to 0.84. Something got worse, but the score doesn’t tell you what. So you start re-running the pipeline, tweaking your inputs, pouring
Feb 12, 20268 min read300
On the new larger context windows
Today the latest LLMs have large context windows up to ~1 million tokens. There are many occasions when this larger context window can be useful: Context engineering: injecting rich system/user conte
Nov 4, 20252 min read317
Your Questions on Building with AI
Since the launch of Airbyte 1.0 with AI Assist, hundreds of new Airbyte connectors have been built. We recently joined our partners at Airbyte for the Data Bytes meetup where we answered questions from an audience of data engineers, AI enthusiasts, a...
Nov 13, 20247 min read257
Using Historical Codebase To Build An Eval Suite
How thousands of pull requests let Fractional automate annotating a ground truth set for a new product.
Nov 11, 20243 min read216