underscored

@underscored

2 clips · 1 follower

Follow
Tag:ai-evalsClear

This process, where reviewing examples changes your definition of good, is called criteria drift. For example, if we'd prompted the agent to look for this 'objection handling' failure up front when we gave it traces to check, the agent would have caught the error automatically. But we'd only know to add 'objection handling' to our criteria after seeing this trace ourselves.

— Hamel Husain
1mo ago

Underscored — save the words that stop you in your tracks.

Start saving quotes →