Vibedia.
All vendors

Show only the vendors you work with. This applies to every page, and is remembered.

Glossary

Evals

A repeatable test set you score a model against, so a change can be shown to help rather than assumed to.

Also written: eval, evaluation, benchmark, benchmarks.

An eval is a fixed set of inputs with a way to score the outputs. Without one, tuning a prompt is guesswork: you change something, the next three answers look better, and you have learned nothing.

Twenty examples you actually care about, scored by hand, beat any public leaderboard for deciding what to ship. Published benchmark numbers are worth little for this purpose — they are often self-reported, the test sets leak into training data, and a model tuned to score well on them is tuned for something other than your problem.

ShareOpen LinkedIn