Post
16
maybe one of the hard parts of my kind of fine tuning is evals.
where do you want model to go? how do you find true answer of a hardly debated issue?
instead of manually writing answers in many domains (which i can't do, i don't know many answers in many domains) i rely on a few tricks:
- find other aligned llms and get ideas from them
- rank many llms in AHA leaderboard and get ideas from top ones also rejecting the worst ones
- do mixture of agents of the above to get a collective answer
- and lately, compare the answers coming from my fine tunes with base models, assume my fine tune is preferred if there is a difference
these still can't find perfect answers but they kick the model in the right direction. and that may be a big deal. can't claim my fine tune (ostrich) knows every truth. but it may make more sense to claim if the base differs with fine tune most probably fine tune is the better answer.
making better evals ends up training better models. and produce better AHA ranking. which further sharpens evals. this feedback loop is going to be useful for a while.
where do you want model to go? how do you find true answer of a hardly debated issue?
instead of manually writing answers in many domains (which i can't do, i don't know many answers in many domains) i rely on a few tricks:
- find other aligned llms and get ideas from them
- rank many llms in AHA leaderboard and get ideas from top ones also rejecting the worst ones
- do mixture of agents of the above to get a collective answer
- and lately, compare the answers coming from my fine tunes with base models, assume my fine tune is preferred if there is a difference
these still can't find perfect answers but they kick the model in the right direction. and that may be a big deal. can't claim my fine tune (ostrich) knows every truth. but it may make more sense to claim if the base differs with fine tune most probably fine tune is the better answer.
making better evals ends up training better models. and produce better AHA ranking. which further sharpens evals. this feedback loop is going to be useful for a while.