File size: 4,507 Bytes
43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 43bfca4 6a3ac93 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | ---
license: cc-by-nc-4.0
base_model: mlabonne/NeuralMonarch-7B
tags:
- generated_from_trainer
- mistral
- instruct
- finetune
- chatml
- gpt4
- synthetic data
- distillation
model-index:
- name: AlphaMonarch-dora
results: []
datasets:
- argilla/OpenHermes2.5-dpo-binarized-alpha
language:
- en
library_name: transformers
pipeline_tag: text-generation
---
# AlphaMonarch-dora
<!-- Provide a quick summary of what the model is/does. -->
AlphaMonarch-dora is a DPO fine-tuned of [mlabonne/NeuralMonarch-7B](https://huggingface.co/mlabonne/NeuralMonarch-7B/) using the [argilla/OpenHermes2.5-dpo-binarized-alpha](https://huggingface.co/datasets/argilla/OpenHermes2.5-dpo-binarized-alpha) preference dataset using DoRA. This model is slightly less performant on the Nous and Openllm leaderboards in comparison to base [AlphaMonarch](https://huggingface.co/mlabonne/AlphaMonarch-7B) and [AlphaMonarch-laser](https://huggingface.co/abideen/AlphaMonarch-laser). I have trained this model for 1080 steps. All hyperparams were kept consist across all these experiments.
## 🏆 Evaluation results
# OpenLLM Benchmark
# Nous Benchmark
### AGIEVAL
| Task | Version | Accuracy | Accuracy StdErr | Normalized Accuracy | Normalized Accuracy StdErr |
|--------------------------------|---------|----------|-----------------|---------------------|-----------------------------|
| agieval_aqua_rat | 0 | 28.35% | 2.83% | 26.38% | 2.77% |
| agieval_logiqa_en | 0 | 38.71% | 1.91% | 38.25% | 1.90% |
| agieval_lsat_ar | 0 | 23.91% | 2.82% | 23.48% | 2.80% |
| agieval_lsat_lr | 0 | 52.55% | 2.21% | 53.73% | 2.21% |
| agieval_lsat_rc | 0 | 66.91% | 2.87% | 66.54% | 2.88% |
| agieval_sat_en | 0 | 78.64% | 2.86% | 78.64% | 2.86% |
| agieval_sat_en_without_passage | 0 | 45.15% | 3.48% | 44.17% | 3.47% |
| agieval_sat_math | 0 | 33.64% | 3.19% | 31.82% | 3.15% |
AVG = 45.976
### GPT4ALL
| Task | Version | Accuracy | Accuracy StdErr | Normalized Accuracy | Normalized Accuracy StdErr |
|--------------|---------|----------|-----------------|---------------------|-----------------------------|
| arc_challenge| 0 | 65.87% | 1.39% | 67.92% | 1.36% |
| arc_easy | 0 | 86.49% | 0.70% | 80.64% | 0.81% |
| boolq | 1 | 87.16% | 0.59% | - | - |
| hellaswag | 0 | 69.86% | 0.46% | 87.51% | 0.33% |
| openbookqa | 0 | 39.00% | 2.18% | 49.20% | 2.24% |
| piqa | 0 | 83.03% | 0.88% | 84.82% | 0.84% |
| winogrande | 0 | 80.98% | 1.10% | - | - |
AVG = 73.18
### TRUTHFUL-QA
| Task | Version | MC1 Accuracy | MC1 Accuracy StdErr | MC2 Accuracy | MC2 Accuracy StdErr |
|---------------|---------|--------------|---------------------|--------------|---------------------|
| truthfulqa_mc | 1 | 62.91% | 1.69% | 78.48% | 1.37% |
AVG = 70.69
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-7
- train_batch_size: 2
- eval_batch_size: Not specified
- seed: Not specified
- gradient_accumulation_steps: 8
- total_train_batch_size: Not specified
- optimizer: PagedAdamW with 32-bit precision
- lr_scheduler_type: Cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 1080
### Framework versions
- Transformers 4.39.0.dev0
- Peft 0.9.1.dev0
- Datasets 2.18.0
- torch 2.2.0
- accelerate 0.27.2 |