UltraBERT: Beyond the Limit of BERT Pre-Training. Pre-trained 150M/380M encoders and a text embedding model, English, 32K context.
Data Mining and Information Systems Lab
dmis-lab
AI & ML interests
None yet
Recent Activity
updated a model less than a minute ago
dmis-lab/UltraBERT-150M liked a model 2 days ago
dmis-lab/UltraBERT-Embed-150M liked a model 2 days ago
dmis-lab/UltraBERT-380MOrganizations
This collection hosts SaMer series introduced in paper, Do All Visual Tokens Matter Equally?
A collection of ablation and final models trained on the Outlier-Safe Pre-Training (OSP) framework.
This collection hosts Meerkat series introduced in paper, Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks.
-
dmis-lab/meerkat-7b-v1.0
Text Generation • 7B • Updated • 70 • 29 -
dmis-lab/llama-3-meerkat-8b-v1.0
Text Generation • 8B • Updated • 684 • • 10 -
dmis-lab/llama-3-meerkat-70b-v1.0
Text Generation • 71B • Updated • 60 • • 6 -
Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks
Paper • 2404.00376 • Published • 5
This collection hosts models introduced in OLAPH: Improving Factuality in Biomedical Long-form Question Answering.
This collection hosts Phrase-reranker models introduced in TouR (ACL 2023 Findings), optimizing test-time query representations for dense retrieval.
This collection hosts BioBERT (Bioinformatics 2020) series, a domain-specific adaptation of BERT pre-trained on biomedical corpora.
This collection hosts CVRR series introduced in paper, Reason Through the Latent! Making Latent Visual Reasoning Necessary
-
Reason Through the Latent! Making Latent Visual Reasoning Necessary
Paper • 2609.06746 • Published • 38 -
dmis-lab/Qwen2.5-VL-7B-CVRR
Visual Question Answering • Updated • 136 -
dmis-lab/Qwen3-VL-8B-CVRR
Visual Question Answering • Updated • 130 -
dmis-lab/Gemma3-4B-CVRR
Visual Question Answering • Updated • 88
This collection hosts MRPO series introduced in paper, Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
-
Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
Paper • 2606.31825 • Published • 25 -
dmis-lab/InternVL3-8B-Instruct-MRPO
Image-Text-to-Text • 613k • Updated • 8 -
dmis-lab/Qwen3-VL-8B-Instruct-MRPO
Image-Text-to-Text • 770k • Updated • 22 -
dmis-lab/Qwen2.5-VL-7B-Instruct-MRPO
Image-Text-to-Text • 849k • Updated • 26
This collection hosts Med-PRM series introduced in paper, Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards
-
dmis-lab/llama-3.1-medprm-reward-v1.0
Text Generation • 8B • Updated • 16.6k • • 17 -
dmis-lab/llama-3.1-medprm-reward-raw-training-set
Viewer • Updated • 11.7k • 41 -
dmis-lab/llama-3.1-medprm-reward-training-set
Viewer • Updated • 11.7k • 175 • 12 -
dmis-lab/llama-3.1-medprm-reward-raw-test-set
Viewer • Updated • 5.47k • 22
This collection hosts ANGEL series introduced in paper, Learning from Negative Samples in Generative Biomedical Entity Linking.
This collection hosts models of Self-BioRAG (ISMB 2024), improving medical reasoning through retrieval and self-reflection.
This collection hosts BioSyn (ACL 2020) series, for learning representations of biomedical entities based on their synonyms.
UltraBERT: Beyond the Limit of BERT Pre-Training. Pre-trained 150M/380M encoders and a text embedding model, English, 32K context.
This collection hosts CVRR series introduced in paper, Reason Through the Latent! Making Latent Visual Reasoning Necessary
-
Reason Through the Latent! Making Latent Visual Reasoning Necessary
Paper • 2609.06746 • Published • 38 -
dmis-lab/Qwen2.5-VL-7B-CVRR
Visual Question Answering • Updated • 136 -
dmis-lab/Qwen3-VL-8B-CVRR
Visual Question Answering • Updated • 130 -
dmis-lab/Gemma3-4B-CVRR
Visual Question Answering • Updated • 88
This collection hosts SaMer series introduced in paper, Do All Visual Tokens Matter Equally?
This collection hosts MRPO series introduced in paper, Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
-
Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
Paper • 2606.31825 • Published • 25 -
dmis-lab/InternVL3-8B-Instruct-MRPO
Image-Text-to-Text • 613k • Updated • 8 -
dmis-lab/Qwen3-VL-8B-Instruct-MRPO
Image-Text-to-Text • 770k • Updated • 22 -
dmis-lab/Qwen2.5-VL-7B-Instruct-MRPO
Image-Text-to-Text • 849k • Updated • 26
A collection of ablation and final models trained on the Outlier-Safe Pre-Training (OSP) framework.
This collection hosts Med-PRM series introduced in paper, Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards
-
dmis-lab/llama-3.1-medprm-reward-v1.0
Text Generation • 8B • Updated • 16.6k • • 17 -
dmis-lab/llama-3.1-medprm-reward-raw-training-set
Viewer • Updated • 11.7k • 41 -
dmis-lab/llama-3.1-medprm-reward-training-set
Viewer • Updated • 11.7k • 175 • 12 -
dmis-lab/llama-3.1-medprm-reward-raw-test-set
Viewer • Updated • 5.47k • 22
This collection hosts Meerkat series introduced in paper, Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks.
-
dmis-lab/meerkat-7b-v1.0
Text Generation • 7B • Updated • 70 • 29 -
dmis-lab/llama-3-meerkat-8b-v1.0
Text Generation • 8B • Updated • 684 • • 10 -
dmis-lab/llama-3-meerkat-70b-v1.0
Text Generation • 71B • Updated • 60 • • 6 -
Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks
Paper • 2404.00376 • Published • 5
This collection hosts ANGEL series introduced in paper, Learning from Negative Samples in Generative Biomedical Entity Linking.
This collection hosts models introduced in OLAPH: Improving Factuality in Biomedical Long-form Question Answering.
This collection hosts models of Self-BioRAG (ISMB 2024), improving medical reasoning through retrieval and self-reflection.
This collection hosts Phrase-reranker models introduced in TouR (ACL 2023 Findings), optimizing test-time query representations for dense retrieval.
This collection hosts BioSyn (ACL 2020) series, for learning representations of biomedical entities based on their synonyms.
This collection hosts BioBERT (Bioinformatics 2020) series, a domain-specific adaptation of BERT pre-trained on biomedical corpora.