3 Pull requests merged by 3 people

• installation.md update

  #1830 merged Jan 17, 2025

• [Docs] new_dataset.md update

  #1827 merged Jan 17, 2025

• [Feature] Add support for InternLM3

  #1829 merged Jan 16, 2025

4 Pull requests opened by 4 people

• fix bug of change_accelerator

  #1819 opened Jan 14, 2025

• [Update] Update LiveMathBench Hard Configs

  #1826 opened Jan 15, 2025

• [Refactor] Code refactoarization

  #1831 opened Jan 17, 2025

• [Fix]: Remove bos_token for vllm

  #1834 opened Jan 17, 2025

1 Issue closed by 1 person

• [Bug] OpenICLEval任务失败--ResourceWarning: unclosed file

  #1835 closed Jan 18, 2025

7 Issues opened by 6 people

• [Bug] 无法使用多卡评测

  #1828 opened Jan 16, 2025

• [Bug] 使用leval测评时报错，求助

  #1825 opened Jan 15, 2025

• [Bug] longbench报错

  #1824 opened Jan 15, 2025

• [Bug] lazy module import error

  #1821 opened Jan 15, 2025

• [Feature] 请问是否能有一个汇总，就是opencompass包含的所有评测数据集、这些数据集的metrics和链接(paper/github)？

  #1820 opened Jan 14, 2025

• [Bug] 部分样本上opencompass的评测方法是否可能有点片面？

  #1818 opened Jan 14, 2025

• [Bug] Unable to load model offline

  #1817 opened Jan 13, 2025

12 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Update support of Yayi API

  #1811 commented on Jan 17, 2025 • 1 new comment

• [Bug] 自定义模型评测配置

  #1746 commented on Jan 13, 2025 • 0 new comments

• [Bug] 使用vllm加速时无法启动多卡

  #1810 commented on Jan 13, 2025 • 0 new comments

• [Feature] dataset for humaneval-multipl

  #1673 commented on Jan 13, 2025 • 0 new comments

• [Feature] 请问使用vllm评测时怎么实现类似HF多卡数据并行？

  #1002 commented on Jan 14, 2025 • 0 new comments

• [Bug] hf推理与vllm推理评测结果不一致

  #1594 commented on Jan 15, 2025 • 0 new comments

• [Bug] The evaluation of configurations requiring a TopKRetriever, such as for flores_datasets, failed with a KeyError: 'metadata'.

  #1686 commented on Jan 16, 2025 • 0 new comments

• [Bug] commonsenseqa_gen_c946f2中的commonsenseqa_datasets数据集报错 TypeError: 'ListWrapper' object is not iterable

  #1760 commented on Jan 16, 2025 • 0 new comments

• [Feature] 是否打算支持mamba或者mamba2的测评吗

  #1778 commented on Jan 16, 2025 • 0 new comments

• [Bug] 无法多卡并行评测数据

  #1755 commented on Jan 17, 2025 • 0 new comments

• [Bug] 推理完成了，但是因为插件版本，在做测评的时候报错了，但是预测的哪些文件都在，怎么重新继续做预测？还有就是运行的时候默认好像是多有的数据集都会测试

  #1761 commented on Jan 17, 2025 • 0 new comments

• [Feature] Support downloading dataset from OpenMind

  #1792 commented on Jan 17, 2025 • 0 new comments