CoQA
Conversational Question Answering Challenge
CoQA is a large-scale dataset for building Conversational Question Answering systems.
How to run
$ npm i -g lettokenburn-cli$ ltb bench pull coqa
External harness. pip install lm-eval && lm_eval --model <hf|vllm|openai-chat-completions> --model_args <model args> --tasks coqa (lm-evaluation-harness task family 'coqa') https://github.com/EleutherAI/lm-evaluation-harness
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
1 sourced entryEvery number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
CoQA is a large-scale dataset for building Conversational Question Answering systems. The goal of the CoQA challenge is to measure the ability of machines to understand a text passage and answer a series of interconnected questions that appear in a conversation.
Available in EleutherAI lm-evaluation-harness as task family coqa (1 tasks).