Auto71%Harbor: pendingCodingInstruction Following

NLC2CMD

The NLC2CMD Competition hosted at NeurIPS 2020 aimed to bring the power of natural language processing to the command line.

How to run

ltb · nlc2cmd
$ npm i -g lettokenburn-cli
$ ltb bench info nlc2cmd

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

1 sourced entry
HarnessSourceConfidenceLink
ChatGPT0.8N/AIndependentmediumFeb 15, 2023arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

The NLC2CMD Competition hosted at NeurIPS 2020 aimed to bring the power of natural language processing to the command line. Participants were tasked with building models that can transform descriptions of command line tasks in English to their Bash syntax.