The goal of Natural-Instructions project is to provide a good quality benchmark for measuring generalization to unseen tasks. This generalization hinges upon (and benefits from) understanding and reasoning with natural language instructions that plainly and completely describe a task (traditionally defined as mapping an input string to an output string). Models equipped with "understanding" language instructions, should successfully solve any unseen task, if they are provided with the task instructions.