Understanding the abilities of LLM s to reason about natural language plans, such as instructional text and recipes, is critical to reliably using them in decision-making systems. A fundamental aspect of plans is the temporal order in which their steps needs to be executed, which reflects the underlying causal dependencies between them.
We introduce CaT-Bench , a benchmark of Step Order Prediction questions, which test whether a step must necessarily occur before or after another in cooking recipe plans. We use this to evaluate how well frontier LLM s understand causal and temporal dependencies. While prompting for explanations and using few-shot examples improve performance, the best F1 result is only 0.
Further, human evaluation of explanations along with answer correctness show that, on average, humans do not agree with model reasoning. Surprisingly, we also find that explaining after answering leads to better performance than normal chain-of-thought prompting, and LLM answers are not consistent across questions about the same step pairs.
Planning is central to decision making and has been studied in various domains such as robotics and embodied environments LaValle ; Jiang et al. To follow, revise, or customize a plan, one must be able to reason about the steps involved as well as their causes and effects Brahman et al. However, real-world natural language plans cannot be executed to test for correctness and reliability. This paper describes a new question-driven evaluation to better study the detailed causal and temporal connections within such plans.
Given a plan, such as making a cake in Figure 1 , one must understand its various aspects to answer questions about it. Answering if ground almonds should be added before stirring the mixture requires understanding that a precondition for mixing evenly is that all ingredients should be added already.
