What is CoT (Chain-of-Thought) Reasoning in AI?

Large Language Models (LLMs) are cool and all, but tasks that require multi-step reasoning can often stump them.

A workaround?

Chain-of-thought prompting, aka guiding AI reasoning with sequential steps – let’s explore.

What is CoT Reasoning? (Guiding AI’s Thought Process)

Chain-of-Thought (CoT) reasoning is a thought-prompting technique designed to encourage an LLM to generate a series of intermediate reasoning steps before providing a final output. Think of it as asking the AI model to “show its work,” moving beyond standard prompting that offers only a direct, no-context answer.

To encourage this thought process, a CoT prompt often includes phrases like “Show me how you landed on that answer in steps,” “Walk me through your reasoning,” or “In your output, describe your reasoning in steps.” What you’re achieving here is to guide the language model not just to a final solution, but to reveal its entire reasoning process.

With this approach, the steps the LLM took to land on its answer aren’t hidden from you. So you can read through, understand, and identify any pitfalls of an LLM’s reasoning, and make any tweaks or suggestions in future prompts for a better model response.

Why CoT Matters: Limiting LLM Errors and Improving Outputs

The value of Chain-of-Thought (CoT) reasoning becomes clear when we look at how it directly addresses the limitations of LLMs.

Addressing LLM Limitations (Hallucinations and Errors)

One of the biggest frustrations for anyone working with LLMs is their tendency towards “hallucinations,” that is to say, confidently presenting incorrect or fabricated information, or providing flawed answers to genuinely complex tasks.

Hallucinations often stem from a lack of transparency in their internal reasoning process, leading to a kind of misalignment between the prompt and an accurate output.

Here’s where CoT does its thing: by explicitly instructing the AI model to break down problems into multiple steps, we are encouraging it to follow a more structured, verifiable path. The benefit? CoT significantly reduces the likelihood of errors, especially in intricate multi-step reasoning scenarios where a direct answer would be prone to logical leaps or factual inaccuracies.

Unlocking Advanced Reasoning Capabilities

Beyond simply reducing errors, chain-of-thought prompting improves the reasoning capabilities of large language models. The technique enables LLMs to move beyond simple responses and engage in more complex forms of reasoning.

We see improvements in areas like:

• Logical Reasoning: The ability to follow a deductive or inductive sequence of arguments.
• Commonsense Reasoning: Applying basic real-world knowledge and intuition to problem-solving.
• Symbolic Reasoning Tasks: Handling abstract concepts, mathematical operations, or coding logic.

TL;DR: Chain-of-thought prompting allows LLMs to tackle complex reasoning tasks that were previously out of reach for a single, direct prompt.

How to Use CoT Reasoning: Chain-of-Thought Prompting Techniques

Now that you’ve wrapped your head around the concept of CoT, allow me to show you how you can tweak your prompts and guide your LLM of choice toward a more structured thinking process.

Zero-Shot CoT Prompting

The simplest and often most effective form of chain of thought prompting, known as zero-shot CoT, is remarkably intuitive: just ask for the steps! As I touched on earlier, simply add a thought prompt phrase like “Let’s think step by step,” “Walk me through your reasoning,” or “Show your work,” and you’ll guide the AI model to generate intermediate reasoning steps before it delivers the final answer.

So instead of prompting like this:

“Tim bought 10 apples. He gave 2 apples to his wife and 2 to his daughter. He then went and bought 5 more apples and ate 1. How many apples did he have left?”

You’d prompt like this:

“Tim bought 10 apples. He gave 2 apples to his wife and 2 to his daughter. He then went and bought 5 more apples and ate 1. How many apples did he have left? Walk me through your reasoning.”

Easy-peasy.

Few-Shot CoT Prompting

While the “step-by-step” cue is effective, Few-Shot CoT Prompting takes it a level further. The Few-Shot technique involves providing the AI model with a few complete examples of input-output pairs that include their intermediate reasoning process.

Few-Shot works by showing the CoT model exactly how you want it to “think.” That way, you provide a clear template for its reasoning path on similar new tasks, which helps the model generalize the desired problem-solving approach, helping it to land on an accurate answer.

Here’s what a Few-Shot prompt might look like:

“Q: Tim has 5 apples. He buys 3 bags of apples, and each bag has 2 apples. How many apples does Tim have now?

A: Tim starts with 5 apples. He buys 3 bags * 2 apples each = 6 apples. Total apples = 5 + 6 = 11 apples. Tim has 11 apples now.

Q: Sarah has 10 books. She gives away 4 books, then buys 2 more. How many books does Sarah have now?”

Notice how the prompt posed a question, then not only provided the answer but also the reasoning behind it. The reasoning was then followed by a problem that you wish the LLM to solve. Ideally, the LLM will follow your lead and use the provided logic to tell you how many books Sarah has left.

Automatic Chain-of-Thought (Auto-CoT)

While Chain-of-Thought prompting with demonstrations (Few-Shot CoT) is, no doubt, effective, manually hand-crafting examples can be time-consuming and prone to suboptimal choices. Automatic Chain-of-Thought (Auto-CoT) is a clever approach designed to eliminate that manual effort.

Auto-CoT uses the capabilities of large language models (LLMs) themselves to automatically generate reasoning chains for demonstrations. The goal is to develop a comprehensive set of self-generated examples that can be used in a few-shot context.

The method involves two main stages:

1. Question Clustering: The initial step involves partitioning a given dataset of questions into a few diverse clusters. The aim here is to demonstrate to the LLM what you’d like its own examples (which it will generate for you) to look like and what reasoning you’d like them to follow.
2. Demonstration Sampling and Generation: From each cluster, a representative question is selected. Then, using Zero-Shot Chain-of-Thought (by adding a simple “Let’s think step by step” prompt), the LLM is instructed to generate its own intermediate reasoning steps and final answer for that question. To ensure these automatically generated demonstrations are high-quality, simple heuristics are often applied – for instance, favoring examples with a concise question length and a manageable number of reasoning steps.

The question cluster will look something like this:

“Q: Tim has 5 apples. He buys 3 bags of apples, and each bag has 2 apples. How many apples does Tim have now?

A: Tim starts with 5 apples. He buys 3 bags * 2 apples each = 6 apples. Total apples = 5 + 6 = 11 apples. Tim has 11 apples now.

Q: Sarah has 10 books. She gives away 4 books, then buys 2 more. How many books does Sarah have now?

A: Sarah starts with 10 books. She gives 4 away, so 10 – 4 = 6 books left. She then buys 2 more books, so 6 + 2 = 8 books. Sarah has 8 books now.”

You’ll then take your examples and give them to the LLM, asking it to generate more like them. Once you have a decent amount, let’s say five or so, you’ll use them within your prompts, including the zero-shot technique.

So, your final prompt might look like:

“Q: Tim has 5 apples. He buys 3 bags of apples, and each bag has 2 apples. How many apples does Tim have now?

A: Walk me through your reasoning. Tim starts with 5 apples. He buys 3 bags * 2 apples each = 6 apples. Total apples = 5 + 6 = 11 apples. Tim has 11 apples now.

Q: Sarah has 10 books. She gives away 4 books, then buys 2 more. How many books does Sarah have now?

A: Walk me through your reasoning. Sarah starts with 10 books. She gives 4 away, so 10 – 4 = 6 books left. She then buys 2 more books, so 6 + 2 = 8 books. Sarah has 8 books now.

Q: Lisa had $25. She spent $12 on lunch and then earned $15 from a friend for helping out. How much money does Lisa have now?

A: Walk me through your reasoning. Lisa starts with $25. She spent $12 on lunch, so $25 – $12 = $13 left. She then earned $15, so $13 + $15 = $28. Lisa has $28 now.

Q: A gardener planted 15 rose bushes. She planted 7 more on Monday and 5 on Tuesday. How many rose bushes has she planted in total?

A: Walk me through your reasoning. The gardener started with 15 rose bushes. She planted 7 more on Monday, so 15 + 7 = 22 rose bushes. She planted 5 more on Tuesday, so 22 + 5 = 27 rose bushes. The gardener has planted 27 rose bushes in total.

Q: A bus started with 30 passengers. At the first stop, 12 passengers got off, and 7 got on. How many passengers are on the bus now?

A: Walk me through your reasoning.”

The aim of the above prompt is to encourage the LLM to answer the last question using the reasoning we used in the examples before it. The answer is 25, by the way. The bus now has 25 passengers.

Conclusion and Next Steps

One more time for those in the back: CoT is a prompting technique that changes how large language models approach problems, guiding them to perform complex reasoning by breaking down challenges into intermediate reasoning steps.

From arithmetic to strategic planning, CoT transforms AI from a black box to a transparent, logical problem-solver. Take it and with it what you will. Have fun!