Can AI summarize hundreds of product reviews?
Yes. Paste a mix of good and bad reviews and it groups them into themes with counts, so you see the real pattern instead of the loudest review.
01Why bot
- Summarizing and grouping text is one of AI's core skills.
- Counting how often a problem appears is more useful than star ratings.
- It only knows the reviews you paste, so include some negative ones.
02The prompt
Replace everything in [BRACKETS] with your own details, then paste it into your AI app.
You are a consumer researcher who reads reviews for patterns. Product: [PRODUCT] Reviews (paste a mix of good and bad): [REVIEWS] Your task: summarize what these reviews say. Rules: - MUST group praise and complaints into themes and count how many reviews mention each. - MUST flag reviews that look fake or off-topic and say why. - NEVER add opinions that aren't in the reviews. Give me: a themes table (theme, how many, good or bad) and a 3-line verdict. Before you send your answer, silently check that you did ALL of these, and fix your answer if not: - Group the points into Praise and Complaints, each with a count of how many reviews mentioned it. - Flag any review that looks fake or off-topic, with a one-line reason. - Use only what the reviews say; add no opinions or facts of your own. - End with a verdict in exactly three lines.
03Steps
- Copy 20 to 50 reviews, including 1-star and 3-star ones.
- Paste them into the prompt.
- Read the themes, then a few reviews on the top complaint.
04What can go wrong
- Garbage in, garbage out: a biased sample gives a biased summary.
- Very long pastes may get cut off; split them if needed.
05What to use
06Our test runs
2 runs, each with a different made-up sample. 0 of 2 passed every check.
The made-up sample:
[PRODUCT]EcoBrew 12‑Cup Programmable Coffee Maker – stainless steel, Wi‑Fi enabled, with auto‑brew timer and reusable filter.[REVIEWS]I love the EcoBrew! It brews quickly, stays hot for hours, and the app lets me start it from bed. The reusable filter is a nice eco touch. The coffee is great, but the water reservoir is hard to lift when full. Also, the Wi‑Fi sometimes drops. Excellent build quality, sleek look. The programmable schedule works perfectly. No more early‑morning trips to the kitchen. Disappointed: the machine leaks from the base after a week. Customer service was slow to respond. Makes decent coffee but the grinder noise is louder than expected. Still, the timer feature saves me time. Fake review? This product changed my life, I never needed coffee before. Best purchase ever!!! I bought it for my office, but the 12‑cup size is too big for a single person. It takes up too much counter space. Off‑topic: The delivery driver was very friendly and offered me a free coffee sample.
- Group praise and complaints into themes with mention counts
- Flag fake or off‑topic reviews with a reason
- Verdict must be exactly three lines
- Never add opinions not in the reviews
- Made up details we never gave it
Details it made up: “Overall, the product has both positive and negative aspects, with 6 reviews mentioning good points and 5 reviews mentioning bad points.”
What came back (excerpt):
Bad | | Water reservoir design | 1 | Bad | | Leaks | 1 | Bad | | Noise | 1 | Bad | | Size | 1 | Bad | | Customer service | 1 | Bad | The following reviews look fake or off-topic: - "This product changed my life, I never needed coffee before.
Read the full answer
Here is the themes table: | Theme | Count | Good/Bad | | --- | --- | --- | | Brew quality/taste | 3 | Good | | Build quality/design | 2 | Good | | Programmable features | 2 | Good | | Eco-friendly features | 1 | Good | | Wi-Fi connectivity | 1 | Bad | | Water reservoir design | 1 | Bad | | Leaks | 1 | Bad | | Noise | 1 | Bad | | Size | 1 | Bad | | Customer service | 1 | Bad | The following reviews look fake or off-topic: - "This product changed my life, I never needed coffee before. Best purchase ever!!!" (looks fake because it's an exaggerated claim) - "The delivery driver was very friendly and offered me a free coffee sample." (off-topic because it's about the delivery driver, not the product) Verdict: The EcoBrew 12-Cup Programmable Coffee Maker receives praise for its brew quality, build, and programmable features. However, some reviewers mention issues with the water reservoir, Wi-Fi connectivity, and leaks. Overall, the product has both positive and negative aspects, with 6 reviews mentioning good points and 5 reviews mentioning bad points.
Ensure theme counts and the three‑line verdict follow the prompt precisely.
The made-up sample (a messier one):
[PRODUCT]EcoBrew 12-Cup Programmable Coffee Maker with built-in grinder and smart app control.[REVIEWS]The grinder works great, coffee is fresh and the app lets me set a brew time. Very convenient. I hate the water reservoir; it's too small and you have to refill it every half hour. Setup was a nightmare, the manual was confusing and the machine kept beeping for no reason. Love the sleek design, fits nicely on my counter, and the programmable timer saved me money on coffee runs. The plastic feels cheap and after a month the brew temperature dropped noticeably. Best coffee maker I've owned! Best purchase ever!!! (looks fake) but I really like it. The app crashes often, can't schedule brew on weekends, need better stability. I ordered a black model but got silver; still works fine though the color mismatch is annoying. Delivery driver was friendly, but the coffee tasted fine. Noise level is high, like a blender, wakes up the whole house early morning.
- Must group praise and complaints into themes and count mentions
- Must flag fake or off‑topic reviews with a one‑line reason
- Never add opinions or facts not in the reviews
- End with a verdict in exactly three lines
- Made up details we never gave it
Details it made up: “Convenient features | 3”, “Design and aesthetics | 2”, “Performance issues | 4”, “Setup and user experience | 2”, “Quality and durability | 2”
What came back (excerpt):
Verdict: The EcoBrew 12-Cup Programmable Coffee Maker receives praise for its convenient features and design. However, it is criticized for performance issues, setup problems, and quality concerns.
Read the full answer
Here is the themes table: | Theme | Count | Good/Bad | | --- | --- | --- | | Convenient features | 3 | Good | | Design and aesthetics | 2 | Good | | Performance issues | 4 | Bad | | Setup and user experience | 2 | Bad | | Quality and durability | 2 | Bad | | App functionality | 2 | Bad | | Delivery and packaging | 1 | Bad | The review "Best coffee maker I've owned! Best purchase ever!!! " looks fake because it uses overly enthusiastic language. The review "Delivery driver was friendly, but the coffee tasted fine." looks off-topic because it mentions the delivery driver. Verdict: The EcoBrew 12-Cup Programmable Coffee Maker receives praise for its convenient features and design. However, it is criticized for performance issues, setup problems, and quality concerns. Overall, the product has both positive and negative aspects, with 7 reviews mentioning problems and 5 reviews mentioning benefits.
Watch for inaccurate theme counts and added summary statements.
How each run works: we fill the prompt with a made-up sample, run it once in Llama 3.3 70B, and GPT-OSS 120B grades the answer against the prompt's own rules. Word limits are counted by code, and every quote flagged as made up is checked against the full answer. Your results can differ. How we test.