Searching Reddit for a product name returns far more than customer complaints. You'll find recommendations, feature questions, jokes, comparisons, and posts that have nothing to do with the product. If every mention becomes an alert, the useful feedback gets buried.
A better approach is to collect relevant conversations, identify possible complaints, and give uncertain cases a place to go before anyone treats them as customer feedback.
In this guide, we'll use Spidra to find Reddit discussions, organize the results, and build a small rules-based classifier that produces a reviewable feed. We'll also look at where the classifier gets things wrong and how to set up ongoing monitoring.
Before collecting data: Reddit restricts automated collection and requires a separate agreement for commercial Data API use. Make sure you have permission for your collection method and intended use before automating it. The classification workflow below also works with Reddit records you already have the right to use. See Reddit's User Agreement, Data API Terms, and our web scraping legal guide.
Start with searches that reveal actual problems
A general search such as Notion or Calendly gives you a useful baseline, but many of the results won't involve dissatisfaction. Add searches that describe a particular experience:
Notion too expensivefor pricing and value discussionscancelled Notionfor accounts of leaving the productswitched from Calendlyfor migration experiencesCalendly not workingfor possible reliability problems
These are discovery queries, not complaint labels. Someone asking whether to switch products isn't necessarily unhappy with the one they use, and a question about an outage isn't proof of an outage.
Spidra provides a Reddit search template that takes search queries and returns posts with their titles, subreddit names, dates, links, and available engagement details. You can also use the Reddit subreddit posts template, sorted by New, to catch discussions that don't use the objection phrases you expected.
If your collection is authorized, the Python SDK can run a search template and save its rows in a consistent shape:
# collect_mentions.py
"""Example for collections permitted under Reddit's applicable terms."""
import json
import os
from datetime import datetime, timezone
from pathlib import Path
from spidra import Spidra
client = Spidra(api_key=os.environ["SPIDRA_API_KEY"])
run = client.run_template("reddit-search", {
"queries": ["Notion too expensive", "cancelled Notion"],
"sort": "new",
})
if run.status not in {"completed", "partial"}:
raise RuntimeError(f"Reddit collection did not complete: {run.status}")
captured_at = datetime.now(timezone.utc).isoformat()
mentions = [
{"product": "Notion", "title": row.get("title", ""),
"text": row.get("text", ""), "url": row.get("url") or row.get("link"),
"subreddit": row.get("subreddit"),
"publishedAt": row.get("posted") or row.get("publishedAt"),
"capturedAt": captured_at}
for row in run.rows
]
Path("mentions.json").write_text(json.dumps(mentions, indent=2), encoding="utf-8")The script deliberately doesn't interpret empty results as evidence that a product has no complaints. Check the run status and any reported failures before trusting the collection. Search results are primarily for discovery; when a post needs closer examination, the Reddit post details and Reddit post comments templates return more context.
If you've already followed our guide to finding product ideas on Reddit, the collection step will look familiar. Here the goal is different: we want to distinguish negative experiences about a known product from every other conversation mentioning it.
Give every mention a consistent record
Before classifying anything, decide which details you'll keep. A useful record contains the product name, post title, optional body text, subreddit, original URL, publication date, and collection time. Keep engagement numbers separately from the complaint decision.
The classifier adds a status (candidate, review, no_signal, or off_topic) and, when the text supports it, a theme such as pricing, billing, reliability, access, or churn.
A post's URL also works as a deduplication key. The same discussion may appear in searches for too expensive, alternative, and cancelled, but it should be one piece of evidence in your feedback feed, not three complaints.
Don't infer a severity level from upvotes or the title alone. A popular discussion might be a joke, and a detailed complaint with few votes can still describe a problem worth investigating.
Separate possible complaints from ordinary mentions
For a first pass, you don't need a complicated sentiment model. Start with rules you can inspect, then measure how they perform against manually reviewed posts.
The script below labels a post as a candidate only when it finds a stronger complaint phrase, and sends ambiguous terms such as down or scam to review. It also checks for the product name and deduplicates posts by their Reddit IDs when a URL is available. Save it as complaints.py:
# complaints.py
"""A small, inspectable classifier for Reddit mentions you may use."""
import json
import re
import sys
from pathlib import Path
from urllib.parse import urlsplit
PHRASES = {
"access": ["locked out", "can't access", "unable to access"],
"billing": ["charged twice", "wrong charge", "unexpected bill"],
"reliability": ["not working", "keeps crashing", "lost my data"],
"pricing": ["too expensive", "not worth it"],
"churn": ["regret ever using", "cancelled because"],
}
REVIEW = ["scam", "down", "alternative", "expensive", "confusing"]
def classify(row):
text = " ".join(str(row.get(k) or "") for k in ("title", "text")).casefold()
names = [row.get("product", ""), *(row.get("aliases") or [])]
on_topic = any(re.search(r"(?<!\w)" + re.escape(str(name).casefold()) + r"(?!\w)", text)
for name in names if name)
if not on_topic:
return "off_topic", None, None
for theme, phrases in PHRASES.items():
for phrase in phrases:
if phrase in text:
return "candidate", theme, phrase
for term in REVIEW:
if re.search(r"\b" + re.escape(term) + r"\b", text):
return "review", None, term
return "no_signal", None, None
def identity(row):
url = str(row.get("url") or row.get("link") or "")
match = re.search(r"/comments/([a-z0-9]+)", urlsplit(url).path, re.I)
return match.group(1).lower() if match else (url.rstrip("/") or (str(row.get("product", "")) + ":" + str(row.get("title", ""))).casefold())
def main(path):
items = json.loads(Path(path).read_text(encoding="utf-8"))
if not isinstance(items, list):
raise ValueError("Input must be a JSON list")
seen, results = set(), []
for row in items:
key = identity(row)
if key in seen:
continue
seen.add(key)
status, theme, signal = classify(row)
results.append({"title": row.get("title"), "url": row.get("url") or row.get("link"),
"status": status, "theme": theme, "signal": signal})
Path("triaged_mentions.json").write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8")
candidates = sum(r["status"] == "candidate" for r in results)
review = sum(r["status"] == "review" for r in results)
print(f"Input: {len(items)} | Unique: {len(results)} | Candidates: {candidates} | Review: {review} | Other: {len(results) - candidates - review}")
if __name__ == "__main__":
main(sys.argv[1] if len(sys.argv) > 1 else "mentions.json")The script accepts a JSON array of mention records. To try it without accessing Reddit, save this small teaching fixture based on the November research as reddit_sample_mentions.json:
[
{
"product": "Notion",
"title": "Notion is probably the biggest productivity scam .",
"url": "https://www.reddit.com/r/Notion/comments/1vr22yp/notion_is_probably_the_biggest_productivity_scam/"
},
{
"product": "Notion",
"aliases": ["Notions"],
"title": "I was one of Notions biggest cheerleaders, now I regret ever using it.",
"url": "https://www.reddit.com/r/Notion/comments/1qhj2cs/i_was_one_of_notions_biggest_cheerleaders_now_i/"
},
{
"product": "Notion",
"title": "Notion down again??"
},
{
"product": "Notion",
"title": "worth switching from Apple Notes?"
},
{
"product": "Notion",
"title": "Notion is probably the biggest productivity scam .",
"url": "https://www.reddit.com/r/Notion/comments/1vr22yp/notion_is_probably_the_biggest_productivity_scam/"
}
]Run it locally:
python complaints.py reddit_sample_mentions.jsonThe local output is:
Input: 5 | Unique: 4 | Candidates: 1 | Review: 2 | Other: 1It also writes triaged_mentions.json, where each retained post has a status, an optional theme, the matched phrase (signal), and a source URL when one was supplied. The repeated first post appears only once. The off_topic and no_signal labels are retained in the output, rather than discarded.
The example is intentionally conservative. A candidate still needs a person to verify that the text describes a real experience, while a review item needs context before it can be promoted or dismissed. The script's labels are not the results of the original research classifier.
Measure the mistakes before using the feed
A small review of real posts shows why the human step matters. In a November 2026 collection covering Notion and Calendly, a rules-based classifier flagged 47 possible complaints. Manual checking confirmed 28, giving it 59.6% precision among flagged items.
That means roughly four in ten flagged posts were not confirmed complaints in that sample. Some were questions about product availability, prospective buyers asking about alternatives, or posts using negative phrases without describing a genuine problem with the product. Other complaints were missed because the author never used one of the phrases the rules looked for.
The point isn't that 59.6% is a benchmark for all complaint classifiers. It's the measured result of one phrase-based approach on one collection. A different product, subreddit, or vocabulary can change the outcome considerably.
To check your own filter, review every candidate in a small batch and record whether it really is a complaint. Then sample some of the review and no_signal posts to see what the filter misses. Change the rules based on actual mistakes, not on a handful of convincing examples.
A good feedback feed should preserve the original URL, the phrase that triggered the label, and the reviewer's final decision. That way, product and support teams can inspect the evidence instead of taking a sentiment label on trust.
Read the comments before deciding what happened
A post can describe the beginning of a problem without explaining how it ended. The comments and any updates may show whether other people experienced something similar, whether the author found a workaround, and whether the issue was resolved.
This is where Spidra's Reddit post details and Reddit post comments templates help with deeper investigation, when you have the necessary permissions. You can examine one relevant thread rather than opening every comment attached to every search result.
The key is to distinguish the original author's account, other users' experiences, and verified facts. Several commenters repeating the same claim aren't necessarily independent confirmation. Keep a link back to each source and summarize the issue in neutral terms.
A useful internal summary might describe the reported problem, who appears affected, any response or update, and what remains uncertain. That tells a team what needs investigation without turning a Reddit discussion into a definitive incident report.
Watch for new mentions without creating an alert for everything
Once the collection and review process works, you can turn a permitted recurring search into a scheduled check. Spidra Monitors can rerun templates, compare results with the previous check, and notify you when new items appear. A plain-language goal can also screen changes for relevance before an alert is sent.
For example, open Templates, run your permitted Reddit search, and choose Watch for changes. Select a schedule and New items as the trigger, then add a goal such as:
Only alert when a new post describes a firsthand problem with the named product. Ignore general recommendations, neutral questions, job posts, and unrelated discussions.
Treat that goal as another filter, not as proof that each alert is correct. When the goal can't make a decision, the Monitor can still send an alert, so a human review process remains necessary. In the November research, a complaint Monitor established its initial baseline, but the recorded checks did not demonstrate a successfully judged and delivered complaint alert. The monitored workflow therefore still needs a successful change-and-notification check before anyone relies on it operationally.
Spidra's Monitor documentation explains scheduling, notification channels, and how the first check establishes a baseline without sending an alert. Review failed checks as failures, not as days when nobody complained.
What a complaint feed can and cannot tell you
Reddit discussions are useful because they can reveal specific frustrations in the language people use themselves. They can help a team find issues to investigate and questions worth asking in customer interviews.
They do not tell you what percentage of all customers are dissatisfied. Reddit users are self-selected, and the same incident can appear across several posts and comments. A surge in mentions may reflect one widely shared story rather than a broad change in customer experience.
Use complaint monitoring to find evidence worth investigating, not as a replacement for support tickets, churn interviews, product analytics, or customer surveys. Keep the classification explainable, review uncertain items, and periodically check whether your source access and permissions still cover the workflow.
Use Spidra Templates to collect the fields you need, run the classifier against a small sample, and make the review queue useful before adding a Monitor.