Build your own social listening tool in a weekend, without paying for one.

Last weekend I wanted to know one thing: what are people on social media saying about the brand I look after in the accounting software space, and how does that compare to competitors? Until then I had been checking by hand, now and again, which is tiring and never thorough. So I figured I would wire up something to listen for me every morning.

I started Saturday afternoon. By Sunday evening, relevant posts and reviews were dropping into a chat channel on their own. This is the part that surprises people: you can wire this up that fast? The answer is yes. Connecting scrapers to an AI is easier than it looks.

But after letting it run on its own for a while, I saw the real lesson. The hard part is not the scraper. Anyone can wire that up in a weekend. The part that takes the time, and makes it trustworthy enough to leave running every morning, is five things people tend to skip, because none of them show up when you run it once to impress someone. They are what has to work when everything is not going to plan. So this guide covers both: how to wire it up, and the five things that make it reliable.

Part 1A weekend is enough to wire it, and that is the trap

At its core, social listening is only three pieces: fetch what people say, filter for what is about you, then send what matters to whoever needs to know. Almost all of it you can assemble from off-the-shelf parts.

Free sources go further than you think

There are more free sources than most people expect. I wired these first, no money and no API keys to beg for:

• News and blogs via Google News RSS, with both Thai and English queries. Free, no key needed.
• Hacker News the free Algolia Search API covers the archive, good for the tech crowd.
• YouTube yt-dlp (the ytsearch command) pulls videos that mention a topic.
• App store reviews the App Store publishes an RSS feed of reviews (via iTunes). This is real customer voice, praising or complaining about the app itself.
• Search trends Google Trends, to see how often people search your brand versus a competitor.
• Forums like Pantip hitting them directly tends to get blocked, so I have Claude search them (site:pantip.com) in the weekly deep pass instead.

The ones people want most are usually X (formerly Twitter) and TikTok. Those cost money, but you do not need the eye-watering official API. I use Apify, which has ready-made pay-per-result actors: for X, kaitoeasyapi at about cents per thousand posts; for TikTok, clockworks/tiktok-scraper at a similar price. Cheap enough to wire in without pain. One thing to watch: some actors block free-plan accounts silently, more on that in point 2.

Let Claude do the filtering

Raw fetched data is a mess. There are millions of posts with the word "accounting" in them, and only a sliver are actually about accounting software. So I send batches of about 40 to Claude (via claude -p) and get back structured JSON: is this relevant, what category, sentiment positive negative or neutral, how urgent, and what aspect it touches, price, feature, bug, or support. Then only the relevant and important items go to the team's chat channel through a Discord webhook.

All of this really does come together in a weekend, and seeing the first results land in chat feels great. But that good feeling is the trap. A tool that runs once and works makes you trust it. And a social listening tool does not do its work while you watch, it works every morning while you are away. That is exactly when it breaks quietly.

Part 2Five things you must get right, or the system lies to you

These five are what I had to add after the scraper already worked. Each comes from a moment where it nearly broke, or actually did. Call them guardrails: the rails that keep the system from going over the edge while it runs unattended.

1. Money needs a ceiling it cannot cross

The moment a system has a connector that can spend money, it can spend money on your behalf. Picture a bug that loops the fetch forever, or a query so broad it returns hundreds of thousands of results. Money leaves at 3am and no one sees it.

The guardrail I set: the scheduler is forbidden from touching anything that costs money. A paid connector runs only when I trigger it by hand, with a key. Anything that can run automatically has a hard monthly cap per brand, a wall, not a warning: hit the cap and it stops. And every cent spent is logged to its own ledger so I can see later where it went. Anything that can spend money, without a ceiling, will one day teach you an expensive lesson.

2. An empty output does not mean nobody is talking

Wiring up X, I hit a classic trap. One scraper returned "no results," which looks exactly like nobody mentioning you at all. But when I opened the run log, it had actually been blocked because the account was on a free plan. Not silence, a block.

The difference is dangerous precisely because the two look identical. Zero results has two meanings: genuinely quiet, or broken. If you cannot tell them apart, you sit there relieved there is no drama, while the feed has been dead for days. The guardrail: always read the run log, never just the output, and swap out any scraper that likes to return ambiguous results for one that says plainly whether it succeeded or failed.

3. Sending to the wrong channel is worse than not sending

I track more than one brand, and each routes to its own channel. The easy mistake is a misconfigured destination, brand A's mentions showing up in brand B's channel. If outsiders are in that channel, now you have a problem.

My guardrail: if the brand's destination cannot be found, stop, do not guess. There is no fallback to a shared channel "just in case," because a shared channel with the wrong brand is a leak. A system that does not know where to send must stay silent, not send anywhere. A tool that puts one brand's voice in another's channel is worse than no tool, because it creates damage that no tool at all would not.

4. A half-written file means lost data

Early on I silently lost old data by overwriting a file carelessly. When the script stumbled midway, the file was left half-written, and days of collected data vanished.

This guardrail is basic for anything that runs itself, yet often skipped. Write to a temporary file first, then rename it over the real one once the write is complete. If it dies midway, it dies on the temp file and the real one is untouched. And when deciding what is new versus old, count from when the item entered the system, not when the post was written, otherwise a backfill will clobber newer data.

5. The filter has to prove it filters

The heart of the system is the AI that decides what is relevant and urgent. But a filter you never test is a filter you do not know is working. The day it says zero urgent items, how do you know today was actually calm, and not that the filter died quietly?

The fix is a positive control. Feed in samples you already know the answer to. Slip in a scorching complaint that should be flagged urgent, and a post that is completely irrelevant. If the filter catches the hot one and drops the irrelevant one, it is still good. If it misses, the pipe is broken, not quiet. A zero you can trust comes from a filter that just proved it catches the right things.

Part 3Build or buy, and what to remember

When to buy, when to build

Off-the-shelf tools like Brandwatch, Sprinklr, or Sprout Social came first and do the job well. If you are a large org, need many languages, share across teams, have people to run it, and need a guarantee it will not go down, buying wins by a mile, no building the guardrails yourself, they are already built in.

But if your scope is narrow, one or two brands in a specific space, the budget is small, and you have someone who can own the guardrails, building it yourself gets you around 80% of the expensive tool at a fraction of the cost. The important condition is the last one: you have to be able to own the guardrails, not just wire up a scraper that runs.

What to remember

If you take one thing from this guide, make it this. Anyone can wire the scraper in a weekend; what makes it trustworthy is the five things above: a spend ceiling, reading the log and not just the output, never routing to the wrong channel, never losing data on a write, and a positive-controlled filter. Get those five in place, then leave it running, because those are the parts that work when you are not watching.

Where to start

You do not need every source on day one. Start small, and put the guardrails in from the first day.

1. Wire one free connector first. Google News RSS alone is enough to see the shape.
2. Have the AI filter and categorize, then positive-control it immediately: feed in a known sample and check it catches it.
3. Add paid sources next, and set the spend ceiling before you turn them on, not after the bill arrives.
4. Let it run on a schedule, but check the log every time to know whether an empty output is quiet or broken.

That gets you a tool that really does listen to social for you. And more importantly, a tool you can believe when it tells you there is nothing to see.