I Built an AI Agent That Monitors My Competitors 24/7
A technical walkthrough of how I built a competitor monitoring agent that tracks pricing changes, feature launches, and content strategy - running autonomously on a 5 EUR server.
The manual version was eating my mornings
Every Monday I used to spend 2-3 hours checking competitor websites. Had they changed their pricing? Launched a new feature? Published blog posts targeting keywords I cared about? Changed their positioning or messaging? The information was valuable, but the process of gathering it was tedious and repetitive.
Tedious and repetitive is exactly what automation is designed for. So I built an agent that does this work every night while I sleep, and surfaces only what actually matters.
What the agent does
The agent runs as a PM2 daemon on the same Hetzner VPS that hosts my products. Every day at 3 AM, it performs the following tasks:
Website change detection. For each monitored competitor (currently 8 across my three products), it fetches their pricing page, features page, and homepage. It compares the current version against the last stored snapshot using a semantic diff rather than a raw text diff. This means it ignores trivial changes (copyright year updates, minor CSS tweaks) and only flags meaningful content changes.
Blog monitoring. It checks competitor RSS feeds (or scrapes their blog index if no RSS exists) for new posts. When a new post appears, the agent summarizes it and identifies which keywords it targets. If a competitor publishes something targeting a keyword I am actively pursuing, that gets flagged as high priority.
Pricing intelligence. The agent maintains a structured record of each competitor’s pricing tiers, feature limits, and positioning. When it detects a change to any pricing element, it generates a comparison showing the old versus new structure and what the change likely signals strategically.
The technical implementation
The architecture is deliberately simple. I wanted something I could maintain in 30 minutes per month, not a complex distributed system.
Cron trigger (node-cron, 3 AM daily)
→ Fetch competitor pages (axios + cheerio for HTML parsing)
→ Generate semantic diff (LLM comparison of old vs new content)
→ Store new snapshot (SQLite, plain text + metadata)
→ If meaningful change detected:
→ Write analysis to Obsidian vault (Markdown file)
→ Append to alerts queue (JSON file)
→ Send Telegram notification (optional)The LLM comparison step is the key innovation over simple text diffing. When a competitor changes their homepage headline from “The fastest email platform” to “The most reliable email platform,” a text diff just shows two different strings. The LLM analysis identifies it as a positioning shift from speed to reliability, notes that this might indicate they received complaints about deliverability, and suggests monitoring their support channels for related discussions.
The total cost of running this agent is essentially zero beyond the server I already pay for. LLM calls cost a few cents per day in API usage.
What I learned from three months of data
After running the agent for three months across 8 competitors, some patterns became visible that I would never have noticed through occasional manual checking.
Competitors change pricing more often than you think. One competitor adjusted their pricing page five times in 90 days. Not dramatic changes (no tier restructures), but quiet adjustments to feature limits and usage caps. Without automated monitoring, I would have missed all of these.
Content cadence signals strategy shifts. When a competitor suddenly doubled their blog output from 2 posts per week to 4, it preceded a feature launch by about three weeks. The content was building awareness for the new capability. Seeing this pattern early gave me time to prepare a response.
Positioning drift is gradual. No competitor made a sudden pivot. However, looking at weekly snapshots side by side, the cumulative drift over three months was significant. One competitor shifted entirely from “developer-focused” messaging to “team-focused” messaging. Week to week it was invisible. Over the quarter it was unmistakable.
Practical considerations
False positives. The first version generated too many alerts because it flagged any text change as meaningful. Adding the LLM analysis step reduced false positives by roughly 80%. The remaining alerts are almost always worth reading.
Rate limiting yourself. Fetching competitor pages daily is fine. Fetching every hour would be aggressive and could trigger blocks. I keep the frequency at once per day, which is sufficient for strategic intelligence without being antisocial.
Storing history matters. The real value is not in any individual snapshot but in the trend over time. Keep everything. Storage is cheap. The query “how has competitor X’s pricing evolved over the last 6 months” is only answerable if you have the historical data.
Could you build this?
The technical requirements are modest. You need basic Node.js or Python skills, an LLM API key, a server that runs 24/7 (any cheap VPS works), and about 2-3 days of focused development time to get the first version running.
Start with one competitor and one page (their pricing page is the highest value target). Get that working reliably before expanding scope. The temptation is to monitor everything immediately, but quality monitoring of one competitor is more valuable than flaky monitoring of ten.
Building autonomous tools for solo founders. Updates on X.