KicksWorthyBot

How to identify us

Almost every request we make carries this User-Agent:

KicksWorthyBot/1.0 (+https://kicksworthy.com/bot; price comparison)

What we fetch

• Public product endpoints only. Where a store publishes a machine-readable catalogue, we read that instead of scraping pages. On Shopify stores that means /products.json.
• One request per store per day in the normal case. We fetch a whole catalogue once rather than one request per product, which is fewer requests, not more.
• Prices, stock status and product names. Nothing else. We do not submit forms, follow checkout flows, create accounts, or touch anything behind a login.

When we run

Once daily, during off-peak hours for Indian e-commerce. Requests are spaced with a delay and randomised jitter.

How we behave

• We read and honour robots.txt.
• If we are blocked, we back off. We do not retry harder, rotate addresses, or hunt for a way around a block. A store that blocks us stays blocked. The single exception is the User-Agent filter described above, which we disclose rather than hide.
• After repeated failures we deactivate the listing automatically rather than keep requesting a dead URL.

How to block us

Add this to your robots.txt and we will stop:

User-agent: KicksWorthyBot
Disallow: /

You do not need to explain why, and you do not need to reply to us first.

Because of the exception above, a rule aimed at our name is not enough on its own everywhere. So we also honour path rules on every site regardless of which User-Agent we sent — a Disallow under User-agent: * covering the pages we read stops us, and so does an email.

Talk to us instead

If something we do is causing you trouble, email kicksworthy@gmail.com. We would much rather adjust our behaviour than be blocked. We can change frequency, narrow what we fetch, or remove your store entirely.

If you would prefer we use an affiliate feed or an API you already publish, tell us and we will switch to it.