ApplyWatchBot

This page is the public identity of the ApplyWatch crawler. It is the address named in the crawler's own User-Agent header.

Identity

  • Name: ApplyWatchBot
  • User-Agent: ApplyWatchBot/<version> (+<this page’s URL>). Each request carries the exact version in that header.
  • Contact page: https://myapplywatch.com/bot
  • Operated by: ApplyWatch operator

Why it visits

ApplyWatch shows students application facts taken from official university pages, with a link to the source. The crawler re-reads a small, explicitly approved set of those official pages so ApplyWatch can tell a student with a saved programme when a published deadline, fee or requirement changes. It reads only publicly available pages, collects no personal data from them, and never fills in or submits a form.

ApplyWatch is an independent service and is not affiliated with, endorsed by, or sponsored by the institutions listed.

How it behaves

  • It requests /robots.txt first and obeys it, following the ApplyWatchBot group and otherwise the * group. If a site’s robots file cannot be read, no page is fetched.
  • It honours Crawl-delay, and independently waits between requests to the same host, with at most one request to a host at a time.
  • It sends If-None-Match and If-Modified-Since, so an unchanged page is answered with a cheap 304 and no body.
  • It fetches HTTPS pages of approved hosts only, accepts HTML only, caps response size, and limits redirects and retries. It backs off on 429 and 5xx responses.
  • It does not log in, does not attempt paywalled or authenticated content, and does not crawl a site broadly.

How to block or limit it

Add a rule for the ApplyWatchBot user agent to your robots file. It takes effect on the next check, and a blocked page is simply left unread.

User-agent: ApplyWatchBot
Disallow: /

Contact

Questions about crawler traffic, or a request to be excluded: carmenmelodyyyy@gmail.com