How to spot it
It names itself in every request it makes:
PrestophonicBot/1.0 (+https://prestophonic.com/bot)
What it reads
The public pages of venues and event lists that list open mics, jam sessions, songwriter rounds, poetry and comedy nights around Boston, to see whether a night is still on, and on which day and at what time. It reads each page about once a week, one page at a time.
It never signs in, never fills in a form, and never reads Facebook or Instagram.
How it behaves
- It follows your
robots.txt: the rules for PrestophonicBot if you name it, or else the ones for every crawler. It asks forrobots.txtagain each day. - It waits 2 seconds at least between two requests to your site, and longer if your
robots.txtasks for it withCrawl-delay. - It asks for a page only if it changed since it last read it (
If-None-MatchandIf-Modified-Since). - It reads web pages alone, 2 MB of one at most.
What we keep
The facts of a night: its day and time, where it is, and how sign-up works. Never your wording or your photos. Every listing is written in our own words, and says where the night is listed.
Keeping it out
Add this to your site's robots.txt, and it reads nothing more of your site from the next day on:
User-agent: PrestophonicBot Disallow: /
Questions, or a night we have wrong? Email info@prestophonic.com.