ParticleNewsBot

ParticleNewsBot is Particle News's web crawler. ParticleNewsBot visits publicly accessible content to index information for our platform. We always attribute content to the original source and include links to the source material.

ParticleNewsBot will respect all requests to stop or limit crawling. You can request that ParticleNewsBot stop or limit crawling your website via robots.txt (the robots exclusion standard).

How ParticleNewsBot accesses your site

When ParticleNewsBot visits your website, it will send a valid Particle News user agent and connect from our cloud infrastructure.

ParticleNewsBot respects robots.txt and is configured to rate-limit concurrent requests made to your site. Rate limits are configurable per site to ensure responsible crawling. Do not hard code IP addresses in your configuration, because the addresses ParticleNewsBot uses can change without notice.

Our user agent is:

Mozilla/5.0 (compatible; ParticleNewsBot/1.0; +https://particle.news/bot)

ParticleNewsBot only crawls with US-based IP addresses.

Verify ParticleNewsBot

Legitimate ParticleNewsBot requests will:

• Include a valid ParticleNewsBot user agent (shown above)
• Respect your robots.txt file
• Follow reasonable crawling behavior and rate limiting
• Originate from US-based cloud infrastructure

If you receive suspicious traffic claiming to be ParticleNewsBot that doesn't follow these patterns, please contact us.

Restrict or limit ParticleNewsBot from accessing your site

To modify the behavior of ParticleNewsBot, update your site's robots.txt file. Place the robots.txt file at the root of your primary domain. We do not support robots.txt files on subdomains.

Example robots.txt configurations

To block ParticleNewsBot completely:

User-agent: ParticleNewsBot
Disallow: /

To allow ParticleNewsBot but limit specific sections:

User-agent: ParticleNewsBot
Disallow: /private/
Disallow: /internal/
Allow: /

To set a custom crawl delay (in seconds):

User-agent: ParticleNewsBot
Crawl-delay: 10

Still need help?

Contact us at support@particle.news.