robotstxt (Q100982)
From MaRDI portal
 | This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use this page instead for the normal view: robotstxt |
A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker
| Language | Label | Description | Also known as |
|---|---|---|---|
| default for all languages | No label defined |
||
| English | robotstxt |
A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker |
Statements
3 September 2020
0 references
