robotstxt (Q100982)
From MaRDI portal
!
This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:
A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker
| Language | Label | Description | Also known as |
|---|---|---|---|
| default for all languages | No label defined |
||
| English | robotstxt |
A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker |
Statements
3 September 2020
0 references
Kun Ren
0 references
