robotstxt (Q100982): Difference between revisions
From MaRDI portal
Changed an Item |
Changed an Item |
||
Property / MaRDI profile type | |||
Property / MaRDI profile type: MaRDI software profile / rank | |||
Normal rank |
Revision as of 13:24, 7 March 2024
A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker
Language | Label | Description | Also known as |
---|---|---|---|
English | robotstxt |
A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker |
Statements
3 September 2020
0 references