robotstxt

From MaRDI portal

Revision as of 19:56, 12 March 2024 by Import240312060351 (talk | contribs) (Created automatically from import240312060351)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Software:100982

Jump to:navigation, search

CRANrobotstxtMaRDI QIDQ100982

A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker

Peter Meissner, Kun Ren

Last update: 3 September 2020

Software version identifier: 0.7.13

Copyright license: MIT license, File License

Retrieved from "https://portal.mardi4nfdi.de/w/index.php?title=Software:100982&oldid=30790604"