robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Very Simple Template allows you to dissociate very simply HTML code from PHP code. A programer will easily rewrite an HTML project done by a grap ...
ANG GES G P
template engine for php, allows to store your html code apart from your php code.
ANG GES G P
OO interface for generating HTML tables. All HTML 3.2 elements can be defined.
ANG GES G P
Allow send POST headers to any sites, icq, boards, and other...
ANG GES G P
phPlate is a very simple template program designed to make maintaining a web site as easy as possible. There are zillions of other similar program ...
ANG GES G P
i18n ist eine sammlung von klassen um mehrsprachige php seiten besser managen zu können. zahlen, währungen, datumsangaben usw. können ebenfalls ...
ANG GES G P
class.gzip_encode.php is a php class to gzip encode served content to reduce size.
ANG GES G P
Class to support MySQL sessions with many servers
403 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@