robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Embed those pesky little gif files right into your scripts.
ANG GES G P
Creates a Search engine with switches to vary search and results return technique
ANG GES G P
Paranoia was designed to check the validity of the parameters that a php page will receive after a form submision. It can be used to check the v ...
ANG GES G P
Bei cphplib handelt es sich um eine Sammlung von PHP Funktionen. Diese können einfach in bestehende PHP-Scripte eingebunden und sofort genutzt we ...
ANG GES G P
Another MySQL interface class for php. FUNCTIONS: GetDatabases, CreateDB, DropDB, CopyDB, SelectDB, GetTableList, GetFieldList, Delete, Update, ...
ANG GES G P
First version of class for formatting of text (TTF) on images. Will allow you to more easily place ttfboxes on images.
ANG GES G P
Sophisticated XML Management library for XML file that does not rely at all on php pre-defined xml classes.
ANG GES G P
MiniXML provides a simple, API to generating and parsing XML. Its advantages are ease-of-use and the fact that no additional libraries are require ...
567 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@