robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Authentification and user track. A simple session manager for all.
ANG GES G P
A framework for mysql database searches and displaying the results making it quick and easy to set up a html search form.
ANG GES G P
php klassen zum erzeugen von vCard dateien die in outlook & co importiert werden können.
ANG GES G P
php klassen zum erzeigen einer rss 1.0 datei. unterstützt dublic core und SY module
ANG GES G P
Conversion of php sessions class to use Oracle instead of MySQL
ANG GES G P
Class for parsing RSS 0.9 and 0.91 formats. Includes utility functions for walking the parse tree.
ANG GES G P
phpdl-count003.class ist eine Counterklasse Features: - Freeware - deutsche Quellcode - Dokumentation - keine Datenbank wird benötigt - auto ...
ANG GES G P
This class is an HTML parser written in PHP4, by employing which you can parse a rather-standard HTML stream into a tree-structure, that consitut ...
432 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@