robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Creates a Search engine with switches to vary search and results return technique
ANG GES G P
jfile class is a wrapper class for fopen/fclose and flock. With this class, its easy to open new files which are locked and released after use
ANG GES G P
QuickTemplate is a PHP extension for mananging templates and performing multi level variable and block interpolation.
ANG GES G P
Very Simple Template allows you to dissociate very simply HTML code from PHP code. A programer will easily rewrite an HTML project done by a grap ...
ANG GES G P
Class library needed for displaying a big group of records in HTML page
ANG GES G P
Class for parsing RSS 0.9 and 0.91 formats. Includes utility functions for walking the parse tree.
ANG GES G P
Die EasyPhpThumbnail Klasse erlaubt Ihnen, Thumbnails zu erstellen und Bildmanipulation für GIF, JPG und PNG schnell zu handhaben. Die Klasse is ...
ANG GES G P
HtmlReader is a PHP class for mananging html templates. You can insert variables, multi level if-blocks and multi level loops
289 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@