The robots.txt file is the first thing search engine "spiders" look for when indexing a website. The robots.txt file tells search engine spiders (robots) which files and/or directories they are NOT allowed to index. This helps to prevent incomplete site indexing as well as prevent exposing the files and directories that you don't want "all over the world wide web". It's fast, easy and FREE.
Similar scripts
Add Icon With Url In Browser
(Popularity: ) : Add icons that show up in the URL box and with Favorites (bookmarks) in IE5. Here are two methods to do it... along with information on icon creation and troubleshooting. Free Code Generators
(Popularity: ) : Choose from 18 different code generators to add to your own website for FREE! Use this online generator to customize the look and feel of the generator(s) you have selected. Then just cut and paste the generator code snippet into ... Blueshoes Spreadsheet Editor
(Popularity: ) : Blueshoes Spreadsheet Editor is a multi-platform compatible excel-like application. It's written in pure dhtml/javascript and works without installation of any plugins, java code etc. It's a great frontend to edit data that you want to display on a web page, ... The Matrix Codebreaker
(Popularity: ) : The Matrix Codebreaker is a multi-platform compatible script that goes through a set array of numbers/letters, and one that randomly chooses numbers until the correct one is chosen. This script requires MSIE version 5.0 or greater or Netscape version 6.0 ... Javascript Online Generator Center
(Popularity: ) : Javascript Online Generator Center is multi-platform compatible javaScript cut paste code generator that creates a cut paste javascript code effect painlessly & easily by following the simple directions for your dynamic javascript cut and paste code effect. Buttons.
(Popularity: ) : Buttons. is multi-platform compatible. This script gives a variety to standard buttons. It changes color of buttons at different events of the mouse. The buttons have four-state a status. You can use all types of buttons - "button", "submit", "reset". Online Graphing Calculator
(Popularity: ) : A highly effective graphing calculator that works in both Netscape and Internet Explorer. Please obey syntax directions at bottom of page. Email alexander.rhodes@brentwood.bc.ca for questions. Line-drawing version coming soon. Amazing Message Box
(Popularity: ) : Amazing Message Box is multi-platform compatible. A message box is a window control derived from QBox and basically used to display short messages to user. It inherits all properties of QBox and its behavior. This control displays a text message ... Script Loader
(Popularity: ) : This script enables you to dynamically load external JavaScript scripts. Get Current Style Method
(Popularity: ) : Cross-platform method for retrieving style information set in either a CSS style sheet, a script block, or inline.
User reviews
Similar Software
Advanced Robots.txt Generator
(Popularity: ) : Create your robots.txt files in seconds and maintain them easilly and effortlesly with the unique utility Advanced Robots.txt Generator. The software supports 300 robots, works in a WYSWIWYG visual way and comes with full documentation and unlimited customer support. It's ... Robots.txt
(Popularity: ) : Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the robots.txt files required to instruct search engine spiders, which parts of a Web site are not ... Easy RSS Content Generator
(Popularity: ) : Easy RSS Content Generator is an easy-to-use program for creating new unique content out of news feeds (the RSS, Atom, and RDF formats are supported). You can use this product, for instance, to create completely autonomous, automatically updated sites, to ... RoboGen
(Popularity: ) : RoboGen is a visual editor for Robot Exclusion Files. It allows the user to quickly and easily create the ROBOTS.TXT files required to instruct web search engines which parts of a web site are not to be indexed and made ... Robots.txt editor
(Popularity: ) : This program is a visual editor for Robot Exclusion Files and log analyzer software. It allows the user to quickly and easily create the robots.txt files required to instruct web search engines which parts of a web site are not ... Oven Fresh Robots Txt Maker
(Popularity: ) : Just fill in a simple form and the robots.txt code is generated for you. Robots Txt Maker will generate the robots.txt file you need to help prevent the indexing of files and directories that if indexed by themselves could frustrate ... A1 Sitemap Generator
(Popularity: ) : Create text, HTML, RSS and XML sitemaps (Yahoo, Google etc.) to help search engines crawl and index your website. Near-endless website scanning options. Configure amount of simultaneous connections to use. Supports crawler filters, robots.txt, custom connection and read timeout values, ... Deva xFTP
(Popularity: ) : Deva xFTP is a Professional FTP Client software for web developer that integrated Deva CA Center, Deva Web Link Checker , Deva Sitemap Generator and Deva HTML/xHTML Validator Webmaster tools .
Whether you're a novice or a WebMaster , Deva ... Page Promoter
(Popularity: ) : Page Promoter is an ultimate URL registration, search engine optimization and promotion tool for submitting Web sites to thousands search engines and directories as well as preliminary site optimization and effectiveness check. The software allows you to enhance sites` exposure ... TracRobotsTxt
(Popularity: ) : TracRobotsTxt is a module that will serve a robots.txt file from Trac. Mostly useful to tracd users, but works on anything. Just put the data you want in the wiki page RobotsTxt.
Script Library
) : Add icons that show up in the URL box and with Favorites (bookmarks) in IE5. Here are two methods to do it... along with information on icon creation and troubleshooting.