RobotUA

Defines a list of browser "user agent" names that will be recognised as crawler robots (search engine spiders).

Synopsis

Scope

This directive is only available for use in the global (interchange.cfg) configuration file, and will affect all websites running under the Interchange installation.  It will not work in a website's local (catalog.cfg) configuration file. 

Description

The RobotUA directive defines a list of browser "user agent" names that will be recognised as crawler robots (search engine spiders).  Requests coming from the recognised spiders will cause Interchange to alter its behaviour to improve the chance of Interchange-served content being crawled and listed.

Note The NotRobotUA list will be consulted before the list configured with RobotUA to help weed out false positive matches.

This directive accepts a wildcard list;  The "*" character represents any number of characters.  and the "?" character represents any single character.  For example, "fo*ar" would match "foobar".

If a client is recognised as a robot, the following will be performed by Interchange:

• The mv_tmp_session CGI value will be set true, causing sessions to be disabled and therefore avoiding need to read and write session data to from/to the disk.  This also causes Interchange to generate URIs without including a session ID.
• The mv_session_id CGI value will be set to "nsession".
• The mv_no_count CGI value will be set true, causing Interchange to generate URIs without including an incremental "page count" number.

Warning Once you have discovered that you are serving a page to a robot, you should not use this knowledge to massively alter your page content in an attempt to improve your search results ranking.  Doing so will stand you a good chance of being blacklisted by the search engine maintainers.

Example

See also

• NotRobotUA
• RobotHost
• RobotIP