Letting search engines find an archive
- From the Tools menu (cogwheel icon) go to Site Configuration > Archives.
- Select Edit for the relevant archive.
- In the Other section, turn on the Allow crawlers to index archive toggle.
- Select Save. This feature is enabled by default for all new archives that are created.
How this feature works
When web crawling is enabled, FotoWeb makes it possible for crawlers, or robots, to index the entire content of your archives, so that users who search using Google, for example, can find pictures and other material on the site.
It's important to understand that this only applies to archives that are open to the public; an indexing robot cannot access archives that require user authentication. As a result, any assets stored in archives that control access with an access list will be off-limits for external search engines.
Note: When an archive has been indexed by Google or another Internet search engine, turning off the Allow crawlers to index archive toggle does not immediately remove your archive from the search engines' listings. This could take several days, possibly even weeks. However, you may be able to use Google webmaster tools or other utilities to notify a search engine to remove your listing sooner.
HTML tags that are used for crawler instruction
When Allow crawlers to index archive is enabled:
- All subfolders and assets in an archive are indexed, and the following is enabled: INDEX, FOLLOW
- The archive root is not indexed, but the following is enabled to enable all the individual assets and subfolders in the archive to be fully indexed: NOINDEX, FOLLOW
When Allow crawlers to index archive is disabled:
- The archive, the root folder, subfolders, and assets are not indexed: NOINDEX, NOFOLLOW
Irrespective of the above settings
- If an album is showcased on the home page, that album is always indexed: INDEX, FOLLOW
- An album that is NOT showcased on the homepage is not publicly available per se, and the album and assets are therefore not indexed: NOINDEX, NOFOLLOW.
A user who does not want an album to be crawled by a bot should not showcase the album but rather share it with individual users.
Note: The crawler flags set by FotoWeb and in any robots.txt file on the site are accepted by all well-behaved crawlers. Ill-behaved crawlers, however, may ignore these flags and still insist on indexing everything they can get to. Such crawlers can be banned from accessing the site, but the procedure for doing so goes beyond the scope of the FotoWeb documentation.