Getting your movie website indexed by multiple web crawlers, web spiders, web robots, whichever name you choose to ascribe to them, will happen organically over time but there are some ways that you can speed up the process and refine it. We have already discussed 5 Steps to Get Your Movie Website Indexed by Search Engines, which covers many of the ways in which to get indexed by web crawlers quickly but there are others in addition to those.
This plugin makes it possible (and very easy) to:
- Prevent indexing of your search result pages, while still allowing the search engines to follow the links on them, by adding noindex,follow robots meta tags.
- Disallow indexing of subpages to your homepage, category pages, author pages and tag pages, to prevent duplicate content.
- Prevent indexing of your login, register and admin pages by adding noindex robots meta tags.
- Add noodp an noydir meta robots tags, allowing you to opt out of DMOZ and Yahoo! Directory descriptions.
- Prevent Yahoo! and Google from indexing your feeds by adding a meta tag to their head-section.
- Prevent indexing of just your comment feeds.
- Disable author and date-based archives.
- Prevent attachment pages from ranking in the search results over your articles.
- Enforce a trailing slash on archives.
- Edit your .htaccess and your robots.txt from within WordPress.
- Assign robots meta tags to individual posts & pages.
- Verify your site with Google Webmaster Tools, Yahoo! Site Explorer and Bing Webmaster Tools.
- Add noarchive tags to your blog.
Google XML Sitemap
The Google XML Sitemap WordPress plugin will generate
a special XML sitemap which will help search engines like Google, Bing, Yahoo and Ask.com to better index your blog. With such a sitemap, it’s much easier for the crawlers to see the complete structure of your site and retrieve it more efficiently. The plugin supports all kinds of WordPress generated pages as well as custom URLs. Additionally it notifies all major search engines every time you create a post about the new content.
Search for Yourself
Search for your movie website on search engines that you want to crawl your website. Once you do, that particular search engine knows of your website’s existence. Take the largest search engine in China for example, Baidu. Since I couldn’t find a good translation option for the page to search for a blog or website submission option like there is for MSN and other search engines, I simply typed in ProMovieBlogger and searched for my website through its search option. Checking the bots that spider ProMovieBlogger yesterday through Google’s feedburner service, I found that ProMovieBlogger now gets spidered by a Baidu bot. Color me surprised and happy. I do not know for a fact that one caused the other but it can not be a coincidence.
Send an email to movie directories/websites you feel your content is worthy of. If your website is accepted, you may receive a favorable return in traffic.