A few months ago, I reviewed Crawlscore.com, a search engine simulator. Originally, I began using the web service on my blog and article site. Aside from using their spiders to analyze my larger sites,. I have started to utilize them to on my small, targeted niche sites.
Recently, I have been working on two niche sites. The two sites are completely unrelated in market and site structure, but one thing they have in common is the fact I am using Crawlscore for on-page optimization. Crawlscore has helped me to identify pages which were missing keywords, titles, descriptions and other elements which are crucial for on-page tuning.
Add your site to Crawlscore
Once I create a niche site with some content, I add it to my control panel in Crawlscore.com. the service will give you an estimated time frame on how long it will take until a site is spidered once you add it to your control panel. My niche sites are between 10 - 50 page, and they are analyzed within fives minutes. These times can vary depending on their server load and the number of pages in your site.
Use the Crawlscore reports to tune your content
Once the site is added and crawled, I go through an a checklist to make sure my content is looking good for Google and the search engines. Here are the steps I follow:
- 404 Report - First we want to see if the site has any broken links. Malformed URLs are usually the cause for these smaller sites.
- Status 200 Report - This report will give you a glance at all the pages which are accessible and valid. I look at each page and make sure that each page has a title. The URL is displayed so you can clickthrough to see the pages live. Also you can make sure your target term is in the titles.
- Duplicate Titles - This report is found in the Site Optimisation section. Duplicate titles don’t help out much, so this is a good report to identify all the occurrences.
- Without Keywords - Also found in the Site Optimisation section. This report identifies all the pages lacking keywords. You can also find the pages with duplicate keywords in an other report.
- Without Descriptions - Again in the Site Optimisation section. Descriptions don’t hurt, so add them if they are missing.
Crawlscore is a great tool for getting your niche sites in tip-top shape for receiving the organic traffic you are targeting. I believe has helped my niche sites to increase relevancy and position in the search engines. To learn more about the features of Crawlscore, see my previous post.


March 23rd, 2008 at 3:45 pm
[...] Use Crawlscore for Optimizing Niche Web Sites, Jason Bartholme [...]
March 23rd, 2008 at 3:45 pm
[...] Use Crawlscore for Optimizing Niche Web Sites, Jason Bartholme [...]
March 25th, 2008 at 10:29 am
nice find, ill have to give this one a go.