Register
Forgot Password?
Score
-
-
-
  /100
Certificate
SEO Certificate

2.1 Do Search Engines know about you, and do they know enough?

Sorry, you are not logged in. Please login to get full access to this page or register if you're not registered yet.

To put it simple, let's check if Search Engines know about your site and will tell the visitors about it if someone's looking for your offer.

Normally, Search Engines will send a robot (also called spider, or crawler) to look around the Web, to see what's new and what's going on in general. Moreover, the spider will look through web pages and evaluate how good and useful they are. But crawlers only come to the places they know about.

So if you launched a site www.buymyproduct.org and, say, Yahoo! doesn't know about it, you can wait for weeks and months and any longer. The spider will hardly visit you.

Unless you make an invitation. And just a bit later, I'll tell you how this invitation for Search Engines is done.

In 2003, I met a client in Belgium I'm still working with. Cyr is now the big boss of several flower delivery services in Belgium and Netherlands, and he just launched one in Germany. Sure, SEO is #1 in his business weapons arsenal.

But the story starts with how I found him. I went to see my cousin who studied psychology in Brussels, and Cyr turned to be his roommate. He just launched an online flowers delivery store (great idea for a student, I guess!) and was wondering why it didn't work. Well, I made a simple check. He had a nice site, well-designed and correctly written. But Google.be just knew nothing about him!

We agreed on a favourable fee for an SEO campaign, and I began from submitting his website to Search Engines — and he started getting orders in about two weeks!

So, let's see what Search Engines know about your site www.buymyproduct.com. Go to the web page of the Search Engine that is important to you and type in the following query site:your_domain_name. For example, type site:buymyproduct.com.

DO IT NOW! Check if Search Engines see and display your website, and if the number of pages they show is correct.

Now let's see how it's done and what results it can bring. 4 situations are possible.

  • a) No results are found.

  • b) Some pages of the site are listed by the Search Engine, but they make up even less than 50% of the pages the site really has.

  • c) What the Search Engine shows is approximately the real amount of pages the site has.

  • d) What you see is too much, over 150% of how many pages the site really has.

Now, let me be fair: a) b) or d) is a red flag.
If you got no results at all, too few or too many, this means you have a problem.

Here's the good news, though. The problem can be solved, and I'll tell you how. But first, let's see where it really lays. There can be two basic problems that cause your trouble.

Problem 1: Your site hasn't been submitted to Search Engines yet.

Note! This problem is quite uncommon, and it can only be a reason for a): getting no results for the website at all.

Try to remember: did you submit your site to Search Engines, or maybe someone did that for you? Or, did you get a link to your site from some rather respected web page?
If your answer to both questions is "no", you'll need to correct the mistake. But it's simple and I guess it won't take more than 5 minutes to do. Search Engines' robots will then crawl your website the next time they're out and about, and you'll become visible.

33 comments

Add comment Hide commentsShow comments

Please log in or register to leave a comment.

2008-08-07 17:21:35: Wade Weston

Just a quick comment here and not to be down on submitting to search engines but I am of the opinion that if you have to submit to the engines then you are not doing your job. In other words they will find you in short order if you do the right things. For example, get a link on a PR 4 or higher site and your site will be indexed automatically within days. Or create a Squidoo Lens and get your site indexed maybe the same day. Just my opinion not to say it is right or wrong but just saying this to emphasize focusing on the core elements of SEO.

Answer
2008-08-07 18:04:37: Rene Kohl

From my experience: The less Google knows about you the better the Serps!

Answer
2008-08-07 19:06:04: Stefano AGBAGLA

I think having good back link (pr4) will help in fact.
But the most important in my opinion is to start slowly with a related site.

Answer
2008-08-08 02:32:38: Jeremy Chatfield

IME, I've never had to submit a site to anything other than tier 3 SE's. A new site can be quite easily linked to places that will result in a crawl from all the major SE's within a week, if not a day. That should be a lot better fro the site and its ranking than submission. After all, it will already have some inbound links.

Good places to get inbound links for the purposes of crawling? Yahoo's paid directory is pretty good - but most affiliate type sites are unlikely to make it. Relevant, on-topic postings to discussion forums, with a signature that includes the URL for material disclosure, attribution purposes. Google Groups gets crawled quite often...

Answer
2008-08-08 02:56:02: Dan Richmond

Hi everyone,

For example, get a link on a PR 4 or higher site and your site will be indexed automatically within days.
I'm talking about a fat inbound link on the next page - check it if you wish:) And here's why I pay so much attention to submitting a site to search engines: firstly, for a newly-born site it's pretty hard to get a PR4 link (at least for free), and secondly, submit to 3 search engines only takes a couple of minutes and is free.

Good places to get inbound links for the purposes of crawling? Yahoo's paid directory is pretty good -

Thanks Jeremy, this will all get discussed in due time. Look at the greyed out table contents for chapter 5. Link-building tips're coming soon:) And you surely can contribute if you wish. If you'd like to discuss some things, you can post it here or send me messages via the contact form.




Answer
2008-08-08 03:28:17: Dan Richmond

@Rick
From my experience: The less Google knows about you the better the Serps!

You don't mean it in the context of simply making sure that your pages are properly indexed, do you? It won't harm you if all your important pages are indexed by googles.

But if we talk about giving Google too much info due to using Google Analytics, Google Webmaster Tools and Google AdWords, then sure it may have a converse effect. But that's a totally different story and I'll dwell upon this in some advanced section much later.

BTW, what's the experience you're taking about? Could you share this? (You can post here or use the contact form at the page's top to send me a message) TIA

Answer
2008-08-09 19:24:51: Gerben Bouwhuis

It's

"The less google knows about your spamming activities, the better you rank" :-).

google spents a lot of time calculating the linkpopularity and linkpoints to domains, but especially to specific url's and seems to be increasing the power of the anchor test.

I've managed to get a number 1 ranking in only 4 hours by implementing 1 backlink with the specific anchor text needed...

As a lot of people always mention, google isn't rocket science, it's all about being natural.

Answer
2009-04-02 14:27:04: Donna Goodman

Hi Dan,

I would like to comment on Dan's point about submitting to the SE's.

I have registered all four of my sites with the Google Web master tools, Yahoo, and MSN. When you do this they give you a special code to put either in your meta tags or on the home page to your site. They then verify that you are in fact the true owner of your site. The benefits of this are substantial in my mind. First of all they know your site is legitimate and has a real person as an owner. Second they give you the ability to submit a new site map anytime you want, and they usually crawl your site within the hour if you do. I think that is huge. So I like the idea of submitting your site to as many search engines as is possible.

Next I would like to comment on another tactic I learned about getting a new site indexed fairly quickly, as submitting a new site to the SE's doesn't guarantee that it will be indexed.

It is very easy to find some very popular blogs and forums that are PR6, 7, 8 and one 9 that will allow you to post on their forum or blog with a link to your site in your sig. The higher the PR the more often they get indexed and some of them get indexed every five or six minutes. If you have a link in a new post on one of them then the link is followed and usually indexed within 24 hours. I have done this several times on new pages that I have added with fairly regular success.

So I am interested in your thoughts on this one, this is a great SEO guide and I am enjoying it very much. Thanks.

Donna

Answer
2009-04-03 10:22:41: Dan Richmond

@Donna Goodman

Firstly, I do agree with you about using webmaster tools.

And secondly, thanks for providing a brilliant example of how to get a high-PR link to help get your website indexed sooner. I urge all readers to pay attention to this technique!

A very useful comment indeed.

Answer
2009-04-03 15:02:09: Investigator Jobs

Im fairly new to this. Im going through the course trying to figure things out. Its all very interesting to me. I love it.

One Thing I know for sure on getting indexed fast is...
I have a few sites that I have put on the net during my learning experience... and all of them have indexed on Google by the next day (except for 1) - But Nothing showing up on MSN or Yahoo.

I have never submitted a site through an SE. What I did was I just placed an advertisement with a link on a few free classified sites (and I did this the wrong way, I should have added "proper anchor text" - I just added the url) and my pages were indexed in Google.

Now... I hope to learn how to get them indexed by Yahoo and MSN and MOST IMPORTANTLY I HOPE TO LEARN HOW TO get the new changes I have made to my pages (from what I have learned on this training) to show up on Google because the changes I have made so far are not showing up??? - its still my old description and title????

cheers
Andrew


Answer
2009-04-06 08:37:46: Dan Richmond

@Andrew Collins

Hey Andrew,

I'm glad you're enjoying the book. To get your website recrawled by Yahoo! and MSN bots you may want to submit a sitemap to these search engines. Check out this chapter of SEO in Practice for details and feel free to ask if anything is unclear.

Answer
2009-12-07 08:03:18: Reginald Randolph

Dan,

I'm really enjoying this course. Thank you soooo much for the work you put together here.

Problem: I use Firefox; when I type in 'site:razzhairproductions.com' (my site name, without single quote, I get the following message: "Firefox doesn't know how to open this address, because the protocol (site) isn't associated with any program".

When I use IE, the response is "webpage cannot be displayed".

Please advise

Reggie
randolphrj@gmail.com

Answer
2009-12-07 20:49:11: Ken Taylor

@Reginald Randolph: Hi Reggie, I'm using Firefox too. It looks like you accidentally typed 'site:razzhairproductions.com' into the URL browser window. You need to type 'site:razzhairproductions.com' into the "Search" window at the top of the Firefox window.

Here, I took the liberaty of doing it for you. Your good to go.
Here's what it looks like:

http://www.google.co.jp/search?q=site%3Arazzhairproductions.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a

In summary: any time you want to check to see if your site is indexed...type "site:example.com" (with or without quotes doesn't matter) into the "search" box of your browser or in Google search.

About this lesson:
I played with Google's Webmaster Tools. Worked fine for me, got my site indexed fairly quick. It may have helped that I already had a couple of backlinks from a PR6.


Answer
2009-12-09 08:06:31: Dan Richmond

@Reginald Randolph and Ken Taylor

Thanks, Ken! That's right - Randolph, you need to enter this command in Google, not in the browser's site address box.

Answer
2009-12-10 06:44:22: Reginald Randolph

Dan and Ken,

Looks like I also need a course in using a search box as opposed to the browser window (^o^).

Thanks.
Reggie

Answer
2009-12-12 09:39:55: Tim Inglis

Hi all. I'm loving this book. Using the demo version of the software, but we'll be purchasing the full version (hopefully before Christmas) and looking forward to getting stuck into some SEO.

I'm just curious about the listings for a site in the search engine results. I did a search for my site (http://www.openedgewebdesign.com) in Google. The top link says, "Open Edge Web Design: Home Page" and the URL at the bottom of the link is http://www.openedgewebdesign.com/portfolio. First question, does this mean that I only have the one page indexed? Second question, why does it say "Home Page" up the top with the URL for the Portfolio page? Third question, this is the only link from the website, all the other links are from forum questions and signatures, does this mean that I have done the wrong thing somewhere?

Thanks in advance for any responses, and keep up the great work Dan.

All the best,

Tim.

Answer
2009-12-13 19:39:15: Ken Taylor

@Tim Inglis

Hey Tim, I think you have a "timing" issue. What I mean is that you may have recently changed or updated your website. The reason is I got a whole different story. I see three of your pages: Home page (From your main page); Contact; Home page (From your portfolio page). I could be wrong.

Actually, I came here to post this comment: If you want to know when the last time Google indexed your pages, after you do [site:you_website.com], click on any "Cached" link. You'll get a new window with the time and date Google last indexed your page along with letting you know that the current page may have changed.

I'm sure Dan has more insight.

Answer
2009-12-13 23:09:27: Tim Inglis

Thanks for that Ken. I'll give that a shot mate.

Answer
2009-12-15 08:34:12: Jimi San

I'm happy to learn from some of you how easy it is to get indexed with Google. But as for my case, it's the opposite. I've Google Webmaster's tools, sitemap, link-building exercise but managed to be indexed by Bing and Yahoo but not Google. Just couldn't understand why Google simply ignore my website. My guess is....most probably "sandboxed". Why??? How the hell am I going to get out of this "sandbox"???

Answer
2009-12-21 10:36:30: Dan Richmond

Could we know your website's URL?

Answer
2009-12-23 12:06:23: Jim Johnstone

Great course, finally managing to get a handle on SEO (been trying long enough). Just a question in your remarks about what the SE's see. I have approx. 250 pages on my site but google comes back with nearly 400. I see you mention that this is a red flag, but you don't seem to explain why this is so and any solutions.

Answer
2009-12-28 10:41:08: Dan Richmond

@Jim Johnstone

If a website's URLs are getting generated dynamically, quite often one page will be available at several different URLs, and they are indexed by Googles.
Problem: duplicate content
Solution:
1. rewriting your CMS template, so that the same URLs are generated for the same page from different locations
or
2. restricting robots' access to the unnecessary URLs with robot.txt

Answer
2009-12-28 11:34:31: Dan Richmond

@Tim Inglis

Google displays the title of your webpage. So you see "Open Edge Web Design: Home page" in Google for the portfolio page because it has the title (please check your code):
Open Edge Web Design: Home page

Right now I'm using the site:www.openedgewebdesign.com
command in Google and I see that it's indexed 41 pages from your website.

Answer
2009-12-30 04:12:43: Paul Watchorn

I read last year that Google was doing away with pagerank, and I did notice that at one point google itself was showing 'page unranked'. It seems to have come back again, so is page rank still important?

By the way, I still don't understand this 'sand box' or what to do about it:(

Answer
2010-01-05 06:03:13: Dan Richmond

@Paul Watchorn

PageRank is as important as before, but it's not displayed correctly to the public, so the PageRank value you can see in the toolbar or check with any kind of tools is not 100% trustworthy and should be used as a pretty relevant factor. For example, if you see that site A has a PR of 2 and site B has a PR of 8, B definitely has more power coming through backlinks. But if A has a PR of 1 and B has a PR of 2, I'd just consider them equal.

Answer
2010-03-08 14:24:54: Ken Kopelman

Great Course and great software.
The course is down-to-earth, and common sense, and the software has the best interface / usability I've seen so far -
and I'd just about given up on finding a SEO suite I could live with.
(A few minor complaints / suggestions, but maybe they will be addressed later...)

One thing I am having trouble reconciling (wrapping my brain around)-
I've been using tools like MicroNicheFinder (excellent), but it suggests using extremely easy / high OCI words.
And it rates the high KEI words in RankTracker as "too hard" to go after. (Fortunately a couple of them I am already doing OK with, so I guess I'm the competition? ;-)

Thoughts on how hard high KEI words are to go after?

Thanks,
Ken

Answer
2010-03-11 09:26:38: Dan Richmond

Ken,

I guess you realize that OCI and KEI are pretty different. Moreover, KEI does not estimate of how hard it is to compete for a word. It's "how worthy it is to compete" both in terms of competition and potential traffic.
As for OCI, it's not really showing how hard it is to compete. It only allows estimating potential profit if you rank top.
Therefore it's pretty hard to compare both values, and it is obvious that high OCI and hight KEI words won't be the same.

Answer
2010-05-29 04:52:55: tradmic zunjing

good ,I love your artical very much ,I can lern more from you vebsite.

Answer
2010-06-09 06:42:36: Mark Lazenby

Hi Dan,

Excellent material. Thank You.

Re too many pages in search engines cache.

I use Wordpress for my site building and the category tags all lead back to the relevant post so if I have 10 tags on a post then I have the identical page listed 10 times.

Also as a newbie I have made a lot of alterations to my webpages and the old pages are still shown cached in google as well as the newer alterations.

Could either of these things be a problem for me?

Thanks,

Mark


Answer
2010-08-31 07:41:59: Dan Richmond

As far as I know Wordpress should automatically put a "rel=canonical" on these pages ( http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html )

There's also an SEO for Wordpress plugin that should do it if Wordpress doesn't do it automatically, you can research that.

Answer
2010-06-10 19:26:04: Thomas Wright

too bad buymyproduct.org doesn't resolve to anything =( I was hoping it was real. =P

Answer
2010-08-27 18:32:09: Haskel Fleisher

Why is d) a problem- What you see is too much, over 150% of how many pages the site really has.
What does it tell us is happening if there are extra pages indexed?
I always thought google indexed more pages than I had because they indexed cache pages as well as current ones?
This is actually the situation on my site handtrucks2go.com
I have a sitemap,good hosting,friendly url's.
However I don't have robot x ,I do have a couple of redirect links and no major javascript issues.
Could any of this be causing that?

Answer
2010-08-31 07:20:51: Dan Richmond

The problem is that most of these pages will be duplicates of existing pages with different URLs. In that case the duplicate content issue arises, you should specify your canonical pages:

http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html

If you run your domain through WebSite Auditor you will see the list of all of your URLs and any possible duplicate content issues.

Answer