Using AltaVista as an Example

We can learn a lot by tailoring our search engine submissions along the same lines as PositionPro. Let's look at AltaVista as an example.

In the past, we were told to submit just one a page a day to AltaVista. But with AV's new submission procedure, we can submit more. How many can we safely submit? Let's look at how does PositionPro handles AltaVista.

"We submit five pages per domain per day, and we don't have any problems. Could we submit ten? Probably. Could we do it over a time period? No," explains Jim.

By the way, it might interest you to know that PositionPro doesn't accept cookies. As you probably know, we've advocated cookie control when working with AV. Is this why they're able to submit five pages a day? Probably not. "I think if you talk to AV, they would be fine with five," Stob says.

What about pages that have had no changes made to them? Does PositionPro resubmit those pages?

"We only do a submission one time to start out. So, with a site that contains 50 pages, it will be fully submitted within 10 days," says Jim. Then, once the pages make it into the index, the service doesn't resubmit them again, at least as far as AltaVista is concerned.

After that, the spider will pick up any changes you make to the pages on its next spider run. But, let's say that you've made significant changes to your pages, and you want those changed pages picked up before the next spider run. At PositionPro, you can tell the service to manually submit your page. However, it won't allow you to resubmit the same page any sooner than 21 days since the last search engine submission.

What if you submit a page and it never makes it into the index? Should you submit over and over again until the page gets in?

"No, that's inappropriate," explains Stob. "If I submit to AltaVista, and if it's not accepted, there's a reason that the page wasn't accepted. It wasn't because of the search engine submission. The engine went in there, and their algorithm judged it not to qualify for their index."

So, what are your options? For one thing, keep in mind that the engines operate on a timetable - their timetable! Be patient and wait to see if the page makes it into the index. If it doesn't, before rushing out to resubmit, look at your page carefully and consider what additional optimization strategies you can employ to boost your chances at getting in the index. Remember: content is what the engines are after.
Note: AltaVista now has a pay inclusion program, where you can pay to have your site indexed by AV.

Safe Search Engine Submission Guidelines

Based on the guidelines that have kept PositionPro out of trouble, let's create a list of "safe submission" guidelines that we can follow for our own sites.

We'll get into engine-specific search engine submission guidelines later in this article, but for now, let's look at some general guidelines.

Submit new pages to the engines, and then wait to see if the pages make it into the engines' indexes. Remember that it can literally take weeks (months?) for pages to get into an index.

If the pages don't make it into the index, don't resubmit them, but look them over carefully to see if you need to employ additional optimization strategies before resubmitting.

Submit all of your important pages to the search engines that allow you to submit more than just the main page of your site.

Stob explains, "None of the spiders crawl as well as they say they do. Sometimes they do an excellent job, but in most cases, they don't."

After a page has made it into the index, unless you make significant changes to the page or the page has dropped from the index, don't resubmit it.

If a page is dropped from the index, wait for two weeks to see if the page is picked back up, and then resubmit it.

If you've made significant changes to a page, resubmit it, but no sooner than 21 days since your last search engine submission. It might benefit you to play it safe!

Consider adding visible links to all of your important pages on the main page of your site. Give the engines something to spider!

What about using hidden links, which is a very popular strategy these days? Stob says to begin to approach this practice with caution. "I believe AV is beginning to look for hidden links, so try to stay away from them. I'm not saying that you will have trouble based upon their spider. However, they do have people reviewing, and I believe they are looking at this closer."

Submit your pages manually, or use a search engine submission service or software that mimics a manual submission. By that, we mean that the software or service waits for verification before submitting another URL. If you use a search engine submission software program, make sure that you're able to choose the schedule for your submissions. If you can't, you may find yourself in trouble with the engines for over-submitting.

Keep in mind that if it takes too long for an engine to access your site, it probably won't get indexed.

Engine-Specific Guidelines and Tips

AltaVista

Submission URL: http://add-url.altavista.com/cgi-bin/newurl?

How long does it take to index a page? a few days with Express Inclusion, their pay inclusion program; 6 weeks through their free Add URL page.

Submission Limit per Day: through free Add URL: 5 pages or more with their new procedure

With AltaVista, rely on the use of links to get the pages in your site indexed. Unless you've made significant changes to your pages, don't resubmit them. Instead, let AV's spider find those changes during its next spider run.

Try submitting your pages using "www" and not using it in the URLs.

AV won't index a site submitted as an IP address. A rep from AltaVista explains, "The spider is doing reverse DNS lookups when an IP site is submitted. You must be registered with Internic and have DNS running to be indexed by the spider."

Does AltaVista have a limit to the number of pages from a particular domain that can be in its index? AV answers, "In reality, the whole site could be indexed with no restriction on depth, except that we have size limitations and stop the spider once the index of the site has a certain size." In other words, they aren't saying. (Is this surprising?)

To check to see if your pages are in the index, enter your URL in the search box like this:

url:yourdomain.com (reveals every page under the root domain)

url:members.prodigy.com/yourwebsite/ (narrows the search to pages within your site, if you're under someone else's domain)

url:members.aol.com/yourwebsite/yourwebpage.htm (finds a specific page at your site)

Google

Submission URL: http://www.google.com/addurl.html

How long does it take to index a page? 1-2 months

Submission Limit per Day: 10 pages

With Google’s supplying Yahoo!'s supplemental index, submitting to Google has become even more important.

Submit all of your important pages to Google. However, with Google's unique way of indexing, your links should be picked up by this engine with no problems. But, why take a chance - submit your important pages.

You can submit up to 10 URLs to Google a day and be safe. "Google doesn't have a number per se," says Stob. "When I first approached them, they asked me to keep it low so we started at 2 per day. They were in the middle of building a new index. Since then we have brought the count to 10 per day."

Stob continues, "Could I submit 50 a day? Probably. Would I consider doing it on a large site? Probably."

With Google, be sure to have other sites link to your Google pages, or those pages may not be picked up in the index.

To check to see if your pages are in the index, search for your domain name like this: www.yourdomainname.com

Then, click on "Find web pages that contain the term." From there, you can choose "Search within results" to narrow down the search to particular pages.

HotBot / Inktomi Engines

Note: Your best bet for submitting to the Inktomi engines is to go through their pay inclusion program. For more information, visit: http://www.positiontech.com/. If you don't want to pay to submit, your next best bet is to submit to Inktomi through Anzwers.com.

Submission URL: http://www.anzwers.com.au/cgi-bin/print_addurl.pl?

How long does it take to index a page? 2 weeks if not through their pay inclusion program; 48 hours if through pay inclusion

Submission Limit per Day: 50 pages

As you know, Inktomi supplies results to HotBot, GoTo, MSN Search, and many more search services. To get your site into the Inktomi engine faster, try submitting to Anzwers.com instead of through HotBot, or again, through their pay inclusion program.

Be sure to submit all important pages of your site to HotBot. Try submitting the same page using different variations of the page. For example:

http://www.HereIAm.com
http://HereIAm.com
http://www.HereIAm.com/index.html

You may find that your sites using the "www" are ranked slightly higher than those without the "www."

When you submit a site to Inktomi, have you noticed that it may be appear in the index for a month or two, then be dropped? Jim explains what might be happening. "Inktomi will analyze a page for inclusion in their index. It may show up for a month or two and then be dropped. It is dropped because Inktomi has not seen it come up in a search and has determined that it doesn't have any value."

So, if the page doesn't get accessed through a search engine, it will be dropped from the Inktomi index?

Stob answers, "Yes for Inktomi, no for AltaVista. Inktomi has two indexes as you probably know."

Have you also noticed that your rankings fluctuate in HotBot or other engines? Stob explains, "Rankings will alter by the hour with some engines. If you lose the spot for more than two weeks, you should be concerned."

To check to see if your pages are in the index, use their Check URL form: http://hotbot.lycos.com/help/checkurl.asp

Or, visit HotBot's main page and choose the "Advanced Search" button, which is on the left-hand side of the page. The SuperSearch page will appear.

Find the Location/Domain box toward the middle of the page, and enter your domain without the "http://www" prefix in the domain box. You can choose other variables, such as the page depth and the number of search results you'll be shown. Click on Search.

Lycos / FAST

Submission URL -- Lycos: http://www.lycos.com/addasite.html

Submission URL -- FAST: http://alltheweb.com/add_url.php3

How long does it take to index a page? 20-40 days (in FAST)

Submission Limit per Day: 50 pages

With both Lycos and FAST, you can safety submit all of your important pages, even though both engines are “deep crawlers” and should find the links on their own.

Try omitting the "www" when submitting your pages to Lycos.

To check to see if your pages are in the index, search for your root domain, such as: thisismysite.com

Some of your pages will usually be listed. If not, search again, but leave off any suffixes, such as ".com" or ".edu" like this: thisismysite

In Conclusion