Because web design and SEO are closely related, it’s a topic that designers are frequently asked about, but there is often a lot of gray area about what is the responsibility of the designer and what is within the designer’s power. While many clients want the designer to create a website that ranks well and attracts targeted visitors, SEO is really an ongoing process that involves much more than the design and coding of the site. Things like keyword research, content development, and link building are also critical aspects of optimizing a website for search engines, and typically these are all ongoing processes.

Although SEO involves much more than just design, there is still a lot that the designer can do to set the foundation for an optimized website. If the designer creates a search engine-friendly site any future and ongoing SEO efforts will have a greater impact. In my opinion, web design and SEO are separate services, and a designer cannot create a truly optimized website without other pieces to the puzzle. The designer should create a search-friendly site and educate clients on what else will need to be done (such as content development and link building) in order to effectively optimize the site for searches.

In this post we’ll take a detailed look at the subject of web designers and SEO and many of the issues that are involved.

What is the Designer’s Responsibility in Terms of SEO?

As was mentioned in the intro, the designer should create a website that is search engine-friendly and will set the foundation for all other SEO efforts. What does it mean to be search engine-friendly? Essentially the term recognizes that design and coding of a site only has so much impact on search engine rankings. A search-friendly site will make it easy for search engines to find the content, determine what the site and pages are really about, and allow the site to rank as high as possible based on the content of the site. However, a search-friendly website needs more work to be truly optimized for search engines. For a detailed look at what makes a website search engine-friendly see How to Create Search Engine Friendly Websites by Cape Cod SEO. We’ll take a detailed look at some of the specifics later in this article.

In addition to building a search-friendly site, the designer should also discuss the subject of search engine rankings with the client to explain what else needs to be done on top of having a search-friendly site, and also should work with the client to identify keywords and phrases to help the client reach the right visitors. In some cases the client will already have identified the words and phrases they want to target, but some clients may not have even considered this before hiring a designer.

Why Does SEO Matter to Designers?

If web design and search engine optimization are really two different services, why should SEO matter to designers? For starters, some designers and agencies offer both services, so naturally there will be some overlap and SEO must be taken into consideration during the design process.

Even for those designers that do not offer specific SEO services, clients hire a designer to get results. While the client’s opinion of the designer will certainly be impacted by the visual appeal and aesthetics of the design, what most businesses really want from a new website design is impact. If it’s an e-commerce site, they will want to sell more products. If it’s a blog, they will want more traffic, subscribers, and interaction in the comments. If it’s a service-oriented business, they will want more inquiries and ultimately more customers/clients that are generated from the site.

If a designer wants to keep clients happy and pick up more referral business, the designer should be equally concerned about the results that are achieved through the site as the visual appeal of the design. Regardless of the type of website being designed and developed, search engine traffic can help to achieve the desired results.

Since search engine traffic is so instrumental in the success of client websites, the designer should be well versed in building search-friendly websites and should give each site a chance to rank well. Otherwise, the designer may create sites that look great but ultimately wind up as a disappointment to clients.

The SEO Limitations of a Designer

Although the design and development process is critical to establishing a solid foundation for search engine optimization, there is only so much that can be done by the designer. Assuming the designer is not also providing services for content development, link building, and social media marketing, the chances of a site ranking well for highly competitive search phrases based solely on the work of the designer/coder are very slim. If the site is already well established in its industry and has considerable number of quality inbound links it will have a chance to rank well, otherwise, building a search-friendly site is just the start.

Web designers are frequently asked by clients to create an optimized website that will rank highly for their targeted search phrases, but unfortunately there is no magic wand that a designer can wave to achieve top rankings without other work. The design and coding of the site is simply one piece to the puzzle. The designer should educate clients on the basics of SEO and what will need to be done to achieve top rankings, or should help the client to identify less-competitive search phrases that may be achievable with a search engine-friendly site and some basic content development.

What Designers Shouldn’t Do

Because the design and coding of the site is really the foundation of SEO, it is important for setting things up properly, but it’s also important to not do anything that could cause this foundation to be unstable. Black hat SEO (anything that attempts to deceive the search engines and violates Google’s guidelines) may help a client to achieve better rankings in the short-term, but could seriously harm the site in the long run. Designers should have a basic understanding of SEO and what could potentially harm the rankings of their clients in order to avoid these situations.

Additionally, designers should avoid making promises like “I can create a site for you that will rank on the first page of Google for X keyword.” Particularly if the designer is providing just design and coding services and not additional SEO services, there is only so much that the designer can control and promises like this will often be difficult to keep. When clients ask for a site that will achieve specific search rankings, the best thing to do is to explain what is involved in optimizing a website, offer your help for the services that you provide, and recommend someone else for specific SEO-related services if there are parts that you do not offer.

Effective SEO for Web Designers

Now that we have discussed the role of the designer in the SEO process and why search traffic should be considered by designers, we’ll take a look at some specific things that designers can do to create a search-friendly site for clients. For the purposes of this article, references to web design are intended to include front end development (HTML/CSS coding) as well as visual design.

1. Thoroughly Discuss the Topic of SEO with Clients and Help Them to Determine Which Keywords Should be Targeted

At the start of any design project it is helpful to discuss the topic of search engine optimization with clients. This is helpful for a few key reasons: 1) If the client has unrealistic expectations about a designer being able to create a site that will rank highly, it is helpful to educate them on the breadth of SEO and the role of design as one aspect, and 2) The designer and the client need to work together to determine what search phrases should be targeted and how the site should be optimized. The client always knows their business and customers better than the designer, so it’s critical that they are involved in determining what search phrases should be targeted.

2. Basic Keyword Research

If the client has not considered or is not sure what keywords and phrases they want to target, the designer can assist them by doing some basic keyword research to determine which phrases searchers are using, what related phrases are also possibilities, and how much competition exists for these phrases. There a lot of tools and software available for keyword research, but the Google AdWords Keyword Tool is sufficient for basic research (and it’s free). With this tool you can enter some words or phrases, receive data on the number of searches that are done on a monthly basis for them, see the level of competition for AdWords (which is often representative of the level of competition for organic search results as well), and receive a list of suggested or related phrases.

Even if the client has already identified the phrases that they want to target, it’s not a bad idea for the designer to do some basic keyword research anyway. There may be other words and phrases not identified by the client that could be better in terms of number of searches or competition for rankings. Additionally, if clients have identified targeted phrases without doing any research it may be helpful to show them the level of competition that they will face by targeting these phrases and how that compares to the level of competition for other similar phrases.

3. Discuss Page Titles with Clients to Create Optimized Titles

The most important on-page factor for SEO is the page title. Ideally, page titles should include the targeted words and phrases in order to provide the best match when someone searches for the targeted word or phrase. Titles can include other text in addition to the targeted phrases, but the targeted phrase should be included in the title. Of course, each page on the site should have its own title and can be intended to target a different search phrase. For example, a photographer’s website may have a homepage with a title that targets the phrase “Philadelphia photographer” and secondary pages could target separate, but related, phrases like “Philadelphia wedding photographer” or “Philadelphia landscape photographer”.

Because page titles are so influential in search rankings, designers should discuss them with clients (at least on key pages) and use the keyword research to aid in determining what should be used for the titles. For clients that use WordPress, designers can install the All in One SEO Pack plugin that will allow users to control page titles on any pages and posts that they create.

4. Clean Code

Part of having a search friendly site is using clean code that makes it easy for search engine spiders to crawl the pages. If search engines have a hard time determining the content of a page it is unlikely to rank very well (see What Beautiful HTML Code Looks Like from CSS Tricks for an example of good code). CSS-based layouts makes it much easier to keep the HTML code clean by separating the content and the design, as opposed to table-based layouts.

5. Keep Primary Content High in the Code

The main content of a page should be located as high as possible within the HTML code of the page. Within the body, the header content will typically appear first, but the main content of the page should appear above sidebar content. Even if the sidebar is shown to the left of the content on the screen, a CSS-based layout can be used so that the content is still placed above the sidebar content in the code. Keeping the main content higher means that search engine spiders will find it quicker and they’ll have an easier time recognizing what they page is truly about.

6. Use Headers and Sub-Headers Appropriately

Heading tags (such as h1, h2, h3, etc.) should be used to indicate headers and sub-headers within pages. While it is possible to achieve the same look by setting font weight, style, or size without using header tags, the header tags indicate importance and structure for search engines. They make it easier for search engines to know what is being emphasized on the page, and they can help the designer to be able to target specific phrases.

7. Use Internal Links to Help Visitors and Search Engines Find the Important Content

Throughout this article we have talked about link building as an important part of SEO. Attracting links from other high-quality sites to pages on your own site (or your client’s site) is a critical part of achieving high search engine rankings, but internal links (a link from one page to another page on the site site) are also important. In general, search engines will assume that the pages that have the most internal links pointing towards them are most important. For instance, if every page on a site includes a link to a particular page that includes detailed information about a product, it would be logical to assume that the product detail page is pretty important.

Using internal links effectively can make it easier for search engine spiders to crawl through the site and find all of the content/pages, plus effective internal linking will make a site easy to use and navigate for visitors, which is ultimately more important than search engine rankings.

8. Use Appropriate Anchor Text

The anchor text (the clickable text in a link) is also important for search engine rankings, both from internal and external links. For the purposes of this article we’ll stick with internal links, since that is what the designer can control. Ideally, anchor text should match up with the targeted phrases or at least include them within a larger phrase of anchor text.

Using keywords and phrases in the anchor text helps to indicate to search engines the topic of the page that is being linked to. For optimization purposes it is better to use keyword-rich anchor text as opposed to “click here”.

9. Set Up a Sitemap

There are two different types of sitemaps and the each have their own purpose. XML sitemaps or Google sitemaps are not intended to be used by human visitors, rather they exist to help search engines find all of the important content of a site. An XML sitemap makes it easier for the search engine spiders to crawl the site and it reduces the chance of a spider missing a certain page and not including it in the index. WordPress-based sites can use the Google XML Sitemaps plugin to automate the process, and for other types of sites you can use a tool like XML-Sitemaps.

HTML sitemaps are intended to be used by human visitors as a way to find specific content on the site, however, they also have an SEO-related purpose. While HTML sitemaps are primarily intended for human visitors, they also provide an opportunity to use internal links to point towards all of the important pages of a site. When the search engine spiders find the sitemap they will be able to easily crawl through the site using the sitemap and the links in the sitemap can help the search engine to determine the important pages on the site (for large sites that don’t include a link to every page on the sitemap).

10. Use Friendly URLs

The URL of a page can have some impact on its search engine rankings, so it is best to use keywords or phrases within the URL. The ideal URL will include the important words or phrases that will tell both human visitors and search engines what the page is about without being ridiculously long. Very long URLs make it harder for users to copy and paste, email, and write down, and search engines also prefer shorter URLs.

Hyphens or underscores are often used as a way to separate words in a URL. This can make it easier for visitors and search engines to read the URL and determine the keywords. Most content management systems will allow you to have some control over the URLs that are created.

11. Use Alt Text on Images

Images and photos used on pages should include alt text for describing the image. The primary purpose of alt text is for accessibility and usability purposes, including screen readers for handicapped users. With effective alt text it is possible to know the purpose or the point of an image without even seeing it. For SEO purposes, keywords or phrases can be used in the alt text for a small boost. Alt text shouldn’t be used strictly for SEO purposes at the expense of usability, but it is helpful to use targeted words or phrases in the alt text when possible and appropriate.

12. Canonical Re-Direct

Search engines view http://example.com and http://www.example.com as two different pages (one has www and one does not). A canonical re-direct is helpful because it indicates to search engines which version of the page is preferred and eliminates the need to view them as two separate pages. Without a canonical re-direct search engine rankings may suffer. A re-direct can be setup in the .htaccess file, see 301 Redirects and www/non-www Canonical Problems.

13. Minimize Page Load Time

Page load time is important for user experience, and also in recent months Google has indicated that it will become more influential on search engine rankings as well. Minimizing load time is a rather large topic that could justify an article of its own (see How to Minimize Load Time for Fast User Experiences) but we’ll quickly look at a few issues here. Things like images, audio and video, as well as calls to a database will increase load time of pages. In general, the connection speed of most internet users has increased in recent years so larger page files and slower loading pages are not as big of a nuisance to most visitors today. However, it is still helpful to optimize files and use tools like YSlow to determine what can be optimized further.

Conclusion:

While design is only a part of the SEO process, there are a number of on-page factors that a designer can, and should, control in order to build a search-friendly website. In situations where clients have unrealistic expectations for what a design can accomplish without efforts for building links or adding optimized content, the designer should communicate with the client to educate them on the basic factors that determine search rankings and what needs to be done in order to rank well for competitive phrases. Feel free to share your own opinions and experiences in the comments.

What do nearly all high ranking websites have in common? As a search engine marketing consultant, I am confronted with the task of evaluating the inbound links of thousands of websites each month. In doing so, I have come to the realization that top ranking websites continue to pay for links despite Google’s public disapproval. Google’s minimalistic efforts to combat paid link building force ethical search engine marketing companies to buy links in order to compete. If done poorly, paid linkers run the risk of having short-lived benefits and potentially harmful consequences. The following tips will help you identify paid links that have positive and long-lasting results on search engine optimization efforts.

If a website requiring payment for a link is less of a directory, and more of an informational resource, Google is unlikely to detect and/or punish website owners for purchasing links from them. Furthermore, it is important that the sale of links be done discretely. A website that does not include a rate sheet for purchasing links is very unlikely to be detected by Google. It takes too much time for Google’s quality control team to pose as website owners and obtain incriminating information. Lastly, I advise against paying for links in directories that offer to submit your link to other directories. These services are Google’s primary targets.

Say I want to be number one for the phrase, “Portland Search Engine Marketing.” I would type this phrase into Google and begin looking for websites that I can get links from. One of the top results is a .org website for a Portland search engine marketing association. If I become a corporate sponsor I can get a link to my blog placed on the right side navigation of this relevant, nonprofit website. It is highly unlikely that Google will ever combat this form of paid linking because membership in a professional organization is a sign of credibility and you are paying for membership rather than a link.

Scrolling down to the bottom of the search results for, “Portland Search Engine Marketing,” I noticed the freelance designer directory. This directory is pretty blatant about selling links, yet Google does nothing about it. Some indicators that this is one of the safer directories are that a free, nonreciprocal option is available, there is an abundance of relevant text, it ranks on the first page for competitive key phrases and Google approved them for AdSense. Getting credit from this site may be short-lived because they are breaking Google’s rules by charging for links and not using the “nofollow” element. However, it is unlikely that websites with links from this directory will get punished because Google will also be punishing those who opted for a free listing.

Since implementing efforts to combat paid linking, Google has done a lousy job of making paid links obsolete. At the current rate, Google will be able to detect intelligent link buying in approximately… never. I consider myself to be an ethical search engine marketing consultant because Google’s unenforceable rules are less important than my ethical obligation to generate leads for my clients. Whether or not their opinions are publicly stated, I can say with confidence that nearly all reputable search marketing companies take part in paid link building. If you don’t believe me, analyze their inbound links.

David Brown is a search engine marketing consultant with Westhill Media.

Rank Checker

Aaron Wall of SEO Book recently released a powerful new Firefox add-on called Rank Checker. If you’re working on moving your site up in the SERPs, you’ll obviously want to know where you stand and to be actively tracking your progress.

As a Firefox add-on this tool is both free and very convenient to use. With Rank Checker you can input the URL that you want to track and a list of keywords that you are targeting (or just researching). You’ll then be able to choose which search engines to check (including all of the international versions of Google). The tool will then go out and get the data and bring it back to you in one convenient place.

There are plenty of other ranking checkers out there, but this one is different for a few reasons. First, it’s a Firefox add-on that can be used easily from within the browser. Also, it has a number of customizable options, including the international versions of Google, other search engine options, and the time delay between queries. Additionally, you can save preset lists of keywords so that you don’t have to enter all of your keywords every time you want to use the tool. This is a great feature if you are working on multiple websites and each one has its own unique keywords to track. What makes this tool even more powerful is that it can be automated to run the reports on its own at set time intervals.

After you have run the Rank Checker you can easily export the results as a CSV file to use in Excel or another program. Aaron created a quick 5 minute video at the download page to explain the tool and how to use it. After spending just a few minutes to download the add-on and watch the video you’ll be able to use this tool with no problem. That’s what I like best about Rank Checker, it’s powerful but very simple to use.

SEO Quake

SEO Quake is another Firefox add-on that you’ll probably want to download. With SEO Quake you can optimize your site by gaining tons of information about other pages that are ranking well. After downloading and installing the add-on, run a Google search for the primary search phrase that you are targeting. Under each of the listings you’ll see addition information provided by SEO Quake, such as the PR and Alexa ranking of the page, plus the number of pages indexed from that URL and the number of inbound links (you can customize a lot of this data to meet your needs).

If you are looking to gain new inbound links to your site, you can find one of your competitors that has a high number of inbound links (you’ll probably want to use the data coming from Yahoo for this) and click on that number, which will open up the page displaying all of those inbound links. This will give you a list of sites that may be interested in also linking to you.

SEO Quake also has plenty of options, including the ability to save data in a CSV file. One thing to be aware of is that SEO Quake will slow down your load time when you’re surfing, but you can easily de-activate it when it is not in use.

Read more about SEO Quake here.

Open Source Link Analysis Tool

Patrick Altoft of BlogStorm has created an open source link analysis tool that does some of the same functions as SEO Quake. Patrick’s tool is not a Firefox extension, so if you are not a Firefox user this may be a good choice. One of the reasons Patrick chose to release this as an open source project is to allow for others to develop and customize it further, so keep your eyes open for continued improvement.

Here is Patrick’s description of what the tool does:

* Uses Yahoo Site Explorer to find all pages on a site

* Pulls in link data for every page on the site & orders results by pages with the most links

* Allows you to drill down 2 levels deep into the link data for pages linking to the target site

* Accepts Google sitemaps imports

* Accepts single URL imports

* Lets you check the rankings for any page on any search engine

What’s Your Experience?

Have you used any of these SEO tools yet? If so, what is your opinion?

Search Engine Optimization is an ongoing process that cannot truly be accomplished by designing a website in a particular way, although a designer can cripple a site’s chances with a poor foundation. Building a search engine-friendly website should be the task of the designer, but there can be some gray areas.

First of all, what makes a website search engine-friendly? Things like optimized page titles, clean coding, proper use of header tags, alt tags, the location in the code of primary content, and anchor text are just some of the elements involved. Equally important, the designer should avoid using elements in the design that harm the site’s ability to be crawled. For a more in-depth look at the construction of search engine-friendly websites, see How to Create Search Engine-Friendly Websites.

When it comes to optimizing a website’s page titles, headers, and similar elements, we step into the gray area of the responsibility of the designer. Of course, the designer should make a considerable effort to build a site that is optimized, but what is the responsibility of the designer and what is the responsibility of the customer?

If you are paying a designer to build a site for you and you’re not giving him or her specific keywords and phrases that you would like to target, you are leaving it up to the designer to determine what words and phrases the site will be optimized for, and in many cases this is not in your best interest.

As a business owner, your website’s optimization is much more important to you than it is for the designer (that’s not to say that a designer doesn’t care, but the designer’s attention will typically be focused on the look and functionality of the site). You know your customers and potential customers better than the designer does, so shouldn’t you have some say in what words and phrases you target?

If the customer is not very knowledgeable about SEO and keyword targeting, and most won’t be, the ideal situation is for the designer and the client to work together on this issue. From my experience, one of the easiest ways to get a customer involved in this way is to simply explain that the words and phrases used throughout the site (titles, headers, etc.) will have a direct impact on the site’s ability to rank well. I typically will ask customers to give me a list of words and phrases that they think potential visitors might use in a search. This can be especially important when designing a site in an industry that I am not very familiar with.

Of course, some research may need to be done, and this can be another gray area. Should the customer take the responsibility to do keyword research or should the designer? My experience is that this research is usually more effective if the customer is involved, but that is not always possible. The designer should be knowledgeable enough to advise customers and should be willing to help, but ultimately it is best for the customer to be as involved as possible. Basically, if the customer is concerned with SEO, being involved in the process should be a reasonable expectation.

Setting realistic expectations may also be a responsibility of a designer. I’ve had some potential customers come to me and say, “I was told that you know how to make my website rank first or second for (insert ultra-competitive keyword here).” In this situation I obviously think it is important to point out that SEO requires ongoing work that can’t usually be accomplished just by creating the site a certain way. I often recommend that customers add a blog to their website to get more content and improve its chances for ranking well.

What is not the responsibility of the designer?

If the designer is simply creating the site, there are a number of SEO factors that really are not affected by the designer. Of course, there are some cases where the design firm is also providing some or all of these services, but that is not usually the case. When a designer is only responsible for creating the site, these SEO factors will not fall within his or her responsibility:

• Link building
• Link baiting
• Content creation
• SMM
• Press release writing
• Sales copy

What is your opinion?

How do you see the responsibility of a designer in regards to SEO? Are there specific tasks that you think a designer or a customer should be responsible for?

Search Engine Optimization is a common concern/priority when building a new website. However, it sometimes receives little attention after the site has been up and running for a while. For website owners and bloggers alike, it can be worthwhile to do a routine check-up on the SEO health of your site periodically. Here are 10 things you can do to help identify potential improvements that can lead to higher rankings and more search engine traffic.

1. Find and Fix Dead Links

Finding dead links on a website can be time-consuming and annoying if you try to do it manually. Dead-Links.com is a free online tool that will crawl your website and search for dead links. Once you know where the dead links are, you can easily correct them.

2. Check for Web Crawl Errors in Google Webmaster Tools

Another way to identify broken links is to login to Google Webmaster Tools and check the crawl errors that are listed. You will see a list of URLs that the Googlebot was not able to find on your site.

3. Check for Missing Title Tags

The title tag of a page is of course one of the most important on-page factors for search engine rankings. Every page on your site should have a unique and descriptive title. If you have a relatively small website, you can easily check for this manually. SEO Book had a free tool available that did this manually, but it has been blocked by Google. Google Webmaster Tools will provide you with this information under “Diagnostics” and “Content Analysis.” WordPress users can install the All-In-One SEO Pack plugin to control page titles throughout the website or blog and ensure that each page/post has a proper title.

4. Find Your Most Productive Search Phrases

Google Webmaster Tools provides some valuable information about your site’s rankings and what phrases searchers are using to find you. Under “Statistics” click on “search queries” and you will see the top 20 search queries in which your site is appearing. This information may help you to find a few phrases for which you didn’t even realize you were ranking well. In this case, you may be able to increase those rankings even more by optimizing your site or a specific page to specifically target that search phrase.

On the right side of the screen you will also see the top 20 queries that were used to actually reach your site. This shows what people are clicking as opposed to simply where you are ranking. On both of these lists, when you see specific queries you may know exactly which page on your site it is referencing. It’s not a bad idea to go to those pages and double check the meta descriptions to be sure that they do an effective job of telling potential visitors what the page is about and enticing them to click through from the SERPs.

5. Add “nofollow” Tags As Needed

Any time you link out to another website that you don’t want to be followed by search engines you can use a nofollow tag on the link. While there is no need to use nofollow on most links from your website or blog, they can be used for links to sites that don’t need your link juice, such as Google, Yahoo, etc. Most bloggers use NoFollow on links to FeedBurner for RSS and email subscriptions.

Additionally, you can also use nofollow tags on internal links to somewhat control how PageRank is passed throughout your site . For example, you will see a lot of website owners and bloggers that use nofollow tags on links to a contact page, since there is no need for the contact page to rank well. For more information about using nofollow on internal links, see Using NoFollow to Control PageRank Flow from SearchNewz.

From time-to-time it’s helpful to go through your site and see where it might be helpful to add nofollow tags. An easy way to do this is by using the NoDoFollow add-on for Firefox (there are several similar add-ons as well) which will show all nofollow links in pink/red as you are surfing.

6. Look for Opportunities to Add Internal Links

Internal links within your website can help to tell search engines which pages are most important. It can potentially help your rankings to add internal links to those pages that you would like to rank higher. If your website continually has new content added (such as with blogs) there will always be opportunities to improve internal linking. Older pages/posts may be well-suited to link to a page that hadn’t yet been created at the time when it was published.

7. Check Your Search Traffic Trends

If your website is suffering from falling search traffic it is obviously helpful to know some information about the trends. Most of us check daily/weekly stats on a regular basis, but longer-term trends aren’t always examined. Using Google Analytics, or a similar program, analyze your search traffic over a span of several months. What are the general trends? Which pages have increased search traffic? Which pages have decreased search traffic? You may be able to identify some problems that need to be addressed or some opportunities that can be taken advantage of by knowing a little bit more about the trends.

8. Check Your Keyword Density

What words and phrases are you targeting with your website? Do they appear on you page more than other words? The keyword density tool from TrafficZap will allow you to enter a URL and it will produce a report of the words and phrases with the most density on the page.

9. Test Your Page from a Spider’s Point-of-View

A spider simulator can help you to quickly see how a search engines sees your page. Summit Media has a nice, free spider simulator that will check several factors and provide you with a brief report. The report may help you to identify some simple improvements that can be made.

10. Check Your Rankings

One of the obvious things that you’ll want to check is your search engine rankings for your targeted keywords or phrases. Mike’s Marketing Tools has a nice rankings checker that will allow you to enter a URL and a search phrase and it will show you where you rank in the leading search engines.

1 – SEM Requires a Solid Foundation

While there are a lot of factors that play a role in determining your website’s chances for success, you’re very unlikely to achieve and maintain high rankings for competitive words and phrases if your site is poorly constructed. Search engine optimization includes far more than page titles, headers and alt tags, but a website built with a properly optimized foundation will have a much better chance for success.

Sure, there is more to it than how your site is coded, but the code is the first piece to the puzzle. Poorly-structured websites, even those that receive significant search engine traffic, are not reaching the potential that exists with some relatively simple planning. If you are building a new website or blog don’t cut corners to get it launched quicker, make sure it is done right. You’ll save yourself the headache of trying to optimize the site after it has been built. (For more information see SEO Basics for Blogs).

2 – SEM Takes Time and Work

If you want to rank well for competitive words and phrases, you will have plenty of competition that is willing to invest time and money into beating you. That’s not to say that you can’t out do the competition, but at least be realistic that it will not be easy.

If you are targeting less competitive long-tail search terms, ranking well may not be as difficult, but time-consuming keyword research and content creation may be needed. If it was easy everyone would have top 10 rankings, and that’s obviously not possible.

3 – There is No Magic Secret

Many people that are new to internet marketing are led to believe that only a handful of people out there hold a precious secret to automatic and almost effortless search engine rankings. Those who are selling e-books on the subject would like you to believe that with their magic formula you can quickly and easily dominate the competition.

The truth is, if it’s easy 1) everyone already knows about it, or 2) it will likely get you banned by search engines if you get caught. Of course there are plenty of great products and writers out there who provide genuine information, but the market in this field seems to be overrun with hype that preys on new marketers looking for fast results.

4 – Most of Your Traffic Will Not Enter Through Your Homepage

If you’ve been online for any amount of time and you pay attention to your website’s statistics, you’ve probably noticed that the majority of visitors who find your site through search engines actually come through secondary pages. This is especially true for blogs, in which case the individual blog posts are usually more optimized for specific search phrases than the blog front page.

If you are focusing all of your optimization efforts on your homepage you are missing valuable opportunities. Each page on your website/blog gives you an opportunity to target different words and phrases. Understand that many of your visitors will not be arriving via your homepage and set up the navigation so that they can still easily make their way through your site without needing the homepage.

5 – High Rankings Are Not Enough. Your Pages Still Need to Convert

High search engine rankings for competitive phrases will bring a lot of traffic. However, unless you’re content to just see high counts of visitors, your work is not done. If you are doing business online there is obviously some ultimate goal that you want to accomplish from your traffic. Do you want them to buy a product, subscribe to your blog/newsletter, click on a PPC ad? Traffic is just the first part of the equation. Converting that traffic is equally important.

The first step is to know what you want from your visitors. Now, set up your pages to lead them to take that action, whatever it may be. (For information on creating effective landing pages, see Copyblogger’s article Seal the Deal: 10 Tips for Writing the Ultimate Landing Page.)

6 – Results Are Not Permanent

Just because your website has good rankings or bad rankings today does not mean that things will not change. Especially for very competitive phrases, even highly-ranked sites need to at least monitor their rankings and be ready to take action if they fall. If you have achieved high rankings you can continue to tweak your pages, or you may want to leave them alone to avoid making changes that negatively affect your rankings. If your website is not ranking well, especially if it is new, don’t worry. Keep working to optimize your site and improvements are likely.

7 – Quantity and Quality Are Both Important

Obviously, having high quality optimized pages is important to gain search engine traffic. But don’t overlook the value of having a large number of pages on a site that are all contributing to the overall traffic numbers. Websites that have established themselves as authority sites in the eyes of the search engines can quickly achieve high rankings for new pages, while other less established websites may have been competing for the same traffic for a much longer time.

Having a huge number of pages indexed by search engines is one way to increase the amount of traffic you receive. Blogs are a great example of this. With each new blog post that is published there is now one more page that is capable of drawing search engine traffic. While this may not make a difference from day-to-day, if you can post frequently and regularly, it will make a big difference in a year’s time. (For more information see Driving Search Engine Traffic to a New Blog)

8 – Having a Game Plan is Crucial

Search engine marketing requires you to have a strategy and an idea of how you are going to accomplish your objectives. Unfortunately, it isn’t enough to just create content and watch the traffic flow. Taking some time at the start of a new project to determine your game plan can save time down the road and it can make your project much more successful. If you’ve already got a website and you’d like to get more visitors from search engines, it’s not too late to develop a game plan.

9 – Keyword Research is Worth Your Time

One of the frustrations with search engine marketing is that it can be very tedious. However, targeting the right keywords can be the difference between failing for extremely competitive phrases or thriving by targeting less competitive phrases. Keyword research is a big part of developing the game plan that was mentioned above.

If keyword research is done at the start of a project you’ll be ahead of the game and able to build the site around the words and phrases that you want to target. Fortunately, there are a number of useful tools to help with this process, and many of them are free. (For more information see 30 Keyword Tools to Use for Your Website)

10 – Unethical Shortcuts Have Consequences

If you are looking to build a website for the long-term, make sure you don’t violate any of the terms of ethical search engine optimization. Unfortunately, the search engines themselves have the right and the ability to determine what they consider to be ethical or unethical. Recently there has been a lot of attention on Google’s “punishments” for websites that buy and/or sell links. This is this most highly-publicized example right now, but other black hat techniques can also result in getting your site banned by search engines.

While this is certainly not an all-inclusive list, I think it does address a number of important issues related to search engine marketing. What other items would you add, or what do you disagree with?

Here’s a quick recap:

In July I received

• 587 visitors from search engines
• 563 visitors from Google (#4 traffic source)

In August I received

• 2,217 visitors from search engines
• 2,173 visitors from Google (#2 traffic source)

And now, the new stats from September

• 3,073 visitors from search engines
• 2,974 visitors from Google (#4 traffic source)

Although the growth wasn’t as drastic as it was in the previous month, it still is steady growth. The most commonly found posts received about the same number of visitors as they did in August, but the growth came from newer pages that didn’t exist in August.

In the original post I explained my strategy for growing search engine traffic, and I think my experience still indicates that creating more content and getting links are two of the most important things you can do as a new blogger.

If you have other experiences with search engines please share them in the comments.

Monday: Vandelay Website Design
Tuesday: Blogging Tips – Be Selfish and Help Someone
Wednesday: Blogtrepreneur – What is Your Best Source of Web Traffic?
Thursday: Super Blogging – Be Community Involved to Get More Traffic
Friday: SiteFever – Drive Traffic By Commenting First

Be sure to subscribe to each one to learn about strategies that are working for other bloggers.

New bloggers and website owners can easily get frustrated with the lack of search engine traffic that they are receiving. Typically, search engines take several months before they will show new sites very prominently in the SERPs (search engine results pages). If your website or blog falls into a very competitive market (like this one does) the situation can be even worse.

From the time period between July 1 and August 31, I experienced a substantial growth in search engine traffic, especially from Google, and I’d like to share the results and my conclusions with you. I think this information will encourage you and show you some ways that you can increase search engine traffic to your own blog.

To start with, here are my stats from July:

• 587 visitors from search engines
• 563 visitors from Google (#4 traffic source)

To give you some background information, this website was launched in February and the blog was posted on very sporadically from then until June. Traffic was very low and little time and effort was devoted to the site. The first week of July two different posts had a great deal of success with delicious and StumbleUpon (16,252 visitors came from StumbleUpon during July). That traffic helped bring some links, which wound up influencing search engine rankings.

As you can see from the stats above, search engine traffic in July wasn’t non-existent, but it also wasn’t substantial.

Here are the stats from August:

• 2,217 from search engines
• 2,173 from Google (#2 traffic source)

From July to August there was a 286% increase in traffic from Google searches, and a 278% increase in total search engine traffic. The graph below shows the visitors sent by Google from July 1 to August 31.

In July:

• Google sent visitors to 65 different pages on my site.
• The most common were:

• • Customize Your Own WordPress Theme – 171 visitors.
  • Find the Perfect Colors for Your Website – 103 visitors.
  • List of Free CSS Navigation Menus – 31 visitors.

• Only 2 visitors came directly to my homepage (which is not the blog homepage) through a Google search.
• 62 different pages had anywhere from 1 visitor to 29 visitors from a Google search.

In August:

• Google sent visitors to 110 different pages on my site.
• The most common were:
  • Find the Perfect Colors for Your Website – 488 visitors.
  • Customize Your Own WordPress Theme – 392 visitors.
  • New WordPress Blogs: 12 Steps to Set Up for Success – 137 visitors.
  • Effectively Testing Your Website in Multiple Browsers – 103 visitors.
  • 30 Keyword Tools to Use for Your Website – 97 visitors.

• Only 14 visits came directly to my homepage through a Google search.
• 106 different pages had anywhere from 1 visitor to 99 visitors from Google.

The lack of search traffic to my homepage isn’t surprising since there is so much competition for website design and my site is still relatively new. Despite traffic to the homepage still being low, search engine traffic overall grew by almost 300%.

These traffic results show that creating content on your website or blog is extremely important. Although my homepage competes directly with thousands of more established websites for phrases like “web design” and “custom website design,” each blog post has its own keywords and phrases that have far less competition and they are much more likely to be highly ranked by search engines. Of course, these pages are still on topic since they relate to some specific aspect of website design and development, blog development, or internet marketing.

The reality is that most of your visitors will not come to your site through the homepage. Every time you write a blog post on a different subject you are creating a page that will compete for search engine traffic from different keywords. As a result, the more posts you create (assuming they have quality content) the more chances you have to draw search engine traffic.

Say for example you have 10 pages on your website and each one draws 10 visitors from search engines each day. In this situation you would have 100 visitors each day that come from search engines. Now assume that you start a blog and for 1 year you add 5 new posts per week. With 50 weeks in the year (to use a round number) that would mean 250 new pages that were created through your blog. Now you have 260 pages on the site and if each one draws 10 visitors a day from search engines that would total 2,600 visitors each day.

Obviously, these numbers are fictional, but it shows the significance of creating new content in order to draw visitors from search engines.

How Can New Blogs Increase the Amount of Traffic They Receive from Search Engines?

1. Start with a solid, SEO-friendly blog structure. Search engine optimization was not something I focused on in the last few months, but rather when I created the site and started the blog. (For more information on this topic see SEO Basics for Blogs.) WordPress users have access to a wide variety of plugins that will enhance the search engine-friendliness of your blog – see our list of the top 10 SEO-related plugins.

2. Post Frequently. I saw an increase in the number of pages that received traffic from Google searches (from 65 to 110 pages) primarily due to new content being added. I expect this to continue as new content is consistently being published. The new pages may not be indexed instantly, but with blogs it may only take days or hours.

3. Get links from other websites and blogs. Most of the pages on my website that drew significant traffic from Google are also among the pages on my site with the most external links. For example, Find the Perfect Colors for Your Website had the most Google traffic in August and it also has the most external links of any page on the website (According to Google Webmaster Tools it has 3,310 links. The homepage in comparison has only 2,169 and the blog front page has only 600.)

To start getting some links you can submit your posts to blog carnivals, find reciprocal link partners, or comment on blogs that have removed the nofollow tags (like this one). You can find a list of “DoFollow” blogs at Courtney Tuttle’s D-List. Another way to draw links is to develop posts that become popular with social media, which is another topic altogether (stay tuned, I think Adnan from Blogtrepreneur has more on this subject on Wednesday).

4. Choose keywords wisely. Each of your blog posts can target different keywords and phrases, which will help you to show up in a greater variety of searches. A collection of valuable keyword tools can be found at 30 Keyword Tools for Your Website. The are a variety of tools listed, many are free, that will help you to find the right words and phrases to target.

I hope that by taking a look at some of these examples you have seen some ways that you can grow search engine traffic to your own blog, particularly if it has just recently been launched. Try applying some of these conclusions to your own blog and see how it affects your traffic.

Fortunately there are a number of tools available online (some free, some not) that can help with the process. I’ve included some basic information about 30 of these tools in this article. Some of these tools are designed specifically for creating and managing pay-per-click campaigns, but they can also be helpful if you’re just trying to optimize your site.

Free Keyword Tools

Keyword Density
Cost: Free
Type in a URL and you’ll see what phrases are used most commonly throughout the page. You have the choice of including meta tags, alt tags, and the title, as well as the choice of 1, 2, or 3 word phrases.

Keyword Research Tool from Apogee Web Consulting
Cost: Free
This tool allows you to see what words and phrases are used in your competitor’s keyword meta tag.

SEO Book’s Keyword Suggestion Tool
Cost: Free
One of the more useful and unique free tools, SEO Book’s Keyword Suggestion Tool not only gives you related words and phrases, but it also shows the search volume and provides links to the results from other keyword tools like Wordtracker and Keyword Discovery. The links help you to save time by not needing to visit a lot of different sites and perform repeated searches.

SEO Book’s Keyword List Generator
Cost: Free
Are you having trouble coming up with a list of keywords for a PPC campaign? Type in your URL and a few keywords and this tool will generate a list for you.

SEO Book’s Keyword Misspelling Generator
Cost: Free
Sometimes searchers will spell words incorrectly. You can benefit from knowing common misspellings and using them in your PPC campaigns. This tool eliminates the need for you to figure out possible misspellings. Type in a word or phrase and you will get a list of possible mistakes.

Yooter Keyword Suggestion Tool
Cost: Free
Find search volume and related phrases with this free tool. It also allows you to export the results into a spreadsheet program (like Excel) via a .CSV file. Requires registration.

McDar Keyword Analysis Tool
Cost: Free
When you enter your URL and a keyword or phrase this tool will provide information on your page and compare it to the top 10 ranked sites by Google for the word or phrase that was entered. The data includes total pages indexed, links and PageRank.

iBusiness Promoter
Cost: Free
IBP is a free tool that shows you search volume and the level of competition for keywords. It also shows you which keywords your competitors are using. Requires download.

Overture’s Keyword Selector Tool
Cost: Free
One of the most frequently used free keyword tools. Enter a word or phrase and Overture will show you related phrases and search volume for each.

Google’s Keywords Tool
Cost: Free
A fairly simple and valuable tool that shows related keywords, search volume, and levels of competition. Provides a large number of suggestions based on pages that are ranking well in Google for the phrase that is entered.

Keyword Suggestion Tool from Wordtracker
Cost: Free
Enter a keyword of phrase and get up to 100 suggestions as well as an estimate of their daily search volume.

Bookmarklet from SEOproToolz
Cost: Free
With the bookmarklet you will drag a link to your browser’s toolbar and the tool will be usable. It will analyze any page that you are visiting to generate related keywords with prices for Google AdWords.

Meta Tag and Keyword Analyzer from Submit Express
Cost: Free
Enter a URL and the tool will show you the meta tags of the page, keyword density, load time, file size, and links found on the page.

Good Keywords
Cost: Free
Includes keyword suggestions, misspelling suggestions, phrase builder and more. One of the more simple programs that still includes multiple tools. Requires download.

Keyword Research Tool from Webmaster Toolkit
Cost: Free
A simple free tool that will help you to find related keywords to a specific word or phrase. It will analyze top-ranked pages in a search engine that you select and report on words and phrases used by those sites.

Keyword Tools with a One-Time Cost

Rapid Keyword
Cost: $69.99 (one-time payment)
Rapid Keyword helps you to find the right keywords by showing results from both Google and Overture’s suggestion tools, as well as by providing possible misspellings. You can analyze the competition by seeing how many competitors are on the major PPC programs for particular words or phrases. Several keyword management tools are included. Offers a one-day free trial and a 60-day money back guarantee.

The Permutator
Cost: $49.99 (one-time payment)
The Permutator includes a keyword suggestions tool as well as various tools to help create and manage lists of keywords. It also includes a typo tool and an ROI calculator. A trial version is available.

Keywords Analyzer
Cost: $97
One of the more popular keyword tools, it will help you find search phrases that have low levels of competition for PPC ad campaigns. Keywords Analyzer also helps you to analyze your competition and manage PPC campaigns.

Web CEO
Cost: There is a free version as well as $189 and $389 versions.
Web CEO is SEO software that includes a keyword tool. The keyword tool provides keyword suggestions, search volume, competing sites, and much more. Requires download.

The Dowser
Cost: There are several tools that range from free to $297
You can research and manage keywords with the free tool, or purchase tools for keyword harvesting, misspellings, and more.

Keyword Elite
Cost: $167
With Keyword Elite you will be able to see the keywords that your competitors are bidding on. You’ll also be able to do keyword list generation and management.

Keyword Cruncher
Cost: $37
A keyword analysis tool that works with Wordtracker. With Keyword Cruncher you can import data from Wordtracker and manipulate and analyze the data. There is a 30-day money-back guarantee.

Keyword Tools with On-Going Costs

Wordtracker
Cost: $299 for a 1-year subscription (weekly and monthly options also available)
One of the leader keyword tools, Wordtracker will help you to find the right keywords and other phrases in which they are commonly used. There is an option for a free trial.

Marketo
Cost: Standard version is $49 per user (the first user per organization is free) plus 7.9% of the amount spent on ads managed by Marketo
Marketo is a complete PPC management program. It features keyword tools, bid management and optimization, and testing tools. There is an option for a 30-day free trial.

Keyword Discovery
Cost: $599
Compiles statistics from over 180 search engines. Features include keyword research, suggestions, misspellings, seasonal search trends, and more.

Adgooroo
Cost: Starts at $99 per month
Adgooroo will show you the search phrases that are generating most of the traffic for your competitors. It’s tools help you to identify more specific phrases that will bring more traffic. There is an option for a 10-day free trial.

NicheBot
Cost: $9.97 – $19.97 per month
NicheBot provides a variety of keyword research tools as well as keyword list management tools. It also includes an affiliate product finder.

SEOmoz Keyword Difficulty Tool
Cost: The tool is limited to premium members, which currently costs $49.00 per month
This tool is used to analyze the competitive landscape that exists for particular search phrases. It is intended to show how difficult it will be to achieve high rankings for specific terms.

HitTail
Cost: Basic version is free, HitTail Plus is $9.95 per month
HitTail shows you real time statistics on what keywords visitors are using to find your website. Widgets are available for Blogger, TypePad, SquareSpace and WordPress.

Wordze
Cost: $35 per month
Wordze offers tools such as keyword research, historical keyword data, analysis of competitors, importing keywords, managing projects and more.

What is Google’s Supplemental Index?

Google’s goal as a search engine is to provide the user (searcher) with the most relevant search results possible. In order to produce the best search results, Goggle excludes certain pages from appearing in the SERPs (search engine results pages). The pages that are excluded are then thrown into what Google calls the supplemental index.

The supplemental index is just what it says, a supplement to the main index. Pages in the supplemental index may show up in search results, but pages in the main index will be given priority.

Why is it Important to Keep Your Pages Out of the Supplemental Index?

With the incredibly massive (and growing) number of searches done through Google, it’s not uncommon for websites to get 60% or more of their traffic from Google. Obviously, if you have more pages from your site being indexed and displayed by Google, you’re likely to get more traffic. Decreasing the number of pages that are in Google’s supplemental index can significantly increase your traffic virtually overnight (see Nathan Metzger’s article about how he increased his traffic by 20%).

Why Do Pages Get Put Into the Supplemental Index?

There are a number of reasons that a page can get thrown into the supplemental index. The most common reasons (and more importantly, solutions) are discussed below.

1 – Duplicate content

The most common cause of a page being condemned to the supplemental index is duplicate content. Google wants the pages in its search results to be unique. Searchers are likely to become frustrated if the information they find through a search is nothing more than the same exact content on different pages and sites. In order to avoid this, Google eliminates pages with duplicate content from its search results by keeping them in the supplemental index.

Solutions:

Duplicate content issues are especially common with blogs. A page on a blog that contains a blog post is duplicated when the post reappears on the homepage, category pages, and date pages. You can get around this by:

- Displaying only excerpts of the posts on the homepage, category pages, and date pages. This shows the search engine only a portion of the post, which is usually not enough to have it flagged as duplicate content.

- Use a robots.txt file to tell search engines to ignore parts of the blog/site that are duplicated. This will help to make sure that the original post page is not caught in the supplemental index. For more information on your blog’s robots.txt, see Not So Boring Life.

- The Duplicate Content Cure plugin is another option for WordPress users. This plugin tells the search engines to ignore the pages that contain duplicate content. By default the plugin prevents search engines from indexing category pages, but that can be changed (which I recommend doing). Category pages can contain strong, related keywords that will produce good search engine rankings.

2 – Lack of text on a page

Pages with little text/content are more likely to be placed in the supplemental index. Pages that only contain links are also likely targets.

Solution:

Add more valuable content (not just links) to any important pages that are in the supplemental index.

3 – Long URLs

Pages with longer URLs and with more variable characters (question marks, hyphens, etc.) are frequently placed in the supplemental index. This is not always the case. Plenty of pages that rank well in Google (especially pages on a blog) have long URLs. The long URL is not always a problem, but when combined with some of the other items on this list it is more likely to cause Google to question the page.

Solution:

For static websites you can give your files shorter names or decrease the number of folders that are used, which can also add to the length of the URL. WordPress users have the option to specify the URL structure that should be used. From your dashboard go to “Options and Permalinks”.

4 – Repetitive page titles

Every page on your website should have a unique title. Simply using the company name or website name for the title on every page does not help the search engines to determine the subject/topic of each individual page.

Solution:

Give each page a unique and descriptive title. WordPress users can download the All In One SEO Pack plugin that will allow you to enter the page title as well as the meta tags for each post.

5 – Orphaned pages

An orphaned page is a page on your site that has no links from other pages on your website. If no other pages link to it, Google assumes it is not important or it has no valuable information.

Solution:

Every page that you hope to have indexed by search engines should have links pointing towards it. One way of accomplishing this is to use a sitemap that includes a link to every page. A sitemap alone is unlikely to make enough of a difference, though. Use links throughout the site wherever they are relevant.

If you have no links to a particular page because you don’t want it to be found, include it in your robots.txt file.

6 – Buried pages

Google takes into consideration how close or far a page is from the site’s homepage (meaning how many clicks from the homepage it takes to reach a certain page). As a general rule of thumb, all pages on your site should be able to be reached from the homepage in two clicks or less (not always possible on larger sites).

Solutions:

- The key is to improve the links to a page throughout the site. Look for relevant pages that can link together. This is the most powerful way to show search engines that the page is important.

- Use links in the text of articles, as well as at the end of articles (to related posts)

- Link to your most important pages from your homepage.

- The sitemap, mentioned under “Buried Pages”, also helps here. Link to the sitemap from your homepage and every page on the sitemap will be only 2 clicks from the homepage.

How can I see which pages are in the supplemental index?

There used to be an easy search to see the pages in the supplemental index, but a few weeks ago Google disabled the search. For now the best way that I know of is type the following in the Google search box: “site:www.yourdomain.com”.

When the search results come up you will see all of your pages that are indexed by Google. This includes pages that are in the main index as well as the supplemental index. The pages from the main index will be listed first. The pages from the supplemental index will say “supplemental result” in green text next to the URL. You will need to click through the pages to find the point where the pages from the main index end and pages from the supplemental index begin. [UPDATE: GOOGLE NO LONGER DISPLAYS WHICH LINKS ARE IN THE SUPPLEMENTAL INDEX]

From my recent personal experience with this site and with a client’s site the robots.txt file, post excerpts, and internal linking have made a huge difference (getting some inbound links from external sites has helped as well). On this particular site we have increased the number of pages in the main index by over 100 in the past 2 or 3 weeks.

Try some of the methods above and you should be able to get your most important pages in the main index and bring in the traffic.