When making a website SEO friendly, one of the first orders of business is analyzing the URL structure. If the website utilizes the proper keywords in the URL filenames, then the next step is to make sure that search engines (and users) are only able to view one version of each page.

Improving URL Structure in Apache / IIS

Below are steps that we take with every server and website that we manage to fix two potential problems common to almost every webmaster.  Our solution will help users improve and work with URL redirects and canonicalization for Apache & IIS webservers.

Please note, that the below steps work very well in tightening up a website’s URL structure, however, only attempt the below steps if you absolutely know what you’re doing.  If this is your first time and you feel confident you can implement the changes, be sure to test the website URLs thoroughly before implementing them on a live website.  Improper usage of these steps for Apache or IIS can really screw up your website.  Mad Fish SEO cannot be held responsible for any damage you cause.

Problem #1: Sometimes search engine crawlers index http://example.com, and http://www.example.com, and consider the two versions different URLs with the same content.  This could result in duplicate content filtering of your website home page.

Solution: 301 Redirect the non-www version of the website to the www version.  This is also known as canonicalization.

Problem #2: Sometimes search engine crawlers index multiple versions of your website’s home page. For example a search engine may index http://www.example.com/ , and http://www.example.com/index.htm, and have to make a decision about which URL is the correct URL for your home page.  The same problem can occur within sub-folders of a website. For example, http://www.example.com/folder/, and http://www.example.com/folder/index.htm.

Solution: 301 Redirect the index file of the root or sub folders to the same URL but without containing the index file in the URL.

Apache Solution

With Apache, the .htaccess file is your best friend for the process.  Chances are you are already using an .htaccess file in your website’s root.  If you have no .htaccess file in your web root, you will need to create one, and upload it to your website’s public root location.

If a .htaccess file already exists for your website, you will need to copy it to your local computer, and append the below directives to it.

If you are using a CMS, you may want to consult your online documentation before making any changes.  The directives may already be present in your .htaccess, but commented out.  Please review any existing .htaccess file before making edits.  Always make sure to backup an existing .htaccess file prior to posting changes.  This way if you encounter a problem, you can always revert back to your original .htaccess file.

If at any point while going through the  steps below your website becomes inaccessible, simply rename the .htaccess file to old.htaccess.  This causes the changes you had previously made to no longer affect your website.  This also holds true to any previous information in the file.  If you already had the file in the webroot, simply remove the information you’ve appended, and overwrite the file on the server.

Add the following directives to your .htaccess file.  You will need to change “example.com” to the domain you are working with.

Make sure the following code is near the top of the .htaccess file (and above any redirect directives).

# Code for .htaccess file
Options +FollowSymLinks
RewriteEngine on

Apache: Redirecting the non-www to the www version

#Code for .htaccess file

RewriteCond %{HTTP_HOST} ^example.com
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

To test, type non-www version of your URL into the browser (in this case it would be http://example.com). If everything is working correctly, the browser should redirect you to the www version.

Apache: Redirect the index file of the domain (or folder) directly to the domain (or folder)

#Code for .htaccess file

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9} /.*index.htm? HTTP/ [NC]
RewriteRule ^(.*)index.htm?$ http://www.example.com/$1 [R=301,L]

** Please note that if your index file is index.php, index.html, or has a file extension other than “.htm”, you will need to update the below code accordingly.

** Please note that you can use this same process if your web site uses a file named default.htm.  Simply change the references of “index”, to “default”, and update the extension accordingly.

IIS Solution

IIS web servers can appear to be daunting for redirects.  Creating the appropriate redirects can be more difficult than just adding code to your htaccess file.  In fact, unless your server has been setup to use an .htaccess file, adding an .htaccess file to your site will do nothing.

This used to make things very complicated for an SEO company trying to implement better URLs on an ASP or Coldfusion website.

Below are step-by-step instructions for fixing the URLs in IIS, and improving the manor in which the search engines index your website.

IIS: Redirecting the non-www to the www version in IIS

1.)   Open the IIS Manager

2.)   Right click on the website/domain you want to add the 301 redirect

3.)   On the “Web Site” tab, click the “Advanced” button to the right of the IP address

4.)   Under “Multiple identities for this website” if there is a record without “www”, (i.e. example.com), select it, and click “Remove”

5.)   Click “OK” to close this window and then click “OK” to close the domain’s “Properties” window

6.)   You will now need to add a new website to the IIS by right clicking on “Web Sites” > “New” > “Website”

7.)   Add a description of something meaningful (i.e. “301 Redirect for example.com)

8.)   In the “Host header for this Web site”, enter the domain name without the “www” (i.e. example.com)

9.)   Select the path for the website.  We recommend pointing this domain to an empty folder somewhere on the server (i.e. c:inetpub301domain).

10.) Once the new website has been added, right click on it in IIS and select properties.

11.)  Select the “Home Directory” tab

12.)  Click the radio button for “A redirection to a URL” and enter the following code into the text field:

http://www.example.com$S$Q (replace “example” with your actual domain name)

13.) Check the box for “The exact URL entered above”, and also check the box next to “A permanent redirection for this resource”.  Click “OK” and “Apply”.

That’s it!  Quite a bit more work than the Apache method.

Now test that things are working by going to http://example.com.  The address bar of your browser should redirect to http://www.example.com.

IIS: Redirect the index file of the domain (or folder) directly to the domain (or folder)

Last but not least we need to make sure that crawlers index the correct version of the website root and folder URLs.

1.)   Download and install “Ionic’s Isapi Rewrite Filter” http://www.codeplex.com/IIRF

2.)   When you install the Isapi Rewrite DLL, you will need to create a blank file called “IsapiRewrite4.ini”, and place it in the same folder as the DLL.

3.)   Open “IsapiRewrite4.ini” in notepad (or a text editor) and paste in the following code:

#if you need to troubleshoot,
#you can uncomment the next two lines to output a log file
#RewriteLog  c:tempiirfLog.out
#RewriteLogLevel 5
RedirectRule ^(.*)/default.asp /$1 [R]

**Please note that you should change default.asp to the correct filename of your website’s home page.  In many cases the filename is index.htm or default.htm

4.)   Save the file

That’s it!  Now navigate through your website, and verify that the redirect is working correctly.  Type http://www.example.com/default.asp and if all has gone well, the address bar should redirect you to http://www.example.com.

Feel free to comment.

Let me know if you feel I missed anything that would be helpful to readers, or if you encounter problems along the way.

~Ben Herman

Ben Herman is the Director of Digital Strategies and resident programming geek at Mad Fish SEO. Follow Ben on Twitter @ben_herman