You are here:  » Enable gzip compression for increased search engine coverage


Enable gzip compression for increased search engine coverage

Submitted by support on Wed, 2007-10-03 21:37 in

Hi everyone,

I've held an SEO theory for some time that is based around the fact that a search engine only has a finite level of resources (bandwidth, disk space etc.) with which to index the web. It follows, therefore, that rather than say "I will only index X number of pages of site Y", an intelligent search engine will say "I will allocate X amount of resource to indexing site Y". Therefore, if you can reduce the amount of effort required to index your site, you should (in theory) be able to increase the number of pages indexed.

One thing you can do, if you are running Price Tapestry on the PHP / Apache combination is to enable gzip compression of your pages. This means that the output from your web server is compressed on the fly, and then decompressed by the browser (or search engine crawler), thus saving on bandwidth - and because of this, Googlebot and other search engines might consider indexing more of your site as a result.

Using gzip compression requires the zlib library to compiled into your PHP installation (not enabled by default). To find out if you have zlib, create a script to view the output of phpinfo() and look for "--with-zlib" in the configure string. If zlib is compiled in to your installation of PHP, then in html/header.php, simply add the following line of code immediately after the opening PHP tag:

ob_start("ob_gzhandler");

That should be all there is to it. To verify that the handler is having an effect, you can use the Web Developer plugin for Firefox. Browse to your homepage, and then use the Web Developer menu to select "Information > View Response Headers". In the headers, look for:

Content-Encoding: gzip

That will confirm that you have enabled compression, and could mean that with everything else being equal, search engines will index more of your site as a direct result!

Cheers,
David.

Submitted by jonny5 on Thu, 2007-10-04 08:30

could this be added to the index page or another page as i have no header.php (well i do but there is nothing in it)

Submitted by support on Thu, 2007-10-04 09:13

Hi Jonny,

On your Price Tapestry sites, header.php is in the html directory of your site. If you want to use the same technique on any other site, you just need to make sure that the line of code to enable compression comes before any HTML is generated - that's all, so it can go inside an index.php file as long as it is right at the top...

Cheers,
David.

Submitted by crounauer on Thu, 2007-10-04 09:53

Hi David,

There is no specific reference to gzip, is this correct?

Date: Thu, 04 Oct 2007 09:23:09 GMT
Server: Apache/2.0.52 (Red Hat)
X-Powered-By: PHP/5.2.4
Vary: Accept-Encoding
Connection: keep-alive
Content-Type: text/html
Content-Length: 13100
Age: 68
Via: HTTP/1.1 GWA (remote cache hit)
X-fetchid: 1201
200 OK

Compare Lingerie

Submitted by support on Thu, 2007-10-04 10:16

Hi Simon,

Those headers don't indicate that gzip is enabled, but they also indicate that the request never made it to your server! The following line:

Via: HTTP/1.1 GWA (remote cache hit)

...means that the "Google Web Accelerator" handled the request entirely. To test your setting you will need to disable GWA temporarily so that you know the request was handled by your server. Also try a deep page just incase your ISP also has a cached version...

Cheers,
David.

Submitted by jonny5 on Thu, 2007-10-04 17:14

Date: Thu, 04 Oct 2007 16:46:38 GMT
Server: Apache/2.0.55 (Red Hat)
X-Powered-By: PHP/4.4.7
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html

200 OK

this is what i get??? has it worked?

thanks

Submitted by support on Thu, 2007-10-04 17:32

Hi Jonny,

That indicates that it hasn't been enabled. I've just double-checked, and it requires the zlib library to be compiled into PHP. If it doesn't exist, you won't get an error, it just won't have any effect. If you're interested in enabling this you would need to contact your host and ask for the zlib extension to be included in your PHP installation. Apologies for not making this clearer, i'll add a note to the original post...

Cheers,
David.

Submitted by jonny5 on Thu, 2007-10-04 17:33

cheers

Submitted by jonny5 on Thu, 2007-10-04 20:55

Hi David , I have checked my account regarding ZLIB and have the below in my account regarding installed modules

I assume this is all enabled already?

ZLib Support enabled
Compiled Version 1.2.3
Linked Version 1.2.1.2

Directive Local Value Master Value
zlib.output_compression Off Off
zlib.output_compression_level -1 -1
zlib.output_handler no value no value

Submitted by support on Fri, 2007-10-05 07:26

Hi Jonny,

They're identical to my settings, so it should be enabled. The other possibility is that your browser isn't sending the appropriate request header that will indicate that a gzipped response is acceptable, which could happen if your ISP's proxy server is having a say in the matter.

If you email me a link to your site I'll check the headers that I'm getting for you...

Cheers,
David.

Submitted by jonny5 on Mon, 2007-10-08 17:19

well David , you are a clever sod.

just checked my sites , one site that was struggling and had 20 pages indexed and was falling, now has over 500 pages indexed as of today!! another now has 300 pages indexed and each site I added this to have all had a large increase in indexing!!!

many thanks

Submitted by crounauer on Tue, 2007-10-09 10:41

Hi David,

What was the outcome of the posts by jonny5 as I have exactly the same response headers for www.germanlanguagecourses.co.uk and just wanted confirmation that the compression is working?

Thanks,
Simon.

Lingerie

Submitted by support on Tue, 2007-10-09 10:43

Hi Simon,

In the file that I checked, the call to ob_start() came after output had started, so Jonny just needed to move it up to the top of the file, and it then worked fine.

Cheers,
David.

Submitted by crounauer on Tue, 2007-10-09 11:29

Hi,

It seems that it has been working all along. Even now when I use the Web Developer tool to view the response headers, no information is given. I assume that the results have somehow been cached??

Anyhow, I used an external web based tool to view the response headers and all is OK!

Thanks,
Simon

Lingerie

Submitted by henk on Thu, 2008-04-03 09:39

Hi

I have this one:

Via: 1.1 SRV-FRIED-03
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Transfer-Encoding: chunked
Date: Thu, 03 Apr 2008 09:27:47 GMT
Content-Type: text/html;charset=iso-8859-1
Server: Apache/1.3.31
X-Powered-By: PHP/4.4.1
Keep-Alive: timeout=15, max=999

200 OK

But server details:

zlib
ZLib Support enabled
Compiled Version 1.1.4
Linked Version 1.1.4

Directive
zlib.output_compression Local Value Off Master Value Off
zlib.output_compression_level Local Value -1 Master Value -1
zlib.output_handler Local Value no value Master Value no value

???

Thx Henk

Submitted by support on Thu, 2008-04-03 09:48

Hi Henk,

Are those headers after adding:

ob_start("ob_gzhandler");

...to your html/header.php?

Might be worth emptying your cache incase it is the same issue mentioned above and then trying again...

Cheers,
David.

Submitted by henk on Thu, 2008-04-03 12:57

Are those headers after adding: yes

I think the cache is clean.

Thx

Henk

Submitted by support on Thu, 2008-04-03 15:58

Hello Henk,

I would create a test page to eliminate anything else....

gzipTest.php:

<?php
  
if (ob_start("ob_gzhandler"))
  {
    print 
"This page should be gzipped!";
  }
  else
  {
    print 
"An error occurred loading gzhandler";
  }
?>

Cheers,
David.

Submitted by henk on Thu, 2008-04-03 17:49

its on :)

Thx

Submitted by GemViper on Mon, 2010-05-03 08:43

I know this is a 2 year old thread but it's a good one, perhaps more important today than it was back then. Here is another code option with options.

<?php
 
if (substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip')) ob_start("ob_gzhandler"); else ob_start(); 
?>

Place that before the DOCTYPE.

Submitted by Rocket32 on Fri, 2013-07-12 21:17

Hello Dave, Hosting company did maintenance and put files on another server/account. I tried suggestions mentioned above with no luck. Now I get following error from pages

Warning: ob_start() [ref.outcontrol]: output handler 'ob_gzhandler' conflicts with 'zlib output compression' in /home/xxxx/public_html/xxxx.com/html/header.php on line 2.

I tried disabling zlib with following lin in .htaccess and I get 500 Internal server error

php_value zlib.output_compression off

Is zlib.output compression an acceptable method of getting pages compressed for search engines? If it is what code do I add to header?

Submitted by support on Sat, 2013-07-13 07:26

Hi Rocket,

zlib.output_compression is an integer setting so that may be why the 500 error occurred when setting it to off in .htaccess - however it does imply that your pages are already being gzip compressed - and it's exactly the same as using ob_gzhandler so all good!

If you wanted to be sure, setting to 0 in .htaccess should do the trick, and then ob_gzhandler() will be doing the job instead:

php_value zlib.output_compression 0

Cheers,
David.
--
PriceTapestry.com

Submitted by Rocket32 on Sat, 2013-07-13 15:15

Thanks David for feedback.

The 500 errror occurs agains. I'll just go with the default zlib since it does the same thing. I should not have to add anything in the header I'm guessing.

Submitted by support on Sat, 2013-07-13 16:33

That's correct - nothing additional to add. If you want to double check that you're sending out gzip compressed pages use a checker such as just to confirm...

Cheers,
David.
--
PriceTapestry.com