Hi David,
I just wanted to run this by you to see if it was relevant or anything to do with how PT works.
I recently installed lfd on server and so get notified of many log issues, which has brought to light the following.
I am gettin alot of messages from the server blocking IPs that attempt too many connections. They get blocked when more than 95 connections are attempted, but as it is happening so frequently, I am wondering whether all the SQL involved in PT means that users will legitimately be making lots of connections to the server?
I have quite a few installations of price tapestry on findallsorts, for category purposes, so I just wondered are all these connection attempts related to all DB in PT and so legitimate visitors being blocked?
My server support at first just increased the limit there and says it is nothing to worry about, but I wondered if it is a PT thing that I need to look into and rectify some setting somewhere. It may be nothing to do with PT and just some server thing, but I thought I would just check with you.
Hi Clare,
There shouldn't be any reason for a normal user or genuine crawler (e.g. Googlebot) to up go 95 connection per user, so this is probably a symptom of unintelligent crawling by scrapers.
This happens to every large site - they don't obey robots.txt so nothing that can really be done about it - in which case it's actually a good thing that your configuration is blocking these requests!
Genuine, well behaved crawlers do not make excessive parallel requests to a single host for this very reason; so there shouldn't be any problem with indexing; and if you never encounter an connection refused error when browsing your site manually I would agree with your host that this is nothing to worry about!
Cheers,
David.