I tend to check the admin each day to check the number of products in each feed, to make sure I do not miss a feed suddenly going bad. I thought there must be a better way, Once you have several feeds, and cron automation setup it is easy not to notice individual feeds not working as you expected without daily checking and even then you might not notice smaller variations.
I thought that if there was an admin option to retain a history of how many products were imported to each feed, and compare that with the current import.
This would make it possible to highlight any feeds that suffer a big reduction in products, or even fall to zero. A report showing the difference between the number of products yesterday, or the average of the week would be great.
I would not suggest retaining history for longer than a week, just enough to be able to identify a failed feed.
What do you think David, is it viable, and do you think it is a good idea?
Cheers
Dave
Hi David
All looking good on the database and the new field populated perfectly.
Only issue I had was not strictly a PT issue at all, as I am on AWS EC2, there are restrictions on sendmail, that PHP mail uses by default. Recommendation is to use pear/Mail
Is there a simple way of making the PHP mail function use that instead?
Cheers
Dave
Hello Dave,
Sure - in place of:
mail($email,"Monitoring Report ".date("Y-m-d"),$body);...if you're able to use the local sendmail program then
<?php
include("Mail.php");
$headers["From"] = $email;
$headers["To"] = $email;
$headers["Subject"] = "Monitoring Report ".date("Y-m-d");
$params["sendmail_path"] = "/usr/lib/sendmail";
$mail = &Mail::factory("sendmail",$params);
$mail->send($email,$headers,$body);Or if there is no local delivery agent available, you can send through an external SMTP server, for example;
include("Mail.php");
$params = array();
$params["host"] = "smtp.example.com";
$params["port"] = "25";
$params["auth"] = true;
$params["username"] = "username";
$params["password"] = "password";
$headers["From"] = $email;
$headers["To"] = $email;
$headers["Subject"] = "Monitoring Report ".date("Y-m-d");
$smtp = &Mail::factory("smtp",$params);
$smtp->send($email,$headers,$body);(you can send mail from a gmail account using a host value of "ssl://smtp.gmail.com" and port 465)
Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
Looks good, I have Pear and Mail.php installed, but no local sendmail, so I will need to smtp setup for google, and remember to open the port (thank-you for the reminder)
Will let you know how I get on
Cheers
Dave
Hi David,
I was wondering if it is possible to monitor error_log files so whenever an error occurs and after the finish of the cron job to be informed by email
Thank you
Steve
Hi Steve,
This would really need to be something set-up outside of the scope of Price Tapestry since there any number of reasons for an Apache error_log entry, especially when multiple sites are running on the same server with a common error_log.
Of course, at any time if you are finding error log entries relating to changes made to Price Tapestry scripts that you are not sure how to resolve, let me know what is displayed and the line referred to and I will try and help out of course...
Cheers,
David.
--
PriceTapestry.com
Hi David,
Thank you very much for the info.
Great support.
Cheers
Steve
Hi Dave,
It would be straight forward to add products_last field to the `feeds` table so that after your cron job you can run a monitoring script and send an email alert if the number of products in any feed has reduced by a certain percentage.
First, create and run the following dbmod.php script to add the new products_last field to the `feeds` table:
<?phprequire("includes/common.php");
$sql = "ALTER TABLE `".$config_databaseTablePrefix."feeds`
ADD `products_last` INT(11) NOT NULL";
database_queryModify($sql,$result);
print "Done.";
?>
Next, edit includes/admin.php and look for the following code at line 714:
$admin_importFeed = $rows[0];...and REPLACE with:
$admin_importFeed = $rows[0];$sql = "UPDATE `".$config_databaseTablePrefix."feeds` SET products_last = products WHERE filename='".database_safe($filename)."'";
database_queryModify($sql,$result);
Finally, create the new file scripts/monitoring.php:
<?phprequire("../includes/common.php");
$sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds` WHERE products_last > 0 ORDER BY filename";
database_querySelect($sql,$rows);
$email = "you@example.com";
$threshold = 10;
$body = "";
foreach($rows as $row)
{
$p = (($row["products_last"] - $row["products"]) / $row["products_last"]) * 100;
if ($p >= $threshold)
{
$body .= $row["filename"]." ".$row["products"]." (".$row["products_last"].")\n";
}
}
if ($body)
{
mail($email,"Monitoring Report ".date("Y-m-d"),$body);
}
?>
Edit the email address and the required threshold to monitor at lines 5-6, so the above would generate a report if the number of products in any feed has reduced by 10%.
Setup monitoring.php to run after cron.php in your cron job, so if your current command ends for example;
/usr/bin/php cron.php...append the command by separating with a semi-colon, e.g;
/usr/bin/php cron.php;/usr/bin/php monitoring.phpHope this helps!
Cheers,
David.
--
PriceTapestry.com