Hi David,
I wonder if you could please point me in the right direction with this?
I run multiple sites and have gone down the route of downloading all merchants into a "master" directory and then making reference to this directory from each individual site. This line of code is added to each config.php file, which references the master directory.
$config_feedDir = "/var/www/vhosts/###/###/feeds/httpdocs/datafeeds/";This all works well except when I do a CRON JOB on a site which is on a different URL. The CRON JOB doesn't throw any errors. This is what I have on SSH...
+ cd /var/www/vhosts/###/###/fashion/httpdocs/scripts/
+ /usr/bin/php /var/www/vhosts/###/###/fashion/httpdocs/scripts/import.php @ALL
backfilling reviews...[done]I think that the problem lies in the fact that I am not making proper reference to the master directory.
This is the import.php file. You will notice references to "$config_feedDir"...
if ($filename == "@ALL")
{
$sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds`";
if (database_querySelect($sql,$rows))
{
foreach($rows as $feed)
{
if (file_exists($config_feedDir.$feed["filename"]))
{
import();
}
}
}
}
elseif($filename == "@MODIFIED")
{
$sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds`";
if (database_querySelect($sql,$rows))
{
foreach($rows as $feed)
{
if (file_exists($config_feedDir.$feed["filename"]) && ($feed["imported"] < filemtime($feed["filename"])))
{
import();
}
}
}
}
else
{
$sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds` WHERE filename='".database_safe($filename)."'";
if (database_querySelect($sql,$rows))
{
$feed = $rows[0];
import();
}
}Thanks in advance.
Simon
Hi David,
I have done that, here was only one occurrence.... and it has been made global too.
MagicParser_parse($config_feedDir.$admin_importFeed["filename"],"admin__importRecordHandler",$admin_importFeed["format"]);and yes you were right in that it didn't import from within admin without making this change.
Thanks again,
Simon
Hi David,
I have narrowed the problem to within the /scripts/import.php file.
If I remove remove this line it imports all the files which is as expected, but I cant figure out why this line of code should be causing it not to work.
This line of code is from within the "MODIFIED" section of the script.
if (file_exists($feed["filename"]) && ($feed["imported"] < filemtime($feed["filename"])))Any ideas please?
Thanks,
Simon
Hi Simon,
I think you need to use your $config_feedDir variable along with the file_exists() and filemtime() functions, otherwise PHP is going to be looking for the file in the current directory and not your central feeds directory. Try this...
if (file_exists($config_feedDir.$feed["filename"]) && ($feed["imported"] < filemtime($config_feedDir.$feed["filename"])))Hope this helps,
Cheers,
David.
Hi David,
After the last post saying it was all sorted, it appears that it works from command line but not when set as a Cron Job.
I have implemented debugging on the shell script itself as follows
#!/bin/sh
_DEBUG="on"
function DEBUG()
{
[ "$_DEBUG" == "on" ] && $@ || :
}
DEBUG echo 'Reading files'
DEBUG set -x
##########
# IMPORT #
##########
cd /var/www/####.co.uk/httpdocs/scripts/
/usr/bin/php /var/www/####.co.uk/httpdocs/scripts/import.php @ALL
DEBUG set +xand also turned php error reporting on in the import.php file by adding this line.
error_reporting(6143);These steps are not throwing up any errors. This makes me believe that the error lies in the fact that I still need to add a full directory path to the datafeeds somewhere within the import() function as everything needs to be relative when working with Cron Jobs.
Do you perhaps have any other suggestions I could try?
Thanks in advance,
Simon
Hi Simon,
Do the merchants each get dropped to zero products after running the script via CRON? I ask this because, because if they don't, it means it isn't even calling the import function for each file, because part of the import routine is to delete all existing products for that merchant...
If that's the case, it points to scripts/import.php itself, and the only instance of the relative path to feeds in that file is on line 61:
if (file_exists("../feeds/".$feed["filename"]))Is that instance correctly changed to the absolute path? The only other instances of the relative path in that file is for the @MODIFIED option - but it's worth changing that as well for consistency...
Cheers,
David.
Hi David,
Yes both instances have been modified as follows: -
@ALL
if (file_exists($config_feedDir.$feed["filename"]))
{
import();
}@MODIFIED
if (file_exists($config_feedDir.$feed["filename"]) && ($feed["imported"] < filemtime($config_feedDir.$feed["filename"])))
{
import();
}also the merchants products are not being set to zero after a Cron Job. This is the confirmation email I get after the Cron Job has run.
Reading files
+ cd /var/www/####/####.co.uk/httpdocs/scripts/
+ /usr/bin/php /var/www/####/####.co.uk/httpdocs/scripts/import.php @ALL
backfilling reviews...
backfilling reviews...[done]
+ DEBUG set +x
+ '[' on == on ']'
+ set +Thanks,
Simon
I just tried an experiment by commenting out the following line within the @MODIFIED script
if (($config_feedDir.$feed["filename"]) && ($feed["imported"] < filemtime($config_feedDir.$feed["filename"])))and it has worked via Cron Job.
Simon
Just a quick thought, could this be something to do with some sort of restriction within PHP to access scripts / files within a different domain?
Thanks,
Simon.
Hi Simon,
The script must have access to the files in the common directory as it works when the above line is commented out. That implies that filemtime is somehow being blocked from working on the feeds in the common directory when running by CRON.
Does this imply that @ALL is now working via CRON, but @MODIFIED isn't, because of the above line? The file_exists() test in the @ALL version only exists as a safety check that the registered feed is still present in the directory, but if you can almost guarantee this, you could remove the check, so instead of:
if (file_exists($config_feedDir.$feed["filename"]))
{
import();
}...simply use just:
import();Cheers,
David.
Hi David,
Thanks for the last post.
Making the changes as per your last post, @ALL deletes all the products via CRON and doesn't do an import. This at least means that the import() is being called. I presume that this means that I need to look a bit closer at the admin__importRecordHandler() function perhaps?
Thanks,
Simon
Hi Simon,
Just to recap slightly,
Have you done anything to make sure that the CRON process is changing dir to the scripts directory before running import.php?
Cheers,
David.
Hi David,
Yes I have, the import.sh script looks like this.
#!/bin/sh
##########
# IMPORT #
##########
cd /var/xxx/xxx/xxx.co.uk/subdomains/flowers/httpdocs/scripts/
/usr/bin/php import.php @ALLThanks,
Simon
Hi Simon,
If importing from the admin panel works, that implies that includes/admin.php is working fine with the common feed directory. Could you perhaps email me you latest includes/admin.php and scripts/import.php and i'll take a closer look...
Cheers,
David.
Hi David,
Thanks once again for your amazing support and patience with me!
Just to wrap up this thread...
It was a permission problem and in the end found that I had three choices. However upon investigation found that the third option which invloved using "sudo" in the shell script opened up too many security issues that I was not able to fully deal with.
The first viable solution was to run the CRON job from the same user as the common datafeed directory used. As I use Plesk, this was easy!
The second, as you suggested was to change the file read permissions right through the parent directories of the common feed directory. This was the most straight forward route which I have now fully implemented.
Thanks again for your ongoing help.
Regards,
Simon.
Hi Simon,
It's probably that your common feeds directory is not making it through into the import function in includes/admin.php. If you open that file, do a search for "../feeds/" - all instances of that will need to be replaced with your config variable for the full path (don't forget to global it into the function).
Let me know if you've already fixed up that file, as i'm thinking you probably wouldn't be able to import through the admin interface either if this was the case, but it's the first step...
Cheers,
David.