Hi David, On the Slow Import page there is a link to Import All feeds. Have you developed a similar mechanism to run all Automation Tool jobs in series? I can't get CRON to work on my home environment. It seems like the approach would be very similar to the Slow Import?
I'm on 12/10B, but I could probably merge the changes if you have them on another release.
Or if you have some suggestions on the best way to run the Automation Tool for all files from a home environment (WampServer Version 2.2).
Thank you for any advice in this area!
Thank you David! The command line would not work for me... probably because I have moved my admin and include folders around and complicated things. So I applied your changes to automation_tool_run and it works!
It takes about 10 minutes to download all the feeds for this site. Not something I will want to do all the time, but a good option to refresh my feeds while I'm working on my site on the home server. Much easier than running every job!
Thanks again. You are THE MAN!
Hi David,
Hope you had a good weekend.
I’ve implemented the above on a site running 14_06A version but when I tried the download all feeds option it runs for a while and then displays the error below:
"Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator at webmaster@mysite.mysite.com to inform them of the time this error occurred, and the actions you performed just before this error.
More information about this error may be available in the server error log.
Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request"
You mention in the post above a way to modify the script to incorporate a browser refresh in the same way as ‘slow import’ does. Assuming you think the error below is caused by a time out, please could you let me know if the refresh mod you mention is already detailed somewhere on the forum. If not I wonder if you might possibly be able to explain how to add a 'refresh'?
Thanks in advance.
Best regards
Chris
Hello Chris,
Here's an alternative modification to admin/automation_tool_run.php for the refresh method. Look for the following code at line 12:
automation_run($id);...and REPLACE with:
if ($id == "@ALL")
{
$start = (isset($_GET["start"])?$_GET["start"]:time());
$progress = (isset($_GET["progress"])?$_GET["progress"]:1);
$sql = "SELECT id FROM `".$config_databaseTablePrefix."jobs`
WHERE lastrun < '".database_safe($start)."' LIMIT 1";
if (database_querySelect($sql,$rows))
{
automation_run($rows[0]["id"]);
$progress++;
header("Location: automation_tool_run.php?id=@ALL&start=".$start."&progress=".$progress);
}
else
{
header("Location: automation_tool.php");
}
exit();
}
else
{
automation_run($id);
}Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
Thanks for your quick response and the code above. I installed the code but now get a different error (below):
"The page isn't redirecting properly
Firefox has detected that the server is redirecting the request for this address in a way that will never complete.
This problem can sometimes be caused by disabling or refusing to accept cookies"
Before the message is displayed quite a lot of feeds appear to be downloaded. I thought a different browser may give a different result but Chrome displays a similar message.
I wondered if you might be able to suggest anything I might try to get this working?
Thanks in advance.
Regards
Chris
Hi Chris,
I've updated the above replacement to include a "progress" parameter in the URL so that it is different every time - hopefully that should do the trick...!
Cheers,
David.
--
PriceTapestry.com
Hi David,
I've updated the code but the url only seems to change when the same error message as before is displayed about mid way through collecting all the feeds.
The url changes to:
http://www.example.com/admin/automation_tool_run.php?id=@ALL&start=1471277671&progress=22
I ran it a couple of times on chrome and mozilla and got the same result. Is the url supposed to change aftr each file is collected?
Would be grateful for any suggestions you might have on how I might resolve this.
Best regards
Chris
Hi Chris,
Here's an alternative REPLACEment using HTTP meta refresh (same as Slow Import) instead of 302 which is probably more appropriate...
if ($id == "@ALL")
{
$start = (isset($_GET["start"])?$_GET["start"]:time());
$sql = "SELECT id FROM `".$config_databaseTablePrefix."jobs`
WHERE lastrun < '".database_safe($start)."' LIMIT 1";
if (database_querySelect($sql,$rows))
{
automation_run($rows[0]["id"]);
$refresh = "automation_tool_run.php?id=@ALL&start=".$start;
print "<html><head><meta http-equiv='refresh' content='1;url=".$refresh."' /></head><body></body></html>";
exit();
}
else
{
header("Location: automation_tool.php");
}
exit();
}
else
{
automation_run($id);
}Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
Thanks for the replacement code above. I tried it last night but it was so slow I ended up having to stop it. I could see some files were collected but I have no idea if it would ever have finished. The previous solution above using the progress refresh did seem to work at a reasonable pace until the error was displayed. Also the progress url (ending in 22) was only displayed when it stopped working. I’m wondering if it would be feasible to display the progress count only once each file was collected? Do you think this would help? It sounds from your last post that you think the 302 is more suited to this application anyway.
Best regards
Chris
Hi Chris,
If there are many jobs then it would need to use the meta refresh method due to the browser imposed 302 redirection limit, but it's not actually necessary to have 1 second delay - have a go with the following alternative replacement, without the delay, and I've also re-instated the progress parameter so you should be able to see in the URL how many jobs have been run so far...
if ($id == "@ALL")
{
$start = (isset($_GET["start"])?$_GET["start"]:time());
$progress = (isset($_GET["progress"])?$_GET["progress"]:1);
$sql = "SELECT id FROM `".$config_databaseTablePrefix."jobs`
WHERE lastrun < '".database_safe($start)."' LIMIT 1";
if (database_querySelect($sql,$rows))
{
automation_run($rows[0]["id"]);
$progress++;
$refresh = "automation_tool_run.php?id=@ALL&start=".$start."&progress=".$progress;
print "<html><head><meta http-equiv='refresh' content='0;url=".$refresh."' /></head><body></body></html>";
exit();
}
else
{
header("Location: automation_tool.php");
}
exit();
}
else
{
automation_run($id);
}Hope this helps!
Cheers,
David.
--
PriceTapestry.com
HI David,
Thanks for the above. I tried it but the counter got to approx. 3k before I stopped it. I only have approx. 50 merchants so this does not appear to work either....would I be correct in my understanding that the counter should not go beyond the number of merchants with feeds?
Best regards
Chris
Hello Chris,
Ah - the process could actually go on forever if any of the feeds failed, since lastrun is not updated. Here's an alternative replacement that uses id values rather than time / lastrun:
if ($id == "@ALL")
{
$nextid = (isset($_GET["nextid"])?$_GET["nextid"]:0);
$progress = (isset($_GET["progress"])?$_GET["progress"]:1);
$sql = "SELECT id FROM `".$config_databaseTablePrefix."jobs`
WHERE id > '".database_safe($nextid)."' LIMIT 1";
if (database_querySelect($sql,$rows))
{
automation_run($rows[0]["id"]);
$progress++;
$refresh = "automation_tool_run.php?id=@ALL&nextid=".$rows[0]["id"]."&progress=".$progress;
print "<html><head><meta http-equiv='refresh' content='0;url=".$refresh."' /></head><body></body></html>";
exit();
}
else
{
header("Location: automation_tool.php");
}
exit();
}
else
{
automation_run($id);
}Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
Thanks for the modified code. I think the last version might have gone on for ever!...but the above works perfectly!
Thanks again for all your help getting this working.
Best regards
Chris
Hello Gregor,
It looks like WampServer installs php both for command line access and as an Apache module - if you first go to /admin/ of your local Price Tapestry installation and then Support Info, the Install Path will indicate where the home directory is located, e.g.
C:\wamp\www
Use that as the base, and add \scripts to get the path to the scripts folder, and then open a Command Prompt window and have a go with;
cd C:\wamp\www\scriptsphp fetch.php @ALL
Otherwise, to add "Run All" functionality to the Automation Tool, first edit admin/automation_tool_run.php and look for the following code at line 16 (12/10B) or line 12 (15/09A)
automation_run($id);...and REPLACE with:
if ($id == "@ALL"){
$sql = "SELECT id FROM `".$config_databaseTablePrefix."jobs`";
if (database_querySelect($sql,$rows))
{
foreach($rows as $row)
{
automation_run($row["id"]);
}
}
}
else
{
automation_run($id);
}
And then to add a Run All link / button;
For 12/10B edit admin/automation_tool.php and look for the following code at line 62:
print "<a href='automation_tool_edit.php?id=0'>New Job</a> ";...and REPLACE with:
print "<a href='automation_tool_edit.php?id=0'>New Job</a> ";print "<a href='automation_tool_run.php?id=@ALL'>Run All</a> ";
And for 15/09A edit admin/automation_tool.php and look for the following code at line 70:
admin_tool("New Job","automation_tool_edit.php?id=0",TRUE,FALSE);...and REPLACE with:
admin_tool("New Job","automation_tool_edit.php?id=0",TRUE,FALSE);admin_tool("Run All","automation_tool_run.php?id=@ALL",TRUE,FALSE);
It would be possible to create a version that uses a browser refresh between each feed similar to how the Slow Import Tool works but since sufficient execution time would be required to fetch your largest feed in a single request and you are running locally, if not already the best thing to do would be to configure PHP for unlimited execution time.
To locate the php.ini script being loaded, create a script to run the phpinfo() function:
<?phpphpinfo();
?>
Browse to the script and look for the "Loaded Configuration File" value. Open this file in notepad and then search for "max_execution_time" and change to, or add if not exists;
max_execution_time=0You would need to restart Apache - the WampServer installation may provide an easy way to do this, otherwise a quick reboot would do the trick...
Hope this helps!
Cheers,
David.
--
PriceTapestry.com