You are here:  » Automatic Feed Import


Automatic Feed Import

Submitted by hostcolor on Sat, 2015-08-22 16:47 in

Hello,

I am currently working on making my Feeds updates automatic. I've already setup crons on my server which are downloading, extracting and replacing my feeds while keeping the same names. Could you tell me what Cron I need to use in order to tell PT to automatically Import all of my feeds?

Best Regards,
Alexader

Submitted by support on Sat, 2015-08-22 17:15

Hi,

Even if you have your own fetching process in place, you can still just set-up a single invocation of scripts/cron.php which:

1) Runs all Automation Tool jobs

2) Imports all feeds

If there are no Automation Tool jobs configured then (1) is simply skipped, and cron.php, without any further parameters will just import all feeds. It is equivalent to invoking scripts/import.php with @ALL as the <filename> parameter.

The /admin/ area of your Price Tapestry installation has a helper area that attempts to automatically work out the correct command line to use as your CRON job to invoke cron.php, if you go to Setup > CRON, if Option 1 is showing as available, it will be something like this:

cd /path/to/scripts/;/usr/bin/php cron.php

It is important to include both a cd (change directory) and separate php command (notice how they are separated by semi-colon) because PHP is not always configured to internally change directory into the current folder...

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by hostcolor on Sat, 2015-08-22 17:27

Hello David,

Thank you very much. Now I would be able to make my Feeds update on "Set and Forget" :)

Best Regards,
Alexander

Submitted by richard on Mon, 2017-10-09 22:29

Hi David

I have an interesting problem that has just arisen on various sites hosted on a managed server with 1and1.

On non wordpress PT installations I have used option 2 for invoking cron.php successfully every day for many years.

For wordpress based PT installations it has been necessary to use option 1.

Since the 5th October, option 1 is no longer available and option 2 results in a perpetual fetching and importing of feeds - as soon as one cycle has finished the process restarts. For smaller sites the interval is every 11 minutes.

I have checked the SSH for timings of cron jobs but they are as they should be.

Have you experienced this issue of the continuous loop and do you have any suggestions? 1and1 do not believe they have changed anything.

Best regards

Richard

Submitted by support on Tue, 2017-10-10 09:11

Hi Richard,

If there has been a configuration change that prevents the Setup > CRON tool from deriving the option 1 command line (for example by security settings preventing file_exists() from checking for a file outside of the PHP user's home folder) it is often the case that the option 1 command will still work, and you can derive it from the Install Path, which is display on the Tools > Support Info page.

The command line is most likely to be;

cd YOUR_INSTALL_PATHscripts/;/usr/bin/php cron.php

So for example if your Install Path is:

/home/username/public_html/

Then the option 1 command line would be:

cd /home/username/public_html/scripts/;/usr/bin/php cron.php

If that doesn't work, try /usr/local/bin/php e.g.

cd /home/username/public_html/scripts/;/usr/local/bin/php cron.php

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 09:18

Hi David

I have just received this reply as I have ascertained that 1and1 did migrate my server last week.

We had a look over your crontab & the various jobs, in your scripts you are pointing to: /usr/local/bin/php. Recently we have added a new feature where you can specify which PHP version you want to use. So your scripts need to point to the specific PHP folder. For example; /usr/local/bin/php/php5.5/

Would I have amend anything else in addition to the fetch.sh request

Regards

Richard

Submitted by support on Tue, 2017-10-10 09:28

Hi Richard,

Ah - that would explain things! Best thing to do then, would be to put the exact path to your chosen PHP version in config.advanced.php - look for the following code at line 50:

  $config_CRONPrograms["php"] = array("/usr/bin/php","/usr/local/bin/php");

...and REPLACE with:

  $config_CRONPrograms["php"] = array("/usr/local/bin/php/php5.5/php");

Option 1 command line should then be displayed in Setup > CRON. If fetch.sh is invoking any of the php scripts you would need to change /usr/local/bin/php to /usr/local/bin/php/php5.5/php also...

Cheers,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 09:44

Many thanks David :)

If I use option 2, thinking of other sites, would I also need to amend

$config_CRONPrograms["wget"] = array("/usr/bin/wget","/usr/local/bin/wget");

I would normally experiment but the past few days has shown that the cron job just goes in to a loop and eventually freezes up the server!

Best regards

Richard

Submitted by support on Tue, 2017-10-10 10:12

Hi Richard,

Paths to wget shouldn't be affected by the change so they can almost certainly stay as they are...

Cheers,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 11:32

I have tried suggested solution and many permutations of without success to get Option 1 showing as opposed to saying N/A. I can ignore this for the time being as I can always fallback on Option 2 for cron job.

My biggest concern is the looping of the cron job once finished. Do you think the choice of php is the problem or something else?

Submitted by support on Tue, 2017-10-10 11:41

Hi Richard,

That would imply that /usr/local/bin/php/php5.5/php is not correct for the PHP binary - thinking about it, the trailing / may have been accidentally included and php5.5 is the actual name of the binary (which thinking about it would make sense with multiple versions on the server) - have a go with;

  $config_CRONPrograms["php"] = array("/usr/local/bin/php/php5.5");

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 13:22

Hi David

I had tried that option and have even tried the php version of 5.6.31 that the PT setup suggests.

I will give it some more thought as two PT installations appear to be ok but the other two I am interested in aren't!!!

I intend working through all pt installations and note the differences and see if I can work out why two were affected and the other two were not.

Best regards

Richard

Submitted by support on Tue, 2017-10-10 13:38

Hi Richard,

Can you see the directory listing of /usr/local/bin/php/php5.6.31/ - from a command prompt:

cd /usr/local/bin/php/php5.6.31/
ls

If you have command line (SSH) access what does the ls command show?

Cheers,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 14:03

Hi David

I tried the cd command but no such directory.

I can switch to another directory such as /kunden/homepages/1/xxxxxxx/htdocs/ so command prompt is ok.

Rgds

Richard

Submitted by support on Tue, 2017-10-10 14:13

Hi Richard,

What about the level up e.g.

cd /usr/local/bin/php/
ls

That should indicate exactly what command line versions are available (which may be different to Apache module versions although I wouldn't have thought that would be the case) in which case if you could change directory down into the most recent 5.x branch and capture the ls output from there that should indicate what is available...

Cheers,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 15:07

Hi David,

I have found a number of php with the latest being php5.5 and php7.0 but not php5.6 or php5.6.31

Regards

Richard

Submitted by support on Tue, 2017-10-10 15:14

Hi Richard,

Please could you copy and post the output of the long format directory listing for /usr/local/bin/php/ using;

cd /usr/local/bin/php/
ls -l

Thanks,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 15:19

I'm not able to go any further than /usr/local/bin/.

The command ls in above location outputs php, php5.5, php7.0 but none of these are valid directories

Submitted by support on Tue, 2017-10-10 15:24

Hi Richard,

Ah - in that case it sounds like the binaries are in the /usr/local/bin/ folder rather than /usr/local/bin/php/ in which case have a go with the following in config.advanced.php:

  $config_CRONPrograms["php"] = array("/usr/local/bin/php5.5");

Cheers,
David.
--
PriceTapestry.com

Submitted by richard on Tue, 2017-10-10 15:47

Hi David

I have tried that but path for option 1 does not appear but I will run the cron job and I will see if it goes in to a loop.

So the fetch file is

cd /kunden/homepages/1/xxxxxx/htdocs/domain/pt/scripts;/usr/local/bin/php5.5 cron.php

and Adv config file is

  $config_CRONPrograms["php"] = array("/usr/local/bin/php5.5");

This will take a while to determine if it loops

I will let you know the outcome :)

Thank you for your time and assistance.

Best regards

Richard

Submitted by richard on Tue, 2017-10-10 17:14

Hi David,

Great news :))

Using option 1, even though the set up screen says N/A, works when used as shown above and does NOT loop continuously.

Option 2 loops continuously but that is not a problem as I will ensure option 1 is used for initiating cron.

THANK YOU again for your help.

Best regards

Richard