No import and double feeds

Hi Jonny,

Both symptoms point towards your server having recently been upgraded; resulting in PHP running to a slightly different configuration; and also possibility as a different user.

With PHP running as a different user, your automation script may have problems overwriting the previous versions when your automation script ran under a different user. You may be able to resolve this by changing permissions using your FTP program. In the remote window, right-click on one example of a duplicate that cannot be removed and look for Permissions... or maybe Properties... and then Permissions. Then give WRITE access to all users (Owner/Group/World); and finally try deleting the file again.

If that works; you should be able to mutli-select all duplicate feeds and then perform the permissions update / delete in one go.

Regarding feeds not importing; this would indicate that PHP is no longer changing directory internally to folder containing the script. In the code you posted, the last line is:

This may now need to be split into 2 operations; first a cd (change directory) and then the php command to run import.php. Have a go with:

Don't forget to change example.co.uk with your domain.

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

i changed the permissions as you stated and deleted the duplicate feed , it then deletes the other feed that is registered.

i tried to delete the entire feed folder but it just removes my registered feeds

i tried to delete a duplicate feed and i get the message 550 buyagift.xml: No such file or directory

seems a bit weird

Hi Jonny,

First of all - don't click De-Register at all!

One way to reset everything would be as follows:

1) Rename feeds to feeds_old

2) Create a new directory called feeds and using your FTP program make sure is writable to all users as above

3) Let your automation process run with the modification to the last section as described above. After the process has run all feeds should have been restored and import as expected...

Cheers,
David.
--
PriceTapestry.com

thinking about it if i just backup the database registration and then delete the datebase and then create a new one it may just be easier

Hi jonny,

We posted simultaneously above - check my last post but yes backing up registrations and recreating should work but you may not need to do that...

Cheers,
David.
--
PriceTapestry.com

right i renamed folder and create new feed folder , run fetch script and it downloads feeds but still doesnt import them , tried the old import all and the version you mentioned

just noticed as well when logged into the ftp all the pricetapestry files the date on them is fine then the time next to them all is 00:00 , does this mean anything ?

Hi Jonny,

That's strange but shouldn't cause any problems. It does sound like your hosting platform has been upgraded and files restored as part of that process.

Are you able to capture the output of your automation process - in other words are you able to see what happens when import.php @ALL is run? From the command line any output should be displayed; or if running via CRON often you are able to receive an email showing the output of the process which might give a clue...

Do you have command line (SSH) access to run the automation script manually?

Cheers,
David.
--
PriceTapestry.com

no ssh access , well not without going through a long process, set the import to email me the output , email never arrived but i have just checked my host and the server my site is on they are having problems with so I assume this is the problem.

I will leave it till tomorrow and try again.

cheers for the help

No worries Jonny,

I did suspect there was an ongoing hosting issue but I prefer never to assume that - sometimes things are best left for 24 hours to settle down when "strange" things happen. Let me know of course if you're still not sure once the server has settled...

Cheers,
David.
--
PriceTapestry.com

Hi David,

Right below is the email received once the fetch script run last night.
I have been checking my others sites and probably 5 to 10 of them have exactly the problem , ie the feeds downloaded but havent imported for days/weeks etc so i assume the host has changed something.

{code saved}

Hi Jonny,

Thanks for the transcript - things have changed but I'm familiar with each of the problems; if you could please email me your current automation script (the script being run by cron) I'll make the changes that should work around the new set-up.

Specifically, CRON (or rather the shell program now being invoked by CRON) is no longer accepting MS-DOS style end of line characters in your .sh script, so they must be converted to Unix style. In addition, it looks like the command line PHP binary has changed, most likely from php to php-cli.

I'll make both changes in your current version for you to try.

Cheers,
David.
--
PriceTapestry.com

i cant email at the mo but my fetch script is below, then each feed is added, if you could post the change to this because if it works i will need to make the change to several sites

many thanks

Hi jonny,

The problem is slightly more subtle than being able to post the changes i'm afraid as it is specific to how the end of lines are represented in the file.

In the computing world there are 2 "standard" ways that the end of a line in a text file is represented. Unix systems typically use a line-feed "LF" character on its own, but other systems, including the operating system on which Microsoft Windows is based use carriage-return followed by line-feed "CRLF".

When you set-up a CRON job in your hosting control panel, the script that you enter (in this case your fetch.sh script) is passed to a "shell" program for interpretation. Most shell programs will accept either form of end of line representation, but what has happened here is that your host has changed the way CRON works, and your script is now being handled by a program that only accepts the Unix style of end of line; so it is trying to interpret every carriage-return character as additional input into the the command on each line of your script, which is what is causing the problems.

One way to work around this would be to make sure that when uploading your fetch.sh script from your local computer to your server that your FTP program is set to make the transfer in ASCII mode. When an ASCII mode FTP transfer is made, the receiving FTP server will always save the file using the end of line representation that is native to the local operating system - so this process should perform the conversion automatically for you.

It's probably best to check your FTP program configuration and set the transfer mode to "Automatic".

As a final change, it looks like the php command line version has changed from /usr/bin/php to what I suspect will be /usr/bin/php.cli

To implement this change, and also to handle the situation where the new PHP configuration has changed to no longer change directory internally to the same directory as the script, in your fetch.sh where you have at the last line:

...REPLACE with:

Finally upload your modified fetch.sh with your FTP program in Automatic (or to make doubly sure "ASCII" mode) and fingers crossed that should do the trick...

Cheers,
David.
--
PriceTapestry.com

many thanks , will give this a go later when im home

many thanks