Hi David,
Since the cj used for Referral Period would help me to give a directions for auto download into cj ftp?
Revision : 12/10B
Rgds,
Hi David,
Based on http://www.pricetapestry.com/node/76
I tested with clixGalore worked fine. tested for CJ but never succeed. ERROR - TRANSFER FAILED. both tested with unzip: yes and no
Regards
Hi Martin,
The first thing I would suggest is to take the FTP URL that you constructed for your CJ feed and paste it into the address bar of your web browser. That will double check that everything is correctly formatted and your browser should begin to download the file if so.
If that's all fine, the next thing I would suggest is to try curl as the automation handler. In your config.advanced.php change line 43 as follows:
$config_automationHandler = "curl";Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
I tested on browser, it worked fine,... i can download the file.
tested on automation tool : ERROR - TRANSFER FAILED
I changed line 43 :
from $config_automationHandler = "php"; into
$config_automationHandler = "curl";
and tested automation tool again.
But I ve got the same msg : ERROR - TRANSFER FAILED
Rgds,
Hi Martin,
Here's how to obtain detailed error information from CURL which should help resolve the problem. In includes/automation.php, look for the following code at line 151:
$retval = curl_exec($ch);...and REPLACE with:
$retval = curl_exec($ch);
if (!$retval)
{
global $curlError;
$curlError = curl_error($ch);
}Finally, look for the following code at line 60:
$status = "ERROR - TRANSFER FAILED";...and REPLACE with:
$status = "ERROR - TRANSFER FAILED";
global $curlError;
$status .= " ".$curlError;If you could let me know the additional error message reported that should help diagnose the problem...
Edit: updated for 20/10A
Thanks,
David.
--
PriceTapestry.com
Hi David,
I changed automation.php as above. than running again automation tool
ftp://CJ_id:CJ_password@datatransfer.cj.com/outgoing/productcatalog/path/CJ_files_Catalog.txt.gz
Status : ERROR - TRANSFER FAILED
no additional error message
Rgds,
Hi Martin,
I'm sorry it's proving awkward, I'll follow up by email...
Cheers,
David.
--
PriceTapestry.com
Hi David,
I have the same problem with CJ. Could make the solution public as soon as solved?
Thanks,
Daniel
Hi Daniel,
Thanks for your post (apologies for the delay appearing, I moderate all posts before publication) - Martin posted the working CJ link in the Affiliate Network Product Feed URLs thread here. The solution is to change the transport method in your CJ account to HTTP, and use URLs of the following format:
http://[Username]:[Password]@datatransfer.cj.com/datatransfer/files/[Username]/outgoing/productcatalog/[MerchantID]/Merchant_Name_Product_Catalog.txt.gzCheers,
David.
--
PriceTapestry.com
Hi David,
I am having the same issue. I first tried with the feed hosted on my server, but got the same error as above. So I tried your suggestion of loading it from CJ's servers, but am still getting the same error message.
I can confirm that the URL is formatted correctly and I can download the feed manually through my browser.
Any help would be appreciated.
Thanks,
Marc.
Hello Marc, and welcome to the forum!
Just as you posted your comment I posted the above reply to another user regarding using the HTTP method (rather than FTP) to access CJ feeds.
If you've already tried that; please could you confirm the status of the Automation Handlers and Automation Unzip Programs from the Support Info page of the /admin/ area of your Price Tapestry installation; (they should either be showing "Available" or "Unavailable") and if you could also confirm whether you have successfully used the Automation Tool with other feed URLs...?
Thanks!
David.
--
PriceTapestry.com
Hi David, thanks for the welcome.
I am relatively new to the software, and have only just installed this morning. Here is the status taken from the support info page;
Automation Handlers
php Available
curl Available
Automation Unzip Programs
Warning: file_exists() [function.file-exists]: open_basedir restriction in effect.{code saved}
Warning: file_exists() [function.file-exists]: open_basedir restriction in effect.{code saved}
unzip Not Available
gzip Not Available
Hi Marcus,
The above indicates that your hosting account is running in a relatively restricted environment - the Automation Handlers (php/curl) both use internal PHP functions; and are showing as available - so that's good; but the distribution version of the automation library relies on PHP being able to shell out to the unzip / gzip programs in order to decompress files where this is required.
However, I have a version of includes/automation.php that uses PHP's internal zlib functions to un-gzip files, and since CJ compressed feeds are .gz compressed this should do the trick for you, which I will email to you now... please check your email in a few minutes...
Cheers,
David.
--
PriceTapestry.com
Hey David,
I am also having trouble using the automation tool with CJ. The ftp url is correct. The error I was getting was Error - Transfer Failed. I then opened my config.advanced.php file and changed 'php' for 'curl' and am now getting Error - Unzip Failed. My Automation Handlers and Automation Unzip Programs both say "Avaliable":"Not Available".
Not sure if I need the includes/automation.php file you mentioned above, or if I should just switch to http feed.
Thanks,
Katelyn
Hi Katelyn,
For you and the benefit of others who may not have command line gzip available, here's the modification to includes/automation.php to use PHP's built-in gzip functions. First, open the file and add the following new code to the end of the script; just before the closing php tag:
function automation_unzip_phpgzip($tmp)
{
$gfp = gzopen($tmp,"r");
if (!$gfp) return FALSE;
$fp = fopen($tmp.".ungzipped","w");
if (!$fp) return FALSE;
while(!gzeof($gfp)) fwrite($fp,gzread($gfp,2048));
fclose($fp);
gzclose($gfp);
unlink($tmp);
rename($tmp.".ungzipped",$tmp);
return TRUE;
}With that in place and the modified file uploaded, edit config.advanced.php and look for the following setting at line 47:
$config_automationUnzipPrograms["gzip"] = "/bin/gzip";...and REPLACE with:
$config_automationUnzipPrograms["phpgzip"] = "";Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Thanks David,
Seems like that did the trick! Status says OK!
http://forum.liftersloot.com/
Hi David,
Hope all is going well.
I've just started using feeds from cj but am having a few issues and wondered if you might be able to help...
I'm fetching the files using the automation tool and the http as detailed above. The files are fetched fine but I don't seem to be able to unzip them...I followed the mod above to use the php gzip function but the files are not uncompressed. I just noticed in my PT install support page that unzip is available but phpgzip is not so I'm guessing the php/zip mod will not work for me? I wondered if you could suggest an alternative solution?
Thanks in advance.
Regards
Chris
Hi Chris,
CJ passwords can often contain "special" characters that must be URL encoded if using within the username:password part of an FTP or HTTP URL - please check out this comment and make any translations as required; but if that doesn't appear to be the cause let me know and I'll check it out further with you of course...
Cheers,
David.
--
PriceTapestry.com
Hi David,
The encoding was about the only thing in the posts I hadn't tried as I thought that since I could download the file the user/password was fine...I encoded it as per the linked post and it now works perfectly!
Thanks again for your help.
Regards
Chris
Hi David,
Just changing a feed on CJ as they have moved across to google shopping format. The datafeed has also moved to SFTP from FTP.
Is there anything else I should do for the automation tool to work for the following
sftp://userid:password@datatransfer.cj.com/outgoing/productcatalog/xxxxx/argos_co_uk-Argos_Google_Feed-shopping.xml.zip
I can access the file ok via cuteftp but not using the automation tool.
Thanks again
Richard
Hi Richard,
If your current setting for $config_automationHandler (line 43, config.advanced.php) is "auto" and CURL is installed that would imply that the version of the CURL library built-in to PHP does not include sftp support.
What you could try is changing to "php" and then in the URL use the ssh2.sftp protocol e.g:
ssh2.sftp://userid:password@datatransfer.cj.com/outgoing/productcatalog/xxxxx/argos_co_uk-Argos_Google_Feed-shopping.xml.zip
Cheers,
David.
--
PriceTapestry.com
Hi David,
Sorry for the late reply.
$config_automationHandler was preveiously set to curl. I have tried both auto and php and using ssh2.sftp in the url without success.
The password has a "%" sign that I have changed to "%25", is that correct?
Best regards,
Richard
Hello Richard,
A 3rd option would be to add a new automation tool handler "wget" which shells out to the wget command which has had sftp support for some time. The code and instructions are in this comment.
Note that in this case, just use the URL beginning sftp:// as per your original post above - the ssh2. prefix is php protocol wrapper specific.
%25 is correct for the "%" character in the password - there's a table of the more common password characters that require encoding in this comment...
Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David
Just tried this option without success.
Very strange that the file cannot be retrieved now that it is encrypted. I will try adding argos to the fetch file and do that one feed separately with the cron job.
Thanks again,
Richard
I wonder if it is linked to php version I am running? I think I am using 5.5.
Is there anything I should be aware of upgrading to php 7 as I think the pt installation could easily be 5 years old!
Hi Richard,
PHP upgrade information and patches for older distributions can be found in this thread...
Are you able to fetch the sftp:// CJ URL from the command line? If so, what command are you using? It should be no problem to make an Automation Tool handler to execute the same command...
Cheers,
David.
--
PriceTapestry.com
Hi David,
Thank you for the links to the patches.
I have now tried various options from the command line, and none were successful. I will retry later when I have upgraded php.
Thank you again.
Best regards,
Richard
Hi David. I have the same problem that Richard using SFTP with new google shopping format.
If I try
wget "sftp://user:password@datatransfer.cj.com/path/to/merchant_google_all-shopping.txt.zip"
It says: "Unsupported scheme ‘sftp’"
Any ideas?
Hi,
Can you confirm what $config_automationHandler you are currently using? As it stands, the "wget" handler (linked to above) wouldn't work as that would be the equivalent of what you have tried from the command line.
Have you tried for this feed using "curl" with the URL as-is, and also "php" using the ssh2.sftp:// prefix?
Cheers,
David.
--
PriceTapestry.com
My $config_automationHandler = "wget";
I have also tried with curl and it doesn't work
Hi,
Is it also not working using
$config_automationHandler = "php";
...and then as the URL:
ssh2.sftp://user:password@datatransfer.cj.com/path/to/merchant_google_all-shopping.txt.zip
If that _does_ work, but you need curl or wget handler for more flexibility using other feeds it would be no problem to hard code an override for this feed, just let me know...
Cheers,
David.
--
PriceTapestry.com
Still no luck :(
'ERROR - TRANSFER FAILED'
automation.php
function automation_handler_php($src,$dst)
{
return copy($src,$dst);
}I am trying with "CJ HTTP" rather than SFTP and with WGET and it seems to work.
https://user:passwordencoded@datatransfer.cj.com/path/to/merchant_google_all-shopping.txt.zip
Hello Martin,
You should be able to use the Automation Tool - the easiest way to construct the required URL is to use FileZilla to connect to datatransfer.cj.com and navigate to the file you wish to transfer, and then in the remote window right-click on the filename and use the Copy URL(s) to Clipboard option which will create the required FTP URL minus your password, e.g.
ftp://userid@datatransfer.cj.com/outgoing/productcatalog/MerchantID/Merchant-Name.txt.gz...and finally for use within the Automation Tool as the download URL, insert your password as follows:
ftp://userid:password@datatransfer.cj.com/outgoing/productcatalog/MerchantID/Merchant-Name.txt.gzIf this is for a merchant that you already have registered and imported don't forget to use the existing filename as it is in your /feeds/ folder as Filename when setting up the job...
Hope this helps!
Cheers,
David.
--
PriceTapestry.com