Hi David,
I have feeds that change names every week and what I want to do is strip some characters from the front, so I would like to convert the following:
10273676-PC_Mall_Affiliate_Advantage_Network.xml
10440897-Newegg_com.xml
10385645-OnSale.xml
10549103-Dell_Small_Business.xml
10387719-Buy_com_USA_and_Buy_com_CA.xml
to:
PC_Mall_Affiliate_Advantage_Network.xml
Newegg_com.xml
OnSale.xml
Dell_Small_Business.xml
Buy_com_USA_and_Buy_com_CA.xml
basically just stripping the digits and the hyphen from the front.
I've messed around with rename but I just can't figure it out...
cheers,
Peter
The original feed is a single .zip file with the various filenames mentioned inside of it.
cheers,
Peter
Ah, ok. I presume you are automating the download of this - is it to a Linux server? What commands are you using to download?
Cheers,
David.
Yes, I'm preparing my import script... This is my plan so far:
from my scripts folder:
1) wget -O "../feed_temp/feeds.zip" "ftp://server.com/xxxxxxxxxxxx.zip"
2) unzip ../feed_temp/feeds.zip
3) find . -size 1M -name \*Newegg_com.xml | xargs rm - r (remove a useless newegg file)
After this I want to rename the files, and my import script can go about it's business.
Yes I'm preparing a script, my plan is:
calling from /scripts directory:
1) wget -O "../feed_temp/feeds.zip" "ftp://xxxx.zip"
2) unzip ../feed_temp/feeds.zip
3) (remove a few useless files from the zip)
And then I would like to rename those files and strip the numbers so import.php script can do it's magic.
cheers,
Peter
Hi Peter,
It would probably take me quite a while to work out how to do this using a shell script; but the renaming can be done quite easily with a simple PHP script. Something like the following; to be executed within /scripts/ would scan ../feeds/ for anything with "-" in the filename, and then rename it to the part after the "-":
rename.php
<?php
$dir = "../feeds/";
if ($dh = opendir($dir))
{
while (($file = readdir($dh)) !== false)
{
if (strpos($file,"-"))
{
$newfile = substr($file,strpos($file,"-")+1);
rename($dir.$file,$dir.$newfile);
}
}
closedir($dh);
}
?>
(note that PHP will need write access to the feeds directory for this to work; but if it is being executed under the same process as you are running wget this shouldn't be a problem)
Hope this helps!
Cheers,
David.
Thanks David, I'll play around with that. I've also asked CJ if they can just keep the same filenames every week.
cheers,
Peter
Hi Peter,
Keep pestering CJ as they can provide a fixed name for their feeds as this is what they have done for me. This makes life a lot easier!
Cheers.
Keeop
Hi Keeop,
I messaged them if they can change the feed names but I didn't explain it clear enough and they thought I meant I wanted to change the name of the .zip file.. jeez..
Anyway I've already finished my script thanks to David's script and now waiting for crontab to run the script now. *crosses fingers*
Peter
Hi David,
I have my script completed and it runs perfectly; downloads the feeds and imports them, when I run it (ie. from the scripts directory with "./fetch.sh"
Now, I have set up a cronjob, and the wget program is failing due to a permission error.
-08:37:01-- ftp://xxxxxxx:*password*@datatransfer.cj.com/outgoing/productcatalog/xxxxx/xxxxxxx_xxxxx_*.zip
=> `/var/www/vhosts/mysite.com/httpdocs/shop/public/temp/feeds.zip'
Resolving datatransfer.cj.com... 216.34.209.22
Connecting to datatransfer.cj.com|216.34.209.22|:21... connected.
Logging in as xxxxxxx ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD /xxxxxxx/outgoing/productcatalog/xxxxxx ... done.
==> PASV ... done. ==> LIST ... done.
.listing: Permission denied
unlink: No such file or directoryHere are the first three lines of the script:
#!/bin/sh
PATH="/var/www/vhosts/mysite.com/httpdocs/shop/public"
/usr/bin/wget -O "$PATH/temp/feeds.zip" "ftp://username:password@datatransfer.cj.com/outgoing/productcatalog/xxxxx/xxxxxxx_xxxxx_*.zip"Here are the permissions of the temp directory to where I want to download to.
drwxrwxrwx 2 username psacln 4096 Oct 17 08:37 tempcheers,
Peter
Sorry David,
I've solved the issue, I had to run the cronjob as root.
Peter
Bob L.
Hi David hope all is well.
Something I'd like to be able to do is take a zip file and rename it to another name.
Example have a file that is 1234.txt and rename it to merchant_net_id.txt
Would that be possible with the rename.php script using a set of array?
Example name1 => myname1, name2 => myname2, etc?
Thanks in advance
Bob L.
Hello Bob,
Certainly - bear in mind that if using a fetch.sh script then the downloaded filenames can be specified in the wget command using the -O parameter, but sure - a rename.php script is straight forward. Assuming the script will be run from your scripts/ folder, with the files to rename in feeds/, something like this would do the trick:
<?php
$rename["1234.txt"] = "merchant1_net1.txt";
$rename["5678.txt"] = "merchant2_net2.txt";
// etc.
foreach($rename as $from => $to)
{
rename("../feeds/".$from,"../feeds/".$to);
}
?>
Cheers,
David.
--
PriceTapestry.com
Bob L.
Hi David,
Wasn't as easy as I thought it would be.
Looks like I need to check if the file I want to rename exist first and then drop the name that I want to rename it to, unless there's a way to force the renamed file to replace the one from the cron before.
example $rename["1234.txt"] = "merchant1_net1.txt";
merchant1_net1.txt already exist. But I wouldn't want to drop this file unless the 1234.txt has been received.
Hi Bob,
Sure - have a go with;
<?php
$rename["1234.txt"] = "merchant1_net1.txt";
$rename["5678.txt"] = "merchant2_net2.txt";
// etc.
foreach($rename as $from => $to)
{
if (file_exists("../feeds/".$from))
{
unlink("../feeds/".$to);
rename("../feeds/".$from,"../feeds/".$to);
}
}
?>
Cheers,
David.
--
PriceTapestry.com
Hi Peter,
What method are you using to download the files?
Rather than renaming the file once downloaded, it's normally easier to force the download with the filename required. For example, when using wget, the -O parameter can be used to specify the name of the downloaded file, as the following example:
wget -O "/home/httpd/mysite.com/feeds/Example_Merchant.xml" "http://www.example.com/getfeed.asp?merchant=1234"Cheers,
David.