Is there a way to deal with i feed containing multiple merchants?
the feed contain a field Merchant name, but can not think of the best way to deal with this
Hi Paul,
Thanks for the info - I wasn't aware that CJ bundled all the merchants that you subscribe to into a single feed.
Creating a feed split utility should be reasonably straight forward providing that you have shell access to the server; and can setup a directory with write access to the PHP process.
This would have to be run as a script executed from the command line as it may take several minutes for Magic Parser to autodetect on a 250MB XML file; and then to process all records and write out the split files will take another few minutes.
I'll get thinking about the best way to do it...
I managed a little work around, which works fine, except the merchant do not show on the merchant index
i made the 'extras' mod here: http://www.pricetapestry.com/node/175
Import the feed into a single merchant say CJ and put the merchant name in extras field
i modified the product page so it shows merchant 'CJ MerchantName' i geuss it would be possible using code to strip the 'CJ' and show just the merchant name
as i say that was just a temp solution whilst i tested feeds from someone, worked fine for me
i would have looked at splitting the xml fiole somehow, but the feed contained an undefined number of merchants and it was not possible to select by merchant, so i could have ended up with thousands of merchants to register
Here's something that might work in the mean time...
250MB is a big feed; but if you're able to register the feed with a known format string (so that auto-detection is not required), one option would be to add a new filter that does the opposite of "Drop Record"; i.e. include the record only if the selected field equals a certain value.
Then, you simply make copy the feed (or make symbolic links on a Linux server) once for each merchant, then register the copies and configure the filter differently for each one.
Here's the code for an opposite to the "Drop Record" filter - just paste into includes/filters.php:
<?php
/*************************************************/
/* dropRecordIfNot */
/*************************************************/
$filter_names["dropRecordIfNot"] = "Drop Record (If Not)";
function filter_dropRecordIfNotConfigure($filter_data)
{
print "Drop record if field DOES NOT contain text:<br />";
print "<input type='text' size='40' name='text' value='".widget_safe($filter_data["text"])."' />";
widget_errorGet("text");
}
function filter_dropRecordIfNotValidate($filter_data)
{
}
function filter_dropRecordIfNotExec($filter_data,$text)
{
global $filter_dropRecordFlag;
if($filter_dropRecordFlag)
{
return $text;
}
else
{
$filter_dropRecordFlag = ($filter_data["text"] !== $text);
}
return $text;
}
?>
To create symbolic links on Linux rather than a copy; use the "ln -s" command, something like this:
$cd feeds
$ln -s original.xml merchanta.xml
$ln -s original.xml merchantb.xmlIn this example, if you had the multi-merchant file "original.xml", you would then have 2 symbolic links to the same file but with the "virutal" filenames "merchanta.xml" and "merchantb.xml", which you can then register separately and use the new Drop Record (If Not) filter to only import products for the required merchant.
I get this when adding the code to filters.php
Parse error: parse error, unexpected '}' in /home/pricespi/public_html/includes/filter.php on line 178
ooops... there was a missing semi-colon on this line:
$filter_dropRecordFlag = ($filter_data["text"] !== $text);
I've fixed it in the code above - that should help...
Thanks David,
one question, how can I tell it to drop if not merchant name?
Filter only gives me the following choices:
Product Name
Product Description
Image URL
Buy URL
but none of these has any relation to the merchant...
Robert
Yes - you're absolutely right. I'm not having a good weekend here! I'll figure out a way to pull in the merchant name in this scenario so that this technique will work. It isn't one of the registered fields so it isn't currently possible to register a filter against it... Sorry about that...
Just wondering if you had any luck with this so far? Got about 20 feeds in one csv file so would be handy...
Cheers
Robert
www.pricespin.co.uk
Hi Robert,
As it is a CSV feed that you need to split there is a reasonably easy way to do this from the command line using the "grep" command, and by piping the output to a new file.
The basic syntax would be:
grep "MerchantName" <filename> > MerchantName.csv
For example:
grep "John Lewis" bigfeed.csv > JohnLewis.csvTo test it out first without doing the redirection into a new file; just experiment with grep on its own:
grep "John Lewis" bigfeed.csvThe quotes around MerchantName are essential if the merchant name contains spaces. I don't know why I didn't think of mentioning this the other day; but it should work fine...!
Cheers,
David.
Hello David,
I can download them individually but when I need to download 25-30 feeds or one large one it would be handy to be able to import one large feed and have it distinguise it via a field 'Merchant' if you know what I mean...
Robert
www.pricespin.co.uk
Does dropRecordIfNot only drop the record if the field is not *exactly* equal to the one entered? In other words, if my product categories look like this:
/Products/webstore/house/bed_bath/bathroom/towels/sets
/Products/webstore/women/accessories/small_goods/wallets
can I get it to drop all records where "house" is not contained anywhere within the category field?
When I attempt to apply this filter in this way, I just get a blank screen on the trial import.
I actually ended up using your selectiveImport function and reversed the arguments for strpos.
I changed this:
$filter_dropRecordFlag = (strpos($filter_data["text"],$text) === FALSE);
to this:
$filter_dropRecordFlag = (strpos($text,$filter_data["text"]) === FALSE);
I kept trying to use the function as it was written, but after looking at the strpos function on php.net, I decided I needed to switch the arguments. I'm not sure I understand how it worked before, but this seems to do ok for me.
Problem solved!
Hi guys,
Just curious, most feeds have a "Merchant" column within them.
Would'nt it be easier to change the process of importing feeds so that when you register a feed you have a "Merchant" and "user_Merchant" as you do category and brand? Then you could name some feeds and select from the dropdown on others...... I know that would help me out quite a bit....
Mark Bellavanc
Hello Mark,
I haven't done what you're talking about, but you'd probably have to start by modifying "admin/feeds_register_step2.php" & "includes/admin.php" to get that to work.
Michael
What if I want to have the products that have "a" or "b" in a certain field, but the rest has to be dropped?
When I input two filters, one for "a" and one for "b", there will be (logically) no products (because the fields do not contain an "A" and a "B", only 1 letter.
How can I fix this, so that I can input A and B, and that PT keeps all the products witn an A or B, and drops the rest?
Hi,
What you need is a new filter that drops the record if the value is not in given list. Here's a new "Drop Record Unless" filter that should fit the bill:
<?php
/*************************************************/
/* dropRecordUnless */
/*************************************************/
$filter_names["dropRecordUnless"] = "Drop Record Unless";
function filter_dropRecordUnlessConfigure($filter_data)
{
print "Drop record unless field contains text: (comma separated list)<br />";
print "<input type='text' size='40' name='text' value='".widget_safe($filter_data["text"])."' />";
widget_errorGet("text");
}
function filter_dropRecordUnlessValidate($filter_data)
{
}
function filter_dropRecordUnlessExec($filter_data,$text)
{
global $filter_dropRecordFlag;
if($filter_dropRecordFlag)
{
return $text;
}
$parts = explode(",",$filter_data["text"]);
$filter_dropRecordFlag = (!isset($parts[$text]));
return $text;
}
?>
To use it, enter the items that you want to allow in a comma separated list, for example "A,B". The filter will then drop any record where the selected field is not A or B.
Hope this helps,
Cheers,
David.
Hmm.. No matter what I put into the field, when I import the feed there will be 0 products imported, so something's wrong here?
Does the script work at your PT install?
Hi,
Sorry - the $parts variable needs to be converted into an associative array rather than the numeric array returned by explode(). Here's the corrected version:
<?php
/*************************************************/
/* dropRecordUnless */
/*************************************************/
$filter_names["dropRecordUnless"] = "Drop Record Unless";
function filter_dropRecordUnlessConfigure($filter_data)
{
print "Drop record unless field contains text: (comma separated list)<br />";
print "<input type='text' size='40' name='text' value='".widget_safe($filter_data["text"])."' />";
widget_errorGet("text");
}
function filter_dropRecordUnlessValidate($filter_data)
{
}
function filter_dropRecordUnlessExec($filter_data,$text)
{
global $filter_dropRecordFlag;
if($filter_dropRecordFlag)
{
return $text;
}
$parts = explode(",",$filter_data["text"]);
foreach($parts as $k=>$v) $parts2[$v] = 1;
$filter_dropRecordFlag = (!isset($parts2[$text]));
return $text;
}
?>
This works for an exact match on the selected field, just tested on my development server....
Cheers,
David.
Mark said:
Hi guys,
Just curious, most feeds have a "Merchant" column within them.
Would'nt it be easier to change the process of importing feeds so that when you register a feed you have a "Merchant" and "user_Merchant" as you do category and brand? Then you could name some feeds and select from the dropdown on others...... I know that would help me out quite a bit....
Mark Bellavanc
_____________________________________-
Hello Mark (and any others)
The method described in Mark Bellavanc's post sounds like a nice straightforward way to read in a single file containing data from multiple merchants (identified in the file). Has anyone done it? (i.e has anyone got any code I could crib?!) Thanks
Alastair
Hi Alastair,
I have made this mod before for another user; but it's quite a complex change.
I have, however, thought of a slightly easier way to do the same thing, so I'll check this out and then document the changes through the forum...
Cheers,
David.
That's great David
I'm sure lots of people will benefit.
Hey David
Forgot to wish you Happy Easter!
Thanks - Happy Easter to all too!
Cheers,
David.
Hello David
I wondered if you had had any further thoughts on how to load in a single feed containing data from multiple merchants? I would find this particularly useful as I find I have to manipulate data quite a lot to "normalise" the product name. I do this in Excel and I find that I have to repeat the process for each merchant's feed. For those merchants in a network such as Affiliate Window, it would be really useful to be able to just manipulate a single file in Excel and then upload with PT taking the Merchant names from a data filed in the feed.
Hi Alastair,
I'll drop you an email with the multi-merchant patch...
Cheers,
David.
Hey David, just got and installed the script, but I am having the same problem described in this thread: CJ feed containing multiple merchants in a single TXT file...
Will the "multi-merchant patch" help with this issue? and if so could I also receive it?
Thanks a lot.
Hi,
Multi Merchant patched sent by email as requested...
Cheers,
David.
hi would it be posible to have the Multi Merchant patch as well
Many thnaks
Mark
Could I have this patch too for http://www.360voice.co.uk
Thank you.
Hi Dave,
Would it be possible if can also have this patch to deal with CJ Feeds
Kind Regards
Darren
This would be a good feature, Commission Junction put all thier feeds in to single file. My 5 large merchants = 250mb xml file. Something to split the large file into sperate files for each merchant would be ideal, i don't wish to import all of them.