Hi David,
I hope life is treating you well. I've run into a situation where I'll need some assistance. About two years ago I set up a shop that featured a specialty datafeed from one merchant with a very large datafeed. The specialty feed covered the one category of my shop (and the domain) so if they sold widgets for example, these were the "all widget designs for age group 3 to 10 only". The shop covered about 4,800 products from their datafeed and from two other merchants with similarly targeted products. OK, now that mega merchant has consolidated their datafeed into over 400,000 products. I'm not able to justify the hit on the server to parse out under 5,000 products, but due to the theme, the name of the shop, I can't add a wider group of their products.
I am able to select the categories or departments for a download. I looked at the mega datafeed, but it is not properly categorized to download only the categories I would need, it is sorted by designs and not ages now so the products I have featured in that shop are now sorted by design.
Their answer was to use the network's API to integrate only the same line of products I've been using there because I could select only specific product names such as "Kids Blue Widgets" or "Kids Round Widgets". So my question is - how can I integrate API calls via the cron scripts? The API requests would be in the form of "&search_results_fields=Merchant+Name|Product+Name|" for multiple product names or keywords in product names. CSV is one of the output options. The network says it is commonly done, but I don't see a way to add API calls in place of URLs for the cron setup. I've never used APIs though I do have keys for it. The link to more information and a description of requirements is here: {link saved}
Do you think I can use this with my 13a version? I didn't update to a newer version because I created a responsive design for the old one that works fine with only html5/css3. I'm probably missing out on new features, but "if it ain't broke don't fix it" works well for most things. That shop is currently no-indexed.
Thank you,
2fer
Hi David,
I don't know what a .sh script is, sorry. I need to get back to my contact at the network, she said I should be able to do all keyword searches with one request, maybe this script stuff is what she meant. My previous cron was set up using the Automation included in the script where I enter the URL with the parameters to download the datafeed and it returned the cron command line. I have not done any API requests yet because I am not clear where the output would show up, in what form or what to do with it when it was returned. As I said, I have no idea how to use the API.
I will need to see what specific information I can get from the network and then get back to you.
Thanks,
2fer
Hi 2fer,
No problem - a method that works nicely that I have put in place for other users in similar scenarios is a local script that makes the API calls and returns the content, which you can then set-up just as any other Automation Tool job.
If you could perhaps email me an example of the API call that needs to be made I'll see if that would be an appropriate option for you...
Thanks,
David.
--
PriceTapestry.com
Hi David,
I can't see what will be output using this request until it is in the shop. I don't have a way to download and preview so I have no idea of what will be "delivered", but I do have the request format and I'll email it to you. Thank you so much. I am going to include their self-generated code to view the products, though similar code is not working for me.
Thank you,
2fer
Hi David,
Thank you so much for the perfect solution!
Just one question sort of related to this shop. Some time ago you gave me an extra search script that would allow a specific and relevant term for these products. I see I never finished getting it to work. I think I can see how to offer that option and it would help to narrow down the number of results with these broad categories. My question: would that more specific search also need to be added to the htaccess file the same as the one for the merchants.php for example? That one looks like this:
RewriteRule ^merchant/$ merchants.php
RewriteRule ^merchant/(.*)/$ search.php?q=merchant:$1:&rewrite=1&%{QUERY_STRING} [L]
RewriteRule ^merchant/(.*)/(.*).html$ search.php?q=merchant:$1:&page=$2&rewrite=1&%{QUERY_STRING} [L]It would be easy to duplicate that set of rules with the term.php in place of the merchants.php and maybe that's why I never got it running before (?)
(It might just as well have been a problem of being offline for months at a time, then trying to pick up where I'd left off...)
At any rate, I wanted to thank you for the outstanding support you offer -
Thank you!
2fer
Hello 2fer,
Sure, you could create a page of terms that relate well to search results on your site, and for search engine friendly URLs link to them using e.g. /terms/Blue-Widgets/. Create your page based on example.php from the distribution saved as terms.php. In the content area, generate your links as required, using hyphens in place of spaces, for example:
<ul>
<li><a href='/terms/Blue-Widgets/'>Blue Widgets</a></li>
<li><a href='/terms/Red-Widgets/'>Red Widgets</a></li>
</ul>And finally, create a new rewrite block in .htaccess as follows:
RewriteRule ^terms/$ terms.php
RewriteRule ^terms/(.*)/$ search.php?q=$1&rewrite=1&%{QUERY_STRING} [L]
RewriteRule ^terms/(.*)/(.*).html$ search.php?q=$1&page=$2&rewrite=1&%{QUERY_STRING} [L]Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
Yes, that looks like what I was needing to get it going. I'm still fine tuning some things, but I think that takes care of this question. As always, thank you for the superb support!
2fer
HI 2fer,
I would suggest creating a .sh script using wget to fetch each API request (with CSV response format selected) to stdout and the >> (concatenation) operator to construct a single .csv file combining the API responses required. For example if you create in scripts/ folder as merchant.sh:
wget -qO- http://www.example.com/api?merchant=Merchant&output=csv&category=Category1 > ../feeds/merchant.csvwget -qO- http://www.example.com/api?merchant=Merchant&output=csv&category=Category2 >> ../feeds/merchant.csv
wget -qO- http://www.example.com/api?merchant=Merchant&output=csv&category=Category3 >> ../feeds/merchant.csv
(note that the first wget command uses ">" which will create the file, subsequent commands use ">>" will append to the existing file)
If you have an existing fetch.sh script, simply add the call to the above along with your other fetch commands e.g.;
/bin/sh merchant.shHope this helps! If you're not sure, let me know how you currently have your CRON process set-up, and if possible if you could include a couple of example API call URLs (I'll remove before publishing your reply) I'll check it out further with you...
Cheers,
David.
--
PriceTapestry.com