You are here:  » Small DataFeeds


Small DataFeeds

Submitted by multiz on Tue, 2006-07-04 00:59 in

Hello,
Thanks for the great script. I've using Overstock's datafeed but they only allow you to access 50 products at a time. Can I make it so Price Tapestry adds to the 50 products--the next 50 and so on. They use a variable to access the next set, but how would this be possible with Price Tapestry?

Thanks,
Michael
Shop.Multiz.com

Submitted by support on Tue, 2006-07-04 07:25

Hello Michael,

I think the easiest way to deal with this would be to download (or write a script to download) each chunk of their feed into a single file, and then register / import the single big file into Price Tapestry.

This is easiest to do if it is a text format file, and - specifically - it doesn't have a head row; otherwise you might have to deal with that using filters.

On a Linux server, you can download a file using wget and append it to the end of an existing file as folows:

$wget -O - "http://www.example.com/filename.txt" >> extisting_file.txt

So, for example, a feed that could only be obtained in blocks of 50, with a parameter ?start=x on the URL, you could create one big feed like this:

$wget -O - "http://www.example.com/feed.txt?start=0" >> feed.txt
$wget -O - "http://www.example.com/feed.txt?start=50" >> feed.txt
$wget -O - "http://www.example.com/feed.txt?start=100" >> feed.txt

If this isn't an option; a Windows based solution would be to download the files individually and then combine them using a spreadsheet. Finally, export from the spreadsheet (I recommend pipe as a separator) and import the combined file.

These are non-PT solutions of course - i'll have a look at the code to see if there would be an easy hack to make this work but i'm not sure it's going to be that straight forward...

Cheers,
David.

Submitted by multiz on Tue, 2006-07-04 17:31

Thank you very much. I'll see how it works once my fetch.sh file updates the shopping site tonight.

Michael
Shop.MultiZ.com