How to manage a URL feed that changes every day (date in URL)?

Submitted by sirmanu on Mon, 2017-01-23 16:31 in Feed Management and Administration

Hi again David.

I am pretty sure that this topic has been already asked, but I can't find any useful information.

I have a feed which name is changing according to its date. In fact, they have a "base" feed and then, they push updates every day to reflect this modifications.

For example:

1 January: base_feed.xml (20.000 products).
2 January: feed_delta_01_02_17.xml (1000 products). Only products that changed.
...
1 February: base_feed.xml (xxx products).

The information of delta_feed is the same as base_feed, but as you can see, the problem is that the URL is changing every day! I have thought about creating a "tmp" cron job, registered, and then, modify via SQL the filename according to the date.

So, do you have an elegant solution?
Thank you.

Hi, Support for a %DATE%

Submitted by support on Mon, 2017-01-23 17:04

Hi,

Support for a %DATE% placeholder in the URL can be added easily - edit includes/automation.php and look for the following code at line 22:

$job = $jobs[0];

...and REPLACE with:

    $job = $jobs[0];

    $job["url"] = str_replace("%DATE%",date("%m-%d-%y"),$job["url"]);

Then use %DATE% in the URL field of an Automation Tool job e.g.

http://www.example.com/export/feed_delta_%DATE%.xml

Note: make sure that %m (month) and %d (day of the month) are the correct way round according to the date format used by the merchant in the call to PHP's date() function in the replacement...

Cheers,
David.
--
PriceTapestry.com

LOL! When I said elegant

Submitted by sirmanu on Mon, 2017-01-23 20:02

LOL! When I said elegant solution I wasn't expecting just one line modification. I' ll try it and see how works. Thank you!

Hi! I have not tested yet, I

Submitted by sirmanu on Tue, 2017-01-24 14:32

Hi! I have not tested yet, I was only thinking before implement it. But, is not going to be unregistered every new filename??

Hi, Placeholder substitution

Submitted by support on Tue, 2017-01-24 14:49

Hi,

Placeholder substitution will only affect the job URL - the Filename to Save As... will remain unchanged.

However, just reviewing your original post there may need to be further considerations, as by default the import process is DELETE/ INSERT, but if I understand correctly now, you ONLY have access to a change feed on a regular basis?

In which case, how does the change feed indicate deletions? Because unless you always have access to the master copy, you would never have any visibility of deletions...

Cheers,
David.
--
PriceTapestry.com

Yes, I have access to the

Submitted by sirmanu on Wed, 2017-01-25 21:21

Yes, I have access to the monthly copy, and also daily copy.
You are right, the filename is not necessary to be changed. Sorry!