You are here:  » Import central/decentral feeds

Support Forum



Import central/decentral feeds

Submitted by marco@flapper on Wed, 2011-02-23 13:11 in

Hi,
I'm setting up some cron jobs for some feeds and was wondering how the timing works with master/slave price tapestry websites?

I guess normally the cron jobs gets the datafeeds and after the feed is downloaded it gets imported.

But how does that work if I have a master/slave websites? I set the cron jobs on the central master price tapestry website and the decentral slave websites also get imported if the master website is imported?

How much time is there between the import of the master and the import of the slave website or is this immediately?

Submitted by support on Wed, 2011-02-23 15:03

Hi Marco,

What I tend to do is set-up the fetch cron, and then look at the modified times of the files to see how long the fetch process is generally taking. Then, schedule your import cron jobs for, say, 1 hour after that - so there will always be plenty of time for the fetch to complete before being imported.

Cheers,
David.
--
PriceTapestry.com

Submitted by marco@flapper on Wed, 2011-02-23 15:48

Hi,
I fetch the datafeeds at the central master site. There I do a import cron too. But do i need to set up an import cron at the slave sites too?

Submitted by support on Wed, 2011-02-23 16:08

Hi Marco,

That's correct, so you might have:

1:00AM
Fetch Master
Import Master

3:00AM
Import Slave 1
Import Slave 2

If you fetch / import process is all in one script, so the processes take place in turn then actually there's not reason not just to do it all in one go, e.g.

1:00AM
Fetch Master
Import Master
Import Slave 1
Import Slave 2

Cheers,
David.
--
PriceTapestry.com

Submitted by marco@flapper on Thu, 2011-02-24 09:43

Hi,
I'm trying to understand if I have a problem or that I didn't configure it right.
I run a cron (combined fetch and import) at the master site via cron. And do a cron with only import at the slave site.

In the admin of my master website I have a feed which I didn't register yet at the master site. I did register it at the slave site but I noticed that the content of the feeds are not current. So I checked the fetched content of the feed at the master site. I noticed that the content of that feed were updated.

So I went to the admin of the master site and saw that the feed was modified at: 2011-02-24 00:30:09. But if I look to the date (with ftp) I see a date: 2011-02-24 01:30:09

I checked by registering the feed at the master site and then I saw the content of yesterday instead of today. But the strange thing is that if I look on the feeds folder, I only see the feed with today's content?

Submitted by support on Thu, 2011-02-24 11:11

Hi Marco,

I wouldn't worry about the 1 hour time difference being observed - that's most likely down the FTP program taking time zone into account differently which some clients do. I would double check by actually looking in the feed to see if it has been updated by the provider (merchant / affiliate network); maybe a transient error there has meant that content wasn't updated in their overnight process perhaps?

Also check the modified / imported time as displayed in the slave installation - double check that Imported is later than Modified, and that both are consistent with when you were expecting the CRON jobs to run...

Cheers,
David.
--
PriceTapestry.com