I'm currently working on a series of PT-websites, which will all be targeted at related niches and therefore use the same datafeeds.
Instead of having to set up all datafeeds for each website (and - when a datafeeds-structure has changed - make changes on every website), I would like to do the Feed Registration on a central place, whereafter the other admin-functions (e.g. category mapping, product mapping, ...) are taken care of at each individaul website.
Thus, a construction where each PT-install takes the both the feeds, as well as the registry-settings & filters from one central DB, whereafter I can hit "import" or "de-import" for every install.
Any thoughts how to start working on something like this?
Yes, that sounds good! Although, I do have some remarks / questions:
Would it be possible to add some "Global" filters to certain feeds? For instance, if a certain feed would need a UTF8-encode, it would be handy to enter that once, instead of for each individual website. Perhaps this is something that can be "hardcoded" in the files somewhere?
Also, would it be possible to enter (for each website) which feeds need to be imported, and which not (or another work-around to achieve the same)?
Hi,
I'll get started on this and follow up by email for you...
Cheers,
David.
Hi David,
I'm almost done with some non-work related stuff and about to start working on this shortly. Did you manage to take a closer look into this?
Hi,
Sorry for the delay, I do now have global filters implemented in the development version which I can back port for you, i've sent you an email to request the required files for modification...
Cheers,
David.
--
PriceTapestry.com
David, thanks again for your superior support!
Earlier in this topic you mentioned the idea to have one "master"-admin and copy all feed-settings to each of the slave-installs. Any thoughts how this can be done the best? Is there some sort of script that copies some parts of the DB to each of the slaves automatically?
I have a main site and now want to set up several smaller sites and I would like them all to use the same admin section including feeds, product mapping, settings etc.
All feeds would be imported and the products displayed on wordpress sites.
So I would have feedsdomain.com to store and run all the product feeds then site1.com site2.com site3.com
Would this be possible?
Hi all,
What I normally suggest is making use of the Database Tool (admin menu) to copy settings across from a master to slave installations, where each shares a common feed directory.
Let's say you have the following set-up:
example.com/master/feeds/
example.com/site1/
example.com/site2/
In config.advanced.php for site1/, site2/ etc., you would have
$config_feedDirectory = "../../master/feeds/";
Then, in /master/admin/ manage your feed registrations, product mapping, filters etc., (but don't import) and then use the Database Tool to export all settings. Then, use Database Tool again on each of site1/admin/ etc. to copy the settings across. Note that this shouldn't have to be a regular operation - normally only required after adding a new feed and creating any mappings required etc.
Kimarie - you mentioned using the same product mapping, filters etc. for all installations so it may be that you don't need to create separate Price Tapestry slave installations as the separation can simply be done in the way in which you use the external scripts to pull the required content into your external sites - let me know if you're not sure...
Cheers,
David.
--
PriceTapestry.com
Finally. I have time to work on this project again!
If the parent- and child-sites all use the same /feed/-directory, and all the products are in seperate databases (config_databaseName differs), would it be possible to configure it so that all the child-sites "look" at the parent-DB for Feed Registration (includes filters)?
I was planning to do export / import after each change, but A) it will take a lot of time if I have to import for (for example) 20 sites when I have one minor change, and B) when importing in a child-site, all global filters are gone (which is necessary to determine what will be included).
Thanks in advance!
Hi,
Provided that the database username and password are the same, this should be possible by prefixing all database references to the `feeds` and `filters` table with the database name, e.g for `feeds` replace:
SELECT * FROM `".$config_databaseTablePrefix."feeds`with:
SELECT * FROM `masterdatabasename.".$config_databaseTablePrefix."feeds`...and for the `filters` table, replace:
SELECT * FROM `".$config_databaseTablePrefix."filters`with:
SELECT * FROM `masterdatabasename.".$config_databaseTablePrefix."filters`References to the `feeds` table are as follows (you could just do Search and Replace in your text editor with the above - make sure you have everything backed up first...!
admin/feeds_filters.php
Line 51
admin/feeds_filters_configure.php
Line 18
admin/feeds_import_slow.php
Line 150,165,169,258,289,322
admin/index.php
Line 59
includes/admin.php
Line 31,41,53,86,95,374,455,487,496
scripts/import.php
Line 59,74,89
References to the `filters` table are as follows:
admin/feeds_filters.php
Line 26,68
admin/feeds_filters_configure.php
Line 31,65
admin/feeds_filters_delete.php
Line 15
admin/feeds_import_slow.php
Line 26
includes/admin.php
Line 99,421,505,509,517
Cheers,
David.
--
PriceTapestry.com
Would it also be possible to have feed-filters from BOTH databases?
So that all the filters from the parent-website will be automatically included in the child-sites, while it is also possible to add feed-filters for each of the child-sites?
I was planning to filter out products for each child-site using Global Filters, but that option isn't advanced enough (i.e. I need feed-specific filters as well).
Hi,
You could have a go with leaving most of the filters table references to the local database, but then the instance where the filters are loaded during import, at the following code on line 421 of includes/admin.php
$sql = "SELECT * FROM `".$config_databaseTablePrefix."filters` WHERE filename='".database_safe($admin_importFeed["filename"])."' ORDER BY created";..REPLACE that with:
$sql = "SELECT * FROM `".$config_databaseTablePrefix."filters`,`masterdatabasename.".$config_databaseTablePrefix."filters` WHERE filename='".database_safe($admin_importFeed["filename"])."' ORDER BY created";Hope this helps!
Cheers,
David.
--
PriceTapestry.com
It appears to be not working, because I don't see any difference. This is what I've done:
1. Replaced code in admin.php of the child-site (and edited the masterdatabasename to the right_name)
2. Added a filter on the master-site to test.
3. Imported products at the child-site, but I didn't see any difference with before (the filter was not taken into account).
Any thoughts of how to check what the problem is?
Hi,
In place of the above replacement, have a go with:
$sql = "SELECT `".$config_databaseTablePrefix."filters`.*,`masterdatabasename.".$config_databaseTablePrefix."filters`.* FROM `".$config_databaseTablePrefix."filters`,`masterdatabasename.".$config_databaseTablePrefix."filters` WHERE filename='".database_safe($admin_importFeed["filename"])."' ORDER BY created";If still no joy, i'll set-up an equivalent situation on my test server and check it out for you...
Cheers,
David.
--
PriceTapestry.com
It's still not working, but I noticed that my original code (which I replaced) is slightly different compared to yours:
<?php
$sql = "SELECT * FROM `".$config_databaseTablePrefix."filters` WHERE filename='".database_safe($sourceFilename)."' ORDER BY created";
?>
I've tried various adjustments, but so far it didn't help.
Hi,
That's the code from the copyFilters function - make sure that it's line 421 (11/09A distribution) within the admin_import() function that you're replacing, and revert the first example as a first test, e.g.
$sql = "SELECT * FROM `".$config_databaseTablePrefix."filters` WHERE filename='".database_safe($admin_importFeed["filename"])."' ORDER BY created";...REPLACE with
$sql = "SELECT * FROM `".$config_databaseTablePrefix."filters`,`masterdatabasename.".$config_databaseTablePrefix."filters` WHERE filename='".database_safe($admin_importFeed["filename"])."' ORDER BY created";Cheers,
David.
--
PriceTapestry.com
Since I have added various other mappings (Brandmapping etc), there was nothing at line 421. There is (next to the one I used before) one other line that "looks like" the one you're referencing to:
<?php
$sql = "SELECT * FROM `".$config_databaseTablePrefix."filters` WHERE filename='".database_safe($admin_importFeed["filename"])."' OR filename='' ORDER BY filename,created";
?>
Still, not exactly the same. Since replacing this line create multiple mysql_fetch_array-errors, I assumed I would need to replace the other line.
Hi,
If you could email me your includes/admin.php I'll set-up the same scenario
on my test server and then modify your version as required...
Cheers,
David.
--
PriceTapestry.com
Again, thanks for your support!
I've just added the modification so that the feed-registration
of the master-site will be used as well. In the /admin/-screen
of the child-site I now see the product-count and import-date
of the master site, and I'm looking for a way how to change
that to information regarding the child-site. In other
words: I would like to see how many products (of each
feed) are imported on that particular child-site.
Hi,
I think this should do it...
First of all, based on the includes/admin.php that I emailed earlier, look for the following code at line 632:
$productCount = $rows[0]["productCount"];...and REPLACE that with:
$productCount = $rows[0]["productCount"];
$sql = "UPDATE `".$config_databaseTablePrefix."feeds` SET products='".$productCount."' WHERE filename='".database_safe($filename)."'";
if (!database_queryModify($sql,$insertId))
{
$sql = "INSERT INTO `".$config_databaseTablePrefix."feeds` SET products='".$productCount."',filename='".database_safe($filename)."'";
database_queryModify($sql,$insertId);
}Then in admin/index.php, look for the following code beginning at line 61:
if (database_querySelect($sql,$rows))
{
foreach($rows as $feed)
{
$feeds[$feed["filename"]] = $feed;
}
}...and REPLACE with:
if (database_querySelect($sql,$rows))
{
foreach($rows as $feed)
{
$feeds[$feed["filename"]] = $feed;
}
$sqlSlave = "SELECT * FROM `".$config_databaseTablePrefix."feeds`";
database_querySelect($sqlSlave,$rowsSlave);
foreach($rowsSlave as $feedSlave)
{
$feeds[$feedSlave["filename"]]["products"] = $feedSlave["products"];
}
}What this will do is that after the initial query to the feeds table (from the master installation), a second query will be made to the local version of the feeds table which will just hold filenames and product counts, which are then updated in the rows selected from the master site...
Hope this helps!
Cheers,
David.
--
PriceTapestry.com
I've added both modifications, but I don't see any change so far. It still shows the same import date and products as the parent-site.
Hi,
As it stands, the counts (not imported time) will only be updated as each feed on the slave site is imported, until then it will show the details from the master site.
To make things clearer (and to add the imported date from the slave installation), in place of the modification in
includes/admin.php use;
$productCount = $rows[0]["productCount"];
$sql = "UPDATE `".$config_databaseTablePrefix."feeds` SET products='".$productCount."' WHERE filename='".database_safe($filename)."'";
if (!database_queryModify($sql,$insertId))
{
$sql = "INSERT INTO `".$config_databaseTablePrefix."feeds` SET products='".$productCount."',imported='".time()."',filename='".database_safe($filename)."'";
database_queryModify($sql,$insertId);
}admin/index.php use;
if (database_querySelect($sql,$rows))
{
foreach($rows as $feed)
{
$feed["products"] = 0;
$feed["imported"] = 0;
$feeds[$feed["filename"]] = $feed;
}
$sqlSlave = "SELECT * FROM `".$config_databaseTablePrefix."feeds`";
database_querySelect($sqlSlave,$rowsSlave);
foreach($rowsSlave as $feedSlave)
{
$feeds[$feedSlave["filename"]]["products"] = $feedSlave["products"];
$feeds[$feedSlave["filename"]]["imported"] = $feedSlave["imported"];
}
}Cheers,
David.
--
PriceTapestry.com
Before (script posted at 2010-12-15 15:34) it showed numbers of the
master-site instead of the child-site, even after an import on the
child-site.
With the new script, both Imported and Products are empty, even
after importing a feed.
Any thoughts on this? Thanks! :-)
Hi,
The admin/index.php mod looks fine - in place of the mod in includes/admin.php, have a go with:
$productCount = $rows[0]["productCount"];
$sql = "DELETE FROM `".$config_databaseTablePrefix."feeds` WHERE filename='".database_safe($filename)."'";
$sql = "INSERT INTO `".$config_databaseTablePrefix."feeds` SET products='".$productCount."',imported='".time()."',filename='".database_safe($admin_importFeed["filename"])."'";
database_queryModify($sql,$insertId);Cheers,
David.
--
PriceTapestry.com
Sorry, even with this code both "Imported" and "Products" stay empty, regardless of importing the products..
Hi,
Could you email me the modified files and I'll set the same thing up my test server for you...
Cheers,
David.
--
PriceTapestry.com
Currently the feed-registration and the filters are pulled from the parent-DB, which works great!
If I would like to pull the category-mapping the same way, which files should I be editing? In other words: first the mapping of the parent-DB should be applied, whereafter the mappings from the child-DB should be applied.
Hi,
Should be straight forward in pretty much the same way as we
made feed registration show the imported / product count from
the slave database.
In includes/admin.php, look for the following
code around line 383:
$admin_importCategoryMappings = array();...and REPLACE that with:
$sql = "SELECT * FROM masterdatabasename.`".$config_databaseTablePrefix."categories`";
if (database_querySelect($sql,$rows))
{
foreach($rows as $category)
{
$alternates = explode("\n",$category["alternates"]);
foreach($alternates as $alternate)
{
$alternate = trim($alternate);
$admin_importCategoryMappings[$alternate] = $category["name"];
}
}
}You'll see that the above is then immediately followed by the
existing same code but to select from the configured database...
Cheers,
David.
--
PriceTapestry.com
Thanks, that works great!!
However, when I apply the same to other mappings (e.g. brand mapping / color mapping; mappings I've added), I get many of these errors during importing:
Warning: Invalid argument supplied for foreach() in /home/***/domains/***.nl/public_html/includes/admin.php on line 251.
All the mappings work fine the normal way. These modifications have been made based on another topic.
Here's an example of how I modified the brand mapping based on THIS topic:
$sql = "SELECT * FROM masterDB.`".$config_databaseTablePrefix."brandsmap`";
if (database_querySelect($sql,$rows))
{
foreach($rows as $brandsmap)
{
$alternates = explode("\n",$brandsmap["alternates"]);
foreach($alternates as $alternate)
{
$alternate = trim($alternate);
$admin_importBrandMappings[$alternate] = $brandsmap["name"];
}
}
}
// --- //
$sql = "SELECT * FROM `".$config_databaseTablePrefix."brandsmap`";
if (database_querySelect($sql,$rows))
{
foreach($rows as $brandsmap)
{
$alternates = explode("\n",$brandsmap["alternates"]);
foreach($alternates as $alternate)
{
$alternate = trim($alternate);
$admin_importBrandMappings[$alternate] = $brandsmap["name"];
}
}
}I've added the first part, and deleted "$admin_importBrandMappings = array();".
Hi,
Rather than delete $admin_importBrandMappings = array(); that
code does actually need to be there (or equivalent) before each mapping is
loaded from the database. That ensures that even if there are no mappings
for a particular feed you won't get any "invalid argument" errors...
That should be all it is...
Cheers,
David.
--
PriceTapestry.com
It has been a while since I've been working on this part. Right now I'm trying to figer out why the following is happening, but so far I'm in the dark..
I can register and import the feed at the parent site (and after import the products are indeed visible on the parent site).
After registering at the parent site, the feeds apear to be ready for import at the child-site. All though, after hitting "import" at the child-site, no products are being imported..
Are you willing to take a look at my admin.php (I think the problem is in there), to see if you can spot an error? If so, the files are in your inbox!
Hi,
No probs - i'll follow up by email...
Cheers,
David.
--
PriceTapestry.com
Hi,
I've just had a think about this - the objective of course is to have to manage just one instance of the `feeds` table.
As you are probably already aware, the first stage of course would be to use a common /feeds/ folder. This is configurable in the latest distribution within config.advanced.php. This can be specified as relative to /admin/, or a fully qualified path on your server. For example, if your site are just going to be in sub-directories of a common domain name, such as
/site1/
/site2/
...then at the same level you could have
/feeds/
...and in config.advanced.php (line 4):
$config_feedDirectory = "../../feeds/";Alternatively, if different websites but on the same server; choose one installation as your "master" installation and on each other installation, use a $config_feedDirectory such as:
$config_feedDirectory = "/home/username/domains/example.com/public_html/feeds/";(you can find out exactly what value to use by going to /admin/ on your master installation and using the Install Path value under Support Info - with "feeds/" on the end)
If that all sounds OK, I've considered changes throughout the code to reference a different database for the `feeds` table but that's going to be quite tricky. How about a utility within /admin/, for use on every installation EXCEPT your master installation, to Copy all feed registrations from a specified "master" installation? Would that be a suitable approach? There would be no need to use the utility on a day-to-day basis; just once on each "slave" installation whenever you have made changes on the "master" installation. Each "slave" installation would then of course have its own filters, category and product mapping etc.
How does that sound?
Cheers,
David.