UID mapping fails due to script run time limit and termination

Submitted by Jomich on Mon, 2017-03-27 07:23 in Price Tapestry

Hi David,

PT is working fine!
We have only one problem with the script run time of the uidmap.php
Our hoster sets a limit of 3 minutes for PHP scripts. The uidmap.php would run longer due to the amount of products that are imported in the db. It is terminated in phase 1 already.

Phase 1:BWT 815480 Vida White (2)

Phase 1:, Passend Oszilloskopvorsatz DSO-220 USB, 12 18 07 - 4R (64000)

<br />

<b>Warning</b>:  mysqli_real_query(): MySQL server has gone away in <b>/path/to/pt/scripts/uidmap.php</b> on line <b>78</b><br />

<br />

<b>Warning</b>:  mysqli_real_query(): Error reading result set's header in <b>/path/to/pt/scripts/uidmap.php</b> on line <b>78</b><br />

We tried out your suggestions of several other threads, like node/5034, but this didn't help.

Do you have any other suggestion for us?

Thanks,
Jochen

Hello Jochen, "MySQL server

Submitted by support on Mon, 2017-03-27 10:15

Hello Jochen,

"MySQL server has gone away" is something of a mysterious error which I think could be somewhat tricky to handle when occurring in the middle of a buffered read; however the point at which the error occurs indicates that phase 2 is being reached. The first thing I would suggest would be to re-create both database links before phase 2 begins - to do this, edit uidmap.php and look for the following code at line 63:

$sql1 = "SELECT * FROM `".$config_databaseTablePrefix."uidfix`";

...and REPLACE with:

  @mysqli_close($link1);

  @mysqli_close($link2);

  $link1 = mysqli_connect($config_databaseServer,$config_databaseUsername,$config_databasePassword,$config_databaseName);

  mysqli_set_charset($link1,"utf8");

  $link2 = mysqli_connect($config_databaseServer,$config_databaseUsername,$config_databasePassword,$config_databaseName);

  mysqli_set_charset($link2,"utf8");

  $sql1 = "SELECT * FROM `".$config_databaseTablePrefix."uidfix`";

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Hi David, It works

Submitted by Jomich on Tue, 2017-03-28 06:22

Hi David,

It works perfectly.

Thanks a lot,
Jochen

Hi David, After importing

Submitted by Jomich on Thu, 2017-04-06 08:38

Hi David,
After importing plenty of products (more than 2 million) we have the same error message (MySQL server has gone away).
The problem is that this time this happens during phase 2.

We didn't find any solution, yet. Do you have a hint for us?

Thanks a lot,
Jochen

Hello Jochen, Try adding the

Submitted by support on Thu, 2017-04-06 08:50

Hello Jochen,

Try adding the following code immediately after the above replacement to set a long timeout period on the new connections;

  mysqli_query($link1,"SET SESSION wait_timeout = 900");

  mysqli_query($link2,"SET SESSION wait_timeout = 900");

Cheers,
David.
--
PriceTapestry.com

Hi David, This didn't help,

Submitted by Jomich on Fri, 2017-04-07 06:29

Hi David,

This didn't help, unfortunately.
Did you experience problems like this, perhaps due to the size of a table?

The pt_products has now 4.4 GB.

Thanks a lot,
Jochen

Hello Jochen, Please could

Submitted by support on Fri, 2017-04-07 06:37

Hello Jochen,

Please could you email me the uidmap.php script that you have so far and I'll check it out further with you...

Thanks,
David.
--
PriceTapestry.com