Feedback loop between multiple servers with push enabled

[MrPetovan]

Feedback loop between multiple servers with push enabled

[AndyHee]

Wow!!

Converting the tag table to InnoDB made an astonishing difference in terms of performance. We probably should covert the entire DB.

I learned a lot about using phpMyAdmin. It's great!

[AndyHee]

Done.

[MrPetovan]

There has been recent alternatives, but none allow to specify the number of packet sent.

It wouldn't be too much work just to suppress this warning. But I feel like it would be a long-term decision, with more surprises along the way. A local workaround might be more appropriate.

[MrPetovan]

I ended up forking pear/Net_ping and I'm using it in the next version of the directory: https://github.com/MrPetovan/Net_Ping

There still is an undefined index issue when the remote server rejects the ping, so it will require a little bit more work.
Also this: https://github.com/pear/Net_Ping/blob/d1f370b3a6072e5a493ed855804d98aadf818fa7/Net/Ping.php#L982-L987

[MrPetovan]

An initial workaround has been to add a new id auto increment column with a PRIMARY index on it in the sync-push-queue. This requires to change the existing PRIMARY index to UNIQUE before adding the column.

This effectively transforms the table into an actual queue where new items are added to the end of the queue.

The INSERT INTO `sync-push-queue` also has to be changed to REPLACE to avoid duplicate index errors.

[MrPetovan]

I also commented out the DELETE FROM `tag` query to avoid congestion.

[AndyHee]

So do you recommend a specific setting for the time being to avoid the issue.

Everyone only pulls, but doesn't push?

dir.friendi.ca | 0 | 1 | 1523116804
dir.friendica.social | 1 | 1 | 1525699804
socialdir.isurf.ca | 1 | 1 | 1525697404

[AndyHee]

An initial workaround has been to add a new id auto increment column with a PRIMARY index on it in the sync-push-queue. This requires to change the existing PRIMARY index to UNIQUE before adding the column.

Could you commit those changes to your branch, so we can see in more detail what's changed in the DB?

[MrPetovan]

I will tonight EST.

Push is interesting for faster contact discovery, but it currently is unbridled. I would recommend to turn it off until you have changed the behavior.

[MrPetovan]

Did you measure increased performance before changing fields?

[AndyHee]

I didn't exactly measure it, but previously maybe it took about three/ four seconds just to load the directory's main page. It was extremely slow, even from my local network where the server is. Now in a "split second" everything is displayed.

[AndyHee]

Yes and the size of the table was reduced to 8.5 MiB from 517MiB.

[MrPetovan]

The size drop is explained by the enforcement of a unique key (747000 rows before, 3800 after), the switch from profile nurl to profile id, and dropping the useless id column.

I’m glad it’s faster just by changing the data structure!

[AndyHee]

I've noticed this error in the log.

2018/05/08 23:42:20 [error] 24826#24826: *127814 FastCGI sent in stderr: "PHP message: PHP Deprecated:  Non-static method Net_Ping::factory() should not be called statically in /var/www/directory/include/site-health.php on line 113
PHP message: PHP Deprecated:  Non-static method Net_Ping::_setSystemName() should not be called statically in /var/www/directory/vendor/pear/net_ping/Net/Ping.php on line 141
PHP message: PHP Deprecated:  Non-static method Net_Ping::_setPingPath() should not be called statically in /var/www/directory/vendor/pear/net_ping/Net/Ping.php on line 143
PHP message: PHP Deprecated:  Non-static method Net_Ping_Result::factory() should not be called statically in /var/www/directory/vendor/pear/net_ping/Net/Ping.php on line 404" while reading response header from upstream, client: 163.172.11.188, server: dir.hubup.pro, request: "GET /submit?url=68747470733a2f2f667269656e646963612e78797a2f70726f66696c652f77616e6465726e HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.1-fpm.sock:", host: "dir.hubup.pro"

[MrPetovan]

Which PHP version?

[AndyHee]

PHP Version 7.1.16-1

[MrPetovan]

Ugh, this is due to Net_Ping having been written in the previous century.

Either we maintain an updated version of the lib, or we make the calling code uglier.

[AndyHee]

Ahh.. Net_Ping is unmaintained. The last release was written for PHP 4.0.0 almost nine years ago!

I take, it there is no existing alternative, ready for use. How much work would it involve to bring the lib up-to-date?

[annando]

Is that some test code?

[MrPetovan]

Looks like some debug code that was committed in the Pear package source.

[MrPetovan]

Won't fix.