Performance problems

[wandenberg]

Performance problems

[wandenberg]

Which version of nginx and module are you using?
The latest code should avoid this CPU starvation.

A large number on count may slow down the redis.

If you need to purge content frequently may be better to set a lower value
to it instead of purge it actively.
On May 20, 2015 19:29, "Vedran" [email protected] wrote:

Hello,

I have tens of millions of files in cache and purge request causes 1000s
of consecutive SCAN ... MATCH ... COUNT 100 queries being performed against
redis which slows down nginx, making it completely saturate cpu cores
during the purge (which is quite often).

Increasing COUNT from 100 to 1000000 in code resolves this issue (thus it
only needs to execute a couple of queries), however when redis reply is
lengthier (in my case it was over 1000 keys in a single reply) purge
doesn't seem to work and I never get a reply from nginx. I noticed it fails
to send subsequent queries to redis with a new cursor so I guess it just
hangs after reading the first batch from redis. There are no errors in logs
and it continues to issue SETEX commands to redis just fine.

There's one more thnig. If I reload nginx while the purge is being
performed, old process ("worker process is shutting down") goes into loop
(using 100% of cpu) filling the log with "[alert] 8677#0: *696760740 header
already sent, ..." until I kill it (with sigkill, term doesn't work).

Regards,
Vedran

—
Reply to this email directly or view it on GitHub
#1
.

[vedranf]

Both are latest version (nginx 1.8 and up to date selective-cache-purge-module)

I just noticed it's using a separate nginx worker for redis communication so that's why delivery appears to be unaffected by cpu usage. However, would it be possible to specify low scheduling priority (10-15) and specific cpu core (using ngx_setaffinity) so that it doesn't share the same as workers doing content delivery? As currently both delivery and purge workers are competing for cpu time on a same cpu core.

Regards,
Vedran

[vedranf]

Hmm, you can ignore that last comment, it appears those were some stale workers left over after nginx was reloaded. I'm going to close this for now and will look more into it over the next few days.

Thanks,
Vedran