are resources being initialized more often than needed?

[msimerson]

are resources being initialized more often than needed?

[msimerson]

This should be pretty easy to write some tests for and make sure that it's doing The Right Thing today, and that it continues to in the future. I'll have a look when I get some spare cycles.

[msimerson]

I inserted a warning line inside the block of code that loads the PSL, and then restarted dmarc_httpd and watched a few connections:

Resolved [*]:8080 to [0.0.0.0]:8080, IPv4
Binding to TCP port 8080 on host 0.0.0.0 with IPv4
Group Not Defined.  Defaulting to EGID '1001 1001 0 20'
User Not Defined.  Defaulting to EUID '1000'
  Dispatch: / => CODE(0x80823de88)
path: /dmarc/json/validate
loading Public Suffix List
path: /dmarc/json/validate
path: /dmarc/json/validate

I'm not sure what you missed, but it seems to be doing exactly what I'd expect.

[msimerson]

Hmmm, I make have spoken too soon:

loading Public Suffix List
path: /dmarc/json/validate
path: /dmarc/json/validate
path: /dmarc/json/validate
path: /dmarc/json/validate
path: /dmarc/json/validate
loading Public Suffix List
path: /dmarc/json/validate
loading Public Suffix List
path: /dmarc/json/validate
loading Public Suffix List
path: /dmarc/json/validate
path: /dmarc/json/validate
loading Public Suffix List
path: /dmarc/json/validate
loading Public Suffix List
path: /dmarc/json/validate
path: /dmarc/json/validate

Now, as to why it's loading more often than needed, I'm uncertain. And the instances where the PSL got reloaded were minutes after the last validation, so it's highly likely that there's some garbage collection / memory pruning stuff happening by something.

[rjbs]

I'm laid up sick today, so forgive me if it turns out I'm writing this whilst hallucinating.

What I don't see is why this would not always be reloading. I'm going to walk through the code a bit.

Net::DMARC::HTTP dispatches every request (in its if ($path) branch) to a routine with no access back to $self, so the HTTP daemon itself can store no state. Validations go to serve_validator.

If there are no errors, then serve_validator will always create a Mail::DMARC::PurePerl object and call its validate method. The object is stored in a local variable and not cached or closed over that I can see.

validate will call exists_in_dns if it can figure out a domain from From. exists_in_dns will call get_organizational_domain, which will call is_public_suffix. That's defined in Mail::DMARC::Base.

is_public_suffix does cache the public suffix list. It stores it in $self, which in this call chain would be the Mail::DMARC::PurePerl object created in the HTTP. That means the cache would have the life span of exactly one request.

It seems unlikely that you'd feed the test server a lot of cases and for minutes get only requests that bail out before calling is_public_suffix, but I don't see another obvious way that this would be cached.

[msimerson]

So, the reason I saw more than one loading of the PSL is actually fairly easy to explain: Net::Server::PreFork.

11025  0  SJ   0:00.61 /usr/bin/perl ./bin/dmarc_httpd (perl5.18.4)
11031  0  SJ   0:00.08 /usr/bin/perl ./bin/dmarc_httpd (perl5.18.4)
11033  0  SJ   0:00.07 /usr/bin/perl ./bin/dmarc_httpd (perl5.18.4)
14682  0  SJ   0:00.06 /usr/bin/perl ./bin/dmarc_httpd (perl5.18.4)
14920  0  SJ   0:00.00 /usr/bin/perl ./bin/dmarc_httpd (perl5.18.4)
14974  0  SJ   0:00.00 /usr/bin/perl ./bin/dmarc_httpd (perl5.18.4)

Merging @marcbradshaw 's PR didn't fully solve the problem because if we want to only load the PSL once, then we must load it before Net::Server::HTTP forks. Moving the $dmarc object creating out of Mail::DMARC::HTTP into the dmarc_httpd exe, loading up the PSL, and then passing in a $report object solves it nicely. A PR will arrive soon.