Trigger function on Page's requestfinished event

[yujiosaka]

Trigger function on Page's requestfinished event

[yujiosaka]

@sebastienfi
Thanks for the comment! I'm flattered :)

Unfortunately, it does not provide APIs for your use case at this moment.

I made this crawler's requestfinished event different from puppeteer's requestfinished.
The crawler's `requestfinished' is emitted only when finished requesting documents like HTMLs.

However, I would like to make one since your request sounds very reasonable.

I'm thinking about exposing puppeteer's page instance so that you have more controls.
It should go like this:

const crawler = await HCCrawler.launch();
crawler.on('newpage', async page => {
  // this is puppeteer's page
  page.on('requestfinished', async request => {
    const url = request.url()
    ...
  });
});
crawler.queue('https://example.com/');
...

How does that sounds to you?

[sebastienfi]

That sounds great !