Export in database - what would be the way to do it?

[yujiosaka]

Export in database - what would be the way to do it?

[BubuAnabelas]

@panthony I'm not familiar with Bigtable, but since it's NoSQL perhaps JSONLineExporter could help. It writes to a file but since it's done through writable streams maybe it can be changed to pipe it to Bigtable.

[yujiosaka]

@panthony
Good point.
I suggest you using onSuccess callback now because It's way simpler.

I'd like to start thinking about making exporters to DB.
What kind of DBs you have in mind? I believe it should be schema-less, so the main candidate would be MongoDB for example.

[panthony]

@yujiosaka For my use case I had Bigtable in mind.

Ok, for now I'll just try with the onSuccess

[yujiosaka]

Closing this because the problem seems to be solved.
Feel free to open a new issue if you'd like support features to store crawled results to DBs by exporters