How to use Academic API to search archives with language filters?

[myrainbowandsky]

How to use Academic API to search archives with language filters?

[myrainbowandsky]

nohup twarc2 search --archive --start-time 2010-02-01 --end-time 2022-11-30 "red apple lang:en" apple.jsonl 2>&1 | tee log.log &

[igorbrigadir]

The lang should be in quotes for the query, so the inner quotes, for the phrase, "red apple" have to be escaped in your terminal, however you do that normally in bash for example:

twarc2 search --start-time "2010-02-01" --end-time "2022-11-30" --archive "\"red apple\" lang:en" apple.jsonl

(I also tend to put the dates in quotes too, not strictly necessary, but always good to be explicit, and i moved --archive to make it easier to read)

The command then:

Twarc keeps own logs, in twarc.log in the folder where you run it, and shows you a progress bar, so you don't need to tee output to a log, that will give you a log of the progress bar output, so you can run:

nohup twarc2 search --start-time "2010-02-01" --end-time "2022-11-30" --archive "\"red apple\" lang:en" apple.jsonl &

instead. Another alternative way is to run long twarc commands is inside a screen you detach from, or tmux.