Search API Functionality

[digitalmethodsinitiative]

Search API Functionality

[ErikBorra]

Hi @dpwoodford,

search.php can now be run through cron by setting the $cronjob option to true in the script parameters. The script takes care not to insert duplicate tweets. The user needs to estimate what cron settings are appropriate (depends on the volume of the query results).

For a medium-volume query you may want to run search.php every 15 minutes.

*/15 * * * * (cd /var/www/dmi-tcat/capture/search/; php search.php)

Would this be sufficient, or would you prefer to manage it via capture/index.php?

Best,

Erik

[dpwoodford]

Thanks for this Erik, and apologies for my slow reply. We've been under-the-gun lately, but I will check out this implementation over the next couple of days. My concern (and why capture may be better) is how well this will cope with multiple search terms (e.g. a 'bin' of 40 Australian News sites), and that it may get messy managing the tables/cronjobs required to achieve this..

But, I may not have fully understood how you have implemented it, and I'll check back in once I've had a chance to upgrade one of the machines and check it out.

[ErikBorra]

bump

[limogin]

Are there any additional documentation resources to learn on how to do more types of queries with the application? Except this own project documentation I have not found any additional resources. I'm interested in being able to apply the types of queries that are listed in the FAQ. But I can not identify exactly how I can make them.

Thanks,

[ErikBorra]

Hi @limogin,

do you want to retrieve data via search (REST API) or track (streaming API) queries? Or do you want to query the already captured data?

Best,

Erik

[limogin]

Hi Erik, Yes, I want to track via the streaming API. It is said, defining the search into the ../capture/index.php panel

Thanks,

Victor.

[ErikBorra]

Hi @limogin,

have you looked at https://dev.twitter.com/docs/streaming-apis/parameters#track?

Best,

Erik

[limogin]

Hi Erik,

ok, thanks, view.

But I can include in the query manager, in the phases to track a term as "@user" to follow the tweets of a user?.

And I can obtain a relation of tweets by your id?. I don't find nothing related to this feature.
I have set "track" in capture roles, but I should change this to "follow"?
In the documentation I view

Choose a capture role. DMI-TCAT allows you to 'track' tweets based on a set of keywords, 'follow' users based on a set of user ids, or retrieve a 'one percent' sample from the Twitter API. As Twitter only allows you to connect to either one of them for any given IP address, and as most machines only have one IP address, you will have to choose either 'track', 'follow', or 'onepercent' and define that in CAPTUREROLES.

I can execute a php script selecting a ip address?

Thanks for your attention,

Victor.

[dentoir]

For historical reference, adding a @ in a tracking will make TCAT track mentions of those users, not tweets by those users. For this, you should configure TCAT (in config.php) to use the 'follow' role.

The capture/search/timeline.php CLI script can also be used to search for tweets by users.