sckott / discgolf Goto Github PK

View Code? Open in Web Editor NEW

6.0 2.0 3.0 200 KB

Discourse API R client

Home Page: https://sckott.github.io/discgolf/

License: Other

R 97.81% Makefile 2.19%

discourse-api discourse r rstats

discgolf's Introduction

discgolf

discgolf is an R client for the Discourse API

Discourse API docs: https://docs.discourse.org/

discgolf package docs: https://sckott.github.io/discgolf/

Installation

CRAN version

install.packages("discgolf")

Development version

install.packages("remotes")
remotes::install_github("sckott/discgolf")

library("discgolf")

Authentication

The Discourse API is based on using a specific installation of Discourse, which requires the URL of the target Discourse forum, your username and an API key for that installation. Get those, set as option variables in .Rprofile (use discourse_url, discourse_username and discourse_api_key) or environment variables in .Renviron (use DISCOURSE_URL, DISCOURSE_USERNAME and DISCOURSE_API_KEY) or similar file (.bashrc, .zshrc, etc.).

Get latest topics

topics_latest()

topics_latest()$topic_list$topics[,c(1:5)]

Get new topics

topics_new()

Get topics by a specific user

topics_by("cboettig")

Get a single topic by id number

topic(8)

topic(8)$post_stream$posts[,c(1:5)]

Get a post

post_get(90)

Create post, create topic first

topic_create("The problem with blue skies", text = "just saying and all that")
post_create(topic_id = 13, text = "There isn't a problem!")

Wikify a post

post_wikify(x$post_stream$posts$id[4])

Search

dg_search(query = "poo")
dg_search(posts_count = 1)
dg_search(in_ = "posted")
dg_search(status = "open")

List a user

user('sckott')
user('cboettig')

List users

users_list('staff')
users_list('new')

Create a user

(x <- user_create("jane doe", "[email protected]", "jane_doe", "afafasfdasdf"))

Activate a user

user_activate(x$user_id)

Update email address

user_update_email('jane_doe', '[email protected]')

Update user name

user_update_username('jane_doe', 'jane_doe2')

Delete a user

user_delete(x$user_id)

discgolf's People

Contributors

Stargazers

Watchers

Forkers

jacob-ogre rpodcast marionlouveaux

discgolf's Issues

users_list API point changed

Looks like the API point for users_list() has changed, or at least this doesn't work for the ropensci forums:

library(disgolf)
user("steffilazerte") # No problem
users_list("active")  # Error: Not Found (HTTP 404)

Looking at the Discourse API docs it appears that it's a new url and format?

This gets the first 50 users with the option to paginate...

resp <- httr::GET(file.path(Sys.getenv("DISCOURSE_URL"), "directory_items.json"),
            query = list(period = "all", order = "days_visited"))
resp <- httr::content(resp, as = "parsed")

stub out json blobs for tests

probably instead of testing live installation, maybe in addition to

Figure out testing off local discourse installation

see

and don't test against our ropensci install

Make authentication optional for endpoints that don't require it?

I've been inspired by the new RStudio community portal (powered by discourse) to leverage the discourse API for scraping posts and other information, and luckily I found your package! When reviewing the API documentation and doing some quick trial and error, I noticed that a few of the endpoints do not require an API key nor user name. Within discgolf I verified that the following functions could in fact run just as well without the key or user parameters specified: categories, category, category_latest_topics, category_top_topics, tag, topics_latest, topics_new, and topics_by. For these functions I made a slight modification (along with a new utility function) to bypass using the dc function if the user does not specify user and key. For example here's a revised version of topics_latest and the new function use_auth in action:

topics_latest <- function(url = NULL, key = NULL, user = NULL, ...){
  args <- NULL
  if (use_auth(key, user)) {
    args <- dc(list(api_key = check_key(key), api_username = check_user(user)))
  }
  disc_GET(check_url(url), "latest.json", args, ...)
}

use_auth <- function(key = NULL, user = NULL) {
  # if each of key or user are not null, then assume auth is needed
  tmp <- !is.null(key) & !is.null(user)
  return(tmp)
}

Does this sound like a reasonable approach? I'm happy to make updates and would be glad to send a PR if you find it useful.

finish routes

params https://github.com/discourse/discourse_api/blob/master/lib/discourse_api/api/params.rb (later)
invites https://github.com/discourse/discourse_api/blob/master/lib/discourse_api/api/invite.rb (later)
backups https://github.com/discourse/discourse_api/blob/master/lib/discourse_api/api/backups.rb (later)

vignette

finish off remainder of major routes

see if any new routes to add from changelog

https://github.com/discourse/discourse_api/blob/master/CHANGELOG.md

use paginator client?

I'm working on my use cases collecting script. I would need to get all posts from a category and was wondering if by any chance you'd be willing to change things upstream to the following behavior 😇

If page=NULL, fetch all pages?

No worries if not, in that case I'd update my script. 😁

replace httr with crul

Search for topics with a given tag - page limitation

Hi!

First, thank you for creating the discgolf package!
I am searching for topics on the image.sc forum tagged with a given tag ("icy", the name of an image analysis software). I would like to get all topics (there are just over a hundred of them).

Session info and discgolf version

discgolf 0.2.8.91 2021-02-19 [1] Github

- Session info ------------------------------------------------------------------------------------
 setting  value                       
 version  R version 4.0.2 (2020-06-22)
 os       Windows 10 x64              
 system   x86_64, mingw32             
 ui       RStudio                     
 language (EN)                        
 collate  French_France.1252          
 ctype    French_France.1252          
 tz       Europe/Paris                
 date     2021-02-22

I tried to get "icy" topics using the dg_search() function, but it found only the 5 last topics tagged with "icy". I also tried the tag() function, but it only found the 30 latest topics. I managed with some help to get around these limitations by modifying the dg_search() function and get all the topics, but this is far from being optimal. I copy paste the code below in case someone else encounters a similar issue.

R code

# dg_search gives only the 5 last topics with the tag icy
test_query <- discgolf::dg_search(query = "tags:icy")
test_query$topics

# tag() gives access to the latest 30 topics only
all_tags <- discgolf:::tag("icy")
topics <- all_tags$topic_list$topics
nrow(topics)
# There is more than a hundred topics
all_tags$topic_list$tags


# modification of the dg_search() function
get_page <- function (url, verb, endpt, args = list(), body = list(), ...) 
{
  headers_auth <- list(`Api-Key` = discgolf:::check_key(),
                       `Api-Username` = discgolf:::check_user())
  cli <- crul::HttpClient$new(url = url, #as.url(check_url(), endpt), 
                              headers = c(discgolf:::dg_head(), headers_auth),
                              opts = list(...))
  res <- cli$get()
  # res <- switch(verb, GET = cli$get(query = args), 
  # POST = cli$post(query = args, 
  #            body = body), PUT = cli$put(query = args, body = body), 
  #               DELETE = cli$delete(query = args))
  res$raise_for_status()
  discgolf:::check_res(res)
  res_json <- discgolf:::parse_json(res)
  return(res_json)
}

# get topics tagged with "icy" on page number 1
url <- "https://forum.image.sc/search?q=tags%3Aicy order%3Alatest&page=1"
x <- get_page(url = url)
dim(x$topics)
tibble::as_tibble(x$topics)

# get topics tagged with "icy" on page number 2
url2 <- "https://forum.image.sc/search?q=tags%3Aicy order%3Alatest&page=2"
x2 <- get_page(url = url2)
dim(x2$topics)
tibble::as_tibble(x2$topics)

Could a future version of dg_search() take in account the page number?
More precisely, could dg_search() have an optionnal argument page that is added to the URL with the symbol &?