ha-pu / globaltrends Goto Github PK

Download and measure global trends through Google searches

License: Other

R 7.82% HTML 92.18%

globaltrends's Introduction

globaltrends

Google offers public access to global search volumes through its search engine through the Google Trends portal. The globaltrends package downloads search volumes provided by Google Trends and uses them to measure and analyze the distribution of search trends across countries or within countries. globaltrends allows researchers and analysts to investigate patterns within these trends, such as degree of internationalization of firms and organizations or dissemination of political, social, or technological trends across the globe or within single countries.

To measure degree of internationalization, globaltrends offers a wide array of empirical possibilities. It allows researchers to compare degree of internationalization for various organizations on a unified scale. In addition, the time-series nature of Google Trends allows for historical analysis of internationalization patterns and speed within organizations.

The enormous detail of the data opens additional applications in research that are impossible with traditional measures of internationalization. For instance, using globaltrends on a subnational level allows researchers to study proliferation within a country and, for example, to trace a particular market entry. In addition, globaltrends offers applications beyond corporate internationalization, such as data on global interest in products, persons, events, fads or scandals, even academic authors and papers.

globaltrends provides user-friendly access to Google Trends. The package vignette offers additional technical details and a basic tutorial. Please, refer to the package NEWS for change log.

# install ----------------------------------------------------------------------
# current cran version
install.packages("globaltrends")
# current dev version
devtools::install_github("ha-pu/globaltrends", build_vignettes = TRUE)

# packages ---------------------------------------------------------------------
library(dplyr)
library(globaltrends)

# connect to db ----------------------------------------------------------------
initialize_db()
start_db()

# add new control batch --------------------------------------------------------
new_control <- add_control_keyword(keyword = c("gmail", "map", "translate", "wikipedia", "youtube"))

# add new object batch ---------------------------------------------------------
new_object <- add_object_keyword(keyword = c("manchester united", "real madrid"))

# run control download ---------------------------------------------------------
download_control(control = new_control)
download_control_global(control = new_control)

# run object download ----------------------------------------------------------
download_object(object = new_object)
download_object_global(object = new_object)

# compute search score ---------------------------------------------------------
compute_score(control = new_control, object = new_object)
compute_voi(control = new_control, object = new_object)

# compute degree of internationalization ---------------------------------------
compute_doi(control = new_control, object = new_object)

# export data ------------------------------------------------------------------
out_score <- export_score(keyword = "manchester united")
out_voi <- export_voi(keyword = "manchester united")
out_doi <- export_doi(type = "obs", locations = "countries")

# plot data --------------------------------------------------------------------
plot_bar(data = out_score)
plot_ts(data = out_score)
plot_box(data = out_score)
plot_ts(data = out_voi)
plot_box(data = out_voi)
plot_ts(data = out_doi)
plot_box(data = out_doi)
plot_voi_doi(data_voi = out_voi, data_doi = out_doi)

# get abnormal internationalization --------------------------------------------
abnorm_score <- get_abnorm_hist(data = out_score)
plot_bar(data = abnorm_score)
plot_ts(data = abnorm_score)
plot_box(data = abnorm_score)

abnorm_voi <- get_abnorm_hist(data = out_voi)
plot_ts(data = abnorm_voi)
plot_box(data = abnorm_voi)

abnorm_doi <- get_abnorm_hist(data = out_doi)
plot_ts(data = abnorm_doi)
plot_box(data = abnorm_doi)

# disconnect from db -----------------------------------------------------------
disconnect_db()

If you use the globaltrends package, please cite it as: Puhr, H., & Müllner, J. (2021). Let me Google that for you: Capturing globalization using Google Trends (SSRN Working Paper 3969013). Available at https://www.ssrn.com/abstract=3969013.

globaltrends's People

Contributors

Stargazers

Watchers

Forkers

reema-dass26

globaltrends's Issues

table names

batch_terms.rda
batch_time.rda
data_agg.rda
data_con.rda
data_map.rda
data_obj.rda
data_score.rda
data_wrld.rda

Assign to globalenv

https://stackoverflow.com/questions/9726705/assign-multiple-objects-to-globalenv-from-within-a-function

Set non-results to 0

Add location function

Adapt .Rbuildignore

Vignette docx
Vignette pdf
Covid Paper pdf

object names

gtrends_db -> doiGT_DB
gtrends.sqlite -> doiGT_DB.sqlite

Remove: check if empty

add .test_empty to run_remove functions

data_score

adapt to nest(., data = c(date, hits))
adapt to select., geo, date, key, keyword, value)
check for unnest()

Link examples to tests

Use sample data
Use same companies in tests

run_init

suppress src_sqlite warning
add messages for each step -> also for insert_geo

Integrate data_object and data_mapping

Run batches as c(control, object, object, object, object) and call compute_score internally

filter lst_geo in run_agg

Filter for which type of lst_agg the function run_agg should compute doi. Takes either lst_wdi or lst_usa.

Replace str_c by glue

# -------------------------------------------------------------------------------------------------
.add_keyword_batch
@importFrom glue glue
X @importFrom stringr str_c
message(glue("New control batch {new_batch} ({keyword}, {time}) created."))
message(glue("New object batch {new_batch} ({keyword}, {time}) created."))

# -------------------------------------------------------------------------------------------------

run_agg.numeric
@importFrom glue glue
X @importFrom stringr str_c
message(glue("Successfully computed DOI | control: {control} | object: {object} [{object}/{total}]", total = max(terms_obj$batch)))

# -------------------------------------------------------------------------------------------------

run_control.numeric
@importFrom glue glue
X @importFrom stringr str_c
message(glue("Successfully downloaded control data | control: {control} | geo: {.x} [{current}/{total}]", current = which(lst_geo == .x), total = length(lst_geo)))

# -------------------------------------------------------------------------------------------------

run_map.numeric
@importFrom glue glue
X @importFrom stringr str_c
out <- .get_trend(geo = .x, term = c(term_con[[i]], term_obj[[1]]), time = glue("{date_min} {date_max}"))
message(glue("Successfully downloaded mapping data | control: {control} | object: {object} | geo: {.x} [{current}/{total}]", current = which(lst_geo == .x), total = length(lst_geo)))

# -------------------------------------------------------------------------------------------------

run_object.numeric
@importFrom glue glue
X @importFrom stringr str_c
message(glue("Successfully downloaded object data | object: {object} | geo: {.x} [{current}/{total}]", current = which(lst_geo == .x), total = length(lst_geo)))

# -------------------------------------------------------------------------------------------------

run_remove
@importFrom glue glue
X @importFrom stringr str_c

.remove_batch_terms
message(glue("Successfully deleted control batch {batch} from 'batch_terms'."))
message(glue("Successfully deleted object batch {batch} from 'batch_terms'."))

.remove_batch_time
message(glue("Successfully deleted control batch {batch} from 'batch_time'."))
message(glue("Successfully deleted object batch {batch} from 'batch_time'."))

.remove_data_con
message(glue("Successfully deleted control batch {batch} from 'data_con'."))

.remove_data_obj
message(glue("Successfully deleted object batch {batch} from 'data_obj'."))

.remove_data_map
message(glue("Successfully deleted control batch {batch_c} from 'data_map'."))
message(glue("Successfully deleted object batch {batch_o} from 'data_map'."))
message(glue("Successfully deleted control batch {batch_c} and object batch {batch_o} from 'data_map'."))

.remove_data_score
message(glue("Control batch {batch_c} deleted from 'data_score'."))
message(glue("object batch {batch_o} deleted from 'data_score'."))
message(glue("Successfully deleted control batch {batch_c} and object batch {batch_o} from 'data_score'."))

.remove_data_agg
message(glue("Control batch {batch_c} deleted from 'data_agg'."))
message(glue("object batch {batch_o} deleted from 'data_agg'."))
message(glue("Successfully deleted control batch {batch_c} and object batch {batch_o} from 'data_agg'."))

.remove_data_wrld
message(glue("Successfully deleted object batch {batch} from 'data_wrld'."))

# -------------------------------------------------------------------------------------------------

.run_score.numeric
@importFrom glue glue
X @importFrom stringr str_c
message(glue("Successfully computed search score | control: {control} | object: {object} | geo: {.x} [{current}/{total}]", current = which(lst_geo == .x), total = length(lst_geo)))

# -------------------------------------------------------------------------------------------------

.run_wrld.numeric
@importFrom glue glue
X @importFrom stringr str_c
message(glue("Successfully downloaded worldwide data | term: {current}/{total_terms} [{object}/{total_batches}]", current = which(terms == .x), total_terms = length(terms), total_batches = max(terms_obj$batch)))

Put tbl_xxx into package env

Put the tbl_xxx objects into the package environment -> use different name
Copy as tbl_xxx into .GlobalEnv
Functions all call objects that users cannot change

Input for export_xxx functions

Add numeric and list methods, allow numeric inputs of length > 1

add_batch - limit keywords to 5

Remove functions

Functions to remove data

batch_terms & batch_time
dict_obj
data_geo
data_con
data_obj
data_map
data_score
data_agg
data_doi
data_wrld

Build ggmap option into plot_score

Adapt header

Add date
Adapt title

Use package data for tests

Add minimum package versions

Rename data_xxx to tbl_xxx

-> check where applicable
-> not in plot_xxx functions

add_batch -> split input

add_batch(...){
  if (length(keywords) > 5) {
    lst_keywords <- split(keywords, ceiling(seq_along(keywords)/5))
    map(lst_keywords, add_batch)
  }
}

Return list of new batches

Update package data

add_batch

suppress warning from max(terms$batch) if terms is empty

Vignette update

Google Trends

-> What do we compute?

Location volume
Global volume -> VOI
Global distribution -> DOI

Analyze internationalization of firms

Setup and start database

Download control data

Integrate with text from ## Download data from Google Trends

Download and compute local volume

-> Explain about local volume = local interest in topic, local exposure
-> download_local_volume = combine:

download_object
download_mapping
compute_score

Download and compute global volume

-> Explain about global volume = global interest in topic, volume of internationalization
-> Mention abbreviation "VOI"

download_global_control
download_global_volume

Compute degree of internationalization

-> Explain that global distribution = degree of internationalization
-> Mention abbreviation "DOI"

Exports and plots

plot_score
plot_voi_ts
plot_voi_box
plot_doi_ts = plot_ts
plot_doi_box = plot_box
plot_voi_doi = plot_trend

Integrate global data with location data

download_global_control and download_global_object -> wrap download_control and download_object with location = ""

Disconnect from db function

Add compute_score_global function

Define summarise .groups

Define .groups for all dplyr::summarise

function names

run_control -> download_control(control, locations)
run_object -> download_object(object, locations)
run_wrld -> download_worldwide(object)
run_map -> download_mapping(control, object, locations)
run_score -> compute_score(control, object, locations)
run_agg -> compute_doi(control, object, locations)
run_init -> initialize_database()
gtrends_base -> start_database()
run_remove -> remove_data(table, control, object)
add_batch -> add_keywords(type, keywords, time)

Initialize

Run Code from "gtrends_base.r" at current working directory ("./Google_Trends")

Create database
Create base tables
- "data_geo" with "lst_geo" from "WDI" package and "lst_usa" from "gtrendsR" package
- Example data
Create empty tables
- Upload data.frame with one row of example data to initialize table with batch_c and batch_o "0"
  dbWriteTable(conn = gtrends_db, name = "table_name", value = data, append = TRUE)
- Set indize for tables
  tmp <- dbSendQuery(gtrends_db, "CREATE INDEX id_con ON table_name (geo, batch);"); dbFetch(tmp); dbClearResult(tmp)
- Delete data from tables
  dbExecute(conn = gtrends_db, statement = "DELETE FROM table_name WHERE batch_c=0")

separate add_batch into add_control_batch and add_object_batch

Both functions call add_batch with the respective inputs
Set add_batch to internal

run_object(object = x){
  map(object, run_object)
}

Add code junks

Computaiton for

gini
hhi
entropy

test batch function

.test_batch <- function(batch) {
  if (is.integer(batch)) {
  return(TRUE)
  } else if (is.numeric(batch)) {
    if (batch %% 1 == 0) {
    return(TRUE)
    } else {
   stop("Error: Batch number must be an integer value.\nYou supplied a non-integer numeric value.")
  } else {
  stop("Error: Batch number must be an integer value.\nYou supplied a non-integer value.")
}
}

implement pivot_X

replace spread and gather through pivot_wider and pivot_longer

gtrends_base export names

  # write objects to .GlobalEnv ----
  lst_object <- list(gtrends_db, data_geo, batch_terms, batch_time, dict_obj, data_agg, data_con, data_map, data_obj, data_score, data_wrld, lst_wdi, lst_usa, terms_con, time_con, terms_obj, time_obj, dict_obj)
  names(lst_object) <- list("gtrends_db", "data_geo", "batch_terms", "batch_time", "dict_obj", "data_agg", "data_con", "data_map", "data_obj", "data_score", "data_wrld", "lst_wdi", "lst_usa", "terms_con", "time_con", "terms_obj", "time_obj", "dict_obj")
  list2env(lst_object, envir = .GlobalEnv)

add_batch -> return batch#

add_batch(...){
  return(new_batch)
}

obs, sad, trd
gini, hhi, entropy

Import functions

Functions to import data

batch_terms & batch_time
dict_obj
data_geo
Write data.frame into data base

Export functions

Functions to export data from tables by batch

data_con
data_obj
data_map
data_score
data_agg
data_doi
data_wrld

export_doi(keyword, object, control, locations, from, to, type, table, file)

value in keyword overrules value in object, defaults to NULL
object is integer value, defaults to NULL
control is integer value, defaults to NULL
locations = c(countries, us_states), defaults to countries
from and to are dates, defaults to NULL
type = c(score_obs, score_sad, score_trd), defaults to score_obs
table is character indicating the table from where the data is downloaded, defaults to "data_doi"
file is character indicating the storage location of the output file, defaults to NULL
if file == NULL output is a tibble containing the data export, when is.character(file), the output is saved at the given location through writexl::write_xlsx(data, file)

gtrends_base

suppress .GlobalEnv message

Reorganize plot functions

plot_score
plot_voi_ts
- like plot_doi_ts
plot_voi_box
- like plot_doi_box
plot_doi_ts
- replaces current plot_ts
plot_doi_box
- replaces current plot_box
plot_voi_doi
- replaces current plot_trend

Drop normalization plot_score

Run functions

Functions to run batch downloads

run_control
run_object
run_map
run_score
run_agg
run_doi