matsura8/heyshiny

Package: New `shiny` input that translates audio to text

heyshiny - Speech to Shiny Text Input

Add Speech Recognition to your Shiny app! The heyshiny package provides a new Shiny input, the speechInput(). This new input allows your Shiny app to listen to the microphone, recognize the speech, and return it as text.

So, make you Shiny app voice-interactive with heyshiny!

heyshiny is possible thanks to the annyang javascript library.

Installation

heyshiny is currently only available as a GitHub package.

To install it run the following from an R console:

if (!require("remotes")) {
  install.packages("remotes")
}
remotes::install_github("jcrodriguez1989/heyshiny", dependencies = TRUE)

Important!

annyang, and thus heyshiny, depends on that the used browser supports speech recognition. The RStudio viewer pane does not support heyshiny, so addins created with heyshiny should be opened in a new window (which opens a supported browser).

heyshiny works nicely with Google Chrome.

Frequently Asked Questions

For a FAQ with respect to the Speech Recognition tool, annyang, please refer to its FAQ page, where you can find:

• What languages are supported?
• Why does the browser repeatedly ask for permission to use the microphone?
• What can I do to make speech recognition results return faster?
• How can I contribute to annyang’s development?
• Why does Speech Recognition repeatedly starts and stops?
• Can annyang work offline?
• Which browsers are supported?
• Can annyang be used to capture the full text spoken by the user?
• Can I detect when the user starts and stops speaking?
• Can annyang be used in Chromium or Electron?
• Can annyang be used in Cordova?

Example

library("shiny")
library("heyshiny")

ui <- fluidPage(
  useHeyshiny(language = "en-US"), # configure the heyshiny package
  titlePanel("Hey Shiny!"),
  speechInput(inputId = "hey_cmd", command = "hey *msg"), # set the input
  verbatimTextOutput("shiny_response")
)

server <- function(input, output, session) {
   # read the speech input
  observeEvent(input$hey_cmd, {
    speech <- input$hey_cmd
    message(speech)
    res <- "Sorry, I don't know how to help with that yet"
    if (grepl("^random number", tolower(speech))) {
      res <- paste0("Here is your random number: ", round(runif(1, 0, 8818)))
    } else if (grepl("^repeat", tolower(speech))) {
      res <- sub("repeat ", "", speech)
    }
    output$shiny_response <- renderText(res)
  })
}

shinyApp(ui, server)