yusuzech/r-web-scraping-cheat-sheet

Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.

r-web-scraping-cheat-sheet's Issues

Some notes on using rsDriver

It seems that most folks out there are interfacing rSelenium with a third-party program called "Docker" and then using the remoteDriver() function rather than the rsDriver() function. rsDriver therefore has less documentation and support. I ran into some problems getting started with the rsDriver method that I eventually resolved, so I thought I'd share my fixes here.

1. I had to install the wdman library. This is a version control package that installs all the latest rSelenium dependencies.
2. The rsDriver() function defaults to using the latest version of Google Chrome, which is a beta version. Simply installing the beta version of Chrome on my computer unfortunately didn't get the job done, because the function uses the file path for the stable version, not for the beta version. I had to specify my Chrome version using the chromever argument.
3. When troubleshooting rsDriver(), it sometimes executed only halfway. It opened the server port, but it didn't fully initialize the server. If I tried to run the function again, I would get an error message saying the port was already in use. This can be resolved by restarting RStudio.
4. When I attempted to use driver <- rD$client, I got an error message saying I was trying to run a non-function. I resolved this by using driver <- rD[["client"]] instead. To stop the server, use rD[["server"]]$stop() instead of rD$server$stop().