rvest
Scraping javascript website in R
So, RSelenium is not the only answer (anymore). If you can install the PhantomJS binary (grab phantomjs binaries from here: http://phantomjs.org/) then you can use it to render the HTML and scrape it with rvest (similar to the RSelenium approach but doesn’t require java): library(rvest) # render HTML from the site with phantomjs url <- … Read more
Scraping a dynamic ecommerce page with infinite scroll
As @nrussell suggested, you can use RSelenium to programatically scroll down the page before getting the source code. You could for example do: library(RSelenium) library(rvest) #start RSelenium checkForServer() startServer() remDr <- remoteDriver() remDr$open() #navigate to your page remDr$navigate(“http://www.linio.com.co/tecnologia/celulares-telefonia-gps/”) #scroll down 5 times, waiting for the page to load at each time for(i in 1:5){ remDr$executeScript(paste(“scroll(0,”,i*10000,”);”)) … Read more