↧
Answer by vezunchik for Web Crawler not following the links
The main error is that you have css function and xpath selector for next_page: next_page = response.css("//a[@class='btn-next btn']/@href").get()The next problem is that you have yielding request of...
View Article
Image may be NSFW.
Clik here to view.
Clik here to view.
Web Crawler not following the links
I want to crawl news site using Scrapy. The code retrieved related news from current link but not following the next page links. The news site has following link propertyThe code I am following :...
View Article