python - scrapy 再次請(qǐng)求的問(wèn)題
問(wèn)題描述
如:item[’url’]=response.xpath(’a/@href’)分析出一個(gè)鏈接,然后想從這個(gè)鏈接里的網(wǎng)頁(yè)再獲取一些元素,放入item[’other’]。應(yīng)該怎么寫(xiě),謝謝。
問(wèn)題解答
回答1:def parse_page1(self, response): for url in urls:item = MyItem()item[’url’] = urlrequest = scrapy.Request(url,callback=self.parse_page2)# request = scrapy.Request('http://www.example.com/some_page.html',dont_filter=True,callback=self.parse_page2)request.meta[’item’] = itemyield requestdef parse_page2(self, response): item = response.meta[’item’] item[’other’] = response.xpath(’/other’) yield item
最后附上官方文檔https://doc.scrapy.org/en/lat...中文翻譯版http://scrapy-chs.readthedocs...
相關(guān)文章:
