Fork on Github

Companies that are using Scrapy

Check who is using Scrapy to do business and make an impact on the world. Should you be on this list?


Parsely: Uses Scrapy to scrape articles from a hundred of news sites. His CTO talks about Scrapy in this talk.


DirectEmployers Foundation: Uses Scrapy to scrape job postings from many websites, which are published on the My.jobs site.


Weotta: Uses Scrapy to crawl data for post-processing (tweet).


BiteHunter: Uses Scrapy to crawl deal sites and restaurant directory sites, to provide real time aggregator and search engine for dining.


InnerBalloons: Uses Scrapy to scrape data for products including local search content generation, reputation management, web presence management and marketing analytics.


Flax: Is a search consulting company based in Cambridge (UK) that uses Scrapy to power the crawling needs of their solutions (blog post).


Médialab Sciences Po: In Paris is using Scrapy to develop a web mining tool for Social Sciences researchers (announcement here).


Lyst: Uses Scrapy to crawl and scrape the fashion websites they index.


ScraperWiki: Is a data services company based in Liverpool providing bespoke solutions for data scraping and aggregation using Scrapy as a core technology (blog post).


Data.Gov.Uk: UK government data aggregation site (tweet).


Oposicionesaldia: Uses Scrapy to collect data from jobs postings, scholarships and online free courses in Spain.


Iberestudios: Uses Scrapy to collect data from masters degrees, doctorates and postgradute degrees in Spain.


DayWatch: Is an Internet Market Intelligence tool that uses Scrapy to empower real-time business information retrieval from Daily Deal sites.


PriceWiki: Uses Scrapy to scrape various websites for cost of living information.


Dealshelve: Uses Scrapy to scrape daily deals from many sites.


CareerBuilder: Uses Scrapy to scrape job offers from many sites.


GrabLab: Is a Russian company which specializes in web scraping, data collection and web automation tasks.


SimpleSpot: Uses Scrapy to build their geolocalized information service.


Monetate: Uses Scrapy daily to collect catalog information from their clients.


ClanSlots: Uses Scrapy daily to collect levels and plugins for games they host.


Alışveriş Robotu: Is a Turkish price comparison site that uses Scrapy to collect data from hundreds of retailers everyday.


Tuvalabs: Uses Scrapy to scrape the web and find the most interesting articles of significant news stories taking place around the world and transforming them into interactive math learning units.


Alistek: Uses Scrapy for updating partner related information in their OpenERP based back-office system, by scraping various data sources, both on the web and off-line.


Zhitongba: Is a company trying to help people better commute within big cities in China. They use Scrapy to scrape ride-sharing information from multiple sources.


Offertazo: Uses Scrapy to scrape offers from many Spanish websites.


Lionseek: Is a search engine that uses Scrapy to find items for sale in forums.


Stilivo: Is a discovery shopping site that uses Scrapy to collect product information from e-commerce sites.


Mapado: Uses Scrapy to find local activities on the web.


Oony: Is a deal aggregator in more than 16 countries. They currently have more than 500 Scrapy spiders running to gather their information.


Woppu: Is using Scrapy to collect product information from online shopping malls in Malaysia and Singapore.


Jobuzu: Uses Scrapy to scrape over 100,000 jobs daily from UK job boards.


Zopper: Uses Scrapy to crawl hundreds of ecommerce portals for classifying the products sold online.


BookRadar: Uses Scrapy to scrape information form many book retailers.


WP Rocket: Uses Scrapy to preload the cache of all customer sites.


Cavucador: Uses Scrapy to collect real estate data from many different websites in Brazil. They scrape more than 100,000 real estates daily and generate a search engine and statistics database.


Competera: Is a price intelligence service that uses Scrapy to collect price, availability and promo data from over the million product pages every day.


Tarlabs: Uses Scrapy for information and text processing and automated testing.


Jobijoba: Uses Scrapy to scrape job offers daily from many job boards. Operate in France, several European countries,Russia, Mexico and Australia.


Data Quarry: Designs data scrapers specifically to address the needs of e-commerce users and offers custom scrapy devlopment.


El Útero de Marita: Peruvian leading news blog uses Scrapy to download public documents from governmental institutions in Peru for data journalism.


Shimply: Building world's largest online marketplace connecting sellers and buyers. Parsing over 500 large and small sites daily. (tweet).


Allclasses: Uses Scrapy to collect over 100,000 online and local classes from recreational through advanced education sites - providing more accessibility to education.

Want to be part of this list?

or

Fork this Github repo, add yourself, and then send a pull request.