Fork me on GitHub

Companies using Scrapy

Parsely uses Scrapy to scrape articles from a hundred of news sites. His CTO talks about Scrapy in this talk
Direct Employers Foundation uses Scrapy to scrape job postings from many websites, which are published on the My.jobs site.
Weotta uses Scrapy to crawl data for post-processing (tweet)
Mydeco uses Scrapy to scrape more than 4000 furniture retailer sites daily
Médialab Sciences Po in Paris is using Scrapy to develop a web mining tool for Social Sciences researchers (announcement here)
BiteHunter uses Scrapy to crawl deal sites and restaurant directory sites, to provide real time aggregator and search engine for dining
Inner Balloons uses Scrapy to scrape data for products including local search content generation, reputation management, web presence management and marketing analytics.
Flax is a search consulting company based in Cambridge (UK) that uses Scrapy to power the crawling needs of their solutions (blog post).
JetRank uses Scrapy to crawl websites for detecting SEO issues
Lyst uses Scrapy to crawl and scrape the fashion websites they index
ScraperWiki is a data services company based in Liverpool providing bespoke solutions for data scraping and aggregation using Scrapy as a core technology (blog post).
World Government Data - UK government data aggregation site (tweet)
Oposicionesaldia uses Scrapy to collect data from jobs postings, scholarships and online free courses in Spain.
Iberestudios uses Scrapy to collect data from masters degrees, doctorates and postgradute degrees in Spain.
DayWatch is an Internet Market Intelligence tool that uses Scrapy to empower real-time business information retrieval from Daily Deal sites.
MarketMate RS uses Scrapy to scrape job offers from many sites
Pricewiki.com uses Scrapy to scrape various websites for cost of living information
Dealshelve uses Scrapy to scrape daily deals from many sites
Zinmoo uses Scrapy to scrape thousands of real state properties a per day (blog post)
CareerBuilder.com uses Scrapy to scrape job offers from many sites
GrabLab is a Russian company which specializes in web scraping, data collection and web automation tasks.
SimpleSpot uses Scrapy to build their geolocalized information service
Monetate uses Scrapy daily to collect catalog information from their clients
ClanSlots uses Scrapy daily to collect levels and plugins for games they host
Alışveriş Robotu is a Turkish price comparison site that uses Scrapy to collect data from hundreds of retailers everyday
ToTheGo is an aggregator of ads related to homes, jobs and cars trading. They use Scrapy to scrape everyday announces from the biggest portals and organize them inside their database.
JobMistral is an aggregator of jobs classified ads. The use Scrapy to scrape job ads hourly from many sites
TuvaLabs uses Scrapy to scrape RSS feeds of major newspapers from all over the world. They scrape the web to find the most interesting articles of significant news stories taking place around the world and transforming them into interactive math learning units.
Alistek uses Scrapy for updating partner related information in their OpenERP based back-office system, by scraping various data sources, both on the web and off-line.
Zhitongba is a company trying to help people better commute within big cities in China. They use Scrapy to scrape ride-sharing information from multiple sources.
Offertazo uses Scrapy to scrape offers from many Spanish websites.
LionSeek is a search engine that uses Scrapy to find items for sale in forums.
Stilivo is a discovery shopping site that uses Scrapy to collect product information from e-commerce sites.
Mapado uses Scrapy to find local activities on the web
Oony is a deal aggregator in more than 16 countries. They currently have more than 500 Scrapy spiders running to gather their information.
Woppu is using Scrapy to collect product information from online shopping malls in Malaysia and Singapore.
Jobuzu uses Scrapy to scrape over 100,000 jobs daily from UK job boards
Reviews42 uses Scrapy to crawl hundreds of ecommerce portals for classifying the products sold online
BookRadar (from CoEnterprise) uses Scrapy to scrape information form many book retailers
Pazar is a price comparison website that uses Scrapy daily to collect data from many websites
WP Rocket uses Scrapy to preload the cache of all customer sites
Cavucador - Ache seu Imóvel uses Scrapy to collect real estate data from many different websites in Brazil. They scrape more than 100,000 real estates daily and generate a search engine and statistics database
Competera is a price intelligence service that uses Scrapy to collect price, availability and promo data from over the million product pages every day
Tarlabs - Automation & Research uses Scrapy for information and text processing and automated testing
Jobijoba uses Scrapy to scrape job offers daily from many job boards. Jobijoba is a job search engine with a semantic twist that understands searches and matches them to the most relevant jobs. They operate in France, several European countries as well as in Russia, Mexico and Australia.
Data Quarry Data Quarry designs data scrapers specifically to address the needs of e-commerce users and offers custom scrapy devlopment
El Útero de Marita Peruvian leading news blog uses Scrapy to download public documents from governmental institutions in Peru for data journalism.
Shimply Building world's largest online marketplace connecting sellers and buyers. Parsing over 500 large and small sites daily.(tweet)

If you use Scrapy, we'd like to include you on the list. Email info@scrapinghub.com and request to be included, or just fork this GitHub repo, add yourself, and send a pull request.