Companies using Scrapy

Here is a list of companies and web sites using Scrapy, in no particular order

  • Scrapinghub provides a platform for running Scrapy spiders in the cloud (still in closed beta)
  • Mydeco uses Scrapy to scrape more than 4000 furniture retailer sites daily
  • Insophia uses Scrapy for several scraping projects on-demand
  • Médialab Sciences Po in Paris is using Scrapy to develop a web mining tool for Social Sciences researchers (announcement here)
  • ZaoJiao100 - Daily for infants and young children crawling early education information
  • World Government Data - UK government data aggregation site (tweet)
  • Catalystic - used to develop bespoke scraping projects
  • NextWeekVacations uses Scrapy to collect the data of their vacation rentals
  • Oposicionesaldia - uses Scrapy to collect data from jobs postings, scholarships and online free courses in Spain.
  • Iberestudios - uses Scrapy to collect data from masters degrees, doctorates and postgradute degrees in Spain.
  • BiteHunter - real time aggregator and search engine for dining
  • Dig It Deals - to scrape deals from many sites
  • Tryolabs - for its Omega Spider product, and for consulting projects that need web crawling
  • MarketMate RS - to scrape job offers from many sites
  • Pricewiki.com - using it to scrape various websites for cost of living information
  • Weotta - used to crawl data for post-processing (tweet)
  • Dealshelve uses Scrapy to scrape daily deals from many sites
  • Zinmoo uses Scrapy to scrape thousands of real state properties a per day (blog post)
  • CareerBuilder.com uses Scrapy to scrape job offers from certain sites
  • GrabLab is Russian company which specializes in web scraping, data collection and web automation tasks.
  • SimpleSpot uses Scrapy to build their geolocalized information service
  • Monetate uses Scrapy daily to collect catalog information from their clients
  • ClanSlots uses Scrapy daily to collect levels and plugins for games they host
  • Alışveriş Robotu is a Turkish price comparison site that uses Scrapy to collect data from hundreds of retailers everyday
  • Thalasar Ventures uses Scrapy to build bespoke crawlers

If you use Scrapy, we'd like to include you on the list. Send us an email to info@scrapinghub.com.