Companies that are using Scrapy

Check who is using Scrapy to do business and make an impact on the world.
(Should you be on this list? Fork this Github repo, add yourself, and send a pull request!)

Scrapinghub: From the creators of Scrapy, Scrapinghub is a leading technology and professional services company, providing successful web crawling and data processing solutions.

Parsely: Uses Scrapy to scrape articles from hundreds of news sites. Its CTO talks about Scrapy in this talk.

DirectEmployers Foundation: Uses Scrapy to scrape job postings from many websites, which are published on the site.

Weotta: Uses Scrapy to crawl data for post-processing (tweet).

Flax: Is a search consulting company based in Cambridge (UK) that uses Scrapy to power the crawling needs of their solutions (blog post).

Médialab Sciences Po: In Paris is using Scrapy to develop a web mining tool for Social Sciences researchers (announcement here).

Lyst: Uses Scrapy to crawl and scrape the fashion websites they index.

ScraperWiki: Is a data services company based in Liverpool providing bespoke solutions for data scraping and aggregation using Scrapy as a core technology (blog post).

Data.Gov.Uk: UK government data aggregation site (tweet).

Oposicionesaldia: Uses Scrapy to collect data from jobs postings, scholarships and online free courses in Spain.

Iberestudios: Uses Scrapy to collect data from masters degrees, doctorates and postgradute degrees in Spain.

DayWatch: Is an Internet Market Intelligence tool that uses Scrapy to empower real-time business information retrieval from Daily Deal sites.

PriceWiki: Uses Scrapy to scrape various websites for cost of living information.

Dealshelve: Uses Scrapy to scrape daily deals from many sites.

CareerBuilder: Uses Scrapy to scrape job offers from many sites.

GrabLab: Is a Russian company which specializes in web scraping, data collection and web automation tasks.

SimpleSpot: Uses Scrapy to build their geolocalized information service.

Monetate: Uses Scrapy daily to collect catalog information from their clients.

ClanSlots: Uses Scrapy daily to collect levels and plugins for games they host.

Alışveriş Robotu: Is a Turkish price comparison site that uses Scrapy to collect data from hundreds of retailers everyday.

Tuvalabs: Uses Scrapy to scrape the web and find the most interesting articles of significant news stories taking place around the world and transforming them into interactive math learning units.

Alistek: Uses Scrapy for updating partner related information in their OpenERP based back-office system, by scraping various data sources, both on the web and off-line.

Zhitongba: Is a company trying to help people better commute within big cities in China. They use Scrapy to scrape ride-sharing information from multiple sources.

Offertazo: Uses Scrapy to scrape offers from many Spanish websites.

Lionseek: Is a search engine that uses Scrapy to find items for sale in forums.

Stilivo: Is a discovery shopping site that uses Scrapy to collect product information from e-commerce sites.

Mapado: Uses Scrapy to find local activities on the web.

Oony: Is a deal aggregator in more than 16 countries. They currently have more than 500 Scrapy spiders running to gather their information.

Woppu: Is using Scrapy to collect product information from online shopping malls in Malaysia and Singapore.

Jobuzu: Uses Scrapy to scrape over 100,000 jobs daily from UK job boards.

Zopper: Uses Scrapy to crawl hundreds of ecommerce portals for classifying the products sold online.

WP Rocket: Uses Scrapy to preload the cache of all customer sites.

Competera: Is a price intelligence service that uses Scrapy to collect price, availability and promo data from over the million product pages every day.

Tarlabs: Uses Scrapy for information and text processing and automated testing.

Jobijoba: Uses Scrapy to scrape job offers daily from many job boards. Operate in France, several European countries, Russia, Mexico and Australia.

Data Quarry: Designs data scrapers specifically to address the needs of e-commerce users and offers custom scrapy devlopment.

El Útero de Marita: Peruvian leading news blog uses Scrapy to download public documents from governmental institutions in Peru for data journalism.

Shimply: Building world's largest online marketplace connecting sellers and buyers. Parsing over 500 large and small sites daily. (tweet).

Allclasses: Uses Scrapy to collect over 100,000 online and local classes from recreational through advanced education sites - providing more accessibility to education.

MonkeyLearn: Is a cloud platform that allows any company to extract relevant data from unstructured text using machine learning. It uses Scrapy to get data to train its algorithms. GmbH Logo GmbH: Uses Scrapy to crawl client websites, allowing to identify possibe optimization measures with the aim to make websites faster, more accessible, and user-friendly.

LavoWeb SAS Logo

LavoWeb: Uses Scrapy to crawl e-commerce websites, make SEO audit and Magento migration.

SayOne Logo

SayOne: uses Scrapy to crawl data for their clients and thereby develop more customer-centric applications for them. Uses Scrapy to crawl and collect data on fashion products, clothes and accessories. Is a vertical search engine for cars, real estate, jobs and products classified ads, in more than 30 countries. Uses Scrapy to crawl and collect ads on partner classified websites.

Up Hail: Does for taxi services what so many websites have done for other travel costs — compares them in real time to find the best deal. Uses Scrapy to crawl and scrape taxi and transportation sites to gather rates, coverage zones, and deals.

Allogarage: Allogarage is the leading company in France to compare auto mechanics. Uses Scrapy to crawl and scrape car manufacturer sites to gather name, address, phone and timetables for car dealers. (tweet)

Want to be part of this list?

Fork this Github repo, add yourself, and send a pull request!