Fork on Github

Companies that are using Scrapy

Check who is using Scrapy to do business and make an impact on the world. Should you be on this list?

Scrapinghub: From the creators of Scrapy, Scrapinghub is a leading technology and professional services company, providing successful web crawling and data processing solutions.

Parsely: Uses Scrapy to scrape articles from a hundred of news sites. His CTO talks about Scrapy in this talk.

DirectEmployers Foundation: Uses Scrapy to scrape job postings from many websites, which are published on the site.

Weotta: Uses Scrapy to crawl data for post-processing (tweet).

BiteHunter: Uses Scrapy to crawl deal sites and restaurant directory sites, to provide real time aggregator and search engine for dining.

InnerBalloons: Uses Scrapy to scrape data for products including local search content generation, reputation management, web presence management and marketing analytics.

Flax: Is a search consulting company based in Cambridge (UK) that uses Scrapy to power the crawling needs of their solutions (blog post).

Médialab Sciences Po: In Paris is using Scrapy to develop a web mining tool for Social Sciences researchers (announcement here).

Lyst: Uses Scrapy to crawl and scrape the fashion websites they index.

ScraperWiki: Is a data services company based in Liverpool providing bespoke solutions for data scraping and aggregation using Scrapy as a core technology (blog post).

Data.Gov.Uk: UK government data aggregation site (tweet).

Oposicionesaldia: Uses Scrapy to collect data from jobs postings, scholarships and online free courses in Spain.

Iberestudios: Uses Scrapy to collect data from masters degrees, doctorates and postgradute degrees in Spain.

DayWatch: Is an Internet Market Intelligence tool that uses Scrapy to empower real-time business information retrieval from Daily Deal sites.

PriceWiki: Uses Scrapy to scrape various websites for cost of living information.

Dealshelve: Uses Scrapy to scrape daily deals from many sites.

CareerBuilder: Uses Scrapy to scrape job offers from many sites.

GrabLab: Is a Russian company which specializes in web scraping, data collection and web automation tasks.

SimpleSpot: Uses Scrapy to build their geolocalized information service.

Monetate: Uses Scrapy daily to collect catalog information from their clients.

ClanSlots: Uses Scrapy daily to collect levels and plugins for games they host.

Alışveriş Robotu: Is a Turkish price comparison site that uses Scrapy to collect data from hundreds of retailers everyday.

Tuvalabs: Uses Scrapy to scrape the web and find the most interesting articles of significant news stories taking place around the world and transforming them into interactive math learning units.

Alistek: Uses Scrapy for updating partner related information in their OpenERP based back-office system, by scraping various data sources, both on the web and off-line.

Zhitongba: Is a company trying to help people better commute within big cities in China. They use Scrapy to scrape ride-sharing information from multiple sources.

Offertazo: Uses Scrapy to scrape offers from many Spanish websites.

Lionseek: Is a search engine that uses Scrapy to find items for sale in forums.

Stilivo: Is a discovery shopping site that uses Scrapy to collect product information from e-commerce sites.

Mapado: Uses Scrapy to find local activities on the web.

Oony: Is a deal aggregator in more than 16 countries. They currently have more than 500 Scrapy spiders running to gather their information.

Woppu: Is using Scrapy to collect product information from online shopping malls in Malaysia and Singapore.

Jobuzu: Uses Scrapy to scrape over 100,000 jobs daily from UK job boards.

Zopper: Uses Scrapy to crawl hundreds of ecommerce portals for classifying the products sold online.

BookRadar: Uses Scrapy to scrape information form many book retailers.

WP Rocket: Uses Scrapy to preload the cache of all customer sites.

Cavucador: Uses Scrapy to collect real estate data from many different websites in Brazil. They scrape more than 100,000 real estates daily and generate a search engine and statistics database.

Competera: Is a price intelligence service that uses Scrapy to collect price, availability and promo data from over the million product pages every day.

Tarlabs: Uses Scrapy for information and text processing and automated testing.

Jobijoba: Uses Scrapy to scrape job offers daily from many job boards. Operate in France, several European countries,Russia, Mexico and Australia.

Data Quarry: Designs data scrapers specifically to address the needs of e-commerce users and offers custom scrapy devlopment.

El Útero de Marita: Peruvian leading news blog uses Scrapy to download public documents from governmental institutions in Peru for data journalism.

Shimply: Building world's largest online marketplace connecting sellers and buyers. Parsing over 500 large and small sites daily. (tweet).

Allclasses: Uses Scrapy to collect over 100,000 online and local classes from recreational through advanced education sites - providing more accessibility to education.

MonkeyLearn: Is a cloud platform that allows any company to extract relevant data from unstructured text using machine learning. It uses Scrapy to get data to train its algorithms. GmbH Logo GmbH: Uses Scrapy to crawl client websites, allowing to identify possibe optimization measures with the aim to make websites faster, more accessible, and user-friendly.

LavoWeb SAS Logo

LavoWeb: Uses Scrapy to crawl e-commerce websites, make SEO audit and Magento migration.

SayOne Logo

SayOne: uses Scrapy to crawl data for their clients and thereby develop more customer-centric applications for them.

Want to be part of this list?


Fork this Github repo, add yourself, and send a pull request!