Meet the Scrapy pros

The following companies offer support contracts and consultancy services for Scrapy, and can also develop bespoke crawlers to meet your needs:

Zyte was founded by the creators of Scrapy and currently is the largest company sponsoring Scrapy development. We provide market-leading web data extraction products and services, Zyte API, Zyte Data and Scrapy Cloud. Zyte API is our flagship solution that automatically handles bans, extracts data, renders pages, automates browser actions, generates screenshots, manages sessions and cookies, reports on and analyzes usage, and provides transparent, per-website pricing.

ScrapeOps is a DevOps tool for web scraping, which provides a suite of monitoring, error tracking, alerting, scheduling and deployment tools for your Scrapy projects when you install the scrapeops-scrapy extension. ScrapeOps also publishes Scrapy guides & tutorials at The Scrapy Playbook.

Arbisoft scours massive websites several layers deep to collect valuable data powering leading firms around the world. It offers realtime crawling and custom-built fully-managed spiders. Over 6 years of quality service, their Python engineers have come to trust Scrapy as their tool of choice.

Datahut provides Scrapy consulting services across different business verticals like e-commerce, content discovery, lead generation, opinion mining etc. They provide clean ready to use data in the most common formats.

SayOne uses Scrapy to power their web crawling and visualization services. They have a strong team of crawling experts who specialize in crawling, information extraction and application integration. They also offer web & mobile app development. Read more about their data scraping services here.

Lambert Labs specialises in using Scrapy to aggregate structured and unstructured text, images and videos from across the web. It combines Scrapy with Selenium to crawl JavaScript-rendered websites in real time. Lambert Labs also offers web development, automated testing, data science and devops.

Tryolabs is a Boutique dev Shop specialized in building Python apps with Machine Learning components. They embed Scrapy into their customers applications as well as into their own products.

Intoli uses Scrapy to provide customized web scraping solutions, delivering data that is used by clients to power their core products, for lead generation, and for competitor research. They specialize in advanced services such as cross-site data aggregation, user logins, and bypassing captchas.

Companies using Scrapy

Check who is using Scrapy to do business and make an impact on the world.
(Should you be on this list? Fork this GitHub repo, add yourself, and send a pull request!)


Zyte, founded by the creators of Scrapy, offers leading web data extraction products and services. Their flagship solution, Zyte API, provides automated handling of bans, data extraction, page rendering, browser action automation, screenshot generation, session and cookie management, along with usage reporting and analysis. It also features transparent, per-website pricing.


Parsely: Uses Scrapy to scrape articles from hundreds of news sites. Its CTO talks about Scrapy in this talk.


DirectEmployers Foundation: Uses Scrapy to scrape job postings from many websites, which are published on the My.jobs site.


Flax: Is a search consulting company based in Cambridge (UK) that uses Scrapy to power the crawling needs of their solutions (blog post).


Médialab Sciences Po: In Paris is using Scrapy to develop a web mining tool for Social Sciences researchers (announcement here).


Lyst: Uses Scrapy to crawl and scrape the fashion websites they index.


ScraperWiki: Is a data services company based in Liverpool providing bespoke solutions for data scraping and aggregation using Scrapy as a core technology (blog post).


Data.Gov.Uk: UK government data aggregation site (tweet).


Oposicionesaldia: Uses Scrapy to collect data from jobs postings, scholarships and online free courses in Spain.


Iberestudios: Uses Scrapy to collect data from masters degrees, doctorates and postgradute degrees in Spain.


Dealshelve: Uses Scrapy to scrape daily deals from many sites.


CareerBuilder: Uses Scrapy to scrape job offers from many sites.


GrabLab: Is a Russian company which specializes in web scraping, data collection and web automation tasks.


SimpleSpot: Uses Scrapy to build their geolocalized information service.


Monetate: Uses Scrapy daily to collect catalog information from their clients.


the urge is a Fashion Search Engine focussed on using Artificial Intelligence to help shoppers find the fashion they’re looking for. They use Scrapy at scale to crawl retailers websites for Fashion products.


Alistek: Uses Scrapy for updating partner related information in their OpenERP based back-office system, by scraping various data sources, both on the web and off-line.


Zhitongba: Is a company trying to help people better commute within big cities in China. They use Scrapy to scrape ride-sharing information from multiple sources.


Offertazo: Uses Scrapy to scrape offers from many Spanish websites.


Lionseek: Is a search engine that uses Scrapy to find items for sale in forums.


Stilivo: Is a discovery shopping site that uses Scrapy to collect product information from e-commerce sites.


Mapado: Uses Scrapy to find local activities on the web.


Zopper: Uses Scrapy to crawl hundreds of ecommerce portals for classifying the products sold online.


WP Rocket: Uses Scrapy to preload the cache of all customer sites.


Competera: Is a price intelligence service that uses Scrapy to collect price, availability and promo data from over the million product pages every day.


Jobijoba: Uses Scrapy to scrape job offers daily from many job boards. Operate in France, several European countries, Russia, Mexico and Australia.


Data Quarry: Designs data scrapers specifically to address the needs of e-commerce users and offers custom scrapy devlopment.


El Útero de Marita: Peruvian leading news blog uses Scrapy to download public documents from governmental institutions in Peru for data journalism.


Shimply: Building world’s largest online marketplace connecting sellers and buyers. Parsing over 500 large and small sites daily. (tweet).


Allclasses: Uses Scrapy to collect over 100,000 online and local classes from recreational through advanced education sites - providing more accessibility to education.


MonkeyLearn: Is a cloud platform that allows any company to extract relevant data from unstructured text using machine learning. It uses Scrapy to get data to train its algorithms.


neu.land GmbH: Uses Scrapy to crawl client websites, allowing to identify possibe optimization measures with the aim to make websites faster, more accessible, and user-friendly.


LavoWeb: Uses Scrapy to crawl e-commerce websites, make SEO audit and Magento migration.


SayOne: uses Scrapy to crawl data for their clients and thereby develop more customer-centric applications for them.


Videdressing.com: Uses Scrapy to crawl and collect data on fashion products, clothes and accessories.


Up Hail: Does for taxi services what so many websites have done for other travel costs — compares them in real time to find the best deal. Uses Scrapy to crawl and scrape taxi and transportation sites to gather rates, coverage zones, and deals (blog post).


Allogarage: Allogarage is the leading company in France to compare auto mechanics. Uses Scrapy to crawl and scrape car manufacturer sites to gather name, address, phone and timetables for car dealers. (tweet)


TheWebFollower: Uses Scrapy to crawl latest Malayalam language blogs, news, articles, stories published in different websites. (tweet)


Cocon.Se: A tool that shows your website like a crawler sees it. Uses Scrapy to prototype data gathering algorithms, generate specific datasets for machine learning and datavizualisation. (blog post)


WatchPatrol: Uses Scrapy for aggregating private sales listings for used and new watches from the web’s most popular watch enthusiast forums


PolitePol: Online tool for creation of RSS feeds for any web page. Uses scrapy for parsing and processing source pages. This site have open source version you can check out and use absolutely for free.


Retail Shake: is a full automated and daily business competitors monitoring solution.
We are using scrapy to offer a 360° monitoring : from the pricing, to the products comparison, prices benchmarks, promotional offers, geolocated stocks monitoring, …


Lapis Data Analytics: Our Team have great experience in developing custom spiders and building data pipelines. Contact us for professional data crawling solutions.


Simplicity: Uses Scrapy to crawl and scrape data from city web sites. This data is also partly processed using Scrapy and is then displayed to citizens in the Simplicity App both on Android and IOS.


Public.Law: Uses Scrapy to create a legal dictionary and other law apps from official sources. Our Scrapy code and scraped data are open source.


Qwurty: Uses Scrapy to crawl the web for links and incorporate link metadata into the social network for users to view, share, and comment on.


GroupBWT is a company that specializes in the development of data scraping solutions, including digital shelf monitoring, data aggregators, warehouses, and price comparison systems. Uses Scrapy in tasks and collecting essential data from the web.


Improwised: uses Scrapy to help small businesses to consolidate their reviews from different listing platforms.

Want to be part of this list?

Fork this GitHub repo, add yourself, and send a pull request!

Be a part of the Community

Join our channels and collaborate!