📚 Article Index by Tag
🏷️ AI
| Title | Date | Link |
|---|---|---|
| THE LAB #86: Querying Web Data using GPT-Like Web Interface | 2025-06-05 | THE LAB #86: Querying Web Data using GPT-Like Web Interface |
| Scrape like a pro... but not like an AI company | 2025-05-20 | Scrape like a pro... but not like an AI company |
| AI and data: different faces of the same coin | 2025-05-20 | AI and data: different faces of the same coin |
| How AI is changing the web scraping industry | 2025-05-20 | How AI is changing the web scraping industry |
| The AI-Powered web scraping tools landscape | 2025-05-20 | The AI-Powered web scraping tools landscape |
| Building a custom GPT using Firecrawl | 2025-05-20 | Building a custom GPT using Firecrawl |
| About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra | 2025-05-20 | About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra |
| Building a generic scraper for multiple websites | 2025-05-20 | Building a generic scraper for multiple websites |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | Use Cursor as web scraping assistant with MCP servers |
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | Build your web scraping assistant with Claude and Cursor |
| Are LLMs capable of replacing traditional scrapers? | 2025-05-20 | Are LLMs capable of replacing traditional scrapers? |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base |
| Build a RAG Application with ScraperAPI, Gemini, and FAISS | 2025-04-02 | Build a RAG Application with ScraperAPI, Gemini, and FAISS |
| Rethinking the web browser - by Katie Hallett | 2025-01-21 | Rethinking the web browser - by Katie Hallett |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | Is Web Scraping Dead? - by Pierluigi Vinciguerra |
| Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra | 2023-10-13 | Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | Web Scraping experts: Is AI stealing our job? |
| How to create a web scraper with ChatGPT | 2023-10-13 | How to create a web scraper with ChatGPT |
| The state of web scraping and AI - by Pierluigi Vinciguerra | 2023-10-13 | The state of web scraping and AI - by Pierluigi Vinciguerra |
🏷️ API
| Title | Date | Link |
|---|---|---|
| THE LAB #26: From internal API to insights. | 2024-10-31 | THE LAB #26: From internal API to insights. |
🏷️ AWS
| Title | Date | Link |
|---|---|---|
| THE LAB #74: Running scrapers on GitHub Actions | 2025-05-20 | THE LAB #74: Running scrapers on GitHub Actions |
| The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra | 2025-05-20 | The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra |
| The Lab #48: Scraping with AWS Lambda | 2024-10-18 | The Lab #48: Scraping with AWS Lambda |
🏷️ Airbnb
| Title | Date | Link |
|---|---|---|
| THE LAB #66: How to properly scrape a booking website | 2025-05-20 | THE LAB #66: How to properly scrape a booking website |
| The Lab #5 - Scraping Airbnb.com using GraphQL | 2023-05-29 | The Lab #5 - Scraping Airbnb.com using GraphQL |
🏷️ Airflow
| Title | Date | Link |
|---|---|---|
| Scheduling Scrapers with Airflow - by Pierluigi Vinciguerra | 2025-05-20 | Scheduling Scrapers with Airflow - by Pierluigi Vinciguerra |
🏷️ Akamai
| Title | Date | Link |
|---|---|---|
| THE LAB #30: How to bypass Akamai protected website when nothing else works | 2025-06-09 | THE LAB #30: How to bypass Akamai protected website when nothing else works |
| THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies | 2025-05-29 | THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies |
| Scraping Akamai-protected websites with Scrapy | 2025-05-20 | Scraping Akamai-protected websites with Scrapy |
| Scraping Cloudflare websites using an API | 2025-05-20 | Scraping Cloudflare websites using an API |
| Scraping Akamai protected websites | 2024-09-08 | Scraping Akamai protected websites |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | THE LAB 32: hRequests vs anti-bots: a full benchmark |
| hRequests: bypass Akamai with Python requests | 2023-11-12 | hRequests: bypass Akamai with Python requests |
🏷️ AlexsandrasSulzenko
| Title | Date | Link |
|---|---|---|
| Interview #6: Aleksandras Šulženko - Oxylabs | 2023-10-13 | Interview #6: Aleksandras Šulženko - Oxylabs |
| Three web scraping tools just discovered on GitHub | 2023-10-08 | Three web scraping tools just discovered on GitHub |
🏷️ Algolia
| Title | Date | Link |
|---|---|---|
| The Lab #54: Scraping from Algolia APIs | 2025-05-20 | The Lab #54: Scraping from Algolia APIs |
| Algolia and web scraping: an introduction | 2023-12-10 | Algolia and web scraping: an introduction |
🏷️ AlternativeData
| Title | Date | Link |
|---|---|---|
| THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools | 2025-06-20 | THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools |
| Creating a dataset for investors with web scraping: Tesla (TSLA) | 2025-03-30 | Creating a dataset for investors with web scraping: Tesla (TSLA) |
| Web scraping and alternative data for financial markets | 2023-10-13 | Web scraping and alternative data for financial markets |
🏷️ Amazon
| Title | Date | Link |
|---|---|---|
| How to Scrape E-Commerce Websites With Python | 2024-08-02 | How to Scrape E-Commerce Websites With Python |
🏷️ AntiDetectBrowsers
| Title | Date | Link |
|---|---|---|
| The Anti-Detect Browser Royal Rumble - updated with notes | 2025-05-20 | The Anti-Detect Browser Royal Rumble - updated with notes |
| The Browser Automation Landscape in 2025 | 2025-05-20 | The Browser Automation Landscape in 2025 |
| The Lab #36: Bypassing Cloudflare with anti-detect browsers | 2025-04-16 | The Lab #36: Bypassing Cloudflare with anti-detect browsers |
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers |
| The Anti-Detect Browser Royal Rumble - Fingerprint tests | 2024-04-23 | The Anti-Detect Browser Royal Rumble - Fingerprint tests |
| How Can Multi-Accounting Browsers Help with Web Scraping? | 2024-04-16 | How Can Multi-Accounting Browsers Help with Web Scraping? |
| Behind the scenes of anti-detect browsers - by Tamas Deak | 2024-03-05 | Behind the scenes of anti-detect browsers - by Tamas Deak |
| The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 | 2024-01-19 | The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 |
| The rise of antidetect browsers - by Pierluigi Vinciguerra | 2023-10-13 | The rise of antidetect browsers - by Pierluigi Vinciguerra |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | How to by-pass Kasada bot mitigation? |
| From Traditional Browsers to AI-Powered Web Scraping | 2023-10-13 | From Traditional Browsers to AI-Powered Web Scraping |
🏷️ Antrophic
| Title | Date | Link |
|---|---|---|
| Scrape like a pro... but not like an AI company | 2025-05-20 | Scrape like a pro... but not like an AI company |
| AI and data: different faces of the same coin | 2025-05-20 | AI and data: different faces of the same coin |
🏷️ Apify
| Title | Date | Link |
|---|---|---|
| THE LAB #15: Deep diving into Apify world | 2023-10-13 | THE LAB #15: Deep diving into Apify world |
🏷️ Automotive
| Title | Date | Link |
|---|---|---|
| Web data and automotive industry - by Pierluigi Vinciguerra | 2025-05-20 | Web data and automotive industry - by Pierluigi Vinciguerra |
🏷️ AvivBesinky
| Title | Date | Link |
|---|---|---|
| Interview #7: Aviv Besinsky - Bright Data | 2023-10-13 | Interview #7: Aviv Besinsky - Bright Data |
🏷️ BearerToken
| Title | Date | Link |
|---|---|---|
| Scraping APIs with Bearer Token - by Pierluigi Vinciguerra | 2025-05-20 | Scraping APIs with Bearer Token - by Pierluigi Vinciguerra |
🏷️ Botasaurus
| Title | Date | Link |
|---|---|---|
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | THE LAB #73: How to Bypass Cloudflare in 2025 |
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | Testing the new Botasaurus 4 - by Pierluigi Vinciguerra |
| Open source Python libraries for your web scraping projects | 2025-05-20 | Open source Python libraries for your web scraping projects |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | Botasaurus: an anti-ban web scraping framework |
🏷️ BrightData
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | The Great Web Unblocker Benchmark - Cloudflare Edition |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| Bypassing Kasada for web scraping 2024 edition | 2024-09-30 | Bypassing Kasada for web scraping 2024 edition |
| The state of public web data in 2024 | 2024-05-05 | The state of public web data in 2024 |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | The Great Web Unblocker Benchmark: March 2024 |
| Testing the Bright Data Web Unblocker proxy | 2023-12-08 | Testing the Bright Data Web Unblocker proxy |
| Scraping Kasada protected websites | 2023-10-13 | Scraping Kasada protected websites |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | How to by-pass Kasada bot mitigation? |
🏷️ Browser
| Title | Date | Link |
|---|---|---|
| The Browser Automation Landscape in 2025 | 2025-05-20 | The Browser Automation Landscape in 2025 |
| Web Unblocker vs. Browser as a service for scraping | 2025-05-20 | Web Unblocker vs. Browser as a service for scraping |
| Rethinking the web browser - by Katie Hallett | 2025-01-21 | Rethinking the web browser - by Katie Hallett |
| THE LAB #20 - AI powered web scrapers with Nimble Browser | 2023-10-13 | THE LAB #20 - AI powered web scrapers with Nimble Browser |
🏷️ BrowserAPI
| Title | Date | Link |
|---|---|---|
| Google has exclusive access to a browser API | 2025-05-20 | Google has exclusive access to a browser API |
🏷️ BrowserFingerprint
| Title | Date | Link |
|---|---|---|
| Browser Fingerprinting 101 - What it is and how it works | 2025-05-20 | Browser Fingerprinting 101 - What it is and how it works |
| Making Playwright scrapers undetected with open source solutions | 2025-05-20 | Making Playwright scrapers undetected with open source solutions |
| The Lab #55: Checking your browser fingerprint | 2025-05-20 | The Lab #55: Checking your browser fingerprint |
| Google has exclusive access to a browser API | 2025-05-20 | Google has exclusive access to a browser API |
| The Lab #46: Fingerprint injection in Playwright | 2025-01-26 | The Lab #46: Fingerprint injection in Playwright |
| The latest papers in 2023 about browser fingerprinting | 2024-02-11 | The latest papers in 2023 about browser fingerprinting |
| THE LAB 33: Fingerprinting at different connection layers | 2023-11-30 | THE LAB 33: Fingerprinting at different connection layers |
| What is device fingerprinting? A deep dive | 2023-10-13 | What is device fingerprinting? A deep dive |
| Browser fingerprinting and web scraping | 2023-10-13 | Browser fingerprinting and web scraping |
| Browser API: an introduction - by Pierluigi Vinciguerra | 2023-10-13 | Browser API: an introduction - by Pierluigi Vinciguerra |
| Is web scraping becoming harder? - by Pierluigi Vinciguerra | 2023-10-13 | Is web scraping becoming harder? - by Pierluigi Vinciguerra |
| From Traditional Browsers to AI-Powered Web Scraping | 2023-10-13 | From Traditional Browsers to AI-Powered Web Scraping |
| THE LAB #19: How to mask the device fingerprint | 2023-09-11 | THE LAB #19: How to mask the device fingerprint |
🏷️ BrowserForge
| Title | Date | Link |
|---|---|---|
| Bypassing Cloudflare with open source repositories | 2024-09-18 | Bypassing Cloudflare with open source repositories |
🏷️ Business
| Title | Date | Link |
|---|---|---|
| Stuck? More of the Same Won’t Do - by Andrea Squatrito | 2025-05-20 | Stuck? More of the Same Won’t Do - by Andrea Squatrito |
| The importance of scraping inventory levels data in the retail industry | 2025-05-20 | The importance of scraping inventory levels data in the retail industry |
| Is web scraping a profitable industry? | 2025-05-20 | Is web scraping a profitable industry? |
| Three ways to make money with web scraping as a freelancer | 2025-05-20 | Three ways to make money with web scraping as a freelancer |
| THE LAB #31: Scraping location data using a world grid | 2025-01-16 | THE LAB #31: Scraping location data using a world grid |
| How We Scraped Global Hotel Data to Track Economic Trends | 2024-12-17 | How We Scraped Global Hotel Data to Track Economic Trends |
| How Scraping the Web Became an Expensive Business | 2024-12-10 | How Scraping the Web Became an Expensive Business |
| Scraping The Inflation - by Andrea Squatrito | 2024-12-03 | Scraping The Inflation - by Andrea Squatrito |
| THE LAB #26: From internal API to insights. | 2024-10-31 | THE LAB #26: From internal API to insights. |
| Web Scraping from 0 to hero: kickstart your career in web scraping | 2024-05-26 | Web Scraping from 0 to hero: kickstart your career in web scraping |
| 10 years of web scraping: a perspective about selling web data | 2024-03-24 | 10 years of web scraping: a perspective about selling web data |
| The Lab #43: Scraping inventory data: why, how and where | 2024-02-29 | The Lab #43: Scraping inventory data: why, how and where |
| How to monetize web scraping skills on Data Boutique? | 2024-02-08 | How to monetize web scraping skills on Data Boutique? |
| Monetize your web scraping skills: a brief guide | 2024-01-14 | Monetize your web scraping skills: a brief guide |
| From 0 to 2 Billion Prices scraped per months | 2023-10-13 | From 0 to 2 Billion Prices scraped per months |
| THE LAB #28: Deep dive on inventory levels tracking | 2023-09-28 | THE LAB #28: Deep dive on inventory levels tracking |
| THE LAB #27: Scraping stock level data to estimate revenues | 2023-09-13 | THE LAB #27: Scraping stock level data to estimate revenues |
🏷️ CAPTCHA
| Title | Date | Link |
|---|---|---|
| Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra | 2023-10-13 | Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra |
🏷️ CDP
| Title | Date | Link |
|---|---|---|
| The Lab #57: Improving your Playwright scraper and avoid CDP detection | 2025-05-20 | The Lab #57: Improving your Playwright scraper and avoid CDP detection |
🏷️ CSS
| Title | Date | Link |
|---|---|---|
| XPATH and CSS Selectors in Web Scraping | 2024-04-28 | XPATH and CSS Selectors in Web Scraping |
| XPath vs CSS selectors: a comparison | 2023-10-13 | XPath vs CSS selectors: a comparison |
🏷️ Camoufox
| Title | Date | Link |
|---|---|---|
| THE LAB #65: Scraping Datadome protected websites with Camoufox | 2025-05-20 | THE LAB #65: Scraping Datadome protected websites with Camoufox |
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 |
| THE LAB #83: Camoufox as a containerized server | 2025-05-19 | THE LAB #83: Camoufox as a containerized server |
🏷️ Castle
| Title | Date | Link |
|---|---|---|
| Scraping APIs with Bearer Token - by Pierluigi Vinciguerra | 2025-05-20 | Scraping APIs with Bearer Token - by Pierluigi Vinciguerra |
🏷️ ChangeDetection
| Title | Date | Link |
|---|---|---|
| Change detection for web scraping: tools and techniques | 2023-10-15 | Change detection for web scraping: tools and techniques |
🏷️ Changedetectionio
| Title | Date | Link |
|---|---|---|
| Change detection for web scraping: tools and techniques | 2023-10-15 | Change detection for web scraping: tools and techniques |
🏷️ ChatGPT
| Title | Date | Link |
|---|---|---|
| Scrape like a pro... but not like an AI company | 2025-05-20 | Scrape like a pro... but not like an AI company |
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared |
| No-Code Web Scraping with Make.com | 2025-05-20 | No-Code Web Scraping with Make.com |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | Web Scraping experts: Is AI stealing our job? |
| Writing a web scraper with ChatGPT. Is it a good idea? | 2023-10-13 | Writing a web scraper with ChatGPT. Is it a good idea? |
| How to create a web scraper with ChatGPT | 2023-10-13 | How to create a web scraper with ChatGPT |
🏷️ Ciphers
| Title | Date | Link |
|---|---|---|
| THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting | 2023-05-29 | THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting |
🏷️ Claude
| Title | Date | Link |
|---|---|---|
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | Build your web scraping assistant with Claude and Cursor |
🏷️ Cloudflare
| Title | Date | Link |
|---|---|---|
| THE LAB #3: Scraping Cloudflare protected websites | 2025-06-07 | THE LAB #3: Scraping Cloudflare protected websites |
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | The Great Web Unblocker Benchmark - Cloudflare Edition |
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | THE LAB #73: How to Bypass Cloudflare in 2025 |
| Scraping Cloudflare websites using an API | 2025-05-20 | Scraping Cloudflare websites using an API |
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | Testing the new Botasaurus 4 - by Pierluigi Vinciguerra |
| THE LAB #62: Bypassing Cloudflare with Nodriver | 2025-05-20 | THE LAB #62: Bypassing Cloudflare with Nodriver |
| The Lab #36: Bypassing Cloudflare with anti-detect browsers | 2025-04-16 | The Lab #36: Bypassing Cloudflare with anti-detect browsers |
| Bypassing Cloudflare with open source repositories | 2024-09-18 | Bypassing Cloudflare with open source repositories |
| The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 | 2024-01-19 | The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | THE LAB 32: hRequests vs anti-bots: a full benchmark |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | Can Undetected Chromedriver bypass Cloudflare or Datadome? |
| Cloudflare Turnstile: what is that and how it works? | 2023-10-13 | Cloudflare Turnstile: what is that and how it works? |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | THE LAB #21 - Bypass anti-bot challenges with AI |
| THE LAB #29: Bypass Cloudflare Bot Protection with Scrapy | 2023-10-12 | THE LAB #29: Bypass Cloudflare Bot Protection with Scrapy |
| Scraping Cloudflare Protected Websites (early 2023 version) | 2023-06-10 | Scraping Cloudflare Protected Websites (early 2023 version) |
| THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin | 2023-05-29 | THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin |
🏷️ Cloudscraper
| Title | Date | Link |
|---|---|---|
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | THE LAB #73: How to Bypass Cloudflare in 2025 |
🏷️ Codex
| Title | Date | Link |
|---|---|---|
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools |
🏷️ Consultancy
| Title | Date | Link |
|---|---|---|
| Stuck? More of the Same Won’t Do - by Andrea Squatrito | 2025-05-20 | Stuck? More of the Same Won’t Do - by Andrea Squatrito |
🏷️ Container
| Title | Date | Link |
|---|---|---|
| THE LAB #83: Camoufox as a containerized server | 2025-05-19 | THE LAB #83: Camoufox as a containerized server |
🏷️ Costs
| Title | Date | Link |
|---|---|---|
| The Unit Economics of Proxy Providers - by Abed | 2025-05-27 | The Unit Economics of Proxy Providers - by Abed |
| Analyzing the cost of a web scraping project | 2025-05-20 | Analyzing the cost of a web scraping project |
| THE LAB #61: Evaluating your proxy provider | 2025-05-20 | THE LAB #61: Evaluating your proxy provider |
| Optimizing Proxy Usage for Large-Scale Scraping | 2025-05-20 | Optimizing Proxy Usage for Large-Scale Scraping |
| Optimizing costs for large-scale scraping operations | 2025-05-20 | Optimizing costs for large-scale scraping operations |
| The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra | 2025-02-14 | The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra |
| How Scraping the Web Became an Expensive Business | 2024-12-10 | How Scraping the Web Became an Expensive Business |
| Scrapoxy, the super proxy aggregator, how it works? | 2024-02-21 | Scrapoxy, the super proxy aggregator, how it works? |
| How scraping a single website costed thousands of dollars in proxy | 2024-01-28 | How scraping a single website costed thousands of dollars in proxy |
| The true costs of a web scraping project | 2023-11-25 | The true costs of a web scraping project |
| The costs of web scraping - by Pierluigi Vinciguerra | 2023-10-13 | The costs of web scraping - by Pierluigi Vinciguerra |
🏷️ Crawlee
| Title | Date | Link |
|---|---|---|
| The most interesting GitHub Repositories about web scraping (2023) | 2023-10-13 | The most interesting GitHub Repositories about web scraping (2023) |
🏷️ Cursor
| Title | Date | Link |
|---|---|---|
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | Use Cursor as web scraping assistant with MCP servers |
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | Build your web scraping assistant with Claude and Cursor |
🏷️ DataQuality
| Title | Date | Link |
|---|---|---|
| THE LAB #69: Building a dashboard for your scrapers with Grafana | 2025-05-20 | THE LAB #69: Building a dashboard for your scrapers with Grafana |
| Web Scraping from 0 to hero: data cleaning processes | 2024-05-12 | Web Scraping from 0 to hero: data cleaning processes |
| Ensuring data quality in web scraping projects | 2023-10-13 | Ensuring data quality in web scraping projects |
🏷️ Datadoma
| Title | Date | Link |
|---|---|---|
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | Testing the new Botasaurus 4 - by Pierluigi Vinciguerra |
| Web Scraping Idealista and Bypass Idealista Blockers | 2024-08-06 | Web Scraping Idealista and Bypass Idealista Blockers |
🏷️ Datadome
| Title | Date | Link |
|---|---|---|
| THE LAB #65: Scraping Datadome protected websites with Camoufox | 2025-05-20 | THE LAB #65: Scraping Datadome protected websites with Camoufox |
| THE LAB #82: How to scrape Vinted using their internal APIs | 2025-05-20 | THE LAB #82: How to scrape Vinted using their internal APIs |
| THE LAB #2: scraping data from a website with Datadome and xsrf tokens | 2025-03-28 | THE LAB #2: scraping data from a website with Datadome and xsrf tokens |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | Botasaurus: an anti-ban web scraping framework |
| Bypassing Datadome with Web Scraping - End of 2023 Version | 2023-12-06 | Bypassing Datadome with Web Scraping - End of 2023 Version |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | THE LAB 32: hRequests vs anti-bots: a full benchmark |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | Can Undetected Chromedriver bypass Cloudflare or Datadome? |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | THE LAB #21 - Bypass anti-bot challenges with AI |
| How to scrape Datadome protected websites (early 2023 version) | 2023-05-29 | How to scrape Datadome protected websites (early 2023 version) |
🏷️ Datasets
| Title | Date | Link |
|---|---|---|
| THE LAB #86: Querying Web Data using GPT-Like Web Interface | 2025-06-05 | THE LAB #86: Querying Web Data using GPT-Like Web Interface |
| Creating a dataset for investors with web scraping: Tesla (TSLA) | 2025-03-30 | Creating a dataset for investors with web scraping: Tesla (TSLA) |
| How to monetize web scraping skills on Data Boutique? | 2024-02-08 | How to monetize web scraping skills on Data Boutique? |
🏷️ Deals
| Title | Date | Link |
|---|---|---|
| Club Deals - by Pierluigi Vinciguerra | 2025-06-13 | Club Deals - by Pierluigi Vinciguerra |
🏷️ Decodo
| Title | Date | Link |
|---|---|---|
| Hands On #4: Testing the new Smartproxy Site Unblocker | 2023-10-13 | Hands On #4: Testing the new Smartproxy Site Unblocker |
| Tik Tok Scraping: how to do it properly | 2023-10-13 | Tik Tok Scraping: how to do it properly |
🏷️ Discounts
| Title | Date | Link |
|---|---|---|
| Club Deals - by Pierluigi Vinciguerra | 2025-06-13 | Club Deals - by Pierluigi Vinciguerra |
🏷️ E-commerce
| Title | Date | Link |
|---|---|---|
| Web scraping and journalism: the Chiara Ferragni case | 2025-05-20 | Web scraping and journalism: the Chiara Ferragni case |
| Scraping E-Commerce websites 101 - by Pierluigi Vinciguerra | 2023-10-13 | Scraping E-Commerce websites 101 - by Pierluigi Vinciguerra |
🏷️ F5
| Title | Date | Link |
|---|---|---|
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | Can Undetected Chromedriver bypass Cloudflare or Datadome? |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | THE LAB #21 - Bypass anti-bot challenges with AI |
🏷️ FabianoSileo
| Title | Date | Link |
|---|---|---|
| Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra | 2023-10-13 | Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra |
🏷️ Fiddler
| Title | Date | Link |
|---|---|---|
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 |
🏷️ Firecrawl
| Title | Date | Link |
|---|---|---|
| Building a custom GPT using Firecrawl | 2025-05-20 | Building a custom GPT using Firecrawl |
🏷️ GPT
| Title | Date | Link |
|---|---|---|
| THE LAB #86: Querying Web Data using GPT-Like Web Interface | 2025-06-05 | THE LAB #86: Querying Web Data using GPT-Like Web Interface |
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools |
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared |
| Building a custom GPT using Firecrawl | 2025-05-20 | Building a custom GPT using Firecrawl |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | Web Scraping experts: Is AI stealing our job? |
| The state of web scraping and AI - by Pierluigi Vinciguerra | 2023-10-13 | The state of web scraping and AI - by Pierluigi Vinciguerra |
🏷️ Geofencing
| Title | Date | Link |
|---|---|---|
| Bypassing Geo-fencing While Scraping | 2024-03-25 | Bypassing Geo-fencing While Scraping |
| Buy cheaper plane tickets using a VPN: truth or myth? | 2023-09-11 | Buy cheaper plane tickets using a VPN: truth or myth? |
🏷️ GermanasLatvaitis
| Title | Date | Link |
|---|---|---|
| Interview #10 - Germanas Latvaitis | 2023-10-13 | Interview #10 - Germanas Latvaitis |
🏷️ GhostCursor
| Title | Date | Link |
|---|---|---|
| Mouse movements in Playwright with Ghost Cursor | 2024-10-13 | Mouse movements in Playwright with Ghost Cursor |
| Bypassing Datadome with Web Scraping - End of 2023 Version | 2023-12-06 | Bypassing Datadome with Web Scraping - End of 2023 Version |
🏷️ Github
| Title | Date | Link |
|---|---|---|
| THE LAB #74: Running scrapers on GitHub Actions | 2025-05-20 | THE LAB #74: Running scrapers on GitHub Actions |
🏷️ Glovo
| Title | Date | Link |
|---|---|---|
| Scraping food delivery data - by Pierluigi Vinciguerra | 2025-05-20 | Scraping food delivery data - by Pierluigi Vinciguerra |
🏷️ GoLogin
| Title | Date | Link |
|---|---|---|
| Scraping Kasada protected websites | 2023-10-13 | Scraping Kasada protected websites |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | How to by-pass Kasada bot mitigation? |
| Scraping Cloudflare Protected Websites (early 2023 version) | 2023-06-10 | Scraping Cloudflare Protected Websites (early 2023 version) |
| THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin | 2023-05-29 | THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin |
🏷️ Google
| Title | Date | Link |
|---|---|---|
| The Scriptwall: Why Google is hiding its SERP content behind Javascript | 2025-05-20 | The Scriptwall: Why Google is hiding its SERP content behind Javascript |
🏷️ Grafana
| Title | Date | Link |
|---|---|---|
| THE LAB #69: Building a dashboard for your scrapers with Grafana | 2025-05-20 | THE LAB #69: Building a dashboard for your scrapers with Grafana |
🏷️ HTTPToolkit
| Title | Date | Link |
|---|---|---|
| How to Scrape Data from Mobile Apps using HTTP Toolkit | 2025-05-20 | How to Scrape Data from Mobile Apps using HTTP Toolkit |
| Scraping food delivery data - by Pierluigi Vinciguerra | 2025-05-20 | Scraping food delivery data - by Pierluigi Vinciguerra |
| HTTP Toolkit, your best friend for network inspection | 2025-05-20 | HTTP Toolkit, your best friend for network inspection |
🏷️ HistoricalData
| Title | Date | Link |
|---|---|---|
| Scraping Historical Data From the Wayback Machine | 2025-05-20 | Scraping Historical Data From the Wayback Machine |
🏷️ Hotel
| Title | Date | Link |
|---|---|---|
| How We Scraped Global Hotel Data to Track Economic Trends | 2024-12-17 | How We Scraped Global Hotel Data to Track Economic Trends |
🏷️ Hrequests
| Title | Date | Link |
|---|---|---|
| THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools | 2025-06-20 | THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | THE LAB 32: hRequests vs anti-bots: a full benchmark |
| hRequests: bypass Akamai with Python requests | 2023-11-12 | hRequests: bypass Akamai with Python requests |
| HTTP requests in Python explained | 2023-10-13 | HTTP requests in Python explained |
🏷️ IKEA
| Title | Date | Link |
|---|---|---|
| The Kallax Index - Scraping Ikea websites | 2023-10-13 | The Kallax Index - Scraping Ikea websites |
🏷️ Idealista
| Title | Date | Link |
|---|---|---|
| Web Scraping Idealista and Bypass Idealista Blockers | 2024-08-06 | Web Scraping Idealista and Bypass Idealista Blockers |
🏷️ Incognition
| Title | Date | Link |
|---|---|---|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers |
🏷️ Infatica
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | The Great Web Unblocker Benchmark - Cloudflare Edition |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| Hands On #6: Testing the Infatica web scraper | 2023-10-05 | Hands On #6: Testing the Infatica web scraper |
🏷️ Infrastructure
| Title | Date | Link |
|---|---|---|
| THE LAB #72: Advanced logging in Playwright | 2025-05-20 | THE LAB #72: Advanced logging in Playwright |
| Analyzing the cost of a web scraping project | 2025-05-20 | Analyzing the cost of a web scraping project |
| THE LAB #69: Building a dashboard for your scrapers with Grafana | 2025-05-20 | THE LAB #69: Building a dashboard for your scrapers with Grafana |
| THE LAB #74: Running scrapers on GitHub Actions | 2025-05-20 | THE LAB #74: Running scrapers on GitHub Actions |
| THE LAB #71: Sending Scrapy logs to RabbitMQ | 2025-05-20 | THE LAB #71: Sending Scrapy logs to RabbitMQ |
| THE LAB #66: How to properly scrape a booking website | 2025-05-20 | THE LAB #66: How to properly scrape a booking website |
| THE LAB #70: Advanced logging in Scrapy | 2025-05-20 | THE LAB #70: Advanced logging in Scrapy |
| Web DRAGON - LLM-powered web scraping on a distributed cloud | 2023-12-19 | Web DRAGON - LLM-powered web scraping on a distributed cloud |
| The costs of web scraping - by Pierluigi Vinciguerra | 2023-10-13 | The costs of web scraping - by Pierluigi Vinciguerra |
| THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers | 2023-05-29 | THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers |
🏷️ Interview
| Title | Date | Link |
|---|---|---|
| Interview #2: Neil Emeigh - Rayobyte | 2023-10-13 | Interview #2: Neil Emeigh - Rayobyte |
| Interview #5: Veritas - The anti obfuscation master | 2023-10-13 | Interview #5: Veritas - The anti obfuscation master |
| Interview with Uriel Knorovich of Nimble | 2023-10-13 | Interview with Uriel Knorovich of Nimble |
| Interview #7: Aviv Besinsky - Bright Data | 2023-10-13 | Interview #7: Aviv Besinsky - Bright Data |
| Interview #3: Ondra Urban - Apify | 2023-10-13 | Interview #3: Ondra Urban - Apify |
| Interview #4: Martin Ganchev - Smartproxy | 2023-10-13 | Interview #4: Martin Ganchev - Smartproxy |
| Interview #10 - Germanas Latvaitis | 2023-10-13 | Interview #10 - Germanas Latvaitis |
| Interview #6: Aleksandras Šulženko - Oxylabs | 2023-10-13 | Interview #6: Aleksandras Šulženko - Oxylabs |
| Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra | 2023-10-13 | Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra |
| Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra | 2023-10-13 | Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra |
🏷️ InventoryData
| Title | Date | Link |
|---|---|---|
| The importance of scraping inventory levels data in the retail industry | 2025-05-20 | The importance of scraping inventory levels data in the retail industry |
| THE LAB #28: Deep dive on inventory levels tracking | 2023-09-28 | THE LAB #28: Deep dive on inventory levels tracking |
| THE LAB #27: Scraping stock level data to estimate revenues | 2023-09-13 | THE LAB #27: Scraping stock level data to estimate revenues |
🏷️ JSON
| Title | Date | Link |
|---|---|---|
| How to Parse JSON with Python: A Beginner-Friendly Guide | 2025-05-20 | How to Parse JSON with Python: A Beginner-Friendly Guide |
🏷️ JWT
| Title | Date | Link |
|---|---|---|
| THE LAB #64: JWT Tokens and API scraping | 2025-05-20 | THE LAB #64: JWT Tokens and API scraping |
🏷️ Ja3Proxy
| Title | Date | Link |
|---|---|---|
| THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies | 2025-05-29 | THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies |
🏷️ Javascript
| Title | Date | Link |
|---|---|---|
| The Scriptwall: Why Google is hiding its SERP content behind Javascript | 2025-05-20 | The Scriptwall: Why Google is hiding its SERP content behind Javascript |
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | Web Scraping and Coding: Five Programming Languages to Check Out |
🏷️ Kameleo
| Title | Date | Link |
|---|---|---|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers |
| The Lab #46: Fingerprint injection in Playwright | 2025-01-26 | The Lab #46: Fingerprint injection in Playwright |
| Behind the scenes of anti-detect browsers - by Tamas Deak | 2024-03-05 | Behind the scenes of anti-detect browsers - by Tamas Deak |
| The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 | 2024-01-19 | The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 |
🏷️ Kasada
| Title | Date | Link |
|---|---|---|
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | Testing the new Botasaurus 4 - by Pierluigi Vinciguerra |
| Bypassing Kasada for web scraping 2024 edition | 2024-09-30 | Bypassing Kasada for web scraping 2024 edition |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | Botasaurus: an anti-ban web scraping framework |
| Scraping Kasada protected websites | 2023-10-13 | Scraping Kasada protected websites |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | Can Undetected Chromedriver bypass Cloudflare or Datadome? |
| Wanted a parka and got an Error 429: Too many requests | 2023-10-13 | Wanted a parka and got an Error 429: Too many requests |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | How to by-pass Kasada bot mitigation? |
| What is Kasada bot mitigation? - by Pierluigi Vinciguerra | 2023-10-13 | What is Kasada bot mitigation? - by Pierluigi Vinciguerra |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | THE LAB #21 - Bypass anti-bot challenges with AI |
🏷️ LLM
| Title | Date | Link |
|---|---|---|
| THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG | 2025-05-20 | THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG |
| How AI is changing the web scraping industry | 2025-05-20 | How AI is changing the web scraping industry |
| The AI-Powered web scraping tools landscape | 2025-05-20 | The AI-Powered web scraping tools landscape |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | Use Cursor as web scraping assistant with MCP servers |
| THE LAB #75: Building self healing scrapers with AI | 2025-05-20 | THE LAB #75: Building self healing scrapers with AI |
| THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2 | 2025-05-20 | THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2 |
| How LLMs are affecting the costs of web scraping | 2025-05-20 | How LLMs are affecting the costs of web scraping |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | Is Web Scraping Dead? - by Pierluigi Vinciguerra |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | Web Scraping experts: Is AI stealing our job? |
| How to create a web scraper with ChatGPT | 2023-10-13 | How to create a web scraper with ChatGPT |
🏷️ LLMScraping
| Title | Date | Link |
|---|---|---|
| Scrape like a pro... but not like an AI company | 2025-05-20 | Scrape like a pro... but not like an AI company |
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared |
| The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 | 2025-05-20 | The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 |
| How AI is changing the web scraping industry | 2025-05-20 | How AI is changing the web scraping industry |
| The AI-Powered web scraping tools landscape | 2025-05-20 | The AI-Powered web scraping tools landscape |
| Building a custom GPT using Firecrawl | 2025-05-20 | Building a custom GPT using Firecrawl |
| About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra | 2025-05-20 | About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra |
| Building a generic scraper for multiple websites | 2025-05-20 | Building a generic scraper for multiple websites |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | Use Cursor as web scraping assistant with MCP servers |
| THE LAB #75: Building self healing scrapers with AI | 2025-05-20 | THE LAB #75: Building self healing scrapers with AI |
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | Build your web scraping assistant with Claude and Cursor |
| Are LLMs capable of replacing traditional scrapers? | 2025-05-20 | Are LLMs capable of replacing traditional scrapers? |
| How LLMs are affecting the costs of web scraping | 2025-05-20 | How LLMs are affecting the costs of web scraping |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | Is Web Scraping Dead? - by Pierluigi Vinciguerra |
| Web DRAGON - LLM-powered web scraping on a distributed cloud | 2023-12-19 | Web DRAGON - LLM-powered web scraping on a distributed cloud |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | Web Scraping experts: Is AI stealing our job? |
| The state of web scraping and AI - by Pierluigi Vinciguerra | 2023-10-13 | The state of web scraping and AI - by Pierluigi Vinciguerra |
🏷️ Lambda
| Title | Date | Link |
|---|---|---|
| The Lab #48: Scraping with AWS Lambda | 2024-10-18 | The Lab #48: Scraping with AWS Lambda |
🏷️ LeadGeneration
| Title | Date | Link |
|---|---|---|
| Web Scraping for Lead Generation and Prospecting | 2025-03-12 | Web Scraping for Lead Generation and Prospecting |
🏷️ Legal
| Title | Date | Link |
|---|---|---|
| AI and data: different faces of the same coin | 2025-05-20 | AI and data: different faces of the same coin |
| Is web scraping legal? - by Pierluigi Vinciguerra | 2025-03-12 | Is web scraping legal? - by Pierluigi Vinciguerra |
| The X vs Bright Data case - by Sanaea Daruwalla | 2024-07-09 | The X vs Bright Data case - by Sanaea Daruwalla |
| Legal Zyte-geist #4: Overview of the EU AI Act | 2024-05-28 | Legal Zyte-geist #4: Overview of the EU AI Act |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | Is Web Scraping Dead? - by Pierluigi Vinciguerra |
| Legal Zyte-geist #3: What the court’s ruling in the Meta v Bright Data case really means for web scrapers | 2024-02-13 | Legal Zyte-geist #3: What the court’s ruling in the Meta v Bright Data case really means for web scrapers |
| Legal Zyte-geist #2: Web Scraping and AI 2023 Legal Wrap-Up | 2024-01-09 | Legal Zyte-geist #2: Web Scraping and AI 2023 Legal Wrap-Up |
| Legal Zyte-geist #1: Step-by-Step Guide to Compliant Web Scraping | 2023-12-05 | Legal Zyte-geist #1: Step-by-Step Guide to Compliant Web Scraping |
| Can I scrape any public data? - by Pierluigi Vinciguerra | 2023-10-13 | Can I scrape any public data? - by Pierluigi Vinciguerra |
| Is it legal to scrape social networks like Facebook or Instagram? | 2023-10-13 | Is it legal to scrape social networks like Facebook or Instagram? |
| Web Scraping Legal Context - by Andrea Squatrito | 2023-10-13 | Web Scraping Legal Context - by Andrea Squatrito |
🏷️ Lightpanda
| Title | Date | Link |
|---|---|---|
| Rethinking the web browser - by Katie Hallett | 2025-01-21 | Rethinking the web browser - by Katie Hallett |
🏷️ LocationData
| Title | Date | Link |
|---|---|---|
| THE LAB #31: Scraping location data using a world grid | 2025-01-16 | THE LAB #31: Scraping location data using a world grid |
🏷️ MCP
| Title | Date | Link |
|---|---|---|
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base |
🏷️ MachineLearning
| Title | Date | Link |
|---|---|---|
| Machine learning models for detecting bot detection triggers | 2025-06-15 | Machine learning models for detecting bot detection triggers |
🏷️ Make
| Title | Date | Link |
|---|---|---|
| No-Code Web Scraping with Make.com | 2025-05-20 | No-Code Web Scraping with Make.com |
🏷️ MarketResearch
| Title | Date | Link |
|---|---|---|
| Web scraping in market research and competitive analysis | 2025-03-12 | Web scraping in market research and competitive analysis |
🏷️ MartinGanchev
| Title | Date | Link |
|---|---|---|
| Interview #4: Martin Ganchev - Smartproxy | 2023-10-13 | Interview #4: Martin Ganchev - Smartproxy |
🏷️ Mistral
| Title | Date | Link |
|---|---|---|
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared |
🏷️ MobileApp
| Title | Date | Link |
|---|---|---|
| How to Scrape Data from Mobile Apps using HTTP Toolkit | 2025-05-20 | How to Scrape Data from Mobile Apps using HTTP Toolkit |
| Scraping food delivery data - by Pierluigi Vinciguerra | 2025-05-20 | Scraping food delivery data - by Pierluigi Vinciguerra |
| HTTP Toolkit, your best friend for network inspection | 2025-05-20 | HTTP Toolkit, your best friend for network inspection |
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 |
| The Lab #58: Intercepting traffic from an App - part 1 | 2025-05-20 | The Lab #58: Intercepting traffic from an App - part 1 |
| THE LAB #1: Scraping data from an app | 2024-12-26 | THE LAB #1: Scraping data from an app |
| THE LAB #12: Reverse-engineering Mobile API | 2023-05-29 | THE LAB #12: Reverse-engineering Mobile API |
🏷️ MobileProxy
| Title | Date | Link |
|---|---|---|
| Comparing Residential And Mobile Proxies for Anti-Bot Evasion | 2025-06-01 | Comparing Residential And Mobile Proxies for Anti-Bot Evasion |
| Building an in-house mobile proxy farm | 2025-05-20 | Building an in-house mobile proxy farm |
| How I've built my home made mobile proxy | 2023-10-13 | How I've built my home made mobile proxy |
🏷️ MouseMovements
| Title | Date | Link |
|---|---|---|
| THE LAB #8: Using Bezier curves for human-like mouse movements | 2023-05-29 | THE LAB #8: Using Bezier curves for human-like mouse movements |
🏷️ Multilogin
| Title | Date | Link |
|---|---|---|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers |
🏷️ NFT
| Title | Date | Link |
|---|---|---|
| THE LAB #9: Scraping OpenSea NFT's data | 2023-05-29 | THE LAB #9: Scraping OpenSea NFT's data |
🏷️ NSTBrowser
| Title | Date | Link |
|---|---|---|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers |
🏷️ NehaSetia
| Title | Date | Link |
|---|---|---|
| Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra | 2023-10-13 | Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra |
🏷️ NeilEmeigh
| Title | Date | Link |
|---|---|---|
| Interview #2: Neil Emeigh - Rayobyte | 2023-10-13 | Interview #2: Neil Emeigh - Rayobyte |
🏷️ NetNut
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
🏷️ News
| Title | Date | Link |
|---|---|---|
| A brief wrap up of the latest news on web scraping | 2023-10-13 | A brief wrap up of the latest news on web scraping |
| The 2022 recap for the Web Scraping industry | 2023-10-13 | The 2022 recap for the Web Scraping industry |
🏷️ Nimble
| Title | Date | Link |
|---|---|---|
| Hands on #3: Building a price comparison tool with Nimble APIs | 2023-10-13 | Hands on #3: Building a price comparison tool with Nimble APIs |
| THE LAB #20 - AI powered web scrapers with Nimble Browser | 2023-10-13 | THE LAB #20 - AI powered web scrapers with Nimble Browser |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | THE LAB #21 - Bypass anti-bot challenges with AI |
| From Traditional Browsers to AI-Powered Web Scraping | 2023-10-13 | From Traditional Browsers to AI-Powered Web Scraping |
🏷️ NoCode
| Title | Date | Link |
|---|---|---|
| No-Code Web Scraping with Make.com | 2025-05-20 | No-Code Web Scraping with Make.com |
🏷️ Nodriver
| Title | Date | Link |
|---|---|---|
| THE LAB #62: Bypassing Cloudflare with Nodriver | 2025-05-20 | THE LAB #62: Bypassing Cloudflare with Nodriver |
🏷️ Octobrowser
| Title | Date | Link |
|---|---|---|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers |
🏷️ OndraUrban
| Title | Date | Link |
|---|---|---|
| Interview #3: Ondra Urban - Apify | 2023-10-13 | Interview #3: Ondra Urban - Apify |
🏷️ OpenAI
| Title | Date | Link |
|---|---|---|
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools |
| AI and data: different faces of the same coin | 2025-05-20 | AI and data: different faces of the same coin |
| No-Code Web Scraping with Make.com | 2025-05-20 | No-Code Web Scraping with Make.com |
| Building a custom GPT using Firecrawl | 2025-05-20 | Building a custom GPT using Firecrawl |
| How to create a web scraper with ChatGPT | 2023-10-13 | How to create a web scraper with ChatGPT |
🏷️ OpenSea
| Title | Date | Link |
|---|---|---|
| THE LAB #9: Scraping OpenSea NFT's data | 2023-05-29 | THE LAB #9: Scraping OpenSea NFT's data |
🏷️ Oxylabs
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | The Great Web Unblocker Benchmark - Cloudflare Edition |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| THE LAB #63: Oxymouse and Playwright | 2025-05-20 | THE LAB #63: Oxymouse and Playwright |
| How to Scrape E-Commerce Websites With Python | 2024-08-02 | How to Scrape E-Commerce Websites With Python |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | The Great Web Unblocker Benchmark: March 2024 |
| Hands On #5: Testing the Oxylabs Web Unblocker | 2023-10-13 | Hands On #5: Testing the Oxylabs Web Unblocker |
| Bypassing Perimeterx in 2023 with code and examples | 2023-09-11 | Bypassing Perimeterx in 2023 with code and examples |
🏷️ Oxymouse
| Title | Date | Link |
|---|---|---|
| THE LAB #63: Oxymouse and Playwright | 2025-05-20 | THE LAB #63: Oxymouse and Playwright |
🏷️ PHP
| Title | Date | Link |
|---|---|---|
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | Web Scraping and Coding: Five Programming Languages to Check Out |
🏷️ Patchwright
| Title | Date | Link |
|---|---|---|
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 |
🏷️ PerimeterX
| Title | Date | Link |
|---|---|---|
| The Lab #56: Bypassing PerimeterX 3 | 2025-05-20 | The Lab #56: Bypassing PerimeterX 3 |
| Bypassing PerimeterX without a browser automation tool | 2024-11-15 | Bypassing PerimeterX without a browser automation tool |
| The Lab #35: Bypassing PerimeterX with Python and Playwright | 2023-12-21 | The Lab #35: Bypassing PerimeterX with Python and Playwright |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | Can Undetected Chromedriver bypass Cloudflare or Datadome? |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | THE LAB #21 - Bypass anti-bot challenges with AI |
| Bypassing Perimeterx in 2023 with code and examples | 2023-09-11 | Bypassing Perimeterx in 2023 with code and examples |
| THE LAB #7: Scraping PerimeterX protected websites | 2023-05-29 | THE LAB #7: Scraping PerimeterX protected websites |
🏷️ PixelWhispererAPI
| Title | Date | Link |
|---|---|---|
| Scraping Cloudflare websites using an API | 2025-05-20 | Scraping Cloudflare websites using an API |
🏷️ Playwright
| Title | Date | Link |
|---|---|---|
| THE LAB #72: Advanced logging in Playwright | 2025-05-20 | THE LAB #72: Advanced logging in Playwright |
| How to start with Scrapy and Playwright - Part 2 | 2025-05-20 | How to start with Scrapy and Playwright - Part 2 |
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 |
| Making Playwright scrapers undetected with open source solutions | 2025-05-20 | Making Playwright scrapers undetected with open source solutions |
| THE LAB #63: Oxymouse and Playwright | 2025-05-20 | THE LAB #63: Oxymouse and Playwright |
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | THE LAB #73: How to Bypass Cloudflare in 2025 |
| The Lab #56: Bypassing PerimeterX 3 | 2025-05-20 | The Lab #56: Bypassing PerimeterX 3 |
| The Lab #55: Checking your browser fingerprint | 2025-05-20 | The Lab #55: Checking your browser fingerprint |
| The 2025 web scraping tech stack - by Pierluigi Vinciguerra | 2025-05-20 | The 2025 web scraping tech stack - by Pierluigi Vinciguerra |
| The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra | 2025-05-20 | The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra |
| The Lab #57: Improving your Playwright scraper and avoid CDP detection | 2025-05-20 | The Lab #57: Improving your Playwright scraper and avoid CDP detection |
| The Lab #46: Fingerprint injection in Playwright | 2025-01-26 | The Lab #46: Fingerprint injection in Playwright |
| THE LAB #11: The Anti-Detect Anti-Bot matrix | 2025-01-01 | THE LAB #11: The Anti-Detect Anti-Bot matrix |
| Mouse movements in Playwright with Ghost Cursor | 2024-10-13 | Mouse movements in Playwright with Ghost Cursor |
| Bypassing Kasada for web scraping 2024 edition | 2024-09-30 | Bypassing Kasada for web scraping 2024 edition |
| Scraping the dark web with Playwright and Brave | 2024-03-07 | Scraping the dark web with Playwright and Brave |
| Web Scraping from 0 to hero: tips and tricks for Microsoft Playwright | 2024-02-18 | Web Scraping from 0 to hero: tips and tricks for Microsoft Playwright |
| Web Scraping from 0 to hero: our first scraper with Microsoft Playwright | 2024-02-04 | Web Scraping from 0 to hero: our first scraper with Microsoft Playwright |
| Web scraping from 0 to hero: Microsoft Playwright | 2024-01-21 | Web scraping from 0 to hero: Microsoft Playwright |
| The Lab #35: Bypassing PerimeterX with Python and Playwright | 2023-12-21 | The Lab #35: Bypassing PerimeterX with Python and Playwright |
| Bypassing Datadome with Web Scraping - End of 2023 Version | 2023-12-06 | Bypassing Datadome with Web Scraping - End of 2023 Version |
| Scraping Kasada protected websites | 2023-10-13 | Scraping Kasada protected websites |
| Selenium vs Playwright, a comparison | 2023-10-13 | Selenium vs Playwright, a comparison |
| HTTP requests in Python explained | 2023-10-13 | HTTP requests in Python explained |
| What do I need for web scraping? - by Pierluigi Vinciguerra | 2023-10-13 | What do I need for web scraping? - by Pierluigi Vinciguerra |
| The starter toolkit for a python web scraping developer (2022) | 2023-10-13 | The starter toolkit for a python web scraping developer (2022) |
| Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra | 2023-10-13 | Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | How to by-pass Kasada bot mitigation? |
| What is Playwright? - by Pierluigi Vinciguerra | 2023-10-13 | What is Playwright? - by Pierluigi Vinciguerra |
| Is web scraping becoming harder? - by Pierluigi Vinciguerra | 2023-10-13 | Is web scraping becoming harder? - by Pierluigi Vinciguerra |
| 5 Playwright useful features for web scraping | 2023-09-30 | 5 Playwright useful features for web scraping |
| Bypassing Perimeterx in 2023 with code and examples | 2023-09-11 | Bypassing Perimeterx in 2023 with code and examples |
| THE LAB #19: How to mask the device fingerprint | 2023-09-11 | THE LAB #19: How to mask the device fingerprint |
| Buy cheaper plane tickets using a VPN: truth or myth? | 2023-09-11 | Buy cheaper plane tickets using a VPN: truth or myth? |
| Scraping Cloudflare Protected Websites (early 2023 version) | 2023-06-10 | Scraping Cloudflare Protected Websites (early 2023 version) |
| THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin | 2023-05-29 | THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin |
| How to scrape Datadome protected websites (early 2023 version) | 2023-05-29 | How to scrape Datadome protected websites (early 2023 version) |
| THE LAB #8: Using Bezier curves for human-like mouse movements | 2023-05-29 | THE LAB #8: Using Bezier curves for human-like mouse movements |
| THE LAB #9: Scraping OpenSea NFT's data | 2023-05-29 | THE LAB #9: Scraping OpenSea NFT's data |
🏷️ PriceMonitoring
| Title | Date | Link |
|---|---|---|
| Web Scraping in Price Monitoring and Dynamic Pricing | 2025-03-12 | Web Scraping in Price Monitoring and Dynamic Pricing |
🏷️ Proxies
| Title | Date | Link |
|---|---|---|
| Comparing Residential And Mobile Proxies for Anti-Bot Evasion | 2025-06-01 | Comparing Residential And Mobile Proxies for Anti-Bot Evasion |
| The Unit Economics of Proxy Providers - by Abed | 2025-05-27 | The Unit Economics of Proxy Providers - by Abed |
| Analyzing the cost of a web scraping project | 2025-05-20 | Analyzing the cost of a web scraping project |
| How to start with Scrapy and Playwright - Part 2 | 2025-05-20 | How to start with Scrapy and Playwright - Part 2 |
| THE LAB #61: Evaluating your proxy provider | 2025-05-20 | THE LAB #61: Evaluating your proxy provider |
| Optimizing Proxy Usage for Large-Scale Scraping | 2025-05-20 | Optimizing Proxy Usage for Large-Scale Scraping |
| Building an in-house mobile proxy farm | 2025-05-20 | Building an in-house mobile proxy farm |
| How to start with Scrapy and Playwright - Part 1 | 2025-05-20 | How to start with Scrapy and Playwright - Part 1 |
| The Dirty Little Secret of Internet's Data | 2025-05-17 | The Dirty Little Secret of Internet's Data |
| Web Scraping with Proxies: How Many IPs Do You Really Need? | 2025-04-29 | Web Scraping with Proxies: How Many IPs Do You Really Need? |
| Five Secrets of the Proxy Industry - by Julia Levi | 2025-03-18 | Five Secrets of the Proxy Industry - by Julia Levi |
| What is a residential proxy? - by Pierluigi Vinciguerra | 2025-03-13 | What is a residential proxy? - by Pierluigi Vinciguerra |
| Where do proxy companies take residential IPs from? | 2025-02-24 | Where do proxy companies take residential IPs from? |
| Web Scraping from 0 to hero: Everything about proxies | 2024-04-14 | Web Scraping from 0 to hero: Everything about proxies |
| Scrapoxy, the super proxy aggregator, how it works? | 2024-02-21 | Scrapoxy, the super proxy aggregator, how it works? |
| How scraping a single website costed thousands of dollars in proxy | 2024-01-28 | How scraping a single website costed thousands of dollars in proxy |
| The costs of web scraping - by Pierluigi Vinciguerra | 2023-10-13 | The costs of web scraping - by Pierluigi Vinciguerra |
| What's a proxy server? - by Pierluigi Vinciguerra | 2023-10-13 | What's a proxy server? - by Pierluigi Vinciguerra |
| On choosing the right proxy provider for scraping | 2023-10-13 | On choosing the right proxy provider for scraping |
| The most interesting GitHub Repositories about web scraping (2023) | 2023-10-13 | The most interesting GitHub Repositories about web scraping (2023) |
| Buy cheaper plane tickets using a VPN: truth or myth? | 2023-09-11 | Buy cheaper plane tickets using a VPN: truth or myth? |
🏷️ Puppeteer
| Title | Date | Link |
|---|---|---|
| How to Improve the Performance of Puppeteer Stealth Evasions | 2024-04-02 | How to Improve the Performance of Puppeteer Stealth Evasions |
🏷️ Pyppetteer
| Title | Date | Link |
|---|---|---|
| THE LAB #11: The Anti-Detect Anti-Bot matrix | 2025-01-01 | THE LAB #11: The Anti-Detect Anti-Bot matrix |
🏷️ Python
| Title | Date | Link |
|---|---|---|
| Scraping Through Tor for Increased Anonymity | 2025-05-25 | Scraping Through Tor for Increased Anonymity |
| Optimizing Python Scripts for High-Traffic Websites | 2025-05-20 | Optimizing Python Scripts for High-Traffic Websites |
| How to Parse JSON with Python: A Beginner-Friendly Guide | 2025-05-20 | How to Parse JSON with Python: A Beginner-Friendly Guide |
| The Lab #47: Scraping real time data with Python | 2025-03-14 | The Lab #47: Scraping real time data with Python |
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | Web Scraping and Coding: Five Programming Languages to Check Out |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | Botasaurus: an anti-ban web scraping framework |
| HTTP requests in Python explained | 2023-10-13 | HTTP requests in Python explained |
🏷️ R
| Title | Date | Link |
|---|---|---|
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | Web Scraping and Coding: Five Programming Languages to Check Out |
🏷️ RAG
| Title | Date | Link |
|---|---|---|
| THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG | 2025-05-20 | THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG |
| THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2 | 2025-05-20 | THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2 |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base |
🏷️ RabbitMQ
| Title | Date | Link |
|---|---|---|
| THE LAB #72: Advanced logging in Playwright | 2025-05-20 | THE LAB #72: Advanced logging in Playwright |
| THE LAB #71: Sending Scrapy logs to RabbitMQ | 2025-05-20 | THE LAB #71: Sending Scrapy logs to RabbitMQ |
🏷️ RaspberryPI
| Title | Date | Link |
|---|---|---|
| How I've built my home made mobile proxy | 2023-10-13 | How I've built my home made mobile proxy |
🏷️ RealEstate
| Title | Date | Link |
|---|---|---|
| Web Scraping Idealista and Bypass Idealista Blockers | 2024-08-06 | Web Scraping Idealista and Bypass Idealista Blockers |
🏷️ Reddit
| Title | Date | Link |
|---|---|---|
| THE LAB #18: How to scrape Reddit with Scrapy | 2023-09-11 | THE LAB #18: How to scrape Reddit with Scrapy |
🏷️ Report
| Title | Date | Link |
|---|---|---|
| The state of public web data in 2024 | 2024-05-05 | The state of public web data in 2024 |
🏷️ Requests
| Title | Date | Link |
|---|---|---|
| Scraping Through Tor for Increased Anonymity | 2025-05-25 | Scraping Through Tor for Increased Anonymity |
| Optimizing Python Scripts for High-Traffic Websites | 2025-05-20 | Optimizing Python Scripts for High-Traffic Websites |
🏷️ ResidentialProxies
| Title | Date | Link |
|---|---|---|
| Comparing Residential And Mobile Proxies for Anti-Bot Evasion | 2025-06-01 | Comparing Residential And Mobile Proxies for Anti-Bot Evasion |
🏷️ Ruby
| Title | Date | Link |
|---|---|---|
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | Web Scraping and Coding: Five Programming Languages to Check Out |
🏷️ SEO
| Title | Date | Link |
|---|---|---|
| Web Scraping for SEO and content marketing | 2025-03-12 | Web Scraping for SEO and content marketing |
🏷️ SERP
| Title | Date | Link |
|---|---|---|
| The Scriptwall: Why Google is hiding its SERP content behind Javascript | 2025-05-20 | The Scriptwall: Why Google is hiding its SERP content behind Javascript |
🏷️ SSLPinning
| Title | Date | Link |
|---|---|---|
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 |
🏷️ ScrapeGraphAI
| Title | Date | Link |
|---|---|---|
| The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 | 2025-05-20 | The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 |
| About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra | 2025-05-20 | About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra |
| Building a generic scraper for multiple websites | 2025-05-20 | Building a generic scraper for multiple websites |
| Open source Python libraries for your web scraping projects | 2025-05-20 | Open source Python libraries for your web scraping projects |
| Build a RAG Application with ScraperAPI, Gemini, and FAISS | 2025-04-02 | Build a RAG Application with ScraperAPI, Gemini, and FAISS |
🏷️ ScrapeOps
| Title | Date | Link |
|---|---|---|
| THE LAB #13: Managing a fleet of scrapers with Scrapeops | 2023-06-10 | THE LAB #13: Managing a fleet of scrapers with Scrapeops |
🏷️ ScrapegraphAI
| Title | Date | Link |
|---|---|---|
| Building a generic scraper for multiple websites | 2025-05-20 | Building a generic scraper for multiple websites |
🏷️ Scraping
| Title | Date | Link |
|---|---|---|
| Machine learning models for detecting bot detection triggers | 2025-06-15 | Machine learning models for detecting bot detection triggers |
| Analyzing the cost of a web scraping project | 2025-05-20 | Analyzing the cost of a web scraping project |
| The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 | 2025-05-20 | The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 |
| THE LAB #81: Scraping Zillow for fun and profit | 2025-05-20 | THE LAB #81: Scraping Zillow for fun and profit |
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 |
| Web scraping and journalism: the Chiara Ferragni case | 2025-05-20 | Web scraping and journalism: the Chiara Ferragni case |
| The Lab #58: Intercepting traffic from an App - part 1 | 2025-05-20 | The Lab #58: Intercepting traffic from an App - part 1 |
| THE LAB #66: How to properly scrape a booking website | 2025-05-20 | THE LAB #66: How to properly scrape a booking website |
| THE LAB #67: Scraping Telegram using its APIs | 2025-05-20 | THE LAB #67: Scraping Telegram using its APIs |
| Web data and automotive industry - by Pierluigi Vinciguerra | 2025-05-20 | Web data and automotive industry - by Pierluigi Vinciguerra |
| THE LAB #64: JWT Tokens and API scraping | 2025-05-20 | THE LAB #64: JWT Tokens and API scraping |
| Build a RAG Application with ScraperAPI, Gemini, and FAISS | 2025-04-02 | Build a RAG Application with ScraperAPI, Gemini, and FAISS |
| Web Scraping typical use cases - by Pierluigi Vinciguerra | 2025-03-13 | Web Scraping typical use cases - by Pierluigi Vinciguerra |
| Web scraping in market research and competitive analysis | 2025-03-12 | Web scraping in market research and competitive analysis |
| Web Scraping in Price Monitoring and Dynamic Pricing | 2025-03-12 | Web Scraping in Price Monitoring and Dynamic Pricing |
| THE LAB #1: Scraping data from an app | 2024-12-26 | THE LAB #1: Scraping data from an app |
| The Lab #48: Scraping with AWS Lambda | 2024-10-18 | The Lab #48: Scraping with AWS Lambda |
| Web Scraping Idealista and Bypass Idealista Blockers | 2024-08-06 | Web Scraping Idealista and Bypass Idealista Blockers |
| The X vs Bright Data case - by Sanaea Daruwalla | 2024-07-09 | The X vs Bright Data case - by Sanaea Daruwalla |
| Web DRAGON - LLM-powered web scraping on a distributed cloud | 2023-12-19 | Web DRAGON - LLM-powered web scraping on a distributed cloud |
| Algolia and web scraping: an introduction | 2023-12-10 | Algolia and web scraping: an introduction |
| The true costs of a web scraping project | 2023-11-25 | The true costs of a web scraping project |
| Web scraping from 0 to hero: a modern tech stack | 2023-11-19 | Web scraping from 0 to hero: a modern tech stack |
| Web scraping from 0 to hero: Introduction to web scraping | 2023-10-22 | Web scraping from 0 to hero: Introduction to web scraping |
| Web scraping and alternative data for financial markets | 2023-10-13 | Web scraping and alternative data for financial markets |
| Web Scraping Legal Context - by Andrea Squatrito | 2023-10-13 | Web Scraping Legal Context - by Andrea Squatrito |
| The Kallax Index - Scraping Ikea websites | 2023-10-13 | The Kallax Index - Scraping Ikea websites |
| Tik Tok Scraping: how to do it properly | 2023-10-13 | Tik Tok Scraping: how to do it properly |
| The state of web scraping and AI - by Pierluigi Vinciguerra | 2023-10-13 | The state of web scraping and AI - by Pierluigi Vinciguerra |
🏷️ ScrapingAPI
| Title | Date | Link |
|---|---|---|
| THE LAB #64: JWT Tokens and API scraping | 2025-05-20 | THE LAB #64: JWT Tokens and API scraping |
| Hands on #3: Building a price comparison tool with Nimble APIs | 2023-10-13 | Hands on #3: Building a price comparison tool with Nimble APIs |
| Hands On #5: Testing the Oxylabs Web Unblocker | 2023-10-13 | Hands On #5: Testing the Oxylabs Web Unblocker |
| Hands On #4: Testing the new Smartproxy Site Unblocker | 2023-10-13 | Hands On #4: Testing the new Smartproxy Site Unblocker |
| Hands On #2: Testing the new Zyte Api | 2023-10-13 | Hands On #2: Testing the new Zyte Api |
| Hands On #6: Testing the Infatica web scraper | 2023-10-05 | Hands On #6: Testing the Infatica web scraper |
🏷️ Scrapoxy
| Title | Date | Link |
|---|---|---|
| Open source Python libraries for your web scraping projects | 2025-05-20 | Open source Python libraries for your web scraping projects |
| Bypassing Geo-fencing While Scraping | 2024-03-25 | Bypassing Geo-fencing While Scraping |
| Scrapoxy, the super proxy aggregator, how it works? | 2024-02-21 | Scrapoxy, the super proxy aggregator, how it works? |
🏷️ Scrapy
| Title | Date | Link |
|---|---|---|
| THE LAB #30: How to bypass Akamai protected website when nothing else works | 2025-06-09 | THE LAB #30: How to bypass Akamai protected website when nothing else works |
| Scraping Akamai-protected websites with Scrapy | 2025-05-20 | Scraping Akamai-protected websites with Scrapy |
| The Lab #54: Scraping from Algolia APIs | 2025-05-20 | The Lab #54: Scraping from Algolia APIs |
| THE LAB #71: Sending Scrapy logs to RabbitMQ | 2025-05-20 | THE LAB #71: Sending Scrapy logs to RabbitMQ |
| Scraping APIs with Bearer Token - by Pierluigi Vinciguerra | 2025-05-20 | Scraping APIs with Bearer Token - by Pierluigi Vinciguerra |
| The 2025 web scraping tech stack - by Pierluigi Vinciguerra | 2025-05-20 | The 2025 web scraping tech stack - by Pierluigi Vinciguerra |
| THE LAB #70: Advanced logging in Scrapy | 2025-05-20 | THE LAB #70: Advanced logging in Scrapy |
| Three ways to make money with web scraping as a freelancer | 2025-05-20 | Three ways to make money with web scraping as a freelancer |
| How to start with Scrapy and Playwright - Part 1 | 2025-05-20 | How to start with Scrapy and Playwright - Part 1 |
| The Framework That Won't Quit: Scrapy's Continued Relevance in Data Extraction | 2025-05-19 | The Framework That Won't Quit: Scrapy's Continued Relevance in Data Extraction |
| The Lab #47: Scraping real time data with Python | 2025-03-14 | The Lab #47: Scraping real time data with Python |
| Bypassing PerimeterX without a browser automation tool | 2024-11-15 | Bypassing PerimeterX without a browser automation tool |
| Scraping Akamai protected websites | 2024-09-08 | Scraping Akamai protected websites |
| The Lab #43: Scraping inventory data: why, how and where | 2024-02-29 | The Lab #43: Scraping inventory data: why, how and where |
| Web scraping from 0 to hero: creating our first Scrapy spider - Part 2 | 2024-01-07 | Web scraping from 0 to hero: creating our first Scrapy spider - Part 2 |
| Web scraping from 0 to hero: creating our first Scrapy spider - Part 1 | 2023-12-17 | Web scraping from 0 to hero: creating our first Scrapy spider - Part 1 |
| Web scraping from 0 to hero: before start scraping | 2023-11-05 | Web scraping from 0 to hero: before start scraping |
| Create your first python scraper with Scrapy | 2023-10-13 | Create your first python scraper with Scrapy |
| HTTP requests in Python explained | 2023-10-13 | HTTP requests in Python explained |
| What do I need for web scraping? - by Pierluigi Vinciguerra | 2023-10-13 | What do I need for web scraping? - by Pierluigi Vinciguerra |
| The starter toolkit for a python web scraping developer (2022) | 2023-10-13 | The starter toolkit for a python web scraping developer (2022) |
| Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra | 2023-10-13 | Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra |
| Wanted a parka and got an Error 429: Too many requests | 2023-10-13 | Wanted a parka and got an Error 429: Too many requests |
| The Kallax Index - Scraping Ikea websites | 2023-10-13 | The Kallax Index - Scraping Ikea websites |
| How to Create Your First Web Scraper with Scrapy: A Step-by-Step Tutorial | 2023-10-13 | How to Create Your First Web Scraper with Scrapy: A Step-by-Step Tutorial |
| What is Scrapy? - by Pierluigi Vinciguerra | 2023-10-13 | What is Scrapy? - by Pierluigi Vinciguerra |
| The most interesting GitHub Repositories about web scraping (2023) | 2023-10-13 | The most interesting GitHub Repositories about web scraping (2023) |
| How to write your first scraper with Scrapy | 2023-10-13 | How to write your first scraper with Scrapy |
| THE LAB #29: Bypass Cloudflare Bot Protection with Scrapy | 2023-10-12 | THE LAB #29: Bypass Cloudflare Bot Protection with Scrapy |
| Three web scraping tools just discovered on GitHub | 2023-10-08 | Three web scraping tools just discovered on GitHub |
| THE LAB #18: How to scrape Reddit with Scrapy | 2023-09-11 | THE LAB #18: How to scrape Reddit with Scrapy |
| THE LAB #13: Managing a fleet of scrapers with Scrapeops | 2023-06-10 | THE LAB #13: Managing a fleet of scrapers with Scrapeops |
| The Lab #5 - Scraping Airbnb.com using GraphQL | 2023-05-29 | The Lab #5 - Scraping Airbnb.com using GraphQL |
| THE LAB #7: Scraping PerimeterX protected websites | 2023-05-29 | THE LAB #7: Scraping PerimeterX protected websites |
| THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers | 2023-05-29 | THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers |
| THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting | 2023-05-29 | THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting |
🏷️ ScrapyD
| Title | Date | Link |
|---|---|---|
| THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers | 2023-05-29 | THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers |
🏷️ ScrapyImpersonate
| Title | Date | Link |
|---|---|---|
| Bypassing PerimeterX without a browser automation tool | 2024-11-15 | Bypassing PerimeterX without a browser automation tool |
| Bypassing Cloudflare with open source repositories | 2024-09-18 | Bypassing Cloudflare with open source repositories |
🏷️ Selectors
| Title | Date | Link |
|---|---|---|
| XPATH and CSS Selectors in Web Scraping | 2024-04-28 | XPATH and CSS Selectors in Web Scraping |
| XPath vs CSS selectors: a comparison | 2023-10-13 | XPath vs CSS selectors: a comparison |
🏷️ Selenium
| Title | Date | Link |
|---|---|---|
| Web Scraping from 0 to hero: Our first scraper with Selenium | 2024-03-17 | Web Scraping from 0 to hero: Our first scraper with Selenium |
| Web Scraping from 0 to hero: Selenium | 2024-03-03 | Web Scraping from 0 to hero: Selenium |
| Selenium vs Playwright, a comparison | 2023-10-13 | Selenium vs Playwright, a comparison |
| What do I need for web scraping? - by Pierluigi Vinciguerra | 2023-10-13 | What do I need for web scraping? - by Pierluigi Vinciguerra |
| What is Selenium? - by Pierluigi Vinciguerra | 2023-10-13 | What is Selenium? - by Pierluigi Vinciguerra |
🏷️ Sitemaps
| Title | Date | Link |
|---|---|---|
| Indexing data in the web: Robots file and Sitemaps | 2023-10-13 | Indexing data in the web: Robots file and Sitemaps |
🏷️ Smartproxy
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | The Great Web Unblocker Benchmark - Cloudflare Edition |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| Scraping Akamai protected websites | 2024-09-08 | Scraping Akamai protected websites |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | The Great Web Unblocker Benchmark: March 2024 |
🏷️ Splash
| Title | Date | Link |
|---|---|---|
| What do I need for web scraping? - by Pierluigi Vinciguerra | 2023-10-13 | What do I need for web scraping? - by Pierluigi Vinciguerra |
| What is Splash? - by Pierluigi Vinciguerra | 2023-10-13 | What is Splash? - by Pierluigi Vinciguerra |
🏷️ TWSC
| Title | Date | Link |
|---|---|---|
| End of year recap for The Web Scraping Club | 2023-12-31 | End of year recap for The Web Scraping Club |
🏷️ Tesla
| Title | Date | Link |
|---|---|---|
| Creating a dataset for investors with web scraping: Tesla (TSLA) | 2025-03-30 | Creating a dataset for investors with web scraping: Tesla (TSLA) |
🏷️ Test
| Title | Date | Link |
|---|---|---|
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | Testing the new Botasaurus 4 - by Pierluigi Vinciguerra |
| The Anti-Detect Browser Royal Rumble - updated with notes | 2025-05-20 | The Anti-Detect Browser Royal Rumble - updated with notes |
| The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra | 2025-02-14 | The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra |
| The Anti-Detect Browser Royal Rumble - Fingerprint tests | 2024-04-23 | The Anti-Detect Browser Royal Rumble - Fingerprint tests |
| Testing the Bright Data Web Unblocker proxy | 2023-12-08 | Testing the Bright Data Web Unblocker proxy |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | THE LAB 32: hRequests vs anti-bots: a full benchmark |
| hRequests: bypass Akamai with Python requests | 2023-11-12 | hRequests: bypass Akamai with Python requests |
| Hands on #3: Building a price comparison tool with Nimble APIs | 2023-10-13 | Hands on #3: Building a price comparison tool with Nimble APIs |
| Hands On #5: Testing the Oxylabs Web Unblocker | 2023-10-13 | Hands On #5: Testing the Oxylabs Web Unblocker |
| Hands On #4: Testing the new Smartproxy Site Unblocker | 2023-10-13 | Hands On #4: Testing the new Smartproxy Site Unblocker |
| Hands On #2: Testing the new Zyte Api | 2023-10-13 | Hands On #2: Testing the new Zyte Api |
| Hands On #6: Testing the Infatica web scraper | 2023-10-05 | Hands On #6: Testing the Infatica web scraper |
🏷️ TikTok
| Title | Date | Link |
|---|---|---|
| Tik Tok Scraping: how to do it properly | 2023-10-13 | Tik Tok Scraping: how to do it properly |
🏷️ Tools
| Title | Date | Link |
|---|---|---|
| A guideline for creating your scrapers with the proper tool | 2023-12-04 | A guideline for creating your scrapers with the proper tool |
🏷️ Tor
| Title | Date | Link |
|---|---|---|
| Scraping Through Tor for Increased Anonymity | 2025-05-25 | Scraping Through Tor for Increased Anonymity |
| Scraping the dark web with Playwright and Brave | 2024-03-07 | Scraping the dark web with Playwright and Brave |
🏷️ Travel
| Title | Date | Link |
|---|---|---|
| THE LAB #66: How to properly scrape a booking website | 2025-05-20 | THE LAB #66: How to properly scrape a booking website |
| Scraping the Skies: Get Insights from Flight Data | 2025-05-20 | Scraping the Skies: Get Insights from Flight Data |
| How We Scraped Global Hotel Data to Track Economic Trends | 2024-12-17 | How We Scraped Global Hotel Data to Track Economic Trends |
| The Lab #5 - Scraping Airbnb.com using GraphQL | 2023-05-29 | The Lab #5 - Scraping Airbnb.com using GraphQL |
🏷️ Turnstile
| Title | Date | Link |
|---|---|---|
| Cloudflare Turnstile: what is that and how it works? | 2023-10-13 | Cloudflare Turnstile: what is that and how it works? |
🏷️ Tutorial
| Title | Date | Link |
|---|---|---|
| Dealing with Rate Limiting Using Exponential Backoff | 2025-06-13 | Dealing with Rate Limiting Using Exponential Backoff |
| Scheduling Scrapers with Airflow - by Pierluigi Vinciguerra | 2025-05-20 | Scheduling Scrapers with Airflow - by Pierluigi Vinciguerra |
| Scraping Historical Data From the Wayback Machine | 2025-05-20 | Scraping Historical Data From the Wayback Machine |
| How to Scrape Data from Mobile Apps using HTTP Toolkit | 2025-05-20 | How to Scrape Data from Mobile Apps using HTTP Toolkit |
| How to start with Scrapy and Playwright - Part 2 | 2025-05-20 | How to start with Scrapy and Playwright - Part 2 |
| Browser Fingerprinting 101 - What it is and how it works | 2025-05-20 | Browser Fingerprinting 101 - What it is and how it works |
| Optimizing Python Scripts for High-Traffic Websites | 2025-05-20 | Optimizing Python Scripts for High-Traffic Websites |
| How to Parse JSON with Python: A Beginner-Friendly Guide | 2025-05-20 | How to Parse JSON with Python: A Beginner-Friendly Guide |
| The 2025 web scraping tech stack - by Pierluigi Vinciguerra | 2025-05-20 | The 2025 web scraping tech stack - by Pierluigi Vinciguerra |
| Scraping the Skies: Get Insights from Flight Data | 2025-05-20 | Scraping the Skies: Get Insights from Flight Data |
| How to start with Scrapy and Playwright - Part 1 | 2025-05-20 | How to start with Scrapy and Playwright - Part 1 |
| Web Scraping with Proxies: How Many IPs Do You Really Need? | 2025-04-29 | Web Scraping with Proxies: How Many IPs Do You Really Need? |
| Web Scraping typical use cases - by Pierluigi Vinciguerra | 2025-03-13 | Web Scraping typical use cases - by Pierluigi Vinciguerra |
| What is a residential proxy? - by Pierluigi Vinciguerra | 2025-03-13 | What is a residential proxy? - by Pierluigi Vinciguerra |
| Web Scraping for SEO and content marketing | 2025-03-12 | Web Scraping for SEO and content marketing |
| What is web scraping? - by Pierluigi Vinciguerra | 2025-03-12 | What is web scraping? - by Pierluigi Vinciguerra |
| Web scraping in market research and competitive analysis | 2025-03-12 | Web scraping in market research and competitive analysis |
| Web Scraping for Lead Generation and Prospecting | 2025-03-12 | Web Scraping for Lead Generation and Prospecting |
| Web Scraping in Price Monitoring and Dynamic Pricing | 2025-03-12 | Web Scraping in Price Monitoring and Dynamic Pricing |
| Web Scraping from 0 to hero: kickstart your career in web scraping | 2024-05-26 | Web Scraping from 0 to hero: kickstart your career in web scraping |
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | Web Scraping and Coding: Five Programming Languages to Check Out |
| Web Scraping from 0 to hero: data cleaning processes | 2024-05-12 | Web Scraping from 0 to hero: data cleaning processes |
| Web Scraping from 0 to hero: Everything about proxies | 2024-04-14 | Web Scraping from 0 to hero: Everything about proxies |
| What is a web unblocker and how does it work? | 2024-04-07 | What is a web unblocker and how does it work? |
| How to Improve the Performance of Puppeteer Stealth Evasions | 2024-04-02 | How to Improve the Performance of Puppeteer Stealth Evasions |
| Why my scraper is getting blocked? | 2024-03-31 | Why my scraper is getting blocked? |
| Web Scraping from 0 to hero: Our first scraper with Selenium | 2024-03-17 | Web Scraping from 0 to hero: Our first scraper with Selenium |
| Web Scraping from 0 to hero: Selenium | 2024-03-03 | Web Scraping from 0 to hero: Selenium |
| Web Scraping from 0 to hero: tips and tricks for Microsoft Playwright | 2024-02-18 | Web Scraping from 0 to hero: tips and tricks for Microsoft Playwright |
| Web Scraping from 0 to hero: our first scraper with Microsoft Playwright | 2024-02-04 | Web Scraping from 0 to hero: our first scraper with Microsoft Playwright |
| Web scraping from 0 to hero: Microsoft Playwright | 2024-01-21 | Web scraping from 0 to hero: Microsoft Playwright |
| Web scraping from 0 to hero: creating our first Scrapy spider - Part 2 | 2024-01-07 | Web scraping from 0 to hero: creating our first Scrapy spider - Part 2 |
| Web scraping from 0 to hero: creating our first Scrapy spider - Part 1 | 2023-12-17 | Web scraping from 0 to hero: creating our first Scrapy spider - Part 1 |
| A guideline for creating your scrapers with the proper tool | 2023-12-04 | A guideline for creating your scrapers with the proper tool |
| Web scraping from 0 to hero: a modern tech stack | 2023-11-19 | Web scraping from 0 to hero: a modern tech stack |
| Web scraping from 0 to hero: before start scraping | 2023-11-05 | Web scraping from 0 to hero: before start scraping |
| Web scraping from 0 to hero: Introduction to web scraping | 2023-10-22 | Web scraping from 0 to hero: Introduction to web scraping |
| The costs of web scraping - by Pierluigi Vinciguerra | 2023-10-13 | The costs of web scraping - by Pierluigi Vinciguerra |
| Selenium vs Playwright, a comparison | 2023-10-13 | Selenium vs Playwright, a comparison |
| Create your first python scraper with Scrapy | 2023-10-13 | Create your first python scraper with Scrapy |
| Web scraping and alternative data for financial markets | 2023-10-13 | Web scraping and alternative data for financial markets |
| What's a proxy server? - by Pierluigi Vinciguerra | 2023-10-13 | What's a proxy server? - by Pierluigi Vinciguerra |
| What do I need for web scraping? - by Pierluigi Vinciguerra | 2023-10-13 | What do I need for web scraping? - by Pierluigi Vinciguerra |
| The starter toolkit for a python web scraping developer (2022) | 2023-10-13 | The starter toolkit for a python web scraping developer (2022) |
| Scraping E-Commerce websites 101 - by Pierluigi Vinciguerra | 2023-10-13 | Scraping E-Commerce websites 101 - by Pierluigi Vinciguerra |
| 3 THINGS + 1 TO DO BEFORE STARTING CODING YOUR SCRAPER | 2023-10-13 | 3 THINGS + 1 TO DO BEFORE STARTING CODING YOUR SCRAPER |
| What is device fingerprinting? A deep dive | 2023-10-13 | What is device fingerprinting? A deep dive |
| Browser fingerprinting and web scraping | 2023-10-13 | Browser fingerprinting and web scraping |
| What is Splash? - by Pierluigi Vinciguerra | 2023-10-13 | What is Splash? - by Pierluigi Vinciguerra |
| Is it legal to scrape social networks like Facebook or Instagram? | 2023-10-13 | Is it legal to scrape social networks like Facebook or Instagram? |
| What is Selenium? - by Pierluigi Vinciguerra | 2023-10-13 | What is Selenium? - by Pierluigi Vinciguerra |
| Browser API: an introduction - by Pierluigi Vinciguerra | 2023-10-13 | Browser API: an introduction - by Pierluigi Vinciguerra |
| What is Playwright? - by Pierluigi Vinciguerra | 2023-10-13 | What is Playwright? - by Pierluigi Vinciguerra |
| What is Undetected Chromedriver? - by Pierluigi Vinciguerra | 2023-10-13 | What is Undetected Chromedriver? - by Pierluigi Vinciguerra |
| What is Kasada bot mitigation? - by Pierluigi Vinciguerra | 2023-10-13 | What is Kasada bot mitigation? - by Pierluigi Vinciguerra |
| How to Create Your First Web Scraper with Scrapy: A Step-by-Step Tutorial | 2023-10-13 | How to Create Your First Web Scraper with Scrapy: A Step-by-Step Tutorial |
| Indexing data in the web: Robots file and Sitemaps | 2023-10-13 | Indexing data in the web: Robots file and Sitemaps |
| Is web scraping becoming harder? - by Pierluigi Vinciguerra | 2023-10-13 | Is web scraping becoming harder? - by Pierluigi Vinciguerra |
| Tik Tok Scraping: how to do it properly | 2023-10-13 | Tik Tok Scraping: how to do it properly |
| What is Scrapy? - by Pierluigi Vinciguerra | 2023-10-13 | What is Scrapy? - by Pierluigi Vinciguerra |
| How to write your first scraper with Scrapy | 2023-10-13 | How to write your first scraper with Scrapy |
| Three web scraping tools just discovered on GitHub | 2023-10-08 | Three web scraping tools just discovered on GitHub |
🏷️ Undetectable
| Title | Date | Link |
|---|---|---|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers |
🏷️ UndetectedCromedriver
| Title | Date | Link |
|---|---|---|
| THE LAB #11: The Anti-Detect Anti-Bot matrix | 2025-01-01 | THE LAB #11: The Anti-Detect Anti-Bot matrix |
| Bypassing Kasada for web scraping 2024 edition | 2024-09-30 | Bypassing Kasada for web scraping 2024 edition |
| Bypassing Cloudflare with open source repositories | 2024-09-18 | Bypassing Cloudflare with open source repositories |
| Scraping Kasada protected websites | 2023-10-13 | Scraping Kasada protected websites |
| What do I need for web scraping? - by Pierluigi Vinciguerra | 2023-10-13 | What do I need for web scraping? - by Pierluigi Vinciguerra |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | Can Undetected Chromedriver bypass Cloudflare or Datadome? |
| What is Undetected Chromedriver? - by Pierluigi Vinciguerra | 2023-10-13 | What is Undetected Chromedriver? - by Pierluigi Vinciguerra |
| Bypassing Perimeterx in 2023 with code and examples | 2023-09-11 | Bypassing Perimeterx in 2023 with code and examples |
| Scraping Cloudflare Protected Websites (early 2023 version) | 2023-06-10 | Scraping Cloudflare Protected Websites (early 2023 version) |
🏷️ UrielKnorovich
| Title | Date | Link |
|---|---|---|
| Interview with Uriel Knorovich of Nimble | 2023-10-13 | Interview with Uriel Knorovich of Nimble |
🏷️ Veritas
| Title | Date | Link |
|---|---|---|
| Interview #5: Veritas - The anti obfuscation master | 2023-10-13 | Interview #5: Veritas - The anti obfuscation master |
🏷️ Vinted
| Title | Date | Link |
|---|---|---|
| THE LAB #82: How to scrape Vinted using their internal APIs | 2025-05-20 | THE LAB #82: How to scrape Vinted using their internal APIs |
🏷️ Wappalyzer
| Title | Date | Link |
|---|---|---|
| Change detection for web scraping: tools and techniques | 2023-10-15 | Change detection for web scraping: tools and techniques |
🏷️ WayBackMachine
| Title | Date | Link |
|---|---|---|
| Scraping Historical Data From the Wayback Machine | 2025-05-20 | Scraping Historical Data From the Wayback Machine |
🏷️ WebData
| Title | Date | Link |
|---|---|---|
| Is web scraping a profitable industry? | 2025-05-20 | Is web scraping a profitable industry? |
| How We Scraped Global Hotel Data to Track Economic Trends | 2024-12-17 | How We Scraped Global Hotel Data to Track Economic Trends |
| 10 years of web scraping: a perspective about selling web data | 2024-03-24 | 10 years of web scraping: a perspective about selling web data |
| How to monetize web scraping skills on Data Boutique? | 2024-02-08 | How to monetize web scraping skills on Data Boutique? |
| Monetize your web scraping skills: a brief guide | 2024-01-14 | Monetize your web scraping skills: a brief guide |
🏷️ WebRTC
| Title | Date | Link |
|---|---|---|
| Bypassing Geo-fencing While Scraping | 2024-03-25 | Bypassing Geo-fencing While Scraping |
🏷️ WebUnblocker
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| Web Unblocker vs. Browser as a service for scraping | 2025-05-20 | Web Unblocker vs. Browser as a service for scraping |
| The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra | 2025-02-14 | The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra |
| How to Scrape E-Commerce Websites With Python | 2024-08-02 | How to Scrape E-Commerce Websites With Python |
| What is a web unblocker and how does it work? | 2024-04-07 | What is a web unblocker and how does it work? |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | The Great Web Unblocker Benchmark: March 2024 |
| Testing the Bright Data Web Unblocker proxy | 2023-12-08 | Testing the Bright Data Web Unblocker proxy |
🏷️ XPATH
| Title | Date | Link |
|---|---|---|
| XPATH and CSS Selectors in Web Scraping | 2024-04-28 | XPATH and CSS Selectors in Web Scraping |
| XPath vs CSS selectors: a comparison | 2023-10-13 | XPath vs CSS selectors: a comparison |
🏷️ Zenrows
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | The Great Web Unblocker Benchmark - Cloudflare Edition |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | The Great Web Unblocker Benchmark: March 2024 |
🏷️ Zillow
| Title | Date | Link |
|---|---|---|
| THE LAB #81: Scraping Zillow for fun and profit | 2025-05-20 | THE LAB #81: Scraping Zillow for fun and profit |
🏷️ Zyte
| Title | Date | Link |
|---|---|---|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | The Great Web Unblocker Benchmark - Cloudflare Edition |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | The Great Web Unblocker Benchmark: Kasada edition |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | The Great Web Unblocker Benchmark: March 2024 |
| Hands On #2: Testing the new Zyte Api | 2023-10-13 | Hands On #2: Testing the new Zyte Api |
🏷️ browserautomation
| Title | Date | Link |
|---|---|---|
| The Browser Automation Landscape in 2025 | 2025-05-20 | The Browser Automation Landscape in 2025 |
| Web Unblocker vs. Browser as a service for scraping | 2025-05-20 | Web Unblocker vs. Browser as a service for scraping |
🏷️ curl_cffi
| Title | Date | Link |
|---|---|---|
| Three web scraping tools just discovered on GitHub | 2023-10-08 | Three web scraping tools just discovered on GitHub |
🏷️ fake-fingerprint
| Title | Date | Link |
|---|---|---|
| Three web scraping tools just discovered on GitHub | 2023-10-08 | Three web scraping tools just discovered on GitHub |
🏷️ opensource
| Title | Date | Link |
|---|---|---|
| Bypassing Cloudflare with open source repositories | 2024-09-18 | Bypassing Cloudflare with open source repositories |
🏷️ recaptcha
| Title | Date | Link |
|---|---|---|
| THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools | 2025-06-20 | THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools |
🏷️ telegram
| Title | Date | Link |
|---|---|---|
| THE LAB #67: Scraping Telegram using its APIs | 2025-05-20 | THE LAB #67: Scraping Telegram using its APIs |
🏷️ tlsfingerprint
| Title | Date | Link |
|---|---|---|
| THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies | 2025-05-29 | THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies |
| THE LAB 33: Fingerprinting at different connection layers | 2023-11-30 | THE LAB 33: Fingerprinting at different connection layers |
| THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting | 2023-05-29 | THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting |
🏷️ tool
| Title | Date | Link |
|---|---|---|
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | Botasaurus: an anti-ban web scraping framework |