web-poet

web-poet is a Python 3.8+ implementation of the page object pattern for web scraping. It enables writing portable, reusable web parsing code.

Warning

web-poet is in early stages of development; backward-incompatible changes are possible.

Getting started

• Overview
• Installation
• Tutorial
• From the ground up

Writing page objects

• Page objects
• Inputs
• Items
• Rules
• Fields
• Additional requests
• Input validation
• Using page params
• Stats
• Tests for page objects
• Frameworks

Writing frameworks

• Framework specification
• Rules
• Additional requests
• Retries
• Frameworks

Reference

• API reference
• Contributing
• Changelog
• License