See more Amazon Details

Contents Listing

Foreword
Preface
1. Introduction to Web Automation
The Web as Data Source
History of LWP
Installing LWP
Words of Caution
LWP in Action
2. Web Basics
URLs
An HTTP Transaction
LWP::Simple
Fetching Documents Without LWP::Simple
Example: AltaVista
HTTP POST
Example: Babelfish
3. The LWP Class Model
The Basic Classes
Programming with LWP Classes
Inside the do_GET and do_POST Functions
User Agents
HTTP::Response Objects
LWP Classes: Behind the Scenes
4. URLs
Parsing URLs
Relative URLs
Converting Absolute URLs to Relative
Converting Relative URLs to Absolute
5. Forms
Elements of an HTML Form
LWP and GET Requests
Automating Form Analysis
Idiosyncrasies of HTML Forms
POST Example: License Plates
POST Example: ABEBooks.com
File Uploads
Limits on Forms
6. Simple HTML Processing with Regular Expressions
Automating Data Extraction
Regular Expression Techniques
Troubleshooting
When Regular Expressions Aren't Enough
Example: Extracting Links from a Bookmark File
Example: Extracting Links from Arbitrary HTML
Example: Extracting Temperatures from Weather Underground
7. HTML Processing with Tokens
HTML as Tokens
Basic HTML::TokeParser Use
Individual Tokens
Token Sequences
More HTML::TokeParser Methods
Using Extracted Text
8. Tokenizing Walkthrough
The Problem
Getting the Data
Inspecting the HTML
First Code
Narrowing In
Rewrite for Features
Alternatives
9. HTML Processing with Trees
Introduction to Trees
HTML::TreeBuilder
Processing
Example: BBC News
Example: Fresh Air
10. Modifying HTML with Trees
Changing Attributes
Deleting Images
Detaching and Reattaching
Attaching in Another Tree
Creating New Elements
11. Cookies, Authentication, and Advanced Requests
Cookies
Adding Extra Request Header Lines
Authentication
An HTTP Authentication Example: The Unicode Mailing Archive
12. Spiders
Types of Web-Querying Programs
A User Agent for Robots
Example: A Link-Checking Spider
Ideas for Further Expansion
A. LWP Modules
B. HTTP Status Codes
C. Common MIME Types
D. Language Tags
E. Common Content Encodings
F. ASCII Table
G. User's View of Object-Oriented Modules
Index