This book comprehensively introduces the development process of directional crawlers, cracking methods of various anti-crawler mechanisms and related techniques of crawler development. The book has 13 chapters in total, including introduction, Python basics, regular expressions and file operations, simple web crawler development, high-performance HTML content parsing, Python and databases, asynchronous loading and request headers, simulated login and verification codes, packet capture and man-in-the-middle crawlers, Android native App crawlers, Scrapy, Scrapy applications, legal and ethical issues in crawler development, etc. Except for Chapters 1, 12, and 13, there are hands-on exercises at the end to help readers consolidate what they have learned in this chapter and previous chapters. For difficult content in the book, there are also video explanations to better demonstrate the relevant operations.
Reader comments
The content is comprehensive, covering all aspects from basics to practical implementation, and each chapter has hands-on practice to help consolidate what you have learned. I especially like the parts about anti-crawler mechanisms and decryption methods in the book, which I find very challenging. This reminds me of the book "Hackers and Painters". They are both full of technical information.
Entry-friendly, theoretical and practical guidance, effective learning
The content is comprehensive, covering all aspects from basics to practical implementation, and each chapter has hands-on practice to help consolidate what you have learned. I especially like the parts about anti-crawler mechanisms and decryption methods in the book, which I find very challenging. This reminds me of the book "Hackers and Painters". They are both full of technical information.
Entry-friendly, theoretical and practical guidance, effective learning