Python Coder with a Lot of Scraping Experience using Python

We are currently seeking 2-3 highly skilled and experienced Python coders (for web scraping) to join our team. As a part of our team, you will be responsible for scraping data from hundreds of online retail stores, using our Framework.

The ideal candidates are independent, proactive, and possess a strong background in web scraping techniques and tools.

You must be able to work 40 hours per week, starting on 11/13/2023 (Monday) or before, and be committed to a long-term collaboration (minimum 6 months, but we prefer years).

Responsibilities:

*Develop Python-based web scrapers for hundreds of online retail stores.

*Implement rotating proxies (datacenter and residential) provided by us.

*Work with GitHub repositories assigned for scraper development.

*Bypass antibot measures, captchas, Cloudflare, expiring tokens, fingerprinting, and other blocking methods efficiently.

*Handle product variations using combinations of attributes.

*Collaborate with the team via Slack and maintain problem/solution logs.

*Follow standard operating procedures and provide regular updates on scraper development.

Requirements:

*Extensive experience in web scraping; technical aspects need not be taught.

*Proficiency in Python for developing scrapers.

*Familiarity with manipulating website DOM, preferably using bs4.

*Knowledge of reverse engineering web apps to find API endpoints.

*Experience with browsers DevTools to intercept and replicate network traffic.

*Ability to build complex scrapers for small and large sites.

*Strong communication skills and adherence to deadlines.

Additional Skills (Preferred):

*Familiarity with ecommerce platforms like BigCommerce, Demandware, Magento, and WooCommerce.

*Familiarity with Selenium grid.

*Knowledge of REST APIs and GraphQL.

The scrapers you create with our Framework, must:

*List every product on the site

*MUST work using plain requests, without automated browsers

*Access each product

*Extract every variation for each product

*Extract each of a list of datapoints for each product and variation

NOTE: Variations are every possible combination of attributes a product can have.

If a product has the Colors: [Red, Blue], and the Sizes: [S,M,L], combinations will be:
Red – S
Red – M
Red – L
Blue – S
Blue – M
Blue – L

Each of the above is considered a variation even though there is more than one attribute for each variation.

Please read through the additional job description details in the link below and answer ALL questions thoroughly. We cannot consider your application without complete and thorough answers.

Click to view/document/d/1WNoDsTKykLMvrSV3UIVQna_XcGVO5QxcGFINd00FJBg/edit?usp=sharing

Once you reply appropriately, if you appear to be a good candidate, I will message you. If all goes well, you will be sent an NDA and Contract. Once those are both signed, we’ll invite you to our Slack channel and get you set up to start working.