Job Description
This tech company focusing on gaming solutions development is legally regsitered in Manila with more than 200 people. Currently they are hiring for web crawler to join the team. This position requires Chinese speaking and is based in Manila. They can offer competitive salary package, visa sponsorship and housing allowance for right candidate.
Job responsibilities
- Design, develop and maintain high-performance web crawler systems for data collection and information capture.
- Implement human-machine confrontation technology to deal with the website’s anti-crawler mechanism and ensure stable data crawling.
- Responsible for the architectural design and optimization of the crawler system to ensure reliability, scalability and security.
- Perform data cleaning and preprocessing to ensure the quality and consistency of the collected data.
- Work with the team to develop and implement data collection strategies and resolve issues that may arise.
- Track and understand the latest anti-crawler technologies and trends, and continuously optimize the crawler system.
- Write technical documents and reports to record the design and implementation details of the crawler system.
- Bachelor's or master's degree in computer science, information technology, or related field.
- Have more than 5 years of work experience in web crawler development and data collection.
- Proficient in commonly used programming languages and technologies, such as Python, Scrapy, Selenium, etc.
- Have experience in human-machine confrontation and anti-crawler, and be familiar with common anti-crawler mechanisms and countermeasures.
- Have excellent data analysis and problem-solving skills.
- Have a basic understanding and awareness of network security and privacy protection.
- Good team spirit and communication skills.