Web Scraping Project for Expert Programmer with Feasibility Study

€30-250 EUR

Closed

Posted

4 months ago

€30-250 EUR

Paid on delivery

We are seeking a highly skilled and experienced web scraper to undertake a complex project involving scraping a specific website for detailed product information. This project will be divided into two phases and requires the development of a comprehensive Intranet site to manage and parse the scraped data effectively. Project Overview: Initial Requirement: Conduct a feasibility study before starting the project. Successful completion will lead to future collaboration opportunities. Expertise Demanded: Applicants must demonstrate their expertise with samples of previous work. The quality and efficiency of the code will be rigorously tested. Phase 1: Data Scraping and Database Integration Input: Read an Excel sheet listing products. Target Website: Navigate to the specified website for product information. Search Parameters: Flexibility in searching for products using name, EAN, Manufacturer Number, or product code. Data Retrieval: Extract product pages and gather information including text boxes, specifications, descriptions, images, PDFs, and videos. Database Integration: Organize and store the collected data in a database, repeating the process for each product. Phase 2: Data Export and Mapping Field Mapping: Ability to remap data fields (e.g., mapping 'Internal Memory' to 'Memory' or 'RAM'). Export Capabilities: Export all product data from the database, organizing binary files in folders named after the product, and exporting text data to a master Excel file. Additional Requirements: Obstacle Navigation: The scraper must efficiently handle redirection scripts, cookies, leech protection, and other potential barriers. Image Retrieval: Secure full-size, uncompressed images from manufacturer websites. Proxy Rotation: Utilize a proxy list to rotate queries, avoiding IP-based limits on HTTP requests, with configurable request limits per proxy. Application Instructions: Interested applicants should provide proof of their expertise, including samples of past projects and a detailed approach to this specific project. The selection process will emphasize the quality, efficiency, and robustness of previous work. This project is challenging and requires a programmer with exceptional skills in web scraping and database management. If you have the necessary expertise and are ready for a rewarding challenge, we encourage you to apply. ATTACHED THERE IS THE EAN LIST THAT WE NEED INFORMATIONS THE WEBSITE TO SCRAPE FROM IS [login to view URL]

Project ID: 37691983

About the project

45 proposals

Remote project

Active 3 mos ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

45 freelancers are bidding on average €162 EUR for this job

@ahmadayaz

With over a decade of experience in Full Stack Development, WordPress Theme and Plugin Development, Mobile App Design and Development, and more, I have a proven track record of delivering successful projects to organizations of all sizes. I am highly skilled in web scraping and database management, making me the perfect fit for your project. To better understand your requirements, I would like to gather some additional information. Could you please clarify the purpose of the feasibility study you mentioned? Additionally, what is the expected volume of data to be scraped, and are there any specific websites you would like to include in the scraping process? Once I have these details, I can provide a more accurate timeline and cost estimate for the project. Now, let me share my proposed approach for your project: Phase 1: Data Scraping and Database Integration - Conduct a thorough feasibility study to identify any potential challenges. - Develop a reliable web scraping solution to extract product information from the specified website using various search parameters. - Implement an efficient database integration system to store and organize the scraped data effectively. Phase 2: Data Export and Mapping - Create a field mapping module to customize the data fields as per your requirements. - Enable easy export of product data from the database, organizing files and exporting text data to an Excel file. Throughout the project, I will handle obstacles like redirection scripts, cookies, leech protection, and IP-based limits on HTTP requests. I will also ensure the retrieval of full-size, uncompressed images from manufacturer websites and implement proxy rotation as needed. I have successfully completed similar projects in the past, and I am committed to delivering a high-quality solution that meets your needs. I look forward to establishing a long-term relationship with you. If you have any additional information or questions, please feel free to share.

€225 EUR in 5 days

5.0

(308 reviews)

8.7

@mantislin

Hi sir, This is Lin and I am scraping expert, I scraped 600+ websites, please check my reviews then you will know. I will create scraping tool with PHP & CURL that extracts all requested data, And provides data in the required format (JSON, CSV, EXCEL, SQL...). Can we discuss more details about this project? Then I can provide sample data/script, we can start project after you checked sample. Thanks, Lin

€250 EUR in 5 days

5.0

(603 reviews)

8.4

@schoudhary1553

Top 1% in Freelancer.com Hi, Greetings! ✅checked your project details: ✅Completed Time: In project deadline We have worked on 900 + Projects. I have 6 + years of the experience in same kind of projects. If you are looking for a true Freelancer, I am the Right person for you. I am available almost 24-7 and am very responsive. I feel proud that I am a trusted Freelancer who pleases almost every single client. You can rest assure, your work will be delivered well in advance of others, with passion and accuracy. I guarantee you instant communication & responses when you need me. Why choose me? I think every client is the reason for my success. I only take projects which I am sure I can do quickly. My Portfolio Items: https://www.freelancer.com/u/schoudhary1553 I would really like to work with you on this project. If interested, Kindly contact me via chat for further details and discussion. Thank you Sandeep

€220 EUR in 4 days

4.9

(657 reviews)

8.6

@imtyzooel71n

Hi, I am Senior Python script developer with 10 years of experience. I can scrape your required website by python script/bot with your instructions very short time. Can we discuss please? Thanks

€140 EUR in 7 days

4.9

(255 reviews)

7.6

@atitudesam

At your service is the team of skilled and experienced Web Scrapers and Developers you've been searching for! With a strong background in PHP, we understand the intricacies of extensive data scraping and complex database integrations. Over our many combined years in the field, we have completed numerous projects using similar technologies such as Laravel and HTML (among others), which guarantees our profound understanding in handling potential barriers such as redirection scripts, cookies, leech protection. Our dynamic Excel handling skills ensures that field mapping won't be an issue as we can easily remap data fields accordingly. In addition to this, we have a comprehensive knowledge of utilizing proxy rotation to avoid IP-based limits and ensuring smooth HTTP requests without compromising on time-efficiency. Lastly, we're well-equipped to handle exporting data as your project demands – organizing binary files impeccably into product-based folders while keeping text-data in a structured Excel file. Through our partnership with you, not only will you receive clean and quality codes but also dedicated post-deployment support - because getting the job done is one thing, but having someone to rely on afterwards is equally important.

€250 EUR in 2 days

5.0

(165 reviews)

7.3

@OmarFarukBD

I am poised to begin work immediately on this challenging web scraping project involving the extraction of detailed product information from the specified website, https://icecat.biz. My commitment starts with a thorough feasibility study, ensuring a robust foundation for the subsequent phases. Drawing on my extensive experience, I will expertly navigate obstacles such as redirection scripts, cookies, and leech protection, guaranteeing efficient data retrieval. In Phase 1, I will employ flexible search parameters to extract diverse product data, organizing and storing it in a meticulously managed database. Phase 2 will see me remapping data fields and exporting information in a structured manner. My ability to handle challenges, coupled with a proactive approach to proxy rotation and image retrieval, positions me as the ideal candidate for this intricate task. I look forward to showcasing my expertise through this project and exploring future collaboration opportunities. Let's embark on this rewarding challenge together.

€180 EUR in 4 days

4.7

(121 reviews)

7.5

@shahajay6434

Hello, you are looking for a expert Web Scraping Project developer to complete your project. With more than a decade of experience, my expertise isn't limited to a single scope. I offer proficiency in PHP, HTML5, and CSS as well as a particular emphasis on React/Node and WordPress development. What sets me apart is my dedication to not just completing projects but exceeding expectations to ensure lasting quality. You can check my portfolio on my freelancer profile = https://www.freelancer.com/u/shahajay6434 I am available for an interview Monday to Friday 10 am to 12pm (IST time zone). I look forward to having the opportunity to discuss the job further Note: Bid price is base price, please discuss time and cost at the chat. With Regards, Ajay Shah

€140 EUR in 7 days

4.8

(108 reviews)

7.5

@uumarkhalid31

Hello! As an expert programmer, I specialize in web scraping and will conduct a thorough feasibility study to ensure the success of your project. I'll develop a customized solution to extract and deliver valuable data efficiently. Let's discuss the details to optimize your web scraping requirements and achieve your goals. Thank you-

€120 EUR in 3 days

4.8

(300 reviews)

7.4

@Aliraza2727

I’m senior developer having 7 years’ experience in WordPress, Shopify, React, Laravel, HTML, CSS, Bootstrap, JavaScript, jQuery, Ajax, PHP and Graphic designing. Please discuss with me in chat so I can start work on it. I have read the things that you want, I’m interested for this task and consider myself perfect for this job. I will you provide 100% satisfied work. We will final price after discussion. Thanks, Ali

€110 EUR in 1 day

4.9

(174 reviews)

6.6

@shrec

Hello Here is a web scraping expert who is proficient in scraping & manipulating data. With over 8 years of experience in data management with various programing language such as Python, PHP, JavaScript and etc, I have built a bunch of tools that scrapes and crawls website. So I am confident I can provide the best result for your demand in a short time. Please send me a message so that I can dive into your project right now. Thanks & Regards

€140 EUR in 7 days

5.0

(17 reviews)

6.3

@GraphicsFamily15

Please consider me and give me a chance to impress you by my quality services. I read job description and I can do your project. I can even work on urgent basis and complete your project soon. Contact me for more information. Regards Kamal

€150 EUR in 7 days

5.0

(16 reviews)

5.0

@devbaloni1983

Hi, I have more then 15 year’ experienced in Development with full stack . As i am expert, I can build it according to your requirements. Because I have a lot of the experiences about these fields, I can build your project as soon as possible perfectly. I offer excellent software quality work and provide the best support on your project. If you give me this job, I will do my best to finish your job until your timeline. If you want to see my past works, I can show you my past works right now. Ready to discuss with you about the task now.

€100 EUR in 7 days

4.7

(13 reviews)

4.5

@Automate007

★★★ PAY AFTER WORK ★★★ Pay only when you got 100% work done ✅ Hi There. I have done the Similar work before. I hope this proposal finds you well. I am writing to express my keen interest in your Python Selenium automation project. With a comprehensive background in test automation and [7 years] of experience working with Python and Selenium, I am confident in delivering a robust and efficient solution tailored to your specific needs. Proposal Highlights: Project Understanding: Having reviewed your project requirements, I am excited about the opportunity to automate your testing processes using Python and Selenium. My approach involves creating scalable and maintainable automation scripts that improve efficiency and reduce manual effort. Selenium Proficiency: I bring a wealth of experience in utilizing Selenium for test automation. My expertise extends to creating dynamic and data-driven test scripts, handling complex scenarios, and ensuring the reliability of test outcomes across different browsers and platforms. I am enthusiastic about the opportunity to contribute to the success of your app project. Please find my detailed proposal attached for your review. Thank you for considering my bid. I look forward to the possibility of collaborating with you on this exciting project.

€130 EUR in 2 days

5.0

(29 reviews)

4.0

@writerLanguage

Hi, I have read the requirements and understand them. I can assure you that if you are to work with me once, you will always work with me on these kinds of project. Would you like to discuss the project with me?

€45 EUR in 1 day

4.7

(8 reviews)

3.3

@akkiwadhwa1995

Hey Hope this message finds you in the best of tech-savvy spirits! As a web scraping specialist and data scientist, I bring together a diverse set of technologies and tools to extract valuable data from the web efficiently and effectively. My expertise encompasses Python-based libraries like BeautifulSoup, Scrapy, and Selenium, ensuring that I can adapt to various scraping challenges and requirements. Additionally, I have experience with data storage and processing technologies such as SQLite, MongoDB, and Pandas, allowing me to handle, organize, and analyze the scraped data with precision. Whether it's e-commerce data, news articles, social media content, or any other web source, I have the technical prowess to craft custom scraping solutions that deliver clean, structured data for your specific needs. Let's collaborate to harness the power of these technologies and create a web scraping solution that provides you with the valuable insights you seek.

€140 EUR in 7 days

4.5

(6 reviews)

3.7

@zahid49ch

As an engineering professional with extensive experience in data analysis and a deep understanding of web scraping, I confidently submit my candidacy for your project. Having worked on numerous data-intensive projects, I have honed my skills in PHP and database management to ensure accurate and efficient data collection, organization and utilization. My knowledge encompasses the use of specific tools such as Excel, scrapers, databases for filtering, remapping and storage of collected information. Beyond my technical aptitude, my team-focused approach is an asset that will be invaluable to the scope of your project. We are adept at navigating complex obstacles such as redirection scripts, cookies, leech protection and more - skills that are undoubtedly crucial for a smooth web scraping process of this scale. Furthermore, our commitment to using high-quality proxies will prevent any potential IP-based limitations on HTTP requests. In summary, with my depth of experience and proven ability to deliver results specifically within web scraping and database integration projects, I am confident in executing this task accurately and efficiently while maintaining the highest standards of quality control. This has been the foundation for valuable long-term collaborations with previous clients like yourself who require meticulous handling of complex datasets. Partner with me to transform your ambitious project into a successful reality!

€300 EUR in 7 days

3.9

(3 reviews)

2.5

@shohelhossainme

As an experienced full-stack web developer and web scraping adept, I can confidently affirm my excellent fit for this project. Throughout my 6-year experience in the field, I have honed advanced skills in core technologies such as HTML5, CSS, JavaScript, PHP, and MySQL, aligned with an adeptness for scraping distinctive data from diverse sources including websites. I am well-versed in handling complex scraping processes that involve navigating through redirection scripts, proxy rotation for query scalability, and overcoming various barriers including cookies and leech protection. My proficiency with data management is an added advantage for your project's database integration aspect. My comprehensive understanding of MySQL ensures that I would be able to effectively and efficiently organize the scraped product data into your system. Not only that, but I am also well-skilled at altering specific data fields to align with your specific requirements. On top of my technical expertise, my commitment to delivering clean coded solutions that align with professional standards will ensure the quality, efficiency, and robustness of the code for this project. Christopher feel free to reach out to me if you have any further queries or concerns about my qualifications. Thank you for considering me for this esteemed opportunity.

€150 EUR in 7 days

5.0

(2 reviews)

1.9

@bilalrana12529

I have 4 years professional field experience and have multiple computer skills like WordPress development Data Entry (Products, CSV, MLS, Newsletter, Content, spreadsheets and databases) Office work, Proficiency in Google Docs, particularly Google Sheets/Excel. I am very passionate and have strong searching skills and solving complex problems. I am very interested to this work. I am a computer science graduate and totally understand your requirement. I am open for long-term work too. You will receive quality work. I'm very happy to bid your project. I am ready to discuss job description and give you an interview.

€140 EUR in 7 days

5.0

(5 reviews)

1.8

@AlexV4Dev

Hi, Sir. I am a programmer with exceptional skills in web scraping and database management. I have read your job description and I am very interested in working on your project. I can development of a comprehensive Intranet site to manage and parse the scraped data effectively. I have ever developed scrapper using Nodejs(puppeteer) to scrapping from pinterest and facebook. I hope to discuss with you more detail Thank you Alessandro

€100 EUR in 2 days

5.0

(1 review)

1.5

@Mxporter

Hello, I am a web crawler expert. I am confident that I am best suited to take on the job and complete it with the highest standards. I am experienced in web scraping, data extraction, HTML, and CSS and I have had completed similar projects before using the web scraping tool. I have focused on projects that require the retrieval of contact information from websites and extracting data from HTML and CSS. My attention to detail ensures accurate extraction of information from websites. My questions are: 1) What is the website URL? 2) Is a certain format of contact information required? 3) Is a specific data structure required? 4) What is the target of the contact information extraction? 5) Is there any specific timeline we should follow? I would love to be considered for this job and I am looking forward to hear from you soon. Best regards,

€240 EUR in 7 days