Find Jobs
Hire Freelancers

Web Scraping to Database

$30-250 USD

Completed
Posted over 2 years ago

$30-250 USD

Paid on delivery
I need a PHP or Python script which will be run periodically to scrape the data I describe in this project and store it locally in a MySQL database running in a typical LAMP environment on a Debian 11 server. The script should start by emptying the table to be used (TRUNCATE), the populate the table only with the results of the new data scraping described in this project. The source data is on a public county GIS mapping server and is strictly limited to the owners of property in the subdivision for which this script is being written. Currently, we do the very tedious work of manually updating our data. To get to the data I need manually, I use these steps: 1: Open web page: [login to view URL] 2: Use the Search Field and enter "Upper Little Warm Spring Creek" 3: On the search results page, click "Select All", then click "Map Selection" 4: The resulting page is shown in the attached screenshot and the purple circles are where the data is--as explained below. PLEASE NOTE that the number of results may change from time to time, so we cannot simply use the URL that results from the search. 5: Use the "Copy to Clipboard" button on the web page to obtain the primary data. 6: Use the "Property Detail" link next to each of the property parcel entries to obtain further data to be added to the primary data. I need this to be fully automated so that simply accessing the script will scrape and update all data in my database. First... After getting to the correct data set, the results of the "Copy to Clipboard" button (circled in purple on attached screenshot) will be 23 tab delimited values for each record, from which I need the first 14 columns which need to be populated into a MySQL database table using the following column names: Parcel, Parent, Account, TaxID, Plat, Lot, Owner, Mail_Street, Mail_CSZ, St_Addr, Deed, Location, Tax_Class, Acerage Once the main data is imported, then every record's "Property Details" link needs to be accessed to get more data for the records already collected (2 "Property Details" links circled in purple on attached screenshot). From each "Property Details" record, I need the following additional columns for each of the records created from the "Copy to Clipboard" data: MV_Year, MV_Total, MV_Land, MV_Improve, AV_Year, AV_Total These values are found by searching the result from each Property Details link for the FIRST line that contains "Market Value" (MV) and the FIRST line that contains "Assessed Value" (AV) extracting the data from lines similar to this example: 2021 Market Value: $ 450,273 ($ 83,886 Land + $ 366,387 Improvements) 2021 Assessed Value: $ 42,776 results in these values: MV_Year = 2021 MV_Total = 450273 MV_Land = 83886 MV_Improve= 366387 AV_Year = 2021 AV_Total = 42776 As a result of both steps each line of the database table should include these columns: Parcel, Parent, Account, TaxID, Plat, Lot, Owner, Mail_Street, Mail_CSZ, St_Addr, Deed, Location, Tax_Class, Acerage, MV_Year, MV_Total, MV_Land, MV_Improve, AV_Year, AV_Total I am available to explain further as needed and to participate in testing. Please do NOT ask me for access to my server as this script needs to run in most any current Linux-based LAMP environment Desired deliverables: 1) MySQL "CREATE TABLE" script to set up the database table I describe in this project. 2) PHP or Python script to accomplish the tasks described in this project. Great performance on this project will likely lead to further projects from me as well.
Project ID: 32283207

About the project

15 proposals
Remote project
Active 2 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
$0 USD in 2 days
5.0 (1179 reviews)
8.6
8.6
15 freelancers are bidding on average $242 USD for this job
User Avatar
Hello ! I am professional Python coder. I am familiar with Python and I have a lot of work experiences in Web Scraping, Python, PHP and MySQL. I can start right away. I want to discuss for this project in detail. Please contact me to discuss more regarding this project. Hope to hear from you soon
$150 USD in 5 days
5.0 (32 reviews)
6.0
6.0
User Avatar
Hi there scraping expert here I've scraped 45000+emails in here(If you want to see, I can show you) I'm quite familiar with Selenium and beautifulsoup in python I've already scraped many shopping sites before if you choose me you can save your time pls ping me Thanks
$200 USD in 3 days
4.9 (13 reviews)
4.0
4.0
User Avatar
Hello there, I can use Selenium to help you scrape the data you need :) It will be ready within 24 hours hopefully and you can use it to replace the manual work you are doing. I can also make it in PHP depends on your preference just let me know! Looking forward to hearing from you.
$150 USD in 2 days
5.0 (7 reviews)
4.0
4.0
User Avatar
Hello. How are you? I am excellent web scraper and I have nice experience in web scraping by using python and node.js, etc. So I am sure I can give you perfect results. I hope to discuss via private chat. I can start working any time you want. Thanks. Best Regards.
$140 USD in 3 days
4.8 (8 reviews)
3.8
3.8
User Avatar
Hello I am a senior developer and I have full experiences in this field. Web scrapping is my major skill and I am very interested in your job. There is no problem about the create database table. There are only one problem in this project. The problem is to analysis the data content in the php and get the correct data. So I can do it. But please pay attention about the budget. Because I am expert and I can do it successfully. Thanks & Regards. Roman
$1,200 USD in 7 days
4.8 (3 reviews)
3.6
3.6
User Avatar
How are you?. I have read your description carefully and very interesting in your project. I'm a web scraping developer with a decade of experience developing software across a wide variety of platforms. My focus now is on distributed systems and back-end development with C#, Python, and Node.js. You can see my top skill about web scrapping on my profile. By hiring me, you will know that you are working with a developer who is working sincerely. I am looking forward to your reply. Best regards.
$140 USD in 7 days
5.0 (3 reviews)
2.9
2.9
User Avatar
Hi, I'm Goli, I read your description and saw the attachment. And I can complete your project with a PHP script that automatically saves data in the database. Please request a chat to negotiate the price and time frame in detail. I'm Proficient in PHP, SQL, SQL SERVER, My SQL, and we could have long-term accomplishments. looking forward to hearing you. Warm regards
$245 USD in 5 days
5.0 (3 reviews)
2.9
2.9
User Avatar
Dear Client Thank you for your project. I've just checked your job description carefully. I'm senior developer with 9+ years of Python. By using Python, I developed AI engine, BOT, Web Scraping Tools, Web Searching Tools AI chat Bot and so on. Python is my major so you will be satisfied. I'm very familiar with Django and Flask, and also I have rich experiences in Selenium, Beautifulshop, and Scrapy I always find my happy and pleasant within client's mind whenever the they are satisfied. So I try harder and harder to develop the projects perfectly. If someone asks me about my happy and pleasant, I always will talk that when the clients are satisfied as I develop their projects excellently and perfectly, I am very happy. I would love to work with you. I can complete your project on time and within your budget. I hope you contact me soon. Thank you. Best Regards.
$250 USD in 7 days
5.0 (1 review)
1.9
1.9
User Avatar
***Scraping Expert*** Hi, pleased to apply for this project. I read your job description and you are seeking a scraping expert. I have vast experience in web scraping with python(Scrapy), C#, JavaScript(Node.js) for more than 10 years. My solid knowledge of algorithm, Neural network, ML will give you the high-quality code. DB management skill(MySQL, PostgreSQL, MongoDB, MS Access) and Cloud computing experience such as AWS, Azure DevOps, Google Cloud will be helpful for the success of this project. My main development tool is GitHub, I am an expert in Slack, Agile, Scrum, Trello. Thanks for this job posting. Regards.
$140 USD in 7 days
5.0 (2 reviews)
0.9
0.9
User Avatar
Hello! I'm glad you reached out! We have experience✅, professional staff✅ and we are highly motivated! ✅ What else do you need? ✅ We have a lot of experience with: MySQL, Web Scraping, PHP and Python ✅ If you activate the chat, I will help you personally! ✅ I will answer all your questions! ✅ I will send you examples of similar work done! ✅ We are a team of 28 highly motivated professionals We are at your disposal! Note: The value and final delivery time will be determined after an agreement is reached.
$223 USD in 26 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hey Mate! I am a web scraping and automation expert. Very familiar in PYTHON and Node.js. It is an enough experiences for me like Web Scraping, PHP, MySQL and Python. I can start right away. I am coding from home. I am flexible with my work hours because freelancing is my passion, and I will not stop working until you're fully satisfied. Please contact me to discuss more about this project. With regards
$100 USD in 3 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Dubois, United States
5.0
35
Payment method verified
Member since Oct 6, 2009

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.