Find Jobs
Hire Freelancers

Node save web page and send to s3 with npm website-scraper and aws-sdk packages

$30-250 USD

Completed
Posted over 5 years ago

$30-250 USD

Paid on delivery
Write a node script which scrapes a webpage using the npm package website-scraper [login to view URL] you must use your own AWS account, the script will use [login to view URL]() to set credentials Use default options and save just the one page / URL to a tmp folder Then use the AWS SDK ( [login to view URL] ) and send the folder to an s3 bucket. For each file set the correct content-type Here is some code to get you started var recursive = require("recursive-readdir"); const options = { urls: ["[login to view URL]"], directory: "/tmp/yahoo" }; scrape(options, (error, result) => { recursive([login to view URL], function(err, files) { for(i = 0; i< [login to view URL];i++){ file = files[i]; file = [login to view URL](1); var params = { Body: // set file content here Bucket: "bucket_name", Key: file, ContentType: // set content type here ACL: 'public-read', }; [login to view URL](params, function(err, data) { if (err) [login to view URL](err, [login to view URL]); // an error occurred else [login to view URL](data); // successful response }); } }); }); Budget is less than $100
Project ID: 18275015

About the project

3 proposals
Remote project
Active 5 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
I am a hobbyist JavaScript developer experienced in NodeJS. I have also worked on a couple personal projects which have involved website scrapping. For this reason, I believe I would be a good fit for this project. For a examples of my past work, I would encourage you to take a look at my other personal projects on my github page [login to view URL] Please message me with further details about the project to see if I can help. Thanks for your consideration, Samuel
$94 USD in 5 days
5.0 (2 reviews)
3.2
3.2
3 freelancers are bidding on average $148 USD for this job
User Avatar
Hi there, I am Node.js AWS expert, I can scrap the the website with your required modules and can save it to S3 bucket and can give you demo on my machine before delivering the code base. I have done several projects with AWS serivces like S3, Lambda, Kinesis, SQS,SES, SNS, EC2 etc. you may have look at my profile. I have done several projects in Node.js with AWS services. I have 4+ years of experience in software development. I am Sr. Full Stack Developer having experience in Node.js, REST APIs, JavaScript, AngularJS(1/2/4/5),ExpressJS, Google Cloud Function,Google Firebase function, AWS ,SQL(MySQL, PostgreSQL, MS SQL Server, Sqlite3), NoSQL(MongoDB, CouchDB), Linux, Git,Nginx. I am good to deliver your code base in $100 Looking forward to work with you Thanks...
$111 USD in 1 day
5.0 (29 reviews)
4.7
4.7
User Avatar
I have worked with node scraped pages using python So it will be an opportunity for me to save webpages via node. I have my AWS account and I am Cloud Engineer by profession ensuring continuous Integration and Delivery of services using AWS and automation tools like Chef, Jenkins, Docker, Kubernetes Specialities: Continuous Integration, Continuous Delivery, Docker Containers, Container orchestration platforms, Infrastructure Automation, Microservices, Monitoring and log management List of Software, Tools used/working on • Automation - Chef, Vagrant, Docker, Kubernetes • CI/CD - Jenkins • Amazon Web Services CloudWatch, S3, EC2, Load Balancers, Auto Scaling, VPC, VPC Peering, VPC - Site VPN, IAM, Cloud Front, Elastic Search, RDS, SES, SQS, SNS, ElastiCache, RDS, EC2 Container Service
$240 USD in 3 days
5.0 (2 reviews)
2.2
2.2

About the client

Flag of UNITED STATES
Mount Sinai, United States
5.0
32
Payment method verified
Member since Jul 30, 2010

Client Verification

Similar jobs

I need help deploying a NetCore + React app to IIS. it works fine on my local but for some reason it does not work on IIS after deplñy
$15-25 USD / hour
Looking for experienced developer in AWS and VB
$30-250 USD
Full-Stack Dev for Music Distribution Startup
₹1500-12500 INR
AWS Lambda & API Gateway for Puppeteer
$10-30 USD
AWS Bedrock Frontend GUI Development - full stack developer
$30-250 USD
React/AWS/Node API Development & Enhancement
$750-1500 USD
Nodejs Scraping Expert Needed
$30-250 USD
EC2 Server Security Enhancement
₹600-1500 INR
Advise on setting up serverless app (AWS)
$30-250 USD
IoT Platform MQTT, Node.js, Next.js with Dynamic Dashboard
$250-750 USD
MQTT Server & Client Setup
$10-30 AUD
AWS Expert required to clear issues from NextJs application Improper Error Handling(Server Banner Disclosure) and Misconfigured CORS
$15-25 USD / hour
Fresh Amazon Account Creator
£10-20 GBP
New hire employee background verification
₹37500-75000 INR
Infographics-Inclusive Business Information Website
₹600-1500 INR
ERPNext Ecommerce with Payment Gateway
₹12500-37500 INR
Deploying AWS CDK build project using Jenkins CI/CD pipeline
$10-20 USD
DevOps engineer to deliver freelance support to free developers from the devops role. We want a skilled experienced freelancer with AWS experience must know Docker, Kubernetes, CD/CI GIT Linux and be able to problem sovle and deliver quickly
£18-36 GBP / hour
Ethereum Blockchain and Dapp Development
₹12500-37500 INR
AWS Architecture Speed Optimisation Consultation
min $50 AUD / hour
Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.