Hi, I have reviewed the project details and understand your requirement of a Python, PySpark developer who has expertise in interactive SQL development and PySpark ETL Pipeline. I am capable of completing this task in a timely and cost-effective manner. Let's connect and discussion more on it.
Short Into about Me:
Currently working in one of the Big4 Companies and solving Big Data Business problem using AWS Cloud and Pyspark Engine.
I have expertise on the following Tools/Technologies:
- Apache Spark (PySpark)
- Python3, unit test script, Numpy, Pandas
- AWS Cloud Services (EMR, DynamoDB, S3, Glue Lambda, EBS, Athena, IAM, Step function, EC2, ECS, secret manager (kms), s3, sns, ses, sqssqs, cloudwatch etc)
- Creation of cloud formation templates for various AWS Services
- CI/CD Pipelines
- SQL/MySQL
- Git, GitHub, BitBucket
- Big Data Pipelines
- IBM DB2
- MS Excel, MS Office
- PyCharm, VSCode, Databricks, Jupyter Notebook, MS Excel, Putty
Please provide more details if it's more than what you have given in the project details. Thank you!
Thanks & Regards,
Prakash