Kindly go through the requirement given below and kindly send me the matching profiles in word document with your minimum expected salary, visa status, current location and availability. You can please email the resumes to firstname.lastname@example.org.
Mail - email@example.com
We need to post a requirement for Cloud Data Engineer., Below are the job duties.
Develop Entity Relationship models and Star schemas for clinical data warehouse to support the needs of customer business analytics.
ð Create data models for MATCH & CTRP using Visual Paradigm.
ð Define data integrity constraints between the entities.
ð Create Aurora MySQL database out of the export SQL from ERD model.
§ Perform ETL activities to process the data from MongoDB and Dynamo DB to Aurora relational database.
ð Writing Lambda functions to process the clinical data from source systems (Mongo DB JSON document collections) to Aurora staging layer.
ð Creating PySpark ETL jobs to process the data into the clinical data warehouse, which supports customer’s reporting dashboard analytics.
§ Develop and perform complex SQL-based queries against the DWH, which is being built in Amazon Redshift database.
ð Create Redshift database defining Sort & DIST keys for the tables
ð Enable encoding for the tables to speed up the data retrieval.
ð Perform vacuum operations regularly on Redshift database.
§ Leverage Amazon Redshift for DWH activities
ð Create views to access the data from reporting.
ð Create custom SQL queries required for data visualization
ð Create ELT script to populate the data across the tables in Redshift.
§ Work in Linux and OS/X based environments
ð Write shell scripts to move JSON & flat files data across different platform servers.
§ Prepare migration document to move the AWS lambdas from development to testing and then to production environments.
§ Collaborate and communicate with other team members towards the development of high-quality technical deliverables.
§ Provide documentation for all developed data models, including process flow and source and target mapping.
§ Perform test plan preparation and unit and system testing.
§ Conduct performance tuning and troubleshooting.
Bachelors in computer science or closely related field.
Years of experience: 1-2 years