Data Engineer

Skills and competencies

• Knowledge of Python and Big Data Processing using Pyspark
• 3-5 years of cloud-based data engineering experience/databases like AWS Data Services like Glue, Lambda, Step functions, DynamoDB, RDS – Postgres, Event triggers, Lake formation
• Ability to understand and operate on CICD Pipelines using Jenkins, Cloud Formation, Terraform etc.
• Ability to build and motivate teams through times of change.
• Excellent communication skills with respect to both technical and non-technical audiences at various levels of the organization
• Passion for end-to-end solutions and delighting the end user through effective listening and expectation management

Education

BE/BTech Computer Science, Engineering, or a related field is required.

Role and resposibilities

• Create AWS Cloud Native components-based solutions for Data Mesh Implementation – Ingestion as a Service module
• Build and enhance the Spark Based Framework for ingestion needs
• Leverage Functional Programming approach to all the modules and Build a platform enabled with different microservices that will be cater to data mesh domain needs
• Help develop baseline technology standards, tools platforms and processes in support of the overall data strategy.
• Work closely with business and technology colleagues to create platforms built using modern, consumer focused technologies
• Develop, build and test data engineering pipelines.
• Contributes to data and IT projects as a business-minded analyst, performing full lifecycle analysis including data profiling, requirements gathering and documenting, data mappings, process and data design flows, data quality controls input, and testing assistance.
• Assist with conceptual data modeling concepts to align to data domains   
• Understands and can communicate data management principles such as Master Data Management, Critical Data Elements, Granularity, Normalization, etc

About the team

Data Engineering will help with the modernization and transformation of the data ecosystem within Moody’s Ratings. This person will build, maintain a range of data initiatives across various business areas and drive the build out modern cloud-based data engineering practices.