Got It

Overview of job

We are seeking a Senior Data Engineer to play a pivotal role in building our data platform from the ground up. You will be responsible for designing, developing, and deploying a scalable, secure, and open-source based data platform on the AWS cloud. This is a unique opportunity to shape the future of our data infrastructure and empower data-driven decision making across the organization. 

• Design and implement a robust data platform architecture on AWS, leveraging open-source technologies whenever possible.
• Evaluate and select appropriate open-source tools and services for data ingestion, transformation, storage, and analytics.
• Develop and maintain data pipelines for efficient movement and processing of large datasets. 
• Ensure data quality and consistency throughout the data lifecycle. 
• Implement data security best practices to protect sensitive information. 
• Collaborate with data scientists, analysts, and other stakeholders to understand data needs and translate them into technical requirements. 
• Automate data platform tasks and processes for improved efficiency and reliability. 
• Monitor and troubleshoot data platform performance to identify and resolve issues proactively. 
• Stay up-to-date on the latest trends and technologies in the data engineering space. 
• Document data platform design, architecture, and processes for future reference.