Company name: ​    Blackstraw.ai

Office location: ​      Chennai

Job Type:​                   Full-time

Experience:​              5 to 7 years

Job Description:

The candidate should have extensive experience in designing and developing scalable data pipelines and
real-time data processing solutions. As a key member of the team, the Senior Data Engineer will play a
critical role in building end-to-end data workflows, supporting machine learning model deployment, and
driving MLOps practices in a fast-paced, agile environment.

Strong expertise in Apache Kafka, Apache Flink, AWS Sage Maker, and Terraform is essential. Additional experience with infrastructure  automation and CI/CD for ML models is a significant advantage.

Key Responsibilities:

• Design, develop, and maintain high-performance ETL and real-time data pipelines using Apache Kafka and Apache Flink.
• Build scalable and automated MLOps pipelines for training, validation, and deployment of models using AWS Sage Maker and associated services.
• Implement and manage Infrastructure as Code (IaC) using Terraform to provision and manage AWS environments.
• Collaborate with data scientists, ML engineers, and DevOps teams to streamline model deployment
  workflows and ensure reliable production delivery.
• Optimize data storage and retrieval strategies for large-scale structured and unstructured datasets.
• Develop data transformation logic and integrate data from various internal and external sources into data lakes and warehouses.
• Monitor, troubleshoot, and enhance performance of data systems in a cloud-native, fast-evolving production setup.
• Ensure adherence to data governance, privacy, and security standards across all data handling activities.
• Document data engineering solutions and workflows to facilitate cross-functional understanding and ongoing maintenance.

Required Qualifications:

• Bachelor’s degree in Computer Science or related field.
• Extensive experience in building data pipelines and streaming applications using Apache Kafka and Apache Flink.
• Strong experience in ETL development, data modeling, and managing data in large-scale environments.
• Proficient in AWS services including Sage Maker, S3, Glue, Lambda, and CloudFormation/Terraform.
• Hands-on expertise with MLOps best practices, including model versioning, monitoring, and CI/CD for ML pipelines.
• Proficiency in Python and SQL; experience with Java is a plus for streaming jobs.
• Deep understanding of cloud infrastructure automation using Terraform or similar IaC tools.
• Excellent problem-solving skills with the ability to troubleshoot data processing and deployment issues.
• Experience in fast-paced, agile development environments with frequent delivery cycles.
• Strong communication and collaboration skills to work effectively across cross-functional teams.

Company Profile:

Blackstraw.ai is an end-to-end technology services company specializing in Artificial Intelligence (AI) and Engineering solutions across Data Science, Data Engineering, LLM/GenAI and LLMOps.

Founded in 2018, we help global enterprises across North America, Europe and Asia to build and operationalize AI systems that create measurable business impact. Our mission is to make AI adoption simpler, faster and scalable through a blend of deep domain expertise, reusable accelerators and proven engineering practices.

With a 450+ strong team of engineers, data scientists and AI specialists, we partner with organizations to deliver real-world outcomes in areas such as predictive analytics, computer vision, natural language processing and Generative AI.

At Blackstraw.ai, we’re passionate about solving complex business challenges through intelligent automation, modern data architectures and next-gen AI applications – enabling enterprises to move from AI experimentation to transformation.

Headquartered in Florida (USA) with operations in Canada and India, Blackstraw.ai continues to empower global enterprises to unlock the true potential of AI.