Contexte du recrutement et définition de poste
We are looking for an experienced DataOps Engineer to join our team. This role focuses on building and maintaining robust on-premise cloud infrastructures and data pipelines. You will work with a diverse range of tools and technologies, ensuring the reliability, scalability, and efficiency of our data and cloud ecosystems.
Key Responsibilities :
- Data Stack Infrastructure Management : Deploy and maintain on-premise cloud environments using Hadoop , Kubernetes , Helm , and Minio .
- CI / CD Implementation : Build and maintain infrastructure as code (IaC) with Terraform , automate configuration management with Ansible , and streamline CI / CD workflows using Jenkins and GitLab CI .
- Monitoring and Observability : Oversee system performance using monitoring tools such as Grafana, Kibana, and ELK Stack to proactively address issues and ensure system health.
- Data Operations Management : Oversee the performance of data pipelines and implement proactive measures to address issues.
- Collaboration : Work closely with data engineers, data scientists, and stakeholders to align data delivery processes with business goals.
- Machine Learning Workflow Optimization : Integrate tools like MLflow to manage and streamline machine learning workflows.
Profil recherché
Bachelor’s degree in Computer Science, Data Engineering, or a related field.Proficiency in managing on-premise cloud environments with Kubernetes , Helm , and Minio .Strong experience with Hadoop ecosystems for big data processing.Experience with CI / CD tools, specifically Jenkins and GitLab Actions .Familiarity with task scheduling and workflow orchestration tools such as Apache Airflow and Celery .Hands-on expertise in Terraform and Ansible for infrastructure automation.Strong problem-solving and communication skills.Bonus :
Experience with monitoring and observability tools like Grafana, Kibana, and ELK Stack.Experience with MLflow for machine learning lifecycle management is a plus.Knowledge of Kafka for real-time data streaming and messaging.