At Datadope, we’re seeking a passionate Data Engineer to design and build scalable, efficient cloud-based data architectures. If you love designing and optimising data pipelines, working with large-scale datasets, and collaborating with ML and engineering teams — we’d love to have you on board!
Key Responsibilities
- Design and implement scalable data pipelines for batch and real-time processing.
- Build and optimise cloud-based data architectures (AWS / Azure).
- Manage data flows from multiple sources ensuring quality and integrity.
- Implement and optimise data storage architectures (data lakes and warehouses: Snowflake, BigQuery, Redshift, ClickHouse).
- Develop and maintain messaging/streaming and orchestration systems (Kafka, Benthos, Airflow, DBT, NiFi).
- Clearly document processes, pipelines, and data architectures for easy reuse across teams.
- Collaborate with ML teams to provide data for training, inference, and monitoring.
- (Advanced plus) Experiment with vector databases and RAG techniques for generative data applications.
Requirements
- Degree in Computer Science, Engineering or related fields.
- 3+ years of experience designing and implementing cloud-based pipelines.
- Proficiency in SQL, Spark, Airflow, and Kafka.
- Experience with relational and NoSQL databases.
- Experience in real-time data processing (Flink or Spark Streaming).
- Familiarity with ETL tools (DBT, NiFi, Talend).
- Strong documentation and technical writing skills.
- Teamwork skills in high-demand environments.
Nice-to-Have
- Data security and governance knowledge.
- Experience in monitoring and performance optimisation of data systems.
- Familiarity with ML data modelling.
- Fluency in English for communication with international teams.