Data Engineer · Skills Database

About the role

At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio. We are currently seeking an Data Engineer to join one of our clients' teams. If you're looking for an exciting opportunity to grow in a innovative environment, this could be the perfect fit for you. Requirements Key Responsibilities: Design, develop, and maintain data ingestion pipelines using Kafka Connect and Debezium for real-time and batch data integration. Ingest data from MySQL and PostgreSQL databases into AWS S3, Google Cloud Storage (GCS), and BigQuery. Implement best practices for data modeling, schema evolution, and efficient partitioning in the Bronze Layer. Ensure reliability, scalability, and monitoring of Kafka Connect clusters and connectors. Collaborate with cross-functional teams to understand source systems and downstream data requirements. Optimize data ingestion processes for performance and cost efficiency. Contribute to automation and deployment scripts using Python and cloud-native tools. Stay updated with emerging data lake technologies such as Apache Hudi or Apache Iceberg. Required Skills and Qualifications: 5+ years of hands-on experience as a Data Engineer or similar role. Strong experience with Apache Kafka and Kafka Connect (sink and source connectors). Experience with Debezium for change data capture (CDC) from RDBMS. Proficiency in working with MySQL and PostgreSQL. Hands-on experience with AWS S3, GCP BigQuery, and GCS. Proficiency in Python for automation, data handling, and scripting. Understanding of data lake architectures and ingestion patterns. Solid understanding of ETL/ELT pipelines, data quality, and observability practices. Good to Have: Experience with containerization (Docker, Kubernetes). Familiarity with workflow orchestration tools (Airflow, Dagster, etc.). Exposure to infrastructure-as-code tools (Terraform, CloudFormation). Familiarity with data versioning and table Originally posted on Himalayas

Skills mentioned in the listing

PythonDockerKubernetesAWSGoogle CloudTerraformPostgreSQLMySQLETLRecruitment

Recognised skills are extracted from the supplied description using canonical names and aliases. They are mentions, not verified essential requirements. Check the source advert for the employer’s full requirements and location restrictions.

Why these skills appear

Python · requirement-context
nce and cost efficiency. Contribute to automation and deployment scripts using Python and cloud-native tools. Stay updated with emerging data lake technologies such as Apache Hudi or Apache Iceberg. Required Skills an

Docker · requirement-context
and observability practices. Good to Have: Experience with containerization (Docker, Kubernetes). Familiarity with workflow orchestration tools (Airflow, Dagster, etc.). Exposure to infrastructure-as-code tools (Ter

Kubernetes · requirement-context
ervability practices. Good to Have: Experience with containerization (Docker, Kubernetes). Familiarity with workflow orchestration tools (Airflow, Dagster, etc.). Exposure to infrastructure-as-code tools (Terraform,

AWS · mentioned
d batch data integration. Ingest data from MySQL and PostgreSQL databases into AWS S3, Google Cloud Storage (GCS), and BigQuery. Implement best practices for data modeling, schema evolution, and efficient partitioning

Google Cloud · mentioned
data integration. Ingest data from MySQL and PostgreSQL databases into AWS S3, Google Cloud Storage (GCS), and BigQuery. Implement best practices for data modeling, schema evolution, and efficient partitioning in the

Terraform · mentioned
tion tools (Airflow, Dagster, etc.). Exposure to infrastructure-as-code tools (Terraform, CloudFormation). Familiarity with data versioning and table Originally posted on Himalayas

PostgreSQL · mentioned
Debezium for real-time and batch data integration. Ingest data from MySQL and PostgreSQL databases into AWS S3, Google Cloud Storage (GCS), and BigQuery. Implement best practices for data modeling, schema evolution,

MySQL · mentioned
nnect and Debezium for real-time and batch data integration. Ingest data from MySQL and PostgreSQL databases into AWS S3, Google Cloud Storage (GCS), and BigQuery. Implement best practices for data modeling, schema e

ETL · requirement-context
ding of data lake architectures and ingestion patterns. Solid understanding of ETL/ELT pipelines, data quality, and observability practices. Good to Have: Experience with containerization (Docker, Kubernetes). Famili

Recruitment · mentioned
Data Engineer At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio. We are currently seeking an Data Engineer to join one of our clients' teams. If you're looking for an exciting