Description:
Total Exp: 8+yrs
Location: Bangalore (Hybrid) / Remote : Rest of India
Joining Preference: Immediate to 15 days joiners
If interested, Share your CV at:
About the Role
We are hiring Senior Data Engineers to lead large-scale AWS migration and data platform initiatives. This is a senior individual contributor role focused on migrating legacy file storage and SQL Server databases into AWS-native environments while ensuring data integrity, scalability, and operational reliability.
You will drive architecture decisions, build migration and reporting pipelines, and define data quality and cutover strategies in a Kubernetes and AWS-based ecosystem.
Key ResponsibilitiesData Migration
- Lead migration of:
- NAS/NFS/SMB file storage to Amazon S3
- SQL Server databases to Aurora PostgreSQL / RDS
- Handle large-scale file migrations (100+ GB files)
- Design incremental sync, reconciliation, checksum validation, rollback, and cutover strategies
- Build multi-hop data flows between S3, Kubernetes processing clusters, and analytics platforms
Data Engineering & Reporting
- Design and build AWS-based ETL/ELT pipelines
- Develop ingestion, transformation, and reporting workflows
- Enable analytics using Redshift, Athena, and Aurora
- Optimize pipeline performance, scalability, and cost
Data Quality & Reliability
- Build validation, reconciliation, and quarantine frameworks
- Implement monitoring, retry, and escalation workflows
- Support DR testing, performance tuning, and migration validation
Infrastructure & CI/CD
- Work within Terraform-managed AWS environments
- Contribute to GitLab CI/CD pipelines
- Support security and compliance initiatives across IAM, KMS, encryption, and secrets management
Required Skills
- 8+ years of Data Engineering experience
- Strong experience with:
- NAS/NFS/SMB → S3 migrations
- SQL Server → Aurora/PostgreSQL migrations
- Strong Python and SQL expertise
- Hands-on experience with:
- boto3
- SQLAlchemy
- ETL scripting
- Strong AWS knowledge:
- S3
- Aurora/RDS
- IAM
- KMS
- CloudWatch
- ECS/EKS
- Experience with Docker, Kubernetes, GitLab CI/CD, and Terraform
- Ability to independently drive technical and architectural decisions
Preferred Skills
- Spark, Airflow, or Kubernetes-based data pipelines
- Exposure to Go or Java microservices
- Pytest, Playwright, or K6 testing frameworks
- Blue/green cutover and rollback planning experience
- Experience supporting ML/data science workloads