Job Title: Senior Azure Data Engineer Location: Austin, TX / Remote Flexibility
Experience: 6+ Years
Employment Type: Full-Time
Job Summary We are seeking a highly skilled and certified
Azure Data Engineer with over 6 years of experience in architecting, developing, and optimizing cloud-based data solutions. The ideal candidate has extensive hands-on experience across
Azure, AWS, Snowflake, and Databricks, and is proficient in building scalable data pipelines and integrating advanced analytics workflows in hybrid cloud environments.
Key Responsibilities - Design, develop, and manage scalable ETL/ELT pipelines using Azure Data Factory, Databricks, and Azure Synapse Analytics.
- Architect and implement real-time data streaming workflows using Azure Event Hub, Logic Apps, and Spark Structured Streaming.
- Build and manage Data Lake and Delta Lake solutions, applying data partitioning, Z-ordering, and optimization techniques.
- Develop distributed data processing workflows using PySpark, Spark SQL, and Scala within Databricks and HDInsight.
- Implement data warehousing solutions using Snowflake and integrate with both Azure and AWS ecosystems.
- Manage real-time and batch data ingestion pipelines across multiple sources including relational, semi-structured, and unstructured data formats.
- Utilize Azure Key Vault, AWS IAM, and Snowflake RBAC for securing data pipelines and environments.
- Design and deploy CI/CD pipelines using Azure DevOps, Git, and Terraform for robust development and deployment workflows.
- Collaborate with cross-functional teams (DevOps, Analysts, Business) in Agile environments to ensure successful delivery of analytics solutions.
- Monitor and troubleshoot systems using Azure Monitoring, Application Insights, and AWS CloudWatch.
Required Skills & Qualifications - Proven experience with Azure Data Services: ADF, Databricks, Data Lake Gen2, Synapse Analytics, Azure SQL, Event Hub.
- Strong working knowledge of AWS Data Tools: AWS Glue, EMR, Redshift, Athena, Kinesis, SageMaker.
- Expertise in Snowflake (DBT, SnowSQL), Python, PySpark, and SQL for data transformation and analytics.
- Proficiency with big data and stream processing frameworks: Kafka, Spark, Delta Live Tables.
- Knowledge of dashboarding and analytics tools: Power BI, Tableau, and AWS QuickSight.
- Experience with DevOps practices and tools: Azure DevOps, GitHub, Bitbucket, JIRA, Terraform.
- Solid understanding of data security, compliance, and cloud governance principles.
- Bachelor's in Computer Science/Engineering or equivalent; Master's in Business Analytics preferred.
Preferred Certifications - Microsoft Certified: Azure Data Engineer Associate
- AWS Certified Data Analytics - Specialty
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.