Data Engineering and QA Team Lead

We are Semrush, a global IT company developing our own product – a platform for digital marketers. New stars are born here, so don’t miss your chance. This is our Data Engineering and QA Team Lead role for those who want to turn ideas into reality and participate in improving Semrush’s analytics tools.

Tasks in the role

Leadership of the team of Data Engineers and QA Engineers to maintain and improve:

• Data Architecture
• Cloud Infrastructure
• CI/CD
• Data Pipelines / Data Modelling
• Data Quality
• QA practices
• Monitoring & Alerting
  
  
  

Team functions:

• Creation Data Architecture Strategy
• Improvement of Incident Management
• Develop and deploy IaaC using Terraform to ensure scalable and reliable data systems
• Create and manage CI/CD pipelines in GitLab for automated testing and deployment of data solutions
• Design, develop, and maintain data pipelines using BigQuery, Airflow & dtb
• Monitor and troubleshoot cloud infrastructure, data pipelines, and workflows to ensure data quality and availability
• Implement data models and optimize data storage, retrieval processes, and costs
• Migrate existing data infrastructure & pipelines from the legacy to the contemporary solutions
• Development and maintenance of ML / AI ecosystem / tools / workflows
• Conduct Code Reviews for Merge Requests (80% SQL, 10% Python, 10% Java)
• Collaborate with analysts, SRE, and other engineers to understand data needs and deliver data solutions
• Conduct Arch Review to enable robust & flexible data architecture
• Propose & drive efficiency-improvement initiatives
• Track and optimize costs of data infrastructure
• Crafting of QA Strategy for Analytics Division
• Orchestrate Testing Activities in the Analytics Division for Dashboards, Data Transformations, Data, Requirements etc.
• Definition & Delivery of Data Quality Metrics
• Introduction of Automation Testing
  
  
  

Biggest projects for the upcoming 6 months:

• Infrastructure migration between cloud regions
• Introduction of DWH 2.0
• Deployment & customization of Data Catalogue & Data Lineage (could be datahub, atlas, or similar)
• CI/CD improvement to downstream checks
• Introduction of QA Strategy
  
  
  

Who we are looking for

Hard Skills

• Hands-on IaaC tools (we use Terraform)
• Proficient in Version control systems (we use GitLab)
• Proficient in CI/CD tools (we use GitLab)
• SQL senior level (proficient in windows functions, optimizations)
• Java & Python middle level
• Proficient in Pipeline Orchestration, incl. Deployment, management and usage (we use Airflow, dbt)
• Proficient in Monitoring & alerting (we use Grafana, Opsginie, Slack)
• Task Management (we use Jira)
• Documentation management (we use Confluence)
• Containers (we use Kubernetes)
  
  
  

Soft Skills

• High project management skills: end-to-end value delivery of data projects: from business requirements gathering, through requirements clarification, requirements decomposition, clarification, to project roadmapping, prioritizing, tasks decomposition, execution, execution tracking, delegation, project communications, project closure, business value delivery
• Proactiveness
• Readiness to make efficient decisions by themselves with minimal micromanagement
• Strategic thinking
• Ability to solve high-level business problems either by themselves or by involving other teammates
• Excellent problem-solving skills and attention to detail
• High Communication Skills
• Stakeholder Management (establishing positive & reliable partnerships)
  
  
  

Experience

• 3+ years of DE experience (or Backend/DataScience 2+ years & DE 1+)
• 2+ years of experience with one of the Top-3 Cloud Providers
• 1+ years of experience in people management
  
  
  

Not required, but a plus

Business Experience

• Experience as DE for b2c SaaS
  
  
  

Tech Experience

• Airflow
• dbt
• Software engineering
• API management
• Tableau / Power BI
• You share our common values: Trust, because we prefer to speak up and be our true selves; Sense of Ownership, because it’s not worth wasting time on something you don’t believe in; and enthusiasm for Constant Changes, because we are always looking to make things better
  
  
  

We will try to create all the right conditions for you to work and rest comfortably

• This offer stands for the remote work format. Digital nomadism, #wfh – call it what you like ;)
• Flexible working day start
• Unlimited PTO
• Hobby benefit
• Breakfast, snacks, and coffee at the office
• Corporate events
• Training, courses, conferences
• Gifts for employees