Middle Data Engineer (with Elasticsearch)
The CHI Software team is not standing still. We love our job and give it one hundred percent of us! Every new project is a challenge that we face successfully. The only thing that can stop us is... Wait, it’s nothing! The number of projects is growing, and with them, our team too. And now we need a Middle Data Engineer (with Elasticsearch).
Requirements:
- At least 5 years building and extending complex data systems: data flows, lakes, warehouses, ETLs, and tooling on a variety of cloud platforms;
- Very strong Python skills, with at least 3 years of experience;
- Strong SQL writing and tuning skills;
- Expert-level understanding of Elasticsearch, including: Configuration;
- Index Management: Creating and optimizing indices for performance and storage efficiency, including the use of index templates and mappings;
- Query Optimization: Building Elasticsearch queries using the Query DSL and optimizing them for performance, including aggregations, filters, and full-text search;
- Data Ingestion: Ingesting data into Elasticsearch from various sources, including RDBMS;
- Experience building data pipelines with at least two industry-standard tools (e.g., Spark, Hive, Airflow, Stitch);
- Demonstrated experience with data pipeline observability strategies and implementations;
- Keen understanding of how to apply modern software development practices (Agile, TDD, CI/CD) to data engineering efforts.
- Administration
Performance Monitoring
Cluster Management
Security
Backup and Restore
Version Management
Scaling
Integrations
Will be a plus:
- Experience building or maintaining streaming platforms with Kafka;
- Experience with PySpark;
- Skills in implementing containerization solutions using Kubernetes, Docker, etc., including containerization observability;
- Experience working with “Big Data as a Service” tools, like AWS’s EMR or Azure’s HDInsight;
- Building and extending ML and Data Science platforms;
- Experience integrating data flows with 3rd party BI platforms, like Domo, Tableau, Sisense, etc.