Spark Scala Data Engineer

About Us

MediaJel is a dynamic software technology company specializing in the digital marketing space. With a dedicated team of professionals, we are on an exciting growth trajectory. We are proud of our innovative approach to technology, utilizing modern tech stacks such as React, Node, GraphQL, Spark, and more. We are also integrating Artificial Intelligence into our platform using Python and OpenAI, placing us at the forefront of technological innovation.

Job Summary:

We are seeking an experienced professional to play a pivotal role in designing, developing, and maintaining our data processing infrastructure. If you have a proven track record in handling large-scale data pipelines, optimizing data workflows, and contributing to the growth of data-centric solutions, we invite you to apply and be part of our innovative journey.

Responsibilities
Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.).
Work closely with our engineering team to integrate and build algorithms.
Process unstructured data into a form suitable for analysis – and then do the analysis.
Support business decisions with ad hoc analysis as needed.
Extract data from a variety of relational databases, manipulate, explore data  using quantitative, statistical and visualization tools.
Create and test data processing pipelines.
Inform the selection of appropriate modeling techniques to ensure that predictive models are developed using rigorous statistical processes.
Establish and maintain effective processes for validating and updating predictive models.
Collaborate with internal business, analytics and data strategy partners to improve efficiency and increase applicability of predictive models into the core software products.
Perform statistical analysis to prioritize in order to maximize success.
Optimize data pipelines, queries, transformations and algorithms to achieve cost saving, performance and stability.
Identify areas for improvement, communicating action plans.
Perform strategic data analysis and research to support business needs.

Requirements, Skills and Experience
Bachelor / Master degree in a technical field such as computer science, computer engineering or related field required.
Big Data experience.
Hands on Spark/Scala programming experience of at least 3+ years.
Participating in agile development projects for batch and real time data ingestion.
2+ years of AWS development experience is preferred.
5+ years experience in Java and Python.
 Machine Learning experience.
Solid understanding of ETL and ELT strategies.
Experience with Apache Airflow is a plus.
Dev Ops experience is a plus.
Ad Tech background desirable.