The Data Engineer is responsible for designing and implementing data pipelines for cloud projects. This position will require working with complex data sources and transforming it into something useful for analysts.
- Bachelor's Degree.
- 5 years of experience.
- Strong background in cloud computing, software engineering and data processing.
- Data management experience.
- Experience in ETL Tools such as Pentaho, Talend, Informatica, Azure Data Factory, Apache Kafka and Apache Camel.
- Experience designing and implementing analysis solutions on Hadoop-based platforms such as Cloudera Hadoop, or Hortonworks Data Platform or Spark based platforms such as Databricks.
- Proficient in RDBMS such as Oracle, SQL Server, DB2, MySQL etc.
- Strong analytical and problem-solving skills.
- Strong verbal and written communication skills.
- Proficient programming skills in Python, SQL NoSQL, and Spark.
- Ability to manage multiple projects essential.
- Ability to work independently or in groups.
- Ability to prioritize time.
- Ability to adapt to a rapidly changing environment.
Preferred Job Qualifications: Experience, education, licensure(s), specialized certification(s). Cloud Certifications in AWS, Azure or GCP.
Responsibilities: • Responsible for design, development and maintenance of data pipelines to enable data analysis and reporting. • Builds, evolves and scales out infrastructure to ingest, process and extract meaning out data. • Write complex SQL queries or python code to support analytics needs. • Manage projects / processes, working independently with limited supervision • Work with structured and unstructured data from a variety of data stores, such as data lakes, relational database management systems, and/or data warehouses. • Combines, optimizes, and manages multiple big data sources. • Builds data infrastructure and determines proper data formats to ensure data is ready for use