Senior Data Engineer

Compensation

: $94,330.00 - $148,460.00 /year *

Employment Type

: Full-Time

Industry

: Information Technology



Loading some great jobs for you...







Senior Data Engineer

Looking for a Data Engineer to help build all data that s fully connected and version controlled. This company is making life easier for people and brands everywhere through trusted conversational AI. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.

In this role, you will:

  • Create and maintain optimal data pipeline architecture.
  • Assemble large, complex data sets that meet a broad range of business requirements.
  • Identify, design, and implement internal process improvements, including automating manual processes, optimizing data flows, and re-designing infrastructure for greater scalability.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS-based big data technologies.
  • Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.

Professional qualifications:

  • Experience using the following software/tools:
    • Big data tools: Hadoop, Spark, Kafka, etc.
    • Relational SQL and NoSQL databases, including Postgres and Cassandra.
    • Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
    • AWS-based cloud services: EC2, EMR, RDS, Snowflake, Redshift, etc.
    • Stream-processing systems: Storm, Spark-Streaming, etc.
    • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
  • Advanced working SQL knowledge
  • Experience building and optimizing big data data pipelines, architectures and data sets.
  • Experience performing root cause analysis
  • Strong analytical skills related to working with unstructured or dirty datasets.
  • Experience building processes supporting data transformation, data structures, metadata dependency, and workload management.
  • A successful history of manipulating, processing, and extracting value from large disconnected datasets.
  • Working knowledge of message queuing, stream processing, and highly scalable big data data stores.
  • Demonstrated ability to work closely with teammates in a highly collaborative environment


Job Requirements:

Associated topics: data analytic, data architect, data integration, data management, data scientist, database, etl, erp, hbase, sybase * The salary listed in the header is an estimate based on salary data for similar jobs in the same area. Salary or compensation data found in the job description is accurate.

Launch your career - Create your profile now!

Create your Profile

Loading some great jobs for you...