Senior Real-Time Data Engineer

Compensation

: $92,790.00 - $131,530.00 /year *

Employment Type

: Full-Time

Industry

: Information Technology



Loading some great jobs for you...





JOB SUMMARY
Responsible for maintaining scalable and reliable data pipelines that support data operations for Reporting, Analytics, Applications, and Data Science by gathering and processing raw data at scale. Delivers solutions by developing, testing, and supporting cloud-based streaming applications. Develops data set processes for data modeling, mining, and consumption.

MAJOR DUTIES AND RESPONSIBILITIES
Actively and consistently supports all efforts to simplify and enhance the customer experience.

Create and maintain scalable, reliable, consistent and repeatable pipelines that support data operations for Reporting, Analytics, Applications, and Data Science.

Gather and process real-time data at scale (including writing APIs, stream processing jobs, and aggregation jobs).

Expertly use cloud-based tools to ingest and process data.

Profile data to measure quality, integrity, accuracy, and completeness.

Manage life cycle of multiple data sources.

Increase speed to delivery by implementing workload/workflow automation solutions.

Interact with various stakeholders to understand their business needs, communicate project status and develop relationships to ensure satisfaction

Perform other duties as assigned.

REQUIRED QUALIFICATIONS
Skills/Abilities and Knowledge
Ability to read, write, speak and understand English
Ability to use a wide variety of open source technologies and cloud services
Strong coding experience using Scala or Java
Strong background in Linux/CentOS installation and administration
Strong knowledge in data storage that demonstrates knowledge of when to use a file system, relational database, or NoSQL variant
Strong experience with Spark or Hadoop/Hive
Experience receiving, converting, and cleansing big data
Ability to identify and resolve end-to-end performance, network, server, and platform issues
Attention to detail with the ability to effectively prioritize and execute multiple tasks
In-depth knowledge of Agile development methodologies

Education
Bachelor's degree in an engineering discipline or computer science

Related Work Experience
3-5 years of designing and building Scala or Java applications
2+ years of designing and building cloud native applications
2+ years of Linux/Unix/CentOS system admin
1+ year(s) of designing and building Kafka or Kinesis applications

WORKING CONDITIONS
Office environment

Associated topics: data architect, data manager, data management, data warehousing, database administrator, etl, erp, mongo database administrator, sql, teradata * The salary listed in the header is an estimate based on salary data for similar jobs in the same area. Salary or compensation data found in the job description is accurate.

Launch your career - Create your profile now!

Create your Profile

Loading some great jobs for you...