Senior Real-Time Data Engineer
Compensation: $92,790.00 - $131,530.00 /year *
Employment Type: Full-Time
Industry: Information Technology
Responsible for maintaining scalable and reliable data pipelines that support data operations for Reporting, Analytics, Applications, and Data Science by gathering and processing raw data at scale. Delivers solutions by developing, testing, and supporting cloud-based streaming applications. Develops data set processes for data modeling, mining, and consumption.MAJOR DUTIES AND RESPONSIBILITIESActively and consistently supports all efforts to simplify and enhance the customer experience.
Create and maintain scalable, reliable, consistent and repeatable pipelines that support data operations for Reporting, Analytics, Applications, and Data Science.
Gather and process real-time data at scale (including writing APIs, stream processing jobs, and aggregation jobs).
Expertly use cloud-based tools to ingest and process data.
Profile data to measure quality, integrity, accuracy, and completeness.
Manage life cycle of multiple data sources.
Increase speed to delivery by implementing workload/workflow automation solutions.
Interact with various stakeholders to understand their business needs, communicate project status and develop relationships to ensure satisfaction
Perform other duties as assigned.REQUIRED QUALIFICATIONSSkills/Abilities and Knowledge
Ability to read, write, speak and understand English
Ability to use a wide variety of open source technologies and cloud services
Strong coding experience using Scala or Java
Strong background in Linux/CentOS installation and administration
Strong knowledge in data storage that demonstrates knowledge of when to use a file system, relational database, or NoSQL variant
Strong experience with Spark or Hadoop/Hive
Experience receiving, converting, and cleansing big data
Ability to identify and resolve end-to-end performance, network, server, and platform issues
Attention to detail with the ability to effectively prioritize and execute multiple tasks
In-depth knowledge of Agile development methodologiesEducation
Bachelor's degree in an engineering discipline or computer scienceRelated Work Experience
3-5 years of designing and building Scala or Java applications
2+ years of designing and building cloud native applications
2+ years of Linux/Unix/CentOS system admin
1+ year(s) of designing and building Kafka or Kinesis applicationsWORKING CONDITIONS
Associated topics: data administrator, data analytic, data engineer, data integration, data integrity, data management, data warehouse, etl, hbase, sql
* The salary listed in the header is an estimate based on salary data for similar jobs in the same area. Salary or compensation data found in the job description is accurate.
Loading some great jobs for you...