About the Job
We are seeking a highly motivated, customer focused; solutions oriented Principal Data Platform Engineer. This position is accountable for developing systems & solutions capable of processing millions of events per second, providing both a real time and historical view into the operation of our wide-array of systems. Design collection and enrichment system components for quality, time lines, scale and reliability. Work on high performance real time data stores and a massive historical data store using best-of-breed and industry leading technology. Build platforms that allow others to design, develop, and apply advanced statistical methods and machine intelligence algorithms.
- Build core components needed to drive the next generation of data platforms and data processing capability.
- Build data products, identifying trouble spots, and optimizing the overall user experience with a robust data architecture capable of providing insights
- Drive standardization and service focused instrumentation
- Build capabilities that analyze massive amounts of data both in real-time and batch processing
- Prototype ideas for new tools, products and services across data management ecosystem
- Employ rigorous continuous delivery practices managed under an agile software development approach
- Ensure a quality transition to production and solid production operation of the platforms
- Raise the bar for the Engineering team by advocating leading edge practices such as CI/CD, containerization and TDD
- Be responsible for end-to-end software development in a DevOps model and enhance DevOps practices to deploy and operate systems
- Troubleshoot and resolve issues in development, test and production environments
- Collaborates with project stakeholders across software engineering teams to identify product and technical requirements. Conducts analysis to determine integration needs.
- Designs new software applications, supports applications under development, and customizes current applications. Assists with the software update process for existing applications, and rollouts of software releases.
- Researches, writes and edits documentation and technical requirements, including software designs, evaluation plans, test results, technical manuals and formal recommendations and reports.
- Lead development for new platform initiatives and provide technical leadership throughout the design process and guidance with regards to practices, procedures and techniques. Serves as a guide and mentor for junior-level Software Development Engineers.
- Works with Quality Assurance team to determine if applications fit specification and technical requirements.
- Displays in-depth knowledge of, and ability to apply, process design and redesign skills. Presents and defends architectural, design, and technical choices to internal audiences.
- Multi-task and prioritize with little to no supervision, providing team leadership skills.
- Bachelors or Masters in Computer Science, Statistics or related discipline is required.
- 8+years Data platform & programming experience
- Experience with Spark(AWS EMR), Spark Streaming &Batch, Hadoop/Cloduera,Hadoop (HDFS, YARN), Hives SQL/ Java, Kafka, MemSQL, Cassandra, HBase, MongoDB (NoSQL), Java, Scala, Git, Maven, Jenkins, Rancher, Puppet, Docker, Kubernetes, Oracle, Teradata and related skills/tools.
- Experience with Python and R is a plus.
- Experience in software development of large-scale distributed systems including proven track record of delivering backend systems that participate in a complex ecosystem.
- Experience in data related technologies and open source frameworks preferred
- Proficient in Unix/Linux environments
- Test-driven development/test automation, continuous integration, and deployment automation
- Enjoy working with data analysis, data quality and reporting
- Requires understanding of complete SDLC and Experience with continuous integration, test-driven/behavior-driven development, and agile, scrum development methodologies
- Able to thrive in a fast-paced agile environment and to work effectively across organizational boundaries
- Excellent communicator, able to analyze and clearly articulate complex issues and technologies understandably and engagingly
- Great design and problem solving skills
- Adaptable, proactive and willing to take ownership
- Keen attention to detail and high level of commitment
- Data mining, joining, and analyzing big data from multiple environments.
- Experience with advanced statistical and data mining techniques.
- Integrating business knowledge with technical functionalities.
- Providing consultative direction to develop long-term objectives to achieving business objectives.
- Working with other members of cross-functional team, third party vendors and business partners to deliver quality products in a timely fashion
- Understanding key business drivers and applying the knowledge of internal/external business issues to improve products and services.
- Anticipating and interpreting customer needs, assessing requirements and identifying solutions.
- Ability to initiate and follow through on complex projects of both short and long term duration required.
- Punctual, regular, and consistent attendance
Comcast is an EOE/Veterans/Disabled/LGBT employer