Senior Data Engineer

<p><strong>Description</strong></p>
<p>Equal Experts is an innovative consultancy specialising in the delivery of custom software solutions for<br>blue-chip enterprise and public sector clients across a range of industry sectors. We work on important<br>projects for our clients, deliver market-leading propositions across the digital, online and mobile<br>channels, and are recognised for our leadership in the application of agile and lean delivery methods to<br>assure delivery. We have offices in Australia, India, South Africa, the UK, the EU and the USA.</p>
<p>We’re looking for Data Engineers with proven senior-level experience to join the Equal Experts<br>network.</p>
<p>Our Data Engineers are highly skilled in creating data pipelines for ingest, transformation and storage of<br>data, usually at scale, and are familiar with standing-up analytics clusters.<br>This role involves using modern agile technical practices - including Continuous Integration and<br>Deployment, TDD, and XP - to deliver timely and pragmatic solutions, as well as helping others to do<br>their jobs in a more effective way.</p>
<p>This role involves working in teams that use modern agile technical practices - including continuous<br>integration and deployment and fast feedback loops - to deliver timely and pragmatic solutions, as well<br>as helping others to do their jobs in a more effective way.<br><br><strong>Responsibilities</strong></p>
<p>● Working with the client and the team to determine data processing and access needs and how<br>best to deliver them.<br>● Creation and support of highly available data pipelines and storage solutions (e.g. data<br>warehouses).<br>● Delivery of software using agile (pair programming, TDD, CI/CD, etc) practices.<br>● Automation of data infrastructure and deployments.<br>● Advocating agile practices to the client organisation.<br>● Mentoring client team members.<br>● Helping improve the team and clients’ data capabilities.<br><br></p>
<p><strong>Required Skills</strong></p>
<p>● Significant experience working on data pipelines, platforms and projects.<br>● Experience in developing solutions in one or more of the main Cloud providers (AWS, GCP,<br>Azure).<br>● Strong background in at least one programming language, Python or Scala preferable.<br>● Working with a variety of databases, data warehouses and distributed file systems in production<br>environments.<br>● Strong experience in SQL and data modeling.<br>● Exposure with data processing at scale.<br>● Knowledge about GDPR and data security. Working with source control and infrastructure as code (Git/Github, Terraform, ansible etc.) and CI/CD tools.<br>● Understanding and being able to support data scientists or business intelligence teams - in data<br>prep, mentoring and support functions.<br>● Strong collaboration and communication skills, including the ability to communicate effectively<br>while working remotely with globally distributed teams.</p>