Job title: Data Engineer
Location: UK
Duration: 6-month contract Inside IR35
Working Mode: Fully Remote
Vetting: Active SC Clearance
Job Description:
Data Engineer Skills and Technologies Data Engineering:
– Strong understanding of data concepts
– data types, data structures, schemas (both JSON and Spark), schema management etc
– Strong understanding of complex JSON manipulation – Experience working with Data Pipelines using a custom Python/PySpark frameworks
– Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data.
– Strong understanding of Data Security principles
– data owners, access controls
– row and column level, GDPR etc including experience of handling sensitive datasets
– Strong problem solving and analytical skills, particularly able to demonstrate these intuitively
Requirements
Required Skills:
Languages / Frameworks
– JSON
– YAML
– Python (as a programming language, not just able to write basic scripts. Pydantic experience would be a bonus.)
– SQL
– PySpark
– Delta Lake
– Bash (both CLI usage and scripting)
– Git
– Markdown
– Scala (bonus, not compulsory)
– Azure SQL Server as a HIVE Metastore (bonus) Technologies
– Azure Databricks
– Apache Spark
– Delta Tables
– Data processing with Python
– PowerBI (Integration / Data Ingestion)
– JIRA
...committed to meeting the Paris Agreements goals by 2050. Legal Counsel Clean Energy Provide advice and counsel to... ...relevant legal experience, preferably at a major law firm or in?house. R enewable energy experience; wind energy, battery storage or...
What We Offer: The Registered Nurse serves as a leader of the healthcare team at Novant Health by establishing an authentic personalized... ...nursing required. If your RN nursing license has been inactive, retired or lapsed for five years or more and you have not been licensed...
...THE ROLE As a key member of the Meat/Seafood department leadership team, the Meat/Seafood Assistant Team Leader has a breadth of responsibilities spanning from team member development and sales generation, to product management and merchandising. In this role, you...