Required Skills and Experience:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field if preferred but not mandatory
• Experience in building real-time analytics platforms
• 9+ Experience in data engineering and data architecture, with a focus on a cloud-based data warehousing solutions around structured (dimensional models) and unstructured data
• Proficiency in Apache Kafka for data streaming and ingestion
• Experience with cloud-based data warehousing solutions such as Apache Hadoop HDFS, Snowflake, Amazon Redshift, Amazon S3, or Google Cloud Storage
• Proficiency in the programming languages as relevant for the platforms involved, such as Java, SQL, Python, or Scala
• Hands-on experience with Apache Spark or Apache Beam for large-scale data processing and transformation
• Experience with data visualization and exploration tools such as Apache Superset, Apache Zeppelin, Metabase, Looker or similar platforms
• Familiarity with security and governance tools and practices in the Hadoop ecosystem, such as Apache Ranger or Apache Sentry
• Excellent communication skills and the ability to work effectively in a collaborative team environment

Key Responsibilities:

• Design and implement scalable and fault-tolerant data streaming and ingestion pipelines using Apache Kafka.
• Architect and develop a cloud-based data warehousing solution using technologies using such as Apache Hadoop HDFS, Snowflake. Apache Hive or similar technologies for the storage and analysis of structured and unstructured data
• Lead the development of data transformation and modeling processes using  DBT, Apache Spark, Apache Beam or similar technologies
• Lead design and implementation of Reporting / Visualisation efforts using Apache Superset, Apache Zeppelin, Metabase, Looker or similar technologies
• Collaborate with front-end developers to embed interactive dashboards and reports using Apache Superset or similar tools into a React front-end
• Ensure data security, governance, and compliance requirements are met by implementing access controls and auditing mechanisms using Apache Ranger, or Apache Sentry or similar technologies
• Mentor and guide junior team members, provide technical leadership, and contribute to the overall technical strategy and roadmap for the analytics platform