Data Lineage Developer

Information Security Responsibilities

• Promote and enforce awareness of key information security practices, including acceptable use of information assets, malware protection, and password security protocols
• Identify, assess, and report security risks, focusing on how these risks impact the confidentiality, integrity, and availability of information assets
• Understand and evaluate how data is stored, processed, or transmitted, ensuring compliance with data privacy and protection standards (GDPR, CCPA, etc.)
• Ensure data protection measures are integrated throughout the information lifecycle to safeguard sensitive information

Role & Responsibilities:

• Analysis of business processes to understand current setup, navigating through multiple systems, products and coordinate with multiple stakeholders.
• Build and implement data lineage frameworks that track the flow of data across multiple systems, databases, and pipelines.
• Work with stakeholders to define and map the end-to-end flow of data, identifying sources, transformations, destinations, and dependencies.
• Implement solutions to catalog metadata from diverse data sources, including MongoDB, Snowflake, and relational databases (e.g., SQL Server, DB2).
• Collaborate with firmwide data governance teams to ensure that lineage metadata is integrated with Collibra for auditing and compliance purposes.
• Develop processes to monitor data quality and ensure that data lineage tracks and reflects accurate data transformations and business rules.
• Design and implement automation tools and scripts to capture and maintain data lineage information and catalog updates on a scheduled basis.
• Create clear, understandable, and accurate documentation of data lineage for business users and technical teams.
• Identify and resolve issues related to data inconsistencies, lineage discrepancies, or errors in data transformation processes.
• Document technical requirements, specifications, and solutions.

Required skills:

• 3+ years of development experience.
• Experience/exposure in data engineering, data governance, or related fields, with a focus on data lineage and metadata cataloging.
• Strong expertise in data modeling, SQL, Stored Procedures/triggers, etc.
• Should possess good knowledge of OO, data structures and algorithms.
• Experience with data modeling, ETL pipelines, and data transformation.
• Experience with SQL and relational databases (e.g., Sybase, DB2, SQL Server).
• Proficiency in Python, Java, or other scripting languages for data integration and automation tasks.
• Strong analytical and troubleshooting skills with the ability to solve complex data flow problems.
• Excellent written and verbal communication skills, with the ability to explain technical concepts to non-technical stakeholders.
• Ability to work in a fast-paced, collaborative environment with cross-functional teams.
• Good to have:
• Experience with Collibra
• Knowledge of metadata management practices, including the ability to extract and catalog metadata from complex data sources such as MongoDB, Snowflake, and relational databases.
• Understanding of data privacy regulations (e.g., GDPR, CCPA) and their impact on data lineage and governance.
• Experience of working in Agile teams.
• Reference Data experience.

Good to Have:

• Experience with Collibra
• Knowledge of metadata management practices, including the ability to extract and catalog metadata from complex data sources such as MongoDB, Snowflake, and relational databases.
• Understanding of data privacy regulations (e.g., GDPR, CCPA) and their impact on data lineage and governance.
• Experience of working in Agile teams.
• Reference Data experience.