Position: Azure Data Engineer

Location: New York City, NY

Duration: Fulltime

Job Description:

Role Objectives
These roles will be part of the Data Strategy team spanning across the client Capital Markets and securities teams, Americas Division's broker-dealer and swap-dealer entities.
These roles will be involved in the active development of the data platform in close coordination with the team, beginning with the establishment of a reference data system for securities and pricing data, and later moving to other data domains.
The consulting team will need to follow internal developments standards to contribute to the overall agenda of the Data Strategy team.
The implementation of this strategic platform on client's Azure Cloud Platform will require solutions and know-how as listed in the qualifications below.
Qualifications and Skills     10+ years of experience in software development using Python, PySpark and its frameworks.
Proven experience as a Data Engineer with experience in Azure cloud.
Experience implementing solutions using -
Azure cloud services
Azure Data Factory
Azure Lake Gen 2
Azure Databases
Azure Data Fabric
API Gateway management
Azure Functions
Well versed with Azure Databricks
Strong SQL skills with RDMS or noSQL databases
Experience with developing API's using FastAPI or similar frameworks in Python
Familiarity with the DevOps lifecycle (git, Jenkins, etc.), CI/CD processes
Good understanding of ETL/ELT processes
Experience in financial services industry, financial instruments, asset classes and market data are a plus.

Job Title: Lead Data Engineer Location: Dallas, TX (Onsite) Duration: 6+ Months Contract Key Responsibilities: <ul> <li>Designing, constructing, installing, testing, and maintaining highly scalable data management systems</li> <li>Familiarity with the Agile development methodology and to be part of Agile scrum team and partner with various stake holders.</li> <li>Responsible for conceptualizing and designing detailed solutions.</li> <li>Works with product team to prioritize features for ongoing sprints and managing a list of data requirements based on industry trends, new technologies and project requirements.</li> <li>Communicating project progress to stakeholders and capable of handling risks and issues affecting the project.</li> </ul>Skills Required: <ul> <li>RDBMS: In-depth understanding of Snowflake. (must), PostgreSQL.</li> <li>Programming Languages: Python.</li> <li>ETL / EL Product: Snap logic</li> <li>Strong knowledge of, and experience with databases, writing medium to high complexity SQL statements, programming (ETL/ELT frameworks).</li> <li>Knowledge with Data Warehouse Architecture to build and articulate conceptual, logical, and physical dimensional data models, and Star Schema artifacts.</li> <li>Understanding of data modelling techniques and working knowledge with OLTP and OLAP systems.</li> <li>Hands on experience working with Python and a good understanding of shells scripts (bash/zsh, grep/sed/awk etc..) is preferred. </li> <li>Experience with data visualization tools like Power BI.</li> <li>Familiarity with cloud platforms like AWS.</li> </ul>Qualifications:<ul> <li>Bachelor's or master's degree in computer science, Information Systems, or related field. </li> <li>7+ years of relevant work experience, including previous experience leading a team.</li> <li>Excellent problem-solving and analytical skills.</li> <li>The individual will need to be hands-on and work closely with all levels in the company.</li> </ul>

Role: Azure Databricks Data Engineer Location: Chicago, IL (Onsite) Job Description: Design/Development: You will design and support the business s database and table schemas for new and existing data sources for the Lakehouse. Create and support the ETL in order to facilitate the movement of data into the Lakehouse. Through your work you will ensure The data platform is scalable for large amounts of data ingestion and processing without service degradation. Processes are built for monitoring and optimizing performance Implement Chaos Engineering practices and measures that allow the end-to-end infrastructure to function as expected, even if individual components fail Collaboration: You will be collaborative - working closely with Product Owners, application engineers, and other data consumers within the business in an attempt to gather and deliver high quality data for business-cases. Work closely with other disciplines/departments and teams across the business in coming up with simple, functional, and elegant solutions that balance data needs across the business Analytics: You will play an analytical role in quickly and thoroughly analyzing business requirements and subsequently translating the emanating results into good technical data designs. Document the data solutions, develop, and maintain technical specification documentation for all reports and processes. Skills you MUST have: 6+ years proven ability of professional Data Development experience 3+ years proven ability of developing with Azure Databricks or Hadoop/HDFS 3+ years of experience with PySpark/Spark 3+ years of experience with SQL 3+ years of experience developing with Python Full understanding of ETL concepts and Data Warehousing concepts Data modeling and query optimization skills, implementation experience of Data Vault, Star Schema and Medallion architecture Experience with CI/CD Experience with version control software Strong understanding of Agile Principles (Scrum) Experience with Azure Experience with Databricks Delta Tables, Delta Lake, Delta Live Tables Bonus Points For Experience In The Following Proficient with Relational Data Modeling Experience with Python Library Development Experience with Structured Streaming (Spark or otherwise) Experience with Kafka and/or Azure Event Hub Experience with GitHub SaaS / GitHub Actions Experience with Snowflake/ Exposure to BI Tooling (Tableau, Power BI, Cognos, etc.) Mandatory skills: Azure Databricks Pyspark Azure Data Factory

Our client is looking Fabric Data Engineer for Long Term Contract project in Newark, NJ (Hybrid. below is the detailed requirements. Job Title : Fabric Data Engineer Location : Newark, NJ (Hybrid) Duration : Long Term Contract Industry : Financial Industry Job summary : We're looking for a Data Engineer to join a diverse team dedicated to providing best in class data services to our customers, stakeholders and partners. As a part of our CDO organization, you will work with our ML Engineers, Data Scientist and various BUs to define solutions for operationalizing data-driven decision making in a cost effective and scalable manner. Job Description: Qualifications <ul> <li>Bachelors degree in computer science, Software Engineering, MIS, or an equivalent combination of education and experience with 10-12 Years of experience. </li> <li>Experience implementing and supporting data lakes, data warehouses, and data applications on AWS or Microsoft Azure for large enterprises (Fabric is plus) Programming experience with Python, Spark and SQL. </li> <li>Experience with Microsoft Azure services, specifically Data Factory, ADLS Gen2, Synapse Analytics, and Synapse Database, KQL. </li> <li>Experience in system analysis, design, development, and implementation of data ingestion pipelines in AWS or Azure. </li> <li>Solid experience implementing solutions on AWS or Azure based data lakes (preferred). </li> <li>Knowledge of ETL/ELT processes. </li> <li>Experience with end to end data solutions (ingest, storage, integration, processing, access) on AWS or Azure (preferred). </li> <li>Architect and implement CI CD strategies for EDP. Implement high velocity streaming solutions. </li> <li>Migrate data from traditional relational database systems, onprem to Azure technologies (preferably in Onelake using Fabric technologies (preferred). </li> <li>Develop data based APIs Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use cases. </li> <li>Any Cloud Certification preferred (Azure AWS preferred) </li> <li>Knowledge of Microsoft Fabric technology focusing on cost and security considerations. </li> </ul> Requirements <ul> <li>8 plus years of experience as a Data Engineer Datalakes . </li> <li>Experience developing business applications using NoSQL SQL databases. </li> <li>Experience working with Object stores and Structured semi structured unstructured data is a must. </li> <li>Experience with Microsoft Azure services Data Factory, ADLS Gen2, Synapse Analytics, and Synapse Database. </li> <li>Develop CI CD pipeline using Terraform and github Microsoft Fabric is plus. </li> <li>Responsibilities Designing, building, and maintaining efficient, reusable, and reliable architecture and code. </li> <li>Ensure the best possible performance and quality of high scale web applications and services. </li> <li>Participate in architecture and system design discussions. </li> <li>Independently perform hands on development and unit testing of the applications. </li> <li>Collaborate with the development team and build individual components into complex enterprise web systems. </li> <li>Work in a team environment with product, frontend design, production operation, QE QA, and cross functional teams to deliver a project throughout the whole software development cycle. </li> <li>Responsible for identifying and resolving any performance issues. Keep up to date with new technology developments and implementations. </li> <li>Participate in code reviews to ensure standards and best practices are met.</li> </ul>

Job Title: Lead Data Engineer Location: Dallas, TX (Onsite) Duration: 6+ Months Contract Key Responsibilities: <ul> <li>Designing, constructing, installing, testing, and maintaining highly scalable data management systems</li> <li>Familiarity with the Agile development methodology and to be part of Agile scrum team and partner with various stake holders.</li> <li>Responsible for conceptualizing and designing detailed solutions.</li> <li>Works with product team to prioritize features for ongoing sprints and managing a list of data requirements based on industry trends, new technologies and project requirements.</li> <li>Communicating project progress to stakeholders and capable of handling risks and issues affecting the project.</li> </ul> Skills Required: <ul> <li>RDBMS: In-depth understanding of Snowflake. (must), PostgreSQL.</li> <li>Programming Languages: Python.</li> <li>ETL / EL Product: Snap logic</li> <li>Strong knowledge of, and experience with databases, writing medium to high complexity SQL statements, programming (ETL/ELT frameworks).</li> <li>Knowledge with Data Warehouse Architecture to build and articulate conceptual, logical, and physical dimensional data models, and Star Schema artifacts.</li> <li>Understanding of data modelling techniques and working knowledge with OLTP and OLAP systems.</li> <li>Hands on experience working with Python and a good understanding of shells scripts (bash/zsh, grep/sed/awk etc..) is preferred. </li> <li>Experience with data visualization tools like Power BI.</li> <li>Familiarity with cloud platforms like AWS.</li> </ul> Qualifications: <ul> <li>Bachelor's or master's degree in computer science, Information Systems, or related field. </li> <li>7+ years of relevant work experience, including previous experience leading a team.</li> <li>Excellent problem-solving and analytical skills.</li> <li>The individual will need to be hands-on and work closely with all levels in the company.</li> </ul>

Job Title: Big Data Engineer Location: Phoenix, AZ OR New York, NY (Day 1 onsite) Duration: 1 year contract Responsibilities: <ul> <li>Experience: 6+ to 9+ Years </li> <li>Should be strong in spark, AWS or GCP </li> <li>Should be very well experienced in Spark Streaming and Scala. </li> <li>Working experience in java and python is good to have.</li> </ul>

Job Title: Data Engineer Location: Delaware or New Jersey (Hybrid Job) Duration: Long Term Contract Job description We are seeking a highly skilled Data Engineer with a strong background in AWS technologies to join our fast-paced and collaborative data team. In this role, you will be responsible for designing, developing, and maintaining scalable data pipelines and architectures that enable data-driven decision-making across the organization. You will work closely with data scientists, analysts, and business stakeholders to ensure that high-performance data systems are optimized to support critical business processes. Required Skills: <ul> <li>Proven experience (5-6 years) as a Data Engineer, with a focus on AWS platforms.</li> <li>Proficiency in AWS services, including AWS Glue, Amazon Redshift, Amazon S3, and Amazon Kinesis.</li> <li>Strong programming skills in languages such as Python, SQL, or Java.</li> <li>2-3 years of experience with data modeling, ETL processes, and data warehousing concepts, especially Snowflake.</li> <li>Familiarity with cloud computing concepts and data security best practices.</li> <li>Excellent analytical and problem-solving skills with a collaborative mindset.</li> </ul> Bonus Skills: <ul> <li>Experience with big data technologies like Apache Spark or Apache Kafka.</li> <li>Knowledge of data visualization tools (e.g., Tableau, Power BI).</li> <li>Understanding of DevOps practices and CI/CD pipelines.</li> </ul>

Role: Big Data Engineer with Snowflake Location: Plano, TX (Day 1 onsite) Job Description: <ol> <li>Proven experience as a Big Data Engineer, with a focus on Snowflake.</li> <li>Relevant Snowflake certification (e.g., SnowPro Core Certification).</li> <li>Strong experience in designing and implementing data pipelines in production environments.</li> <li>Proficiency in SQL and experience with ETL tools and processes.</li> <li>Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and their integration with Snowflake.</li> <li>Familiarity with data modeling, data warehousing concepts, and best practices.</li> <li>Excellent problem-solving skills and attention to detail.</li> <li>Strong communication and collaboration skills.</li> <li>Experience with other big data technologies (e.g., Hadoop, Spark).</li> <li>Knowledge of programming languages such as Python or Java.</li> </ol> Experience with data visualization tools (e.g., Tableau, Power BI

Data Engineer Lorven Technologies Jobs

Azure Data Engineer
December 20, 2024

Lead Data Engineer 6 Months Contract Dallas Tx Onsite
December 19, 2024

Azure Databricks Data Engineer Chicago Il Onsite
December 19, 2024

Fabric Data Engineer Financial Industry Newark Nj Hybrid
December 17, 2024

Lead Data Engineer 6 Months Contract Dallas Tx Onsite
December 16, 2024

Big Data Engineer 1 Year Contract Phoenix Az Or New York Ny Day 1 Onsite
December 16, 2024

Data Engineer Long Term Contract Delaware Or New Jersey Hybrid Job
December 16, 2024

Big Data Engineer With Snowflake
December 16, 2024

Quick filters

Top companies

Data Engineer Lorven Technologies Jobs

Azure Data EngineerDecember 20, 2024

Lead Data Engineer 6 Months Contract Dallas Tx OnsiteDecember 19, 2024

Azure Databricks Data Engineer Chicago Il OnsiteDecember 19, 2024

Fabric Data Engineer Financial Industry Newark Nj HybridDecember 17, 2024

Lead Data Engineer 6 Months Contract Dallas Tx OnsiteDecember 16, 2024

Big Data Engineer 1 Year Contract Phoenix Az Or New York Ny Day 1 OnsiteDecember 16, 2024

Data Engineer Long Term Contract Delaware Or New Jersey Hybrid JobDecember 16, 2024

Big Data Engineer With SnowflakeDecember 16, 2024