Data Engineer experience in SSIS, SSRS Location: Columbia, TN Long Term Project Please drop for the profiles Roles and Responsibilities: . Design and develop database solutions - Creating and maintaining databases in Microsoft SQL Server. . ETL Development - Create and maintain appropriate packages and stored procedures in SSIS to pull data from various source systems and load into destination databases on SQL Server . MDM Master Data Management Development - Create and maintain scripts in MDM Tool Semarchy xDM . Optimize database performance - Ensuring that the database performs optimally by tuning queries, indexes, and other database objects. . Monitor and troubleshoot database issues - Monitoring the database for performance issues, errors, and other issues, and troubleshooting those issues as they arise. . Maintaining database security - Work with DBAs and team to ensuring that the databases are secure by implementing and maintaining appropriate security measures, such as access controls, encryption, and auditing/logging. . Develop DevOps for Data - Work with the DevOps group to help set up appropriate procedures and pipelines for data packages. Qualifications: . Bachelor"s or master"s degree in technical or business discipline or equivalent experience. . 8+ years of professional experience as a Data Engineer. . Hands on Experience with any MDM Tool preferably Semarchy Xdm. . Proven experience as a SQL Database Developer. . Expert in writing complex SQL queries and stored procedures using Microsoft SQL Server. . Experience with database design, optimization, and performance tuning. . Designing, developing, and maintaining ETL process using SQL server Integration services (SSIS). . Extracting data from various data sources, transforming, and loading it into target systems

Title:: Sr. Data Engineer (Teradata ETL with AWS exp.) Location:: Atlanta, GA (Hybrid role, 3x days onsite/week) Type of Hire:: Contract (c2c/w2) Duration:: 12 months Job ID: 38955 Qualifications: Required Skillsets: Teradata Expertise: •In-depth knowledge of Teradata architecture and components (e.g., AMP, Nodes, Parsing Engines, etc.). •Proficiency in Teradata SQL and Teradata utilities. SQL and Scripting: •Strong SQL skills for querying and managing relational databases. •Scripting experience (e.g., Shell, Python) for automating tasks. •Know tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.). ETL Tools: •Experience with ETL tools and technologies used in Teradata environments. Performance Optimization: •Knowledge of query optimization techniques, indexing, partitioning, and workload management in Teradata. Data Warehousing Concepts: •Strong understanding of data warehousing principles, including star/snowflake schemas, OLAP, and ETL processes. Cloud Platform / AWS Expertise: •Proficiency with core AWS services: EC2, S3, RDS, Lambda, CloudWatch, CloudFormation, etc. •Experience with AWS networking (VPC, subnets, security groups, NACLs) and security management. •Familiarity with AWS storage options, such as S3, EFS, and Glacier, in the context of large-scale data storage and retrieval. Teradata and AWS Integration: •Experience deploying and managing Teradata in AWS cloud environments. •Knowledge of AWS services that enhance Teradata performance, such as using S3 for staging data or integrating with Redshift for analytics. Automation and Infrastructure as Code (IaC): •Hands-on experience with AWS CloudFormation, Terraform, or similar tools for automating infrastructure deployment. •Proficiency in automating ETL pipelines and data processing workflows using AWS Lambda, Step Functions, or AWS Glue. Data Security in AWS: •Understanding of encryption and security protocols for data stored and processed in AWS. •Experience implementing compliance standards such as GDPR, HIPAA, or other data protection regulations in cloud environments. Key Responsibilities: Database Management: •Design, implement, and maintain Teradata databases. •Optimize and tune queries for performance. •Monitor and troubleshoot database issues. ETL Development: •Develop and maintain ETL (Extract, Transform, Load) processes for data integration and transformation. •Use tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.). Data Modeling: •Create and maintain data models to support business processes. •Work with stakeholders to understand data requirements and design solutions. Performance Tuning: •Identify and resolve performance bottlenecks in SQL queries, system configuration, or data design. •Regularly monitor and optimize system performance. Data Security & Governance: •Implement data security best practices. •Ensure compliance with data governance policies and regulations. Cloud Integration and Management: •Implement and manage Teradata on AWS infrastructure, leveraging AWS services for scalability and performance. •Manage data migration from on-premises Teradata environments to AWS cloud-based Teradata instances. •Ensure seamless integration between Teradata and AWS services like S3, Redshift, RDS, and Lambda. Cloud Infrastructure: •Design and deploy cloud infrastructure (VPCs, EC2 instances, S3 buckets) to support Teradata environments. •Implement networking and security best practices in AWS to ensure secure data access and storage. Automation and DevOps: •Automate infrastructure provisioning, scaling, and monitoring using AWS tools like CloudFormation, Terraform, or AWS CLI. •Use AWS Lambda for automating ETL processes or handling specific workloads. Data Warehousing and Analytics: •Leverage AWS data services (e.g., Redshift, Athena, Glue) to complement Teradata workloads. •Implement data pipelines using AWS services for ETL processes, integrating data from various sources into Teradata. Security and Compliance: •Implement AWS security best practices, such as IAM roles and policies, to secure Teradata environments in the cloud. •Ensure compliance with relevant data governance policies, including encryption at rest and in transit. Collaboration: •Work closely with data architects, analysts, and other stakeholders to understand data needs and deliver solutions.

Please Note: As of July 22, 2021, our team will require that all candidate submissions include a LinkedIn profile. Please do not submit any candidates that do not have a LinkedIn. Kforce has a client in Brooklyn, NY that is seeking a Senior Data Engineer. Our team is headquartered in Brooklyn but has a remote-first culture and we encourage remote applicants - no preference in time zone. This is a 9-month full-time contractor position reporting to the Engineering Manager, Data Platform. Summary: As a member of our Data Infrastructure team, the Senior Data Engineer will help us build and maintain tools and infrastructure to write, test, and schedule batch data pipelines. Their work will enable other developers, data scientists, and analysts to write the high-performing pipelines that power data science, machine learning, and product development. We primarily write in Scala, Java, and Python and use technologies like Hadoop, Spark, Airflow, Terraform, and Kubernetes, as well as GCP services like Dataproc, Dataflow, and BigQuery. We build highly-performant systems that are maintainable and easy to understand by selecting and integrating with the best of current technologies. We develop robust, highly available, well-monitored data infrastructure. We stay in close communication with our internal customers and make strategic improvements to ensure those that depend on us have a great experience using data. The Senior Data Engineer should have experience building data processing platforms, supporting them at scale, and collaborating with other teams that depend on them. Experience building applications and managing infrastructure using one of the major cloud providers is preferred but not required. We value curiosity, passion, responsibility, and generosity of spirit. Required Skills : Understand that being an effective software engineer is about communicating with people as much as it is about writing code Willing to work with and improve code you did not originally write Generous with your time and experience, and can mentor and learn from other engineers Can tackle unconstrained problems and know when to seek help Strong domain expertise and backend skills (working in Scala, Java and Python) Strong leadership and communication skills Experience collaborating with other engineers and product teams Experience with the following: Advantages and limitations of distributed systems DevOps Writing ETL pipelines Building and monitoring cloud services (GCP services preferred) and infrastructure Using or maintaining data processing environments like Hadoop, Spark, PySpark, and Dataflow. Preference for candidates that have experience working with Airflow Basic Qualification : Additional Skills : candidates that have prior experience in staff-level engineer roles and their resume clearly demonstrates leadership experience. Though we are looking for a Data engineer with prior experience in writing data pipelines, we're also looking for a candidate that has infrastructure/DevOps experience/skillset. Rank :A3 Requested Date :2023-03-08

Title: Senior Data Engineer / Analyst Client: NYC Agency Location: Brooklyn, NY (Remote an option with onsite visits for critical meetings as needed) Duration: 10 Months Work Hours: 35 Hours/ week MANDATORY SKILLS/EXPERIENCE <ul> <li>12+ years of experience as an Analytics Developer, Data Engineer, or similar role, with a focus on developing reports, dashboards, and data models.</li> <li>Strong experience designing and implementing data models and schemas in Snowflake.</li> <li>Define Snowflake databases objects to support efficient data storage and retrieval.</li> <li>Expertise with data visualization tools such as Power BI, Looker, Tableau, or similar.</li> <li>Proficiency in SQL and experience with relational databases (e.g., MySQL, PostgreSQL, Oracle).</li> <li>Proficiency in data modeling concepts and techniques (e.g., dimensional modeling, star schema).</li> <li>Solid understanding of data engineering principles and experience with data integration and ETL processes.</li> <li>Familiarity with cloud platforms, preferably Azure, and experience with integration tools like IICS (Informatica Intelligent Cloud Services).</li> <li>Excellent problem-solving skills and attention to detail.</li> <li>Strong communication and collaboration skills to work effectively in cross-functional teams.</li> </ul>DESIRABLE SKILLS/EXPERIENCE:<ul> <li>Proficiency in programming languages such as Python, R, or Java.</li> <li>Familiarity with big data technologies (e.g., Hadoop, Spark) is a plus.</li> <li>Experience with data integration tools such as Informatica Intelligent Cloud Services and MuleSoft.</li> <li>Experience using Azure services for Security, Blob Storage, Data Lake, Databricks, Data Factory etc.</li> <li>Experience with Azure Monitoring services</li> <li>MS Certified Azure Solutions Architect Expert or a Snowpro Certification or a similar one</li> </ul>

Role: Lead Data Engineer Client: Alimentation Couche-Tard Location: Remote Duration: 6-month contract Requirements: <ul> <li>Looking for 8+ years of experience <ul> <li>Building pods (about 4-5 different pods in org.) all reporting to Pradeep. This person will support one project at first (could be retail, finance operations, loyalty, etc.)</li> <li>Will have 2-6 Data Engineers under the Lead (still reporting to Pradeep)</li> </ul> </li> <li>Python, SQL and PySpark</li> <li>Azure preferred but open to other (GCP, AWS)</li> <li>Databricks, Snowflake, and Fabric <ul> <li>Batch and real-time streaming - building pipelines</li> <li>Gathering requirements from the business</li> <li>Mentoring some of the Data Engineers on the team - room for growth</li> </ul> </li> </ul>Any nice-to-have skills that would make them exceptional? Certifications , 2+ yrs data architecture, data engineering patters , git repo, self-starter and go-getter attitude, software engineering background , ML experience Job Description: As the Technical Lead Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with cross-functional teams, and spearheading initiatives geared towards fortifying our data infrastructure, CI/CD pipelines, and analytics capabilities Responsibilities:<ul> <li>Apply advanced knowledge of Data Engineering principles, methodologies and techniques to design and implement data loading and aggregation frameworks across broad areas of the organization.</li> <li>Gather and process raw, structured, semi-structured and unstructured data using batch and real-time data processing frameworks.</li> <li>Implement and optimize data solutions in enterprise data warehouses and big data repositories, focusing primarily on movement to the cloud.</li> <li>Drive new and enhanced capabilities to Enterprise Data Platform partners to meet the needs of product / engineering / business.</li> <li>Experience building enterprise systems especially using Databricks, Snowflake and platforms like Azure, AWS, GCP etc</li> <li>Leverage strong Python, Spark, SQL programming skills to construct robust pipelines for efficient data processing and analysis.</li> <li>Implement CI/CD pipelines for automating build, test, and deployment processes to accelerate the delivery of data solutions.</li> <li>Implement data modeling techniques to design and optimize data schemas, ensuring data integrity and performance.</li> <li>Drive continuous improvement initiatives to enhance performance, reliability, and scalability of our data infrastructure.</li> <li>Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions.</li> <li>Implement best practices for data governance, security, and compliance to ensure the integrity and confidentiality of our data assets.</li> </ul>Qualifications:<ul> <li>Bachelor's or master's degree in computer science, Engineering, or a related field.</li> <li>Proven experience (8+) in a data engineering role, with expertise in designing and building data pipelines, ETL processes, and data warehouses.</li> <li>Strong proficiency in SQL, Python and Spark programming languages.</li> <li>Strong experience with cloud platforms such as AWS, Azure, or GCP is a must.</li> <li>Hands-on experience with big data technologies such as Hadoop, Spark, Kafka, and distributed computing frameworks.</li> <li>Knowledge of data lake and data warehouse solutions, including Databricks, Snowflake, Amazon Redshift, Google BigQuery, Azure Data Factory, Airflow etc.</li> <li>Experience in implementing CI/CD pipelines for automating build, test, and deployment processes.</li> <li>Solid understanding of data modeling concepts, data warehousing architectures, and data management best practices</li> </ul>

Role- Snowflake Data Engineer Austin, TX- Initial Remote Exp req:-12+ yrs' Sound Knowledge of BI and Data Warehousing Experience in designing and developing ETL data pipelines Hands-on experience in Snowflake database Proficient in writing complex SQLs, expertise in performance tuning of SQLs Basic working knowledge of Python scripting and programming Ability to play a Techno Functional role Hands on experience in database and table design Working experience with huge volumes of data would be an advantage Expected to lead offshore database engineers The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face Apple experience is a plus.

Role: Data Engineer III (549) Location: Cincinnati Ohio 45227 (On-Site) Duration: Contract MUST HAVE: Business Intelligence - Data Engineering. The data engineer designs and builds platforms, tools, and solutions that help the bank manage, secure, and generate value from its data. The person in this role creates scalable and reusable solutions for gathering, collecting, storing, processing, and serving data on both small and very large (i.e., Big Data) scales. These solutions can include on-premises and cloud-based data platforms, and solutions in any of the following domains ETL, business intelligence, analytics, persistence (relational, NoSQL, data lakes), search, messaging, data warehousing, stream processing, and machine learning. Responsible and accountable for risk by openly exchanging ideas and opinions, elevating concerns, and personally following policies and procedures as defined. Accountable for always doing the right thing for customers and colleagues and ensuring that actions and behaviors drive a positive customer experience. While operating within the Bank's risk appetite, achieves results by consistently identifying, assessing, managing, monitoring, and reporting risks of all types. Responsible for design, Development, and Support of data solutions, APIs, tools, and processes to enable rapid delivery of business capabilities. Work closely with IT application teams, Enterprise architecture, infrastructure, information security, and LOB stakeholders to translate business and technical strategies into data-driven solutions for the Bank. Act as a technical Expert addressing problems related to system and application design, performance, integration, security, etc. Conduct research and Development based on current trends and technologies related to the banking industry, data engineering and architecture, data security, and related topics. Work with developers to Build CI/CD pipelines, Self-service Build tools, and automated deployment processes. Evaluate software products and Provide documented recommendations as needed. Provide Support and troubleshooting for data platforms. Must be willing to Provide escalated on-call Support for complicated and/or critical incidents. Participate in the planning process for hardware and software. Plan and work on internal projects as needed, including legacy system replacement, Monitoring and analytics improvements, tool Development, and technical documentation. Provide technical guidance and mentoring for other team members.- Manage and prioritize multiple assignments. Requirements: Bachelor's degree in Computer Science/Information Systems or equivalent combination of education and experience. Must be able to communicate ideas both verbally and in writing to management, business and IT sponsors, and technical resources in language that is appropriate for each group. Fundamental understanding of distributed computing principles Knowledge of application and data security concepts, best practices, and common vulnerabilities. Conceptual understanding of one or more of the following disciplines preferred big data technologies and distributions, metadata management products, commercial ETL tools, BI and reporting tools, messaging systems, data warehousing, SQL, Java (language and run time environment), major version control systems, continuous integration/delivery tools, infrastructure automation and virtualization tools, data testing, major cloud, or rest API design and development.

Data Engineer Technogen Jobs

Data Engineer
December 28, 2024

Sr Data Engineer
December 28, 2024

Senior Data Engineer
December 19, 2024

Senior Data Engineer Analyst
December 19, 2024

Lead Data Engineer
December 19, 2024

Snowflake Data Engineer
December 19, 2024

Data Engineer
December 19, 2024

Quick filters

Top companies

Data Engineer Technogen Jobs

Data EngineerDecember 28, 2024

Sr Data EngineerDecember 28, 2024

Senior Data EngineerDecember 19, 2024

Senior Data Engineer AnalystDecember 19, 2024

Lead Data EngineerDecember 19, 2024

Snowflake Data EngineerDecember 19, 2024

Data EngineerDecember 19, 2024