EncartaLabs

Informatica Data Engineering Integration - Developers

( Duration: 3 Days )

In Informatica Data Engineering Integration - Developers training course, attendees will learn to accelerate Data Engineering Integration through mass ingestion, incremental loads, transformations, processing of complex files, creating dynamic mappings, and integrating data science using Python. Optimize the Data Engineering system performance through monitoring, troubleshooting, and best practices while gaining an understanding of how to reuse application logic for Data Engineering use cases.

By attending Informatica Data Engineering Integration - Developers workshop, attendees will learn to:

  • Mass ingest data to Hive and HDFS
  • Perform incremental loads in Mass Ingestion
  • Perform initial and incremental loads
  • Integrate with relational databases using SQOOP
  • Perform transformations across various engines
  • Execute a mapping using JDBC in Spark mode
  • Perform stateful computing and windowing
  • Process complex files
  • Parse hierarchical data on Spark engine
  • Run profiles and choose sampling options on Spark engine
  • Execute Dynamic Mappings
  • Monitor logs using REST Operations Hub
  • Monitor logs using Log Aggregation and troubleshoot
  • Run mappings in Databricks environment
  • Create mappings to access Delta Lake tables
  • Tune performances of Spark and Databricks jobs

  • Informatica Developer Tool for Big Data Developers

The Informatica Data Engineering Integration - Developers class is ideal for:

  • Developers

COURSE AGENDA

1

Informatica Data Engineering Management Overview

  • Data Engineering concepts
  • Data Engineering Management features
  • Benefits of Data Engineering Management
  • Data Engineering Management architecture
  • Data Engineering Management developer tasks
  • Data Engineering Integration 10.4 new features
2

Ingestion and Extraction

  • Integrating Data Engineering Integration with Hadoop cluster
  • Application Services of Data Engineering Integration 10.4.0
  • Hadoop file systems
  • Ingest data to HDFS and Hive using SQOOP
  • Mass Ingestion to HDFS and Hive – Initial load
  • Mass Ingestion to HDFS and Hive - Incremental load
3

Native and Hadoop Engine Strategy

  • Data Engineering Integration engine strategy
  • Hive Engine architecture
  • MapReduce
  • Tez
  • Spark architecture
  • Blaze architecture
  • Basic Data Engineering Integration Transformations
  • Deployed Applications
4

Data Engineering Development Process

  • Advanced Transformations in Data Engineering Integration Python and Update Strategy
  • Hive ACID Use Case
  • Stateful Computing and Windowing
5

Complex File Processing

  • Data Engineering file formats – Avro, Parquet, JSON
  • Complex file data types – Structs, Arrays, Maps
  • Complex Configuration, Operators and Functions
6

Hierarchical Data Processing Configuration

  • Hierarchical Data Processing
  • Flatten Hierarchical Data
  • Hierarchical Data Processing with Schema Changes
  • Complex Configuration, Operators and Functions
  • Dynamic Ports
  • Dynamic Input Rules
7

Mappings and Mapping Optimization

  • Validation Environments
  • Execution Environment
  • Mapping Optimization
  • Mapping Recommendations
  • Scheduling, Queuing, and Node Labeling
8

Monitoring Logs and Troubleshooting in Hadoop

  • Hadoop Environment Logs
  • Spark Engine Monitoring
  • Blaze Engine Monitoring
  • REST Operations Hub
  • Log Aggregator
  • Troubleshooting
9

Databricks Integration

  • Databricks Integration Overview
  • Run-time Process on the Databricks Spark Engine
  • Databricks Integration Task Flow
  • Pre-requisites for Databricks integration
  • Databricks integration tasks
  • Cluster Workflows
  • Mass Ingestion
10

Intelligent Structure Model

  • Intelligent Structure Discovery Overview
  • Intelligent Structure Model

Encarta Labs Advantage

  • One Stop Corporate Training Solution Providers for over 4,000 Modules on a variety of subjects
  • All courses are delivered by Industry Veterans
  • Get jumpstarted from newbie to production ready in a matter of few days
  • Trained more than 50,000 Corporate executives across the Globe
  • All our trainings are conducted in workshop mode with more focus on hands-on sessions

View our other course offerings by visiting http://encartalabs.com/course-catalogue-all.php

Contact us for delivering this course as a public/open-house workshop/online training for a group of 10+ candidates.

Top