EncartaLabs

Talend Big Data - Spark Streaming

( Duration: 1 Day )

The Talend Big Data - Spark Streaming training course, covers Big Data streaming Jobs that use the Spark streaming framework.

By attending Talend Big Data - Spark Streaming workshop, attendees will learn to:

  • Connect to a Hadoop cluster from a Talend Job
  • Use context variables and metadata
  • Read and write files in HDFS or HBase in a Big Data batch or Big Data streaming Job
  • Read and write messages in a Kafka topic in real time
  • Configure a Big Data batch Job to use the Spark framework
  • Configure a Big Data streaming Job to use the Spark streaming framework
  • Save logs to Elasticsearch
  • Configure a Kibana dashboard
  • Ingest a stream of data to a NoSQL database, HBase

  • Attend Talend Big Data - Essentials course or equivalent experience.

Anyone who wants to use Talend Studio to interact with Big Data systems

COURSE AGENDA

1

Spark in context

  • Concepts
2

Reading and writing messages with Kafka

  • Understanding Kafka basics
  • Creating a new topic in Kafka
  • Publishing messages to a specific topic using a standard Job
  • Consuming messages in a specific topic using a standard Job
  • Publishing messages to Kafka topics in real time using a Big Data Spark Streaming Job
  • Consuming messages to Kafka topics in real time using a Big Data Spark Streaming Job
  • Enriching data using a MySQL table and a lookup in a Big Data Spark Streaming Job
3

Introduction to Spark

  • Understanding Spark basics
  • Analyzing customer data
  • Producing and consuming messages in real time
4

Logs processing use case - monitoring

  • Introduction to the log processing use case
  • Monitoring enriched logs
  • Saving logs to Elasticsearch
  • Using and modifying a Kibana dashboard to visualize data
5

Logs processing use case - reporting

  • Generating reports based on data windows
  • Consuming messages from a Kafka topic
  • Using the tWindow component to schedule processing
6

Logs processing use case - batch analysis

  • Ingesting streams of data
  • Analyzing logs with a batch Job

Encarta Labs Advantage

  • One Stop Corporate Training Solution Providers for over 4,000 Modules on a variety of subjects
  • All courses are delivered by Industry Veterans
  • Get jumpstarted from newbie to production ready in a matter of few days
  • Trained more than 50,000 Corporate executives across the Globe
  • All our trainings are conducted in workshop mode with more focus on hands-on sessions

View our other course offerings by visiting http://encartalabs.com/course-catalogue-all.php

Contact us for delivering this course as a public/open-house workshop/online training for a group of 10+ candidates.

Top