%{search_type} search results

• Cover
• Title Page
• Copyright and Credit
• Dedicated
• Foreword
• Contributors
• Table of Contents
• Preface
• Part 1: Introducing Distributed Tracing
• Chapter 1: Observability Needs of Modern Applications
• Understanding why logs and counters are not enough
• Logs
• Events
• Metrics and counters
• What's missing?
• Introducing distributed tracing
• Span
• Tracing
• building blocks
• Reviewing context propagation
• In-process propagation
• Out-of-process propagation
• Ensuring consistency and structure
• Building application topology
• Resource attributes

• Performance analysis overview
• The baseline
• Investigating performance issues
• Summary
• Questions
• Further reading
• Chapter 2: Native Monitoring in .NET
• Technical requirements
• Building a sample application
• Log correlation
• On-demand logging with dotnet-monitor
• Monitoring with runtime counters
• Enabling auto-collection with OpenTelemetry
• Installing and configuring OpenTelemetry
• Exploring auto-generated telemetry
• Debugging
• Performance
• Summary
• Questions
• Chapter 3: The .NET Observability Ecosystem
• Technical requirements
• Configuring cloud storage

• Using instrumentations for popular libraries
• Instrumenting application
• Leveraging infrastructure
• Configuring secrets
• Configuring observability on Dapr
• Tracing
• Metrics
• Instrumenting serverless environments
• AWS Lambda
• Azure Functions
• Summary
• Questions
• Chapter 4: Low-Level Performance Analysis with Diagnostic Tools
• Technical requirements
• Investigating common performance problems
• Memory leaks
• Thread pool starvation
• Profiling
• Inefficient code
• Debugging locks
• Using diagnostics tools in production
• Continuous profiling

• The dotnet-monitor tool
• Summary
• Questions
• Part 2: Instrumenting .NET Applications
• Chapter 5: Configuration and Control Plane
• Technical requirements
• Controlling costs with sampling
• Head-based sampling
• Tail-based sampling
• Enriching and filtering telemetry
• Span processors
• Customizing instrumentations
• Resources
• Metrics
• Customizing context propagation
• Processing a pipeline with the OpenTelemetry Collector
• Summary
• Questions
• Chapter 6: Tracing Your Code
• Technical requirements
• Tracing with System.Diagnostics or the OpenTelemetry API shim

• Tracing with System.Diagnostics
• Tracing with the OpenTelemetry API shim
• Using ambient context
• Recording events
• When to use events
• The ActivityEvent API
• Correlating spans with links
• Using links
• Testing your instrumentation
• Intercepting activities
• Filtering relevant activities
• Summary
• Questions
• Chapter 7: Adding Custom Metrics
• Technical requirements
• Metrics in .NET
• past and present
• Cardinality
• When to use metrics
• Reporting metrics
• Using counters
• The Counter class
• The UpDownCounter class
• The ObservableCounter class

• 1. Purpose, Scope and Audience
• 2. Executive Call to Action-How Chief Data Officers and Business Sponsors Can Empower Results
• 3. Assessing Conditions, Controls and Capabilities
• 4. Detailed Playbook Activities
• 5. Aligning the Language of Business: The Business Glossary
• 6. The Business Data Governance Toolkit
• 7. Playbook Deployment
• 8. Data Governance as an Operations Process
• 9. Governing Big Data and Analytics
• 10. Rapid Playbook Deployment.
• (source: Nielsen Book Data)

The Data and Analytics Playbook: Proven Methods for Governed Data and Analytic Quality explores the way in which data continues to dominate budgets, along with the varying efforts made across a variety of business enablement projects, including applications, web and mobile computing, big data analytics, and traditional data integration. The book teaches readers how to use proven methods and accelerators to break through data obstacles to provide faster, higher quality delivery of mission critical programs. Drawing upon years of practical experience, and using numerous examples and an easy to understand playbook, Lowell Fryman, Gregory Lampshire, and Dan Meers discuss a simple, proven approach to the execution of multiple data oriented activities. In addition, they present a clear set of methods to provide reliable governance, controls, risk, and exposure management for enterprise data and the programs that rely upon it. In addition, they discuss a cost-effective approach to providing sustainable governance and quality outcomes that enhance project delivery, while also ensuring ongoing controls. Example activities, templates, outputs, resources, and roles are explored, along with different organizational models in common use today and the ways they can be mapped to leverage playbook data governance throughout the organization.
(source: Nielsen Book Data)

"Scaling up and out - and back down again - at a moment's notice is essential for many large scale applications today. Scaling and performance at scale doesn't have to be a nightmare, this collection covers concrete ways to ensure your distributed architecture is resilient, robust, and able to seamlessly interact with databases, APIs, and customers."--Resource description page

This book constitutes the refereed proceedings of the 17th IFIP/IEEE International Workshop on Distributed Systems, Operations and Management, DSOM 2006, held in Dublin, Ireland in October 2006 in the course of the 2nd International Week on Management of Networks and Services, Manweek 2006. The 21 revised full papers and 4 revised short papers presented were carefully reviewed and selected from 85 submissions. The papers are organized in topical sections on performance of management protocols, complexity of service management, ontologies and network management, management of next generation network and services, business and service management, security and policy based management, short papers, and supporting approaches for network management.
(source: Nielsen Book Data)

"This course is an end-to-end, practical guide to using Hive for Big Data processing ... Hive helps you leverage the power of Distributed computing and Hadoop for Analytical processing. Its interface is like an old friend: the very SQL like HiveQL. This course will fill in all the gaps between SQL and what you need to use Hive. End-to-End: The course is an end-to-end guide for using Hive: whether you are analyst who wants to process data or an Engineer who needs to build custom functionality or optimize performance - everything you'll need is right here."--Resource description page

• Chapter 1 Introduction.-
• Chapter 2 Some Theoretical Background.-
• Chapter 3 Spatial Grasp Model.-
• Chapter 4 SGL Detailed Specification.-
• Chapter 5 Main Spatial Mechanisms in SGL.-
• Chapter 6 SGL Networked Interpreter.-
• Chapter 7 Creation, Activation and Management of a Distributed World.-
• Chapter 8 Parallel and Distributed Network Operations.-
• Chapter 9 Solving Social Problems.-
• Chapter 10 Automated Command and Control.-
• Chapter 11 Collective Robotics.-
• Chapter 12 Conclusions.
• (source: Nielsen Book Data)

The book describes a novel ideology and supporting information technology for integral management of both civil and defence-orientated large, distributed dynamic systems. The approach is based on a high-level Spatial Grasp Language, SGL, expressing solutions in physical, virtual, executive and combined environments in the form of active self-evolving and self-propagating patterns spatially matching the systems to be created, modified and controlled. The communicating interpreters of SGL can be installed in key system points, which may be in large numbers (up to millions and billions) and represent equipped humans, robots, laptops, smartphones, smart sensors, etc. Operating under gestalt-inspired scenarios in SGL initially injected from any points, these systems can be effectively converted into goal-driven spatial machines (rather than computers as dealing with physical matter too) capable of responding to numerous challenges caused by growing world dynamics in the 21st century. Including numerous practical examples, the book is a valuable resource for system managers and programmers.
(source: Nielsen Book Data)

• Introduction.- Spatial Grasp Model and Technology.- Spatial Grasp Language (SGL).- Symbiosis of Different Worlds in SGT.- Global Network Management under Spatial Grasp Paradigm.- Simulating Distributed and Global Consciousness under SGT.- Fighting Global Viruses under SGT.- Decision-Centric and Mosaic-Based Organizations under SGT.- Conclusions.
• (source: Nielsen Book Data)

This book investigates new important applications of the Spatial Grasp Technology (SGT) allowing us to effectively simulate and manage large distributed dynamic systems on semantic and holistic levels. This patented technology, developed for decades and in different countries, is based on a completely different philosophy and model allowing us to directly operate in united distributed physical and virtual spaces and provide system solutions much simpler and more compact than under other approaches. The described applications include basic operations suitable for solving many network-related problems, simulation of such mysterious concept as consciousness so important for the design of advanced intelligent systems, modelling the spread of viruses and distribution of antivirus vaccine, and also implementation of the latest decision-centric and mosaic-based organizational concepts important for modern defence and industrial systems. The described technology version with its Spatial Grasp Language can be implemented even within university environments, with communicating language interpreter copies, potentially numbering millions to billions, easily embedded into any existing systems, including Internet, thus converting the whole world into a powerful symbiotic simulation management engine. The book is oriented on system scientists, application programmers, industry managers, and also university students interested in advanced M.Sc. and Ph.D. projects related to distributed system management. .
(source: Nielsen Book Data)

Many modern programming languages include libraries to handle performance, multiple cores, structured data, errors or failures, and other tasks. Ballerina builds many of these tasks directly into the language. With this brief introduction, developers and software architects will learn how Ballerina can speed development and reduce failures in today's cloud native, distributed environments. Ballerina is a general-purpose cloud native programming language that specializes in integration. On the surface, it looks like many other C-style languages, but Ballerina also contains features that incorporate current best practices for web programming, microservices, and Agile- or DevOps-oriented development. In this report, O'Reilly editor Andy Oram helps you understand what Ballerina offers and how it solves modern development problems. You'll explore how: High-level Ballerina features make it easier to conduct network activities This language is designed around DevOps practices with an IDE-based build system Ballerina includes a module for continuous integration and testing Modules for deploying programs on Docker, Kubernetes, or AWS Lambda are included Ballerina also features compiler extensions, security, concurrency, and error checking.

Miniservices provide a valuable middle ground between monoliths and microservices. As Nicholas Keune explains in this report, miniservices are suited for application landscapes involving data-intensive workloads that span monoliths and microservices or cross the traditional boundaries of a service context. Drawn from the work of many development teams, the report gives a model and language to data-centric system attributes so that they can be considered more proactively in the design discussion. Combining monolithic corporate or third-party systems with microservices requires a design pattern to balance both local and global aspects of the data lifecycle. The approach advocated here, called a data discourse, is both flexible and bounded by guiding principles that help bring data discussions into early architectural conversations. Using real-world experiences and use cases, the report focuses on three of the most commonly observed attributes in a miniservice: consistency, transactionality, and proximity. The examples illustrate how design discussions about data discourses lead to miniservice creation, and how miniservices help solve otherwise difficult architectural challenges. With this report, you'll learn: What miniservices are and how they offer solutions to challenges What data discourses are and how to use them How data discourses and miniservices help shift design discussions around data.

• Cover; Copyright; Credits; About the Authors; Acknowledgement; About the Reviewer; www.PacktPub.com; Table of Contents; Preface;
• Chapter 1: The Need for Data Lake; Before the Data Lake; Need for a Data Lake; Defining Data Lake; Key benefits of Data Lake; Challenges in implementing a Data Lake; When to go for a Data Lake implementation; Data Lake architecture; Architectural considerations; Architectural composition; Architectural details; Understanding Data Lake layers; Understanding Data Lake tiers; Summary;
• Chapter 2: Data Intake; Understanding Intake tier zones

• Source System Zone functionalitiesUnderstanding connectivity processing; Understanding Intake Processing for data variety; Transient Landing Zone functionalities; File validation checks; Data Integrity checks; Raw Storage Zone functionalities; Data lineage processes; Deep Integrity checks; Security and governance; Information Lifecycle Management; Practical Data Ingestion scenarios; Architectural guidance; Structured data use cases; Semi-structured and Unstructured data use cases; Big Data tools and technologies; Ingestion of structured data; Ingestion of streaming data; Summary

• Chapter
• 3: Data Integration, Quality, and EnrichmentIntroduction to the Data Management Tier; Understanding Data Integration; Introduction to Data Integration; Prominent features of Data Integration; Practical Data Integration scenarios; The workings of Data Integration; Raw data discovery; Data quality assessment; Data cleansing; Data transformations; Data enrichment; Collect Metadata and track data lineage; Traditional data integration versus Data Lake; Data pipelines; Data partitioning; Scale on demand; Data ingest parallelism; Extensibility; Big Data tools and technologies; Syncsort

• Use case scenarios for SyncsortTalend; Use case scenarios for Talend; Pentaho; Use case scenarios for Pentaho; Summary; Chapter 4: Data Discovery and Consumption; Understanding the Data Consumption tier; Data Consumption
• Traditional versus Data Lake; An introduction to Data Consumption; Practical Data Consumption scenarios; Data Discovery and metadata; Enabling Data Discovery; Data classification; Relation extraction; Indexing data; Performing Data Discovery; Semantic search; Faceted search; Fuzzy search; Data Provisioning and metadata; Data publication; Data subscription

• Data Provisioning functionalitiesData formatting; Data selection; Data Provisioning approaches; Post-provisioning processes; Architectural guidance; Data discovery; Big Data tools and technologies; Data Provisioning; Big Data tools and technologies; Summary; Chapter 5: Data Governance; Understanding Data Governance; Introduction to Data Governance; The need for Data Governance; Governing Big Data in the Data Lake; Data Governance
• traditional versus Data Lake; Practical Data Governance scenarios; Data Governance components; Metadata management and lineage tracking; Data security and privacy

This book is for architects and senior managers who are responsible for building a strategy around their current data architecture. The reader will need a good knowledge of master data management, information lifecycle management, data governance, data product design, data engineering, and systems architecture. Also required is experience of Big Data technologies such as Hadoop, Spark, Splunk, and Storm.
(source: Nielsen Book Data)

• Preface Acknowledgements
• Chapter 1. Introduction
• Chapter 2. Word Security Areas, Bodies, and Measures
• Chapter 3. Spatial Grasp Model and Technology (SGT)
• Chapter 4. Spatial Grasp Language (SGL)
• Chapter 5. Security Related Management Examples under SGT
• Chapter 6. Networked Security Related Solutions
• Chapter 7. Managing Security by Spatial Control of Moving Objects
• Chapter 8. Investigating Nuclear War Dangers under SGT
• Chapter 9. Distributed Mosaic-Based Organizations
• Chapter 10. Conclusions.
• (source: Nielsen Book Data)

In September 2018, the 73rd General Assembly of the United Nations acknowledged that international instability is increasing and that improving global security is among the most important tasks facing the world today. The Assembly concluded that it is extremely important to develop new, effective frameworks and technologies to understand and confront increasingly complex networks of actors, interests, and contexts. Leading international security expert Peter Sapaty meets this challenge head-on and introduces a new, high-level distributed processing and control approach capable of finding real-time solutions for irregularities, crises, and security problems emerging any time and in any part of the world. Drawing upon the principles of Gestalt psychology, this book develops a radically new model of technology, Spatial Grasp Technology (SGT), a self-navigating, self-replicating, self-modifying spatial pattern technology expressed in a special high-level recursive language. Through rigorous theoretical argument and many practical examples, Sapaty shows how SGT can account for millions to billions of nodes distributed worldwide without vulnerable central resources; explains why SGT is hundreds of times shorter, simpler, and faster than other models and languages; and shows that SGT's technology basics are so simple that they can be effectively implemented even in a short time by a small group of system programmers within traditional university environments. Perhaps most importantly, Sapaty demonstrates how SGT is capable of implementing security scenarios not only at run time, but also conceivably ahead of it, allowing in some cases for the prediction and even prevention of local or global crises. For the novelty, simplicity, and wide applicability of its approach, Complexity in International Security is essential reading for system scientists, application programmers, industry managers, security and defence personnel, and university students interested in advanced MSc and PhD projects in the area of holistic and distributed management.
(source: Nielsen Book Data)

The ultimate guide to managing, building, and deploying large-scale clusters with Apache Mesos About This Book * Master the architecture of Mesos and intelligently distribute your task across clusters of machines * Explore a wide range of tools and platforms that Mesos works with * This real-world comprehensive and robust tutorial will help you become an expert Who This Book Is For The book aims to serve DevOps engineers and system administrators who are familiar with the basics of managing a Linux system and its tools What You Will Learn * Understand the Mesos architecture * Manually spin up a Mesos cluster on a distributed infrastructure * Deploy a multi-node Mesos cluster using your favorite DevOps * See the nuts and bolts of scheduling, service discovery, failure handling, security, monitoring, and debugging in an enterprise-grade, production cluster deployment * Use Mesos to deploy big data frameworks, containerized applications, or even custom build your own applications effortlessly In Detail Apache Mesos is open source cluster management software that provides efficient resource isolations and resource sharing distributed applications or frameworks. This book will take you on a journey to enhance your knowledge from amateur to master level, showing you how to improve the efficiency, management, and development of Mesos clusters. The architecture is quite complex and this book will explore the difficulties and complexities of working with Mesos. We begin by introducing Mesos, explaining its architecture and functionality. Next, we provide a comprehensive overview of Mesos features and advanced topics such as high availability, fault tolerance, scaling, and efficiency. Furthermore, you will learn to set up multi-node Mesos clusters on private and public clouds. We will also introduce several Mesos-based scheduling and management frameworks or applications to enable the easy deployment, discovery, load balancing, and failure handling of long-running services. Next, you will find out how a Mesos cluster can be easily set up and monitored using the standard deployment and configuration management tools. This advanced guide will show you how to deploy important big data processing frameworks such as Hadoop, Spark, and Storm on Mesos and big data storage frameworks such as Cassandra, Elasticsearch, and Kafka. Style and approach This advanced guide provides a detailed step-by-step account of deploying a Mesos cluster. It will demystify the concepts behind Mesos.
(source: Nielsen Book Data)

• Table of Contents Breaking the Monolith Edge Services Interservice Communication Client Patterns Reliability Patterns Data Modelling Monitoring Scaling Continuous Integration & Delivery.
• (source: Nielsen Book Data)

Quickly learn and employ practical methods for developing microservices Key Features Get to grips with microservice architecture to build enterprise-ready applications Adopt the best practices to find solutions to specific problems Monitor and manage your services in production Book DescriptionMicroservices have become a popular way to build distributed systems that power modern web and mobile apps. Deploying your application as a suite of independently deployable, modular, and scalable services has many benefits. In this book, you'll learn to employ microservices in order to make your application more fault-tolerant and easier to scale and change. Using an example-driven approach, Microservice Development Cookbook introduces you to the microservice architectural style. You'll learn how to transition from a traditional monolithic application to a suite of small services that interact to provide smooth functionality to your client applications. You'll also learn about the patterns used to organize services, so you can optimize request handling and processing and see how to handle service-to-service interactions. You'll then move on to understanding how to secure microservices and add monitoring in order to debug problems. This book also covers fault-tolerance and reliability patterns that help you use microservices to isolate failures in your applications. By the end of the book, you'll be able to work with a team to break a large, monolithic codebase into independently deployable and scalable microservices. You'll also study how to efficiently and effortlessly manage a microservice-based architecture. What you will learn Learn how to design microservice-based systems Create services that fail without impacting users Monitor your services to perform debugging and create observable systems Manage the security of your services Create fast and reliable deployment pipelines Manage multiple environments for your services Simplify the local development of microservice-based systems Who this book is forMicroservice Development Cookbook is for developers who would like to build effective and scalable microservices. Basic knowledge of the microservices architecture is assumed.
(source: Nielsen Book Data)

A practical guide to help you tackle different real-time data processing and analytics problems using the best tools for each scenario About This Book * Learn about the various challenges in real-time data processing and use the right tools to overcome them * This book covers popular tools and frameworks such as Spark, Flink, and Apache Storm to solve all your distributed processing problems * A practical guide filled with examples, tips, and tricks to help you perform efficient Big Data processing in real-time Who This Book Is For If you are a Java developer who would like to be equipped with all the tools required to devise an end-to-end practical solution on real-time data streaming, then this book is for you. Basic knowledge of real-time processing would be helpful, and knowing the fundamentals of Maven, Shell, and Eclipse would be great. What You Will Learn * Get an introduction to the established real-time stack * Understand the key integration of all the components * Get a thorough understanding of the basic building blocks for real-time solution designing * Garnish the search and visualization aspects for your real-time solution * Get conceptually and practically acquainted with real-time analytics * Be well equipped to apply the knowledge and create your own solutions In Detail With the rise of Big Data, there is an increasing need to process large amounts of data continuously, with a shorter turnaround time. Real-time data processing involves continuous input, processing and output of data, with the condition that the time required for processing is as short as possible. This book covers the majority of the existing and evolving open source technology stack for real-time processing and analytics. You will get to know about all the real-time solution aspects, from the source to the presentation to persistence. Through this practical book, you'll be equipped with a clear understanding of how to solve challenges on your own. We'll cover topics such as how to set up components, basic executions, integrations, advanced use cases, alerts, and monitoring. You'll be exposed to the popular tools used in real-time processing today such as Apache Spark, Apache Flink, and Storm. Finally, you will put your knowledge to practical use by implementing all of the techniques in the form of a practical, real-world use case. By the end of this book, you will have a solid understanding of all the aspects of real-time data processing and analytics, and will know how to deploy the solutions in production environments in the best possible manner. Style and Approach In this practical guide to real-time analytics, each chapter begins with a basic high-level concept of the topic, followed by a practical, hands-on implementation of each concept, where you can see the working and execution of it. The book is written in a DIY style, with plenty of practical use cases, well-explained code examples, and relevant screenshots and diagrams.
(source: Nielsen Book Data)

• Table of Contents Introduction to Apache Spark Apache Spark Installation Spark RDD Spark DataFrame and Dataset Spark Architecture and Application Execution Flow Spark SQL Spark Streaming, Machine Learning, and Graph Analysis Spark Optimizations.
• (source: Nielsen Book Data)

A practical guide for solving complex data processing challenges by applying the best optimizations techniques in Apache Spark. Key Features Learn about the core concepts and the latest developments in Apache Spark Master writing efficient big data applications with Spark's built-in modules for SQL, Streaming, Machine Learning and Graph analysis Get introduced to a variety of optimizations based on the actual experience Book DescriptionApache Spark is a flexible framework that allows processing of batch and real-time data. Its unified engine has made it quite popular for big data use cases. This book will help you to get started with Apache Spark 2.0 and write big data applications for a variety of use cases. It will also introduce you to Apache Spark - one of the most popular Big Data processing frameworks. Although this book is intended to help you get started with Apache Spark, but it also focuses on explaining the core concepts. This practical guide provides a quick start to the Spark 2.0 architecture and its components. It teaches you how to set up Spark on your local machine. As we move ahead, you will be introduced to resilient distributed datasets (RDDs) and DataFrame APIs, and their corresponding transformations and actions. Then, we move on to the life cycle of a Spark application and learn about the techniques used to debug slow-running applications. You will also go through Spark's built-in modules for SQL, streaming, machine learning, and graph analysis. Finally, the book will lay out the best practices and optimization techniques that are key for writing efficient Spark applications. By the end of this book, you will have a sound fundamental understanding of the Apache Spark framework and you will be able to write and optimize Spark applications. What you will learn Learn core concepts such as RDDs, DataFrames, transformations, and more Set up a Spark development environment Choose the right APIs for your applications Understand Spark's architecture and the execution flow of a Spark application Explore built-in modules for SQL, streaming, ML, and graph analysis Optimize your Spark job for better performance Who this book is forIf you are a big data enthusiast and love processing huge amount of data, this book is for you. If you are data engineer and looking for the best optimization techniques for your Spark applications, then you will find this book helpful. This book also helps data scientists who want to implement their machine learning algorithms in Spark. You need to have a basic understanding of any one of the programming languages such as Scala, Python or Java.
(source: Nielsen Book Data)

Data is changing everything. Many industries today are being fundamentally transformed through the accumulation and analysis of large quantities of data, stored in diversified but flexible repositories known as data lakes. Whether your company has just begun to think about big data or has already initiated a strategy for handling it, this practical ebook shows you how to plan a successful data lake migration. You'll learn the value of data lakes, their structure, and the problems they attempt to solve. Using Zaloni's data lake maturity model, you'll then explore your organization's readiness for putting a data lake into action. Do you have the tools and data architectures to support big data analysis? Are your people and processes prepared? The data lake maturity model will help you rate your organization's readiness. This report includes: The structure and purpose of a data lake Descriptive, predictive, and prescriptive analytics Data lake curation, self-service, and the use of data lake zones How to rate your organization using the data lake maturity model A complete checklist to help you determine your strategic path forward.

Learn about the fastest-growing open source project in the world, and find out how it revolutionizes big data analytics About This Book * Exclusive guide that covers how to get up and running with fast data processing using Apache Spark * Explore and exploit various possibilities with Apache Spark using real-world use cases in this book * Want to perform efficient data processing at real time? This book will be your one-stop solution. Who This Book Is For This guide appeals to big data engineers, analysts, architects, software engineers, even technical managers who need to perform efficient data processing on Hadoop at real time. Basic familiarity with Java or Scala will be helpful. The assumption is that readers will be from a mixed background, but would be typically people with background in engineering/data science with no prior Spark experience and want to understand how Spark can help them on their analytics journey. What You Will Learn * Get an overview of big data analytics and its importance for organizations and data professionals * Delve into Spark to see how it is different from existing processing platforms * Understand the intricacies of various file formats, and how to process them with Apache Spark. * Realize how to deploy Spark with YARN, MESOS or a Stand-alone cluster manager. * Learn the concepts of Spark SQL, SchemaRDD, Caching and working with Hive and Parquet file formats * Understand the architecture of Spark MLLib while discussing some of the off-the-shelf algorithms that come with Spark. * Introduce yourself to the deployment and usage of SparkR. * Walk through the importance of Graph computation and the graph processing systems available in the market * Check the real world example of Spark by building a recommendation engine with Spark using ALS. * Use a Telco data set, to predict customer churn using Random Forests. In Detail Spark juggernaut keeps on rolling and getting more and more momentum each day. Spark provides key capabilities in the form of Spark SQL, Spark Streaming, Spark ML and Graph X all accessible via Java, Scala, Python and R. Deploying the key capabilities is crucial whether it is on a Standalone framework or as a part of existing Hadoop installation and configuring with Yarn and Mesos. The next part of the journey after installation is using key components, APIs, Clustering, machine learning APIs, data pipelines, parallel programming. It is important to understand why each framework component is key, how widely it is being used, its stability and pertinent use cases. Once we understand the individual components, we will take a couple of real life advanced analytics examples such as 'Building a Recommendation system', 'Predicting customer churn' and so on. The objective of these real life examples is to give the reader confidence of using Spark for real-world problems. Style and approach With the help of practical examples and real-world use cases, this guide will take you from scratch to building efficient data applications using Apache Spark. You will learn all about this excellent data processing engine in a step-by-step manner, taking one aspect of it at a time. This highly practical guide will include how to work with data pipelines, dataframes, clustering, SparkSQL, parallel programming, and such insightful topics with the help of real-world use cases.
(source: Nielsen Book Data)

Learn about the fastest-growing open source project in the world, and find out how it revolutionizes big data analytics About This Book Exclusive guide that covers how to get up and running with fast data processing using Apache Spark Explore and exploit various possibilities with Apache Spark using real-world use cases in this book Want to perform efficient data processing at real time? This book will be your one-stop solution. Who This Book Is For This guide appeals to big data engineers, analysts, architects, software engineers, even technical managers who need to perform efficient data processing on Hadoop at real time. Basic familiarity with Java or Scala will be helpful. The assumption is that readers will be from a mixed background, but would be typically people with background in engineering/data science with no prior Spark experience and want to understand how Spark can help them on their analytics journey. What You Will Learn Get an overview of big data analytics and its importance for organizations and data professionals Delve into Spark to see how it is different from existing processing platforms Understand the intricacies of various file formats, and how to process them with Apache Spark. Realize how to deploy Spark with YARN, MESOS or a Stand-alone cluster manager. Learn the concepts of Spark SQL, SchemaRDD, Caching and working with Hive and Parquet file formats Understand the architecture of Spark MLLib while discussing some of the off-the-shelf algorithms that come with Spark. Introduce yourself to the deployment and usage of SparkR. Walk through the importance of Graph computation and the graph processing systems available in the market Check the real world example of Spark by building a recommendation engine with Spark using ALS. Use a Telco data set, to predict customer churn using Random Forests. In Detail Spark juggernaut keeps on rolling and getting more and more momentum each day. Spark provides key capabilities in the form of Spark SQL, Spark Streaming, Spark ML and Graph X all accessible via Java, Scala, Python and R. Deploying the key capabilities is crucial whether it is on a Standalone framework or as a part of existing Hadoop installation and configuring with Yarn and Mesos. The next part of the journey after installation is using key components, APIs, Clustering, machine learning APIs, data pipelines, parallel programming. It is important to understand why each framework component is key, how widely it is being u...