This session builds on the foundations of the Partitioning 101 session. &nbsp;Gain a deeper understanding of data partitioning with Ascend, including advanced topics such as how to practice more fine-grained control over data partitioning at ingest. &nbsp; You will learn how Ascend uses metadata to repartition data, and how to use advanced features like partition filtering and partition unions to tune and control processing in your intelligent data pipelines.&nbsp;

Shayaan Saiyed

Partitioning 201

Join this advanced technical session to gain a deeper understanding of the various ways in which custom Python capabilities can enhance your data engineering prowess on Ascend. &nbsp;You will learn how to use custom Python in Read connectors to easily connect to data sources that are not yet natively supported, and explore how Python can power Transform components in the pipeline flow as well. Finally, we will discuss the importance of custom Python in Write connectors, which enables data loading into any variation of destinations for your intelligent data pipelines.

Ayush Maganahalli

Custom Python for Different Components

Most BI analysts have deep knowledge of how data delivers on business needs, and encode that knowledge in BI platforms. Join us for this advanced technical training session where we will demonstrate how business intelligence teams can utilize Ascend to create better, cleaner datasets, for reusability that can be leveraged across teams, all while reducing the running costs of BI platforms.

Tessa Blankenship

Jacob Levy

Data Pipelines for BI Teams

This very advanced technical session is for expert practitioners at data driven organizations seeking additional force multipliers through advanced automation with Ascend.
Join us to explore the concept of using Ascend automation to generate families of related data pipelines, versus creating data pipelines through external scripting using the Ascend SDK. &nbsp;We will introduce the power of harnessing metadata and patterns to drive this approach. You will learn how dynamic pipeline generation can streamline and standardize your data engineering workflows, and see real-world use cases including mass ingestion and flexible schema management.

Qianyi Sha

Jon Saltzman

Dynamic Pipeline Generation: Advanced Automation for Modern Data Challenges

APIs are among the most common interfaces with which to acquire data from data sources. Join us to learn about how to read data from APIs into intelligent data pipelines built and run on Ascend.&nbsp; &nbsp;
This presentation introduces the concept and explores the different types of APIs supported on Ascend. Learn the steps necessary to set up and start reading data from APIs. We will delve into how Ascend helps you with schema enforcement and evolution, which is the process by which the structure of data in APIs changes over time. You will walk away with a strong understanding of how to read data from APIs, and how to manage schema evolution in your intelligent data pipelines.
&nbsp;

Maggie Huang

Reading Data from APIs

This session builds on the Introduction to Ascend CLI. Join us as we review a compressive example of how to use a CI/CD workflow to develop and release intelligent data pipelines via the Ascend SDK and CLI. You will see the complete example end-to-end, including version control, continuous development, testing, and deployment into production.
After this session, Alex White will come on stage to close out the event.&nbsp;

Data Pipeline Automation Summit 2023

Agenda