Acceldata
One platform to author, run, observe, and manage your data pipelines, wherever they run: on-prem, cloud, or both.
COMING SOON:
See the Full Data Pipeline Lifecycle in Action with xLake
What if your Spark job could decide on its own where to run? In this video course I show you how xLake, Acceldata's hybrid data platform, does exactly that. It picks the right cluster based on where your data sits and how past runs went, spins up compute next to the data and removes it again when the job is done. Along the way you'll see the rest of xLake in action too, from one catalog with access policies down to single tables, to monitoring every job in one place and AI agents that propose fixes for your data issues.
Hands-On Agentic Data Management With Acceldata
The most annoying thing for engineers: getting hundreds of alerts and looking through endless dashboards to find issues! Why can't we just ask an AI what's wrong and how to fix it?
Well, actually we can do exactly that. I recently tried out Acceldata's Agentic Data Management Platform. And this is how every tool should work by now.
About Acceldata
A lot of companies call their data platform "hybrid" or "multi-cloud". But most of the time that means two or three separate stacks running side by side, with their own tools, their own monitoring and their own access rules. None of them knows what the others are doing. Acceldata had GLG survey C-level executives at Fortune 1000 and Global 2000 companies. Almost 80 percent run hybrid data operations today, and more than 40 percent say governance fragmentation is their single biggest challenge.
xLake is Acceldata's answer to that. It's one platform, not two stacks for cloud and on-prem stitched together and called hybrid. It's built on Kubernetes, so the same job runs on a server in AWS, in Azure or in your own datacenter, by policy and not by rewrite. And governance isn't bolted on afterwards. You define a policy once, and it's enforced at every stage.
In the course I show you how that works across all four stages:
Author: You'll see how xStore pulls an existing Hive setup into one catalog, schemas and tables included, without typing anything in by hand. Then xGovern puts an access policy on top, down to individual tables. One user runs a query and gets results, another one gets blocked. Exactly what you want.
Run: xCompute gets Spark, Trino, JupyterHub and Airflow running on one Kubernetes cluster in a few clicks. Then comes the part I find really cool. You hit "AI Run", and xRoute recommends where the job should run based on where the data sits and how the previous runs went. It even explains why. It spins up compute right next to the S3 bucket, so there's no cross-region egress, runs the Spark job and tears the cluster down again when it's done.
Observe: xObserve gives you one timeline per job, no matter where it ran. That includes memory, I/O, ideal versus actual runtime and how many compute hours you wasted. On top of that, ADOC watches the data itself and shows every pipeline run stage by stage, so you see exactly where the time goes.
Manage: Agentic Data Management takes over the work nobody can keep up with, like writing quality rules by hand. An agent walks the lineage, finds the root cause and either fixes the issue or proposes a fix for you to approve. Every confirmed fix becomes a prevention rule. You stop writing rules and start approving decisions.
This isn't a step-by-step lab where you rebuild everything yourself. It's a look under the hood at how a hybrid data platform actually works, so you can judge whether it fits your setup.
If your data stack is scattered across on-prem and cloud and nobody can see the whole pipeline lifecycle in one place, this is worth a look.
More Material
