MyDo.io Documentation

Complete guides for data lineage, observability, and the AI Agent. Learn how to connect your data stack, monitor pipelines, and detect failures automatically.

Introduction

MyDo.io is a real-time data lineage and observability platform that helps data teams understand their pipelines, detect failures, and find root causes automatically.

📊 Data Lineage

See where your data comes from, where it goes, and everything in between. Track dependencies across 165+ data tools.

🔍 Observability

Monitor data freshness, quality, and pipeline health in real-time. Get instant alerts when something breaks.

🤖 AI Agent

Ask natural language questions about your data. Get automatic root cause analysis and fix proposals.

Key Features

  • Real-time lineage: Updates every 60 seconds, not daily
  • 3D visualization: Explore complex pipelines with interactive 3D graphs
  • 165+ connectors: Works with Salesforce, BigQuery, dbt, Fivetran, Snowflake, and more
  • Impact analysis: Understand what breaks when a pipeline fails
  • AI-powered RCA: Find root causes in minutes, not hours
  • Multi-tenancy: Separate workspaces for teams or environments

Quick Start

Get up and running in 5 minutes.

Step 1: Sign Up

1. Go to https://app.mydo.io
2. Click "Sign Up"
3. Enter email and password
4. Verify your email

Step 2: Create First Workspace

A workspace is created automatically. You can create additional workspaces for different teams or projects.

Step 3: Connect Your First Source

1. Click "Data Stacks" in sidebar
2. Click "+ Add Connection"
3. Select your data tool (Salesforce, PostgreSQL, BigQuery, etc.)
4. Provide credentials
5. Click "Connect"

Step 4: View Your Lineage

Once connected, click "View Graph" to see your data lineage. The graph updates in real-time as data flows through your pipelines.

💡 Pro tip: Start with one connection to learn the platform. Add more sources once you're comfortable.

Data Lineage

Data lineage shows the complete journey of your data from source to destination.

Understanding the Graph

  • Nodes: Tables, datasets, dashboards, or jobs
  • Links: Data dependencies (A depends on B, B flows to C)
  • Colors: Green (healthy), Yellow (running), Red (failed)
  • Size: Represents data volume or throughput

2D vs 3D Views

  • 2D Graph: Better for understanding structure, linear pipelines
  • 3D Graph: Better for exploring large graphs, seeing patterns, interactive exploration

Switch between 2D and 3D using the toggle in the top-right of the graph.

Digital Twin

The Digital Twin is a real-time visualization of your entire data ecosystem. It updates every 60 seconds with live status.

What You See

  • Real-time status: Green (healthy), Yellow (running), Red (failed)
  • Data flowing: Animated particles show data movement
  • Last update: Hover over nodes to see when data was last refreshed
  • Errors: Click red nodes to see error messages

Replaying History

Use the replay controls to go back in time and see how your pipelines behaved. Useful for debugging incidents.

Data Quality Monitoring

Set up automated checks to catch data quality issues before they impact your dashboards.

Built-in Checks

  • Freshness: Alert if data is stale (e.g., "update every 1 hour")
  • Row count: Alert if row count drops unexpectedly
  • Null values: Alert if nulls exceed threshold
  • Schema drift: Alert if column added/removed

Creating Alerts

1. Click "Settings" → "Quality Rules"
2. Click "+ New Rule"
3. Select dataset and rule type
4. Set threshold and alert frequency
5. Choose notification method (email, Slack, PagerDuty)

AI Agent (Claude)

Ask questions about your data in natural language. The AI Agent queries your lineage and data to provide answers and fix proposals.

Example Questions

  • "Why is the revenue dashboard showing stale data?"
  • "Which pipelines depend on the customer_data table?"
  • "What's the root cause of this failure?"
  • "List all jobs that run daily"

Getting Proposals

The Agent can suggest fixes:

You: "dbt model marketing_funnel failed. What should I do?"

Agent: I found the root cause—the source 
table 'events' is missing the 'user_id' column. 
I suggest reconnecting Fivetran to refresh the schema.

[APPROVE FIX] [VIEW DETAILS]

Incident Management

Incidents are automatically created when data quality rules fail or pipelines error out.

Incident Lifecycle

  • Created: Automatically when threshold exceeded
  • Assigned: Routed to on-call team (configurable)
  • Investigated: Use lineage to understand impact
  • Resolved: Automatically when underlying issue clears

API Reference

MyDo.io provides a REST API for programmatic access to lineage, quality, and incident data.

Base URL: https://api.mydo.io/v1

Authentication

All API requests require an authentication token.

Authorization: Bearer YOUR_API_TOKEN

Get token: Settings → API Keys → Generate New Key

Common Endpoints

GET
/workspaces

List all workspaces for the current user

GET
/workspaces/{workspace_id}/stacks

List all data stacks in a workspace

GET
/lineage?stack_key={stack}

Get full lineage graph for a stack

GET
/incidents

List all incidents

POST
/incidents/{id}/resolve

Mark incident as resolved

Full API docs: https://api.mydo.io/docs (requires authentication)

Error Handling

Code Meaning Solution
401 Unauthorized Check API token is valid and included in request
404 Not found Check resource ID exists
429 Rate limited Wait before retrying. Max 100 requests/minute
500 Server error Contact support if persists

FAQ

How often does lineage update?

Every 60 seconds. We crawl your connected sources and update the graph automatically.

Can I connect on-premise databases?

Yes. For databases not exposed to the internet, you can run a self-hosted agent that connects to MyDo.io.

Is my data secure?

Yes. We use AES-256 encryption for credentials at rest and TLS for data in transit. We never store actual data—only metadata and lineage.

What happens if I disconnect a source?

Historical lineage is preserved. New data won't flow through that connection, and we won't fetch fresh metadata.

Can I export lineage data?

Yes. Use the API to export JSON or CSV format. Contact support for large exports.

Do you offer on-premise deployment?

Yes, for Enterprise plans. Contact sales for pricing and setup.

Troubleshooting

Connection Failed

  • Check credentials are correct
  • Verify IP allowlist (if applicable)
  • Ensure data source is online
  • Check API rate limits aren't exceeded

Lineage Not Updating

  • Click "Refresh" in the graph header
  • Check if connection is still active (Settings → Connections)
  • Verify source data actually changed
  • Contact support if issue persists

High Latency

  • For large graphs (1000+ nodes), try 2D view instead of 3D
  • Zoom to specific sub-graph instead of viewing entire graph
  • Check browser browser performance (Chrome recommended)

Alerts Not Triggering

  • Verify alert rule is enabled (Settings → Quality Rules)
  • Check notification method configured (email, Slack, PagerDuty)
  • Verify threshold settings (maybe data hasn't crossed threshold yet)

Contact & Support

📧 Email

support@mydo.io

Response time: 2-4 hours

💬 Slack

Join our community Slack at mydo.io/slack

Live chat with team and users

📚 Resources

Read our blog →

Deep dives into data lineage, observability, and best practices

Hours of Operation

  • Monday–Friday: 8 AM–6 PM PT
  • Emergency support: 24/7 for Enterprise customers

Enterprise support? Contact sales

Ready to See Your Data Lineage?

Try MyDo.io free for 14 days. No credit card required.

Get Started Free →