API docs by Redocly

VEGA 引擎 · 发现任务 API (0.1.0)

Download OpenAPI specification:

APIs for Vega Backend resource-discovery tasks.

A DiscoverTask is an execution audit record created through one of three paths:

  • Manual trigger: POST /catalogs/{id}/discover, documented at the end of this specification.
  • Scheduled trigger: a worker starts it from a DiscoverSchedule; see discover-schedule.yaml.
  • Resource refresh: POST /resources/{id}/discover; see resource.yaml.

Single-resource count_only supports table, index, and logical_view categories. Dataset counts are computed during reads; unsupported single-resource categories return 400 before task creation. Catalog-level count tasks skip Dataset resources and include them in skipped_count.

queue_priority is a read-only integer computed by the server; larger values have higher priority. The default priorities are 10 for scheduled catalog discovery, 20 for manual catalog discovery, and 30 for resource refresh. Workers dequeue tasks strictly by priority and execute tasks for the same catalog serially.

The worker advances the state machine (pending → running → completed/failed/cancelled) in one direction. Users cannot cancel, retry, or restart a task. The public API exposes only list, get, and cleanup operations.

Bulk deletion follows the endpoint rules in [vega-backend/CLAUDE.md]: IDs are passed as a comma-separated path parameter to DELETE /discover-tasks/{ids}. Parent-resource filters use query parameters such as catalog_id, schedule_id, and resource_id; nested list endpoints are not provided.

List discovery tasks

Returns paginated discovery tasks and supports filtering by catalog_id, resource_id, schedule_id, status, strategy, and trigger_type.

Authorizations:
OAuth2
query Parameters
catalog_id
string

Filter by owning catalog

resource_id
string

Filters by the Resource being refreshed. A non-empty value matches only resource-refresh tasks; catalog-discovery tasks leave this field empty.

schedule_id
string

Filters by the owning DiscoverSchedule. Tasks with trigger_type=manual always have an empty value.

status
Array of strings
Items Enum: "pending" "running" "completed" "failed" "cancelled"

Filters by status. Repeat this parameter for multiple values, which are combined with OR.

strategy
string
Enum: "full_sync" "create_only" "cleanup_only" "count_only"

Filters by discovery strategy.

trigger_type
string
Enum: "manual" "scheduled"

Filters by trigger type.

offset
integer <int64> >= 0
Default: 0

Pagination offset. Must be at least 0 and defaults to 0

limit
integer <int64> [ -1 .. 1000 ]
Default: 20

Page size from 1 to 1000. Use -1 to disable pagination. Defaults to 20.

sort
string
Default: "create_time"
Enum: "create_time" "start_time" "finish_time" "last_progress_time"

Sort field. Results default to descending creation time. A zero timestamp is sorted as a normal database value, first in ascending order and last in descending order.

direction
string
Default: "desc"
Enum: "asc" "desc"

Sort direction

Responses

Response samples

Content type
application/json
{
  • "entries": [
    ],
  • "total_count": 0
}

Get discovery task details

Authorizations:
OAuth2
path Parameters
id
required
string

DiscoverTask ID. DELETE accepts a comma-separated list of IDs.

Responses

Response samples

Content type
application/json
{
  • "id": "string",
  • "catalog_id": "string",
  • "resource_id": "string",
  • "resource_name": "string",
  • "catalog_name": "string",
  • "schedule_id": "string",
  • "strategy": "full_sync",
  • "trigger_type": "manual",
  • "queue_priority": 0,
  • "status": "pending",
  • "progress": 0,
  • "message": "string",
  • "start_time": 0,
  • "finish_time": 0,
  • "last_progress_time": 0,
  • "result": {
    },
  • "creator": {
    },
  • "create_time": 0
}

Delete discovery tasks atomically

The operation is atomic: deletion starts only after every ID passes validation. If any validation fails, nothing is deleted.

Validation order:

  1. If any ID is pending or running, return 409 VegaBackend.DiscoverTask.HasRunningExecution with { running_ids: [...] } in error_details. This status check cannot be bypassed because deleting a task while a worker writes it would leave orphaned data. Wait until the task reaches completed, failed, or cancelled.
  2. If any ID does not exist and ignore_missing is false, return 404 VegaBackend.DiscoverTask.NotFound with { missing_ids: [...] } in error_details.
  3. If all IDs pass, delete them and return 204.
Authorizations:
OAuth2
path Parameters
id
required
string

DiscoverTask ID. DELETE accepts a comma-separated list of IDs.

query Parameters
ignore_missing
boolean
Default: false

Relaxes the existence check: missing IDs are treated as already deleted and silently skipped, while other IDs are deleted normally. This does not bypass the pending/running check.

Responses

Response samples

Content type
application/json
{
  • "error_code": "string",
  • "description": "string",
  • "error_details": null,
  • "metadata": { }
}

Manually trigger catalog resource discovery

Triggers resource discovery for the specified catalog. Asynchronous: the server creates a DiscoverTask with trigger_type=manual, immediately returns its task ID, and lets a worker perform discovery asynchronously. Poll GET /discover-tasks/{id} with the returned ID to observe progress.

The optional body is {"strategy":"full_sync|create_only|cleanup_only|count_only"} and defaults to full_sync. This RPC-style nested endpoint represents an action on a catalog rather than creation of a top-level task.

Metadata discovery requires type=physical. Logical catalogs accept only count_only, which counts their logic views with the registered view service and skips unsupported categories.

Authorizations:
OAuth2
path Parameters
id
required
string

Catalog ID

Request Body schema: application/json
optional
strategy
string
Enum: "full_sync" "create_only" "cleanup_only" "count_only"

Responses

Request samples

Content type
application/json
{
  • "strategy": "full_sync"
}

Response samples

Content type
application/json
{
  • "id": "string"
}