Data Source

Create data source

POST /v2/team/datasets/{id}/datasources

Create a data source in a specified dataset. You can only create data sources in datasets that you have created.

The data source can be in one of the following formats: .csv, .tsv, .md, .mdx, .json, .txt, .pdf, .pptx, .ppt, .doc, .docx, .xls, or .xlsx.

Body Request Parameters

{
  "name": "test.csv",
  "type": "FILE",
  "user_id": "tmm-dsfasdfasdfa",
  "url": "https://s3.amazonaws.com/xxxtest/user/clvl4cad2001q01l1m522hxlu/upload/f9773f1e-cd68-489a-8121-d566ca9218b1.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240924T143419Z&X-Amz-SignedHeaders=host&X-Amz-Expires=599&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20240924%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=9ca0c58d508926a5811818041d557ffb53c64025dae94c0855280d457c7089a2"
}

Request Parameters

Detailed Explanation

id: Target dataset ID.

To query the list of datasets you have access to, call the GET /v2/team/datasets interface.

» url: File URL for public access.

Either url or file_object_key must be specified, but not both.

Only files with the following extensions are supported: .csv, .tsv, .md, .mdx, .json, .txt, .pdf, .pptx, .ppt, .doc, .docx, .xls, or .xlsx.

» file_object_key: The object storage path of the file uploaded locally.

Either url or file_object_key must be specified, but not both.

Supported file extensions include: .csv, .tsv, .md, .mdx, .json, .txt, .pdf, .pptx, .ppt, .doc, .docx, .xls, or .xlsx.

How to obtain the file’s file_object_key:

When using the Upload file interface to complete the file upload, the file_object_key of the file will be returned.

Example Response

{
  "code": 0,
  "data": {
    "id": "datasource-cadsgfsdagasgadsg",
    "dataset_id": "dataset-dagasdgasgasg",
    "name": "test.csv",
    "type": "FILE",
    "status": "synching"
  }
}

Response

Response Data Structure

Status Code 200

Enumeration Values

Response Header

List data sources

GET /v2/team/datasets/{id}/datasources

Return the list of data sources in the specified dataset. When using this interface, please note:

• Ensure the specified dataset and your API Key belong to the same project.
• To view datasets you have access to within the project, call the GET /v2/team/datasets interface.

Request Parameters

Detailed Explanation

id: Target dataset ID.

To query the list of datasets you have access to, call the GET /v2/team/datasets interface.

status: Data source status. If this parameter is specified, only data sources in the specified status will be returned. Optional values include:

• invalid: Pending processing.
• synching: Processing.
• synched: Successfully synchronized.

If not specified, all data sources will be returned.

Multiple statuses can be specified as a comma-separated list, and any matching status’s data source will be returned.

Enumeration Values

Example Response

{
  "code": 0,
  "data": {
    "total_items": 1,
    "page_size": 10,
    "page_number": 1,
    "records": [
      {
        "id": "datasource-cadsgfsdagasgadsg",
        "dataset_id": "dataset-dagasdgasgasg",
        "name": "test.csv",
        "type": "FILE",
        "status": "synching"
      }
    ]
  }
}

Response

Response Data Structure

Status Code 200

Enumeration Values

Response Header

Delete data source

DELETE /v2/team/datasets/{dataset_id}/datasources/{datasource_id}

Delete a data source from the specified dataset. Once deleted, the data source cannot be recovered.

You can only delete data sources in your own datasets.

Body Request Parameters

{
  "user_id": "tmm-dafasdfasdfasdf"
}

Request Parameters

Detailed Explanation

dataset_id: Target dataset ID.

To query the list of datasets you have access to, call the GET /v2/team/datasets interface.

datasource_id: ID of the data source to delete.

To query the data sources in a specified dataset, call the GET /v2/team/datasets/{id}/datasources interface.

Example Response

200 Response

{
  "code": 0,
  "data": {}
}

Response

Response Data Structure

Status Code 200

Response Header

Get data source

GET /v2/team/datasets/{dataset_id}/datasources/{datasource_id}

Retrieve information about a specified data source.

Request Parameters

Detailed Explanation

dataset_id: The dataset ID where the target data source is located.

To query the list of datasets you have access to, call the GET /v2/team/datasets interface.

datasource_id: The target data source ID.

To query the data sources in a specified dataset, call the GET /v2/team/datasets/{id}/datasources interface.

Example Response

{
  "code": 0,
  "data": {
    "id": "datasource-cadsgfsdagasgadsg",
    "dataset_id": "dataset-dagasdgasgasg",
    "name": "test.csv",
    "type": "FILE",
    "status": "synching"
  }
}

Response

Response Data Structure

Status Code 200

Enumeration Values

Response Header

Create data source without specifying a dataset

POST /v2/team/datasources

This interface is used to create a data source directly without specifying a dataset.

When invoking this interface, MAXIR AI will automatically create a dataset for the data source. Please save the dataset ID in the response for future operations, such as associating the dataset with a job for data analysis and exploration.

Body Request Parameters

{
  "name": "test.csv",
  "type": "FILE",
  "user_id": "tmm-dafasdfasdfasdf",
  "file_object_key": "/tmp/sdgsagdsgsadgasdg"
}

Request Parameters

Example Response

{
  "code": 0,
  "data": {
    "id": "datasource-cadsgfsdagasgadsg",
    "dataset_id": "dataset-dagasdgasgasg",
    "name": "test.csv",
    "type": "FILE",
    "status": "synching"
  }
}

Response

Response Data Structure

Status Code 200

Enumeration Values

Response Header

Presign data source

POST /v2/team/datasets/{dataset_id}/datasources/{datasource_id}/presign

This interface is used to generate a pre-signed URL (Presigned URL) for the specified data source, allowing you to download the corresponding data source through this URL.

The pre-signed URL has an expiration period, so be sure to complete the download of the data source before the URL expires.

Body Request Parameters

{
  "expires_in": 600,
  "user_id": "tmm-dafasdfasdfasdf"
}

Request Parameters

Detailed Explanation

dataset_id: The dataset ID where the target data source is located.

To query the list of datasets you have access to, call the GET /v2/team/datasets interface.

datasource_id: The target data source ID.

To query the data sources in a specified dataset, call the GET /v2/team/datasets/{id}/datasources interface.

Example Response

{
  "code": 0,
  "data": {
    "presigned_url": "string",
    "expires_at": "2024-11-13T14:15:22.123Z"
  }
}

Response

Response Data Structure

Status Code 200

Response Header