Create an extraction task

POST

/302/crawler/task

Create a web crawling task

Price: Charged according to the model called

Request

Header Params

Body Params application/x-msgpack

Example

{
    "urls": [
        "https://news.302.ai"
    ],
    "target": "AI model and detailed introduction",
    "schema": {
        "topic": {
            "type": "string",
            "description": "Topic of the article"
        },
        "view": {
            "type": "number",
            "description": "Article views"
        },
        "detail": {
            "type": "string",
            "description": "Article details summary"
        },
        "keywords": {
            "type": "array",
            "description": "Article keywords, up to 5",
            "items": {
                "type": "string"
            }
        },
        "hot": {
            "type": "boolean",
            "description": "Whether it is a popular article with more than 10 views"
        }
    },
    "recursiveConfig": {
        "matchType": "auto",
        "searchModel": "default",
        "outputMd": true,
        "maxDepth": 1,
        "maxUrls": 1
    },
    "proxyConfig": {
        "proxyUrl": "",
        "proxyUsername": "",
        "proxyPassword": ""
    },
    "browserConfig": {
        "headless": true,
        "browserType": "chromium",
        "viewportWidth": 1920,
        "viewportHeight": 1080,
        "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
        "cacheEnabled": false,
        "jsEnabled": false,
        "lightMode": false,
        "textMode": false,
        "cookies": [],
        "headers": {
            "Accept-Language": "zh-CN"
        }
    }
}

Request Code Samples

Shell

JavaScript

Java

Swift

PHP

Python

HTTP

Objective-C

Ruby

OCaml

Dart

curl --location --request POST 'https://api.302.ai/302/crawler/task' \
--header 'Authorization: Bearer ' \
--header 'Model: gpt-4o-mini' \
--header 'Content-Type: application/x-msgpack' \
--data-raw '{
    "urls": [
        "https://news.302.ai"
    ],
    "target": "AI model and detailed introduction",
    "schema": {
        "topic": {
            "type": "string",
            "description": "Topic of the article"
        },
        "view": {
            "type": "number",
            "description": "Article views"
        },
        "detail": {
            "type": "string",
            "description": "Article details summary"
        },
        "keywords": {
            "type": "array",
            "description": "Article keywords, up to 5",
            "items": {
                "type": "string"
            }
        },
        "hot": {
            "type": "boolean",
            "description": "Whether it is a popular article with more than 10 views"
        }
    },
    "recursiveConfig": {
        "matchType": "auto",
        "searchModel": "default",
        "outputMd": true,
        "maxDepth": 1,
        "maxUrls": 1
    },
    "proxyConfig": {
        "proxyUrl": "",
        "proxyUsername": "",
        "proxyPassword": ""
    },
    "browserConfig": {
        "headless": true,
        "browserType": "chromium",
        "viewportWidth": 1920,
        "viewportHeight": 1080,
        "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
        "cacheEnabled": false,
        "jsEnabled": false,
        "lightMode": false,
        "textMode": false,
        "cookies": [],
        "headers": {
            "Accept-Language": "zh-CN"
        }
    }
}'

Responses

🟢201success

application/json

Body

Example

{
    "taskId": "48c4efe2-f890-4b3a-a595-54fb2ad2623a",
    "status": "pending",
    "message": "Task started successfully"
}

Modified at 2025-02-19 02:44:25

Generate Schema

Query extraction progress