Skip to main content
POST
/
knowledge-base
/
learn
Scrape websites and save to knowledge base
curl --request POST \
  --url https://api.cleartalk.ai/knowledge-base/learn \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '
{
  "type": "web",
  "name": "Documentation",
  "urls": [
    "https://docs.example.com/overview",
    "https://docs.example.com/api"
  ],
  "description": "Complete product documentation"
}
'

Documentation Index

Fetch the complete documentation index at: https://docs.cleartalk.ai/llms.txt

Use this file to discover all available pages before exploring further.

Authorizations

​
x-api-key
string
header
required

API Key for authentication

Body

application/json
​
type
string
default:web
required

Type of knowledge base

Example:

"web"

​
name
string
required

Name of the knowledge base

Maximum string length: 100
Example:

"Documentation"

​
urls
string[]
required

Array of URLs to scrape (maximum 100 URLs)

Maximum array length: 100
Example:
[
  "https://docs.example.com/overview",
  "https://docs.example.com/api"
]
​
description
string

Optional description of the knowledge base content

Maximum string length: 500
Example:

"Complete product documentation"

Response

Web scraping completed and knowledge base created successfully