> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cleartalk.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape websites and save to knowledge base



## OpenAPI

````yaml post /knowledge-base/learn
openapi: 3.0.0
info:
  title: ClearTalk API
  description: The ClearTalk API Documentation
  version: '1.0'
  contact:
    name: Developer Support
    email: devsupport@cleartalk.ai
servers:
  - url: https://api.cleartalk.ai
    description: ClearTalk AI API Server
security: []
tags: []
paths:
  /knowledge-base/learn:
    post:
      tags:
        - Knowledge Base
      summary: Scrape websites and save to knowledge base
      operationId: KnowledgeBaseController_saveWebKnowledge
      parameters: []
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateKnowledgeBaseWebDto'
      responses:
        '201':
          description: Web scraping completed and knowledge base created successfully
        '400':
          description: Invalid URLs or data
        '422':
          description: Unable to scrape provided URLs
      security:
        - apiKey: []
components:
  schemas:
    CreateKnowledgeBaseWebDto:
      type: object
      properties:
        type:
          type: string
          description: Type of knowledge base
          example: web
          default: web
        name:
          type: string
          description: Name of the knowledge base
          example: Documentation
          maxLength: 100
        description:
          type: string
          description: Optional description of the knowledge base content
          example: Complete product documentation
          maxLength: 500
        urls:
          description: Array of URLs to scrape (maximum 100 URLs)
          example:
            - https://docs.example.com/overview
            - https://docs.example.com/api
          maxItems: 100
          type: array
          items:
            type: string
      required:
        - type
        - name
        - urls
  securitySchemes:
    apiKey:
      type: apiKey
      in: header
      name: x-api-key
      description: API Key for authentication

````