How to add websites
- Open Knowledge (find it in the sidebar under Configure Or on your Double in homepage)
- Click Website URL in Data Sources section
- Choose Single Page or Multiple Pages
- Paste the website URL
- Click Crawl website and train
What to know
- Single Page - Adds one specific page
- Multiple Pages - Crawls entire site or multiple pages
- Only visible text is imported - Images and hidden content aren’t captured
- Previously crawled pages are excluded - Prevents duplicate content
- Paid articles not supported - Content behind paywalls won’t work
- Social profiles not supported - Use social media connectors instead
Best practices
Add websites that contain:- Services or product pages
- About page and team information
- Blog posts or articles
- Support documentation
- Case studies or testimonials
Managing websites
To delete a website, click the trash icon next to the URL in the Links section. Next steps:- Adding Websites
- Connecting X/Twitter
- Connecting LinkedIn
- Connecting YouTube
- Context Instructions