Question 1

What are Scraped Domains in Deskhero?

Accepted Answer

Scraped Domains lets Deskhero crawl your company's website and use the content as AI knowledge. The page, found under Configurations in the Superpowers section, is described as "Add domains to scrape for data", and the empty state invites you to "Let us scrape your company's website for AI Knowledge". The crawled pages help power AI answers and suggestions in your workspace.

Question 2

How do I add a website for Deskhero to crawl?

Accepted Answer

1) Go to Configurations and open Scraped Domains (visible to Owners and Admins). 2) Click "Add Domain". 3) Enter your domain, for example company.com; you do not need to include http or https, as any protocol prefix is removed automatically. 4) Click "Verify Domain". The domain is added and crawling is kicked off right away.

Question 3

Who can manage Scraped Domains?

Accepted Answer

Only workspace Owners and Admins. The Scraped Domains page is restricted to those roles, and edit and delete actions are disabled for regular users with the message "Not enough rights to do that".

Question 4

Do I need to verify ownership of a domain before Deskhero scrapes it?

Accepted Answer

No separate ownership verification step is required anymore. When you click "Verify Domain" while adding it, the domain is accepted and marked verified right away, and the crawl is scheduled immediately. Deskhero instead controls what gets crawled and blocks domains that are not allowed, such as public email provider domains.

Question 5

Why does Deskhero say my domain is not allowed when I try to add it for scraping?

Accepted Answer

Public email provider domains (like free webmail domains) cannot be added as scraped domains, and Deskhero's own domain is also blocked; these attempts return the error "Domain not allowed". Add your own company website domain instead. Also note that the domain name cannot contain a slash and must be a valid domain of at least 2 characters.

Question 6

Can I add the same domain to Scraped Domains twice?

Accepted Answer

No. Each domain can only exist once in your workspace; trying to add a duplicate fails validation because the domain name already exists. Edit or delete the existing entry instead.

Question 7

How do I change or remove a scraped domain?

Accepted Answer

On the Scraped Domains page: to change it, click the edit button on the row, adjust the domain name in the Edit Email Domain dialog, and click "Update Email Domain". To remove it, click the delete button and confirm "Are you sure you want to delete the domain?" with the "Delete domain" action.

Question 8

Does Deskhero scrape every page on my website, including duplicates in other languages?

Accepted Answer

Deskhero discovers pages from your site (including via its sitemap) and crawls them, but it automatically skips pages that are language or alternate duplicates of another page, and pages excluded by the domain's crawl rules. This keeps the AI knowledge focused on unique content instead of the same page repeated in several languages.

Question 9

Is there a limit on website scraping?

Accepted Answer

Yes. Your plan limits both how many scraped domains you can add (adding one beyond the limit is rejected with a plan usage error) and how many web pages can be scraped, which appears as "Web scraped pages" in your plan details. Check the Plans page under Configurations for your allowances.

Question 10

Can I re-crawl a single page of a scraped domain?

Accepted Answer

Yes. On the domain's detail page, find the page in the list and click its Rescrape this page button (the retry icon). The page returns to the Queued status and Deskhero fetches it again. This is useful when the content on that page has changed and you want Deskhero's knowledge to be up to date.

Question 11

How do I add a specific page of my website to be scraped?

Accepted Answer

On the domain's detail page, click Queue page. Enter the path of the page on that domain (it must start with /, for example /pricing) and confirm. Deskhero adds that page to the crawl queue. If the path cannot be queued (for example because a skip-path rule excludes it) Deskhero tells you it was rejected.

Question 12

How do I control which pages of my website Deskhero crawls?

Accepted Answer

Open the domain's detail page and use the crawl settings panel: Sitemap URL (a path or full URL to your sitemap; leave empty to use /sitemap.xml), Focus on paths (one URL pattern per line, for example /help*, to limit crawling to those areas; leave empty to crawl the whole site), and Skip paths (one pattern per line, for example /sv/*, which take priority over focus paths). Changes save automatically shortly after you stop typing, and Deskhero re-applies the new rules to the pages it already knows about.

Question 13

What is the Primary language setting on a scraped domain?

Accepted Answer

When your site publishes the same page in several languages, the Primary language setting tells Deskhero which language version to keep so it does not store duplicates. Enter a language code (for example en) or leave it empty to let Deskhero auto-detect; the page shows the language it detected. Pages that are duplicate-language copies are marked Ignored (duplicate).

Question 14

Why can't I change a scraped domain's name?

Accepted Answer

The domain name is fixed once the domain is created. The detail page shows it as read-only, with a note that it cannot be changed after creation. If you need to crawl a different domain, add it as a new scraped domain and remove the old one.

Website scraping

Did not find what you were looking for?