# Web resources

This page covers the `webresources_*` options, which connect CKEditor AI On-Premises to a web scraping endpoint you run. You need it only if users attach web pages to conversations or the agent fetches pages during a request.

> **Note**
>
> CKEditor AI On-Premises does not include a web scraping tool. You provide the tool, run it, and connect it through the endpoint below.

To connect your web scraping endpoint, set these options:

* `webresources_enabled` – must be set to `true`.
* `webresources_endpoint` – the URL of your gateway. The service sends the scrape requests to this URL.
* `webresources_request_timeout` – request timeout in milliseconds (optional, default: `30000`).

**JSON**

```json
{
	"webresources_enabled": true,
	"webresources_endpoint": "[WEBRESOURCES_ENDPOINT]",
	"webresources_request_timeout": 30000
}
```

**Environment variable**

```bash
WEBRESOURCES_ENABLED="true"
WEBRESOURCES_ENDPOINT=[WEBRESOURCES_ENDPOINT]
WEBRESOURCES_REQUEST_TIMEOUT=30000
```

<a id="request-format">

## Request format

Your endpoint must accept POST requests with this JSON body:

```json
{
  "url": "https://example.com/page-to-scrape"
}
```

Where:

* `url` (required) – the URL of the page to scrape.

<a id="response-format">

## Response format

Your endpoint must return this response:

```json
{
  "type": "text/html",
  "data": "<html>...</html>"
}
```

Where:

* `type` (required) – the content type of the scraped data. Allowed values: `text/html`, `text/markdown`.
* `data` (required) – the scraped website content.

<a id="next-steps">

## Next steps

* [Web search](web-search.md) – Connect the endpoint that runs the searches models request.
* [Network requirements](requirements.md#network-requirements) – Check the outbound destinations a deployment needs.
* [Required configuration](configuration.md) – Set the secret keys and the database, Redis, and storage options.

---

Full index of the Cloud Services documentation: [llms.txt](../../../llms.txt)
