velven
Menu

VelvenBot

VelvenBot is Velven's crawler and clip recorder. If you see it in your logs, someone listed a page of yours on Velven, or tried to. A listed page stays on your site. Velven plays it in a frame on its Velven page and never hosts a copy.

View as Markdown

How to recognise it

Every request Velven makes to a page listed from your site has VelvenBot in its user agent. The crawler sends:

Crawler
Mozilla/5.0 (compatible; VelvenBot/1.0; +https://velven.ai/docs/bot)

The clip recorder, which is a Chrome browser, sends:

Clip recorder
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36 VelvenBot/1.0 (+https://velven.ai/docs/bot)

The addresses these requests come from change. The VelvenBot token doesn't.

People who play the space on Velven load your page in their own browsers, with their own user agents, as they would on any site that embeds it.

One Velven browser doesn't name VelvenBot. After each new version of a space hosted on Velven itself (at a velven.io address) goes live, Velven looks at that space in a Chrome browser, opening it on its Velven page the way its players get it. This browser never opens a page listed from your site. If a space hosted on Velven loads files from your site, those requests look like a player's.

What it fetches

The crawler fetches only pages that someone listed or tried to list, and it makes only these requests:

  • When a page is listed, claimed or moved to a new address: one GET of the URL. It follows up to 5 redirects and reads at most the first 512 KB of HTML. From that it reads the title, description, preview image, the creator's proof tag, any leaderboard block, and the response headers for framing. It may also fetch /.well-known/velven, an older form of the proof.
  • Every 6 hours after that: one GET of each listed page, plus a second one 2 seconds later if the first fails. It checks that the page still answers and allows framing, reads its leaderboard block, and notes whether the HTML changed.
  • When the creator presses Check again or Check my page now on Velven: the same GET, right away.
  • When a clip is recorded: one GET of the page to read the words it shows, for the text on the space's Velven page.
  • It never crawls a page for links, and fetches nothing beyond the listed page, except what the page itself loads in the clip recorder.

The clip recorder

Velven records a 5-second clip of each space for its tile. To record it, Velven opens the page in a Chrome browser and plays it for up to 10 minutes. The browser loads whatever the page loads, as a visitor's browser would.

It records after the page is listed, when the page has changed, and when the creator asks for a new clip. When a page changes, it's also recorded once more so Velven can check it again, at most once a day. The recorder's user agent names VelvenBot, so blocking VelvenBot stops it too.

Block it

Answer any user agent containing VelvenBot with a 403. Velven then can't list the page or record a clip of it.

The browser that opens spaces hosted on Velven doesn't name VelvenBot, so the block doesn't stop the files your site serves to a hosted space that loads them. Those are the same requests the space's players make.

Note: A page that's already listed stays listed, because Velven's background check treats a 403 as a page that's up but private. To have it taken off, ask Velven as described below.

To have a page of yours taken off Velven now, use Feedback on its Velven page, or email privacy@velven.ai. The privacy page has the rest.