What you'll need before you start
Before you begin, make sure you have two things: a Sitepilot account (the free tier is fine for getting started) and access to your website's HTML or CMS admin panel. You don't need FTP access, a developer on call, or any technical knowledge beyond the ability to paste a snippet of text.
If you're running WordPress, you can paste the script into your theme's footer area or use a plugin like Insert Headers and Footers. Shopify users can add it through the theme editor. Webflow, Squarespace, Wix, and virtually every other website builder have a "custom code" area in their settings — that's where the embed goes.
One thing to check: make sure the pages you want the bot to learn from are publicly accessible. If your pricing page is behind a login or your docs site requires authentication, the crawler won't be able to read them. We'll cover workarounds for that in a later step.
Step 1: Create your bot and paste your URL
After signing into Sitepilot, click New Bot in the top navigation. You'll be asked for a name for your bot — something like "Support Bot" or your company name works fine — and the URL of your website.
The URL you provide is the starting point for the crawler. Sitepilot will follow links from that page outward, discovering your pricing page, feature pages, blog posts, FAQ pages, and documentation automatically. You don't need to manually list every URL you want it to read.
If your website has a sitemap (most do — it's usually at yourdomain.com/sitemap.xml), Sitepilot will use it to prioritise pages more efficiently. This is especially helpful for larger sites where the crawler needs to make smart decisions about which pages to visit first.
Step 2: Wait for the crawl to complete
Once you submit your URL, Sitepilot kicks off a crawling process in the background. Here's what actually happens during those few minutes:
- The crawler visits each page on your site and extracts the visible text content — headings, paragraphs, lists, and other readable elements.
- Each page's content is split into overlapping chunks of roughly 300–400 tokens. This chunking is important because it lets the bot retrieve specific sections of your content rather than dumping an entire page into every response.
- Each chunk is converted into a vector embedding using Azure OpenAI's text embedding model. A vector embedding is a list of numbers that represents the meaning of the text — semantically similar chunks end up with similar numbers.
- Those vectors are stored in a database alongside the original chunk text. When a user asks a question, their query is also converted to a vector, and the database finds the chunks with the most similar meaning.
You'll see a progress bar in the Sitepilot dashboard while the crawl runs. Small sites (under 50 pages) typically finish in under five minutes. Once it's complete, the bot enters "ready" state and you can start testing it immediately.
Step 3: Customise your chatbot
Before you go live, spend a few minutes in the bot settings panel to make the chatbot feel like it belongs on your site. The key customisation options are:
- Bot name: This appears in the chat header. Use your product name, your company name, or something friendly like "Aria" — whatever fits your brand.
- Welcome message: The first message the bot sends when someone opens the chat window. Keep it short and action-oriented: "Hi! Ask me anything about [Product] — I'm here to help."
- Primary colour: The colour used for the chat bubble, the send button, and the bot's message bubbles. Match this to your brand colour using a hex code.
- Widget position: Bottom-right is the convention most users expect. Bottom-left works if your existing UI has elements in the bottom-right corner.
- Placeholder text: The greyed-out hint inside the message input. "Ask a question…" is the default, but "Search our docs…" or "How can I help?" can work better depending on context.
These settings can be changed at any time without re-crawling or updating the embed script on your site.
Step 4: Copy and paste the embed script
Once your bot is configured, navigate to the Embed tab in your bot settings. You'll see a single script tag that looks like this:
<script src="https://cdn.sitepilot.ai/widget.js" data-bot="YOUR_BOT_ID" async></script>
The data-bot attribute uniquely identifies your bot — that's the only thing that varies between accounts. Everything else is handled by Sitepilot's CDN-hosted widget script.
Where to put it:
- Standard HTML sites: Paste it just before the closing
</body>tag in your base template or layout file. - WordPress: Use the Appearance → Theme Editor to add it to your
footer.php, or install the "Insert Headers and Footers" plugin and paste it in the footer section. - Shopify: Go to Online Store → Themes → Edit Code →
theme.liquidand paste before</body>. - Google Tag Manager: Create a new Custom HTML tag, paste the script, and set it to fire on All Pages. This avoids touching your site's code entirely.
- Webflow / Squarespace / Wix: Each has a "Custom Code" or "Footer Code" section in the site settings — paste it there.
Script tag vs iframe: which should you use?
Some chatbot platforms offer an iframe embed as an alternative. Here's why the script tag is almost always the better choice:
The script tag loads asynchronously — notice the async attribute in the snippet above. This means the browser renders your page content first and only loads the chatbot widget after. Your Lighthouse scores and Core Web Vitals are completely unaffected.
An iframe, by contrast, is a separate document embedded inside your page. It creates several problems: it can't share styles or fonts with your site, it breaks on mobile because users get a scroll-within-scroll experience, and it's harder to position correctly across different screen sizes. On mobile specifically, iframes with fixed heights frequently clip content or create double-scrollbar situations that feel broken to users.
The script tag also gives Sitepilot's widget access to the full page context — so it can detect things like the current URL, the user's scroll position, and eventually the page's topic — which enables smarter, context-aware responses.
Step 5: Test the live widget
After pasting the embed script, open your site in a private/incognito window. This ensures you're seeing exactly what a first-time visitor sees, without any cached assets or logged-in state interfering.
You should see the chat bubble appear in the corner within a second or two of the page loading. Click it and run through these test questions:
- Ask your top three most common support questions — the ones you get via email every week.
- Ask about your pricing in a few different ways: "How much does this cost?", "What's the cheapest plan?", "Is there a free trial?"
- Ask something the bot definitely won't know — "What's the weather like?" — and confirm it responds politely rather than making something up.
- Ask a question using different phrasing than your site uses, to test whether the semantic retrieval is working (it should match meaning, not just keywords).
If a question gets a wrong or incomplete answer, the fix is almost always adding more specific content to your website and running a re-crawl. The bot's quality is a direct reflection of your content's quality.
Keeping your chatbot up to date
Your chatbot's knowledge is a snapshot of your website at the time of the last crawl. Every time you add a new feature, change your pricing, publish new documentation, or update your FAQ, you should trigger a re-crawl from the Sitepilot dashboard.
The re-crawl process is identical to the initial crawl — it visits all your pages, re-chunks and re-embeds the content, and replaces the old knowledge base. There's no downtime during re-crawls; the bot continues answering questions using the previous knowledge base until the new one is ready, at which point it switches over automatically.
For sites that update frequently, consider building a re-crawl into your content publishing workflow. If you're using a CMS with webhooks (like Contentful, Sanity, or WordPress with a plugin), you can trigger a re-crawl automatically every time you publish a new page — though for most teams, a weekly or bi-weekly manual re-crawl is sufficient.