# robots.txt User-agent: * Allow: / # Private / auth pages — no indexing needed Disallow: /dashboard/ Disallow: /login # /register intentionally indexable — it's the public sign-up landing # and had inbound links that were wasting equity while blocked # (see May 2026 SEO audit). The page itself adds noindex via metadata # if we ever want to block again without robots.txt. Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page # /video$ — match the call page exactly so /video_animation (public # landing) keeps indexing. Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ # Every page under /feed needs a session (closed 2026-10-12) — the feed, # advisor profiles and the single-post permalink all redirect a visitor # without one to /login. No anchor and no exceptions: this prefix rule is # meant to cover the whole section. They are out of sitemap.xml and the HTML # sitemap for the same reason. Disallow: /feed # /posts/ is the same post body at a different URL — its page is a sibling # of feed/ in the (feed) route group, so it survived the feed closure once # already. Closed with it on 2026-10-12 and out of sitemap.xml too. Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ # AI answer-engine bots — ALLOW the public site so Solvemeet appears in # ChatGPT, Gemini, Claude, Perplexity and Apple Intelligence answers. # # ⚠️ Each group below MUST repeat the private-route Disallow list. A crawler # obeys ONLY the group that names it — it does not inherit from `User-agent: *`. # Before this change these ten agents had a bare `Allow: /` and were therefore # permitted to crawl /api/, /dashboard/, /booking, /invoice, /payment/ and the # whole logged-in feed. User-agent: GPTBot Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: ClaudeBot Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: Google-Extended Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: meta-externalagent Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: Applebot-Extended Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: PerplexityBot Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: YouBot Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: Amazonbot Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: CCBot Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ User-agent: Bytespider Allow: / Disallow: /dashboard/ Disallow: /login Disallow: /forgotpassword Disallow: /reset-password Disallow: /change-password Disallow: /verify-adviser Disallow: /payment/ Disallow: /chat-page Disallow: /video$ Disallow: /video/ Disallow: /notifications Disallow: /auth/ Disallow: /feed Disallow: /posts Disallow: /booking Disallow: /invoice Disallow: /api/ # Sitemap Sitemap: https://www.solvemeet.com/sitemap.xml