# Casa Pattaya crawl policy (Task 7.2) # Keep Disallow paths narrow — do not block public listing/content URLs. # AI User-agent groups do not inherit User-agent: * rules — repeat Disallows. User-agent: * Disallow: /user/ Disallow: /livewire/ Allow: / # Explicit allow for AI / answer-engine crawlers (confirm training use with client). User-agent: GPTBot Disallow: /user/ Disallow: /livewire/ Allow: / User-agent: OAI-SearchBot Disallow: /user/ Disallow: /livewire/ Allow: / User-agent: ChatGPT-User Disallow: /user/ Disallow: /livewire/ Allow: / User-agent: PerplexityBot Disallow: /user/ Disallow: /livewire/ Allow: / User-agent: ClaudeBot Disallow: /user/ Disallow: /livewire/ Allow: / User-agent: Claude-SearchBot Disallow: /user/ Disallow: /livewire/ Allow: / User-agent: Google-Extended Disallow: /user/ Disallow: /livewire/ Allow: / User-agent: Applebot-Extended Disallow: /user/ Disallow: /livewire/ Allow: / Sitemap: https://casapattaya.com/sitemap.xml