web: keep crawlers out of search, preview and the API
This commit is contained in:
1 file changed
+4
@@ -0,0 +1,4 @@
|
||||
User-agent: *
|
||||
Disallow: /sok
|
||||
Disallow: /preview/
|
||||
Disallow: /api/
|
||||
Reference in new issue
Block a user