web: keep crawlers out of search, preview and the API
This commit is contained in:
1 file changed
+4
@@ -0,0 +1,4 @@
|
|||||||
|
User-agent: *
|
||||||
|
Disallow: /sok
|
||||||
|
Disallow: /preview/
|
||||||
|
Disallow: /api/
|
||||||
Reference in new issue
Block a user