diff options
| author | Aaron Ball <nullspoon@oper.io> | 2023-09-06 15:57:44 -0600 |
|---|---|---|
| committer | Aaron Ball <nullspoon@oper.io> | 2023-09-06 15:57:44 -0600 |
| commit | 398b0af22200827771a81b8757625f4c9a6a248b (patch) | |
| tree | 1ecd67d8c7933ba4c3c9e5156bfaaf122c2877bc | |
| parent | c9bfeb885a6b5f4ebea3ef5ff26023ac506daca3 (diff) | |
| download | oper.io-398b0af22200827771a81b8757625f4c9a6a248b.tar.gz oper.io-398b0af22200827771a81b8757625f4c9a6a248b.tar.xz | |
Add robots.txt
This should stop amazonbot from trying to download every single snapshot
of every single ref of every single repo hosted by cgit. :facepalm:
| -rw-r--r-- | robots.txt | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/robots.txt b/robots.txt new file mode 100644 index 0000000..3f4b679 --- /dev/null +++ b/robots.txt @@ -0,0 +1,2 @@ +User-agent: * +Disallow: /src/*/snapshot/*.tar.gz |