🍋 Developer Tools

Robots.txt Generator

Create robots.txt files with AI crawler blocking options

Free Nur im Browser No sign-up

Eingabe

chars words sentences lines

Zweite Eingabe

robots.txt-Optionen

Alle Crawler erlauben

Gesperrte Pfade (kommagetrennt)

Sitemap-URL

Crawl-Verzoegerung (Sekunden)

Ergebnisse

Ergebnis

Ueber Robots.txt Generator

Generate robots.txt files for your website with easy-to-use options. Include rules for search engine crawlers, AI bots, and custom paths. Supports sitemap URL declaration and crawl delay.

So funktioniert es

Configure your robots.txt rules through simple options. Toggle allow/disallow for common paths, add sitemap URLs, set crawl delays, and optionally block known AI crawlers.

Schritt fuer Schritt

1 [
2 '
3 C
4 h
5 o
6 o
7 s
8 e
9
10 t
11 o
12
13 a
14 l
15 l
16 o
17 w
18
19 a
20 l
21 l
22
23 o
24 r
25
26 s
27 e
28 l
29 e
30 c
31 t
32 i
33 v
34 e
35 l
36 y
37
38 d
39 i
40 s
41 a
42 l
43 l
44 o
45 w
46
47 p
48 a
49 t
50 h
51 s
52 '
53 ,
54
55 '
56 A
57 d
58 d
59
60 p
61 a
62 t
63 h
64 s
65
66 t
67 o
68
69 b
70 l
71 o
72 c
73 k
74
75 (
76 e
77 .
78 g
79 .
80 ,
81
82 /
83 a
84 d
85 m
86 i
87 n
88 /
89 ,
90
91 /
92 a
93 p
94 i
95 /
96 )
97 '
98 ,
99
100 '
101 O
102 p
103 t
104 i
105 o
106 n
107 a
108 l
109 l
110 y
111
112 a
113 d
114 d
115
116 y
117 o
118 u
119 r
120
121 s
122 i
123 t
124 e
125 m
126 a
127 p
128
129 U
130 R
131 L
132 '
133 ,
134
135 '
136 S
137 e
138 t
139
140 c
141 r
142 a
143 w
144 l
145
146 d
147 e
148 l
149 a
150 y
151
152 i
153 f
154
155 n
156 e
157 e
158 d
159 e
160 d
161 '
162 ,
163
164 '
165 C
166 o
167 p
168 y
169
170 t
171 h
172 e
173
174 g
175 e
176 n
177 e
178 r
179 a
180 t
181 e
182 d
183
184 r
185 o
186 b
187 o
188 t
189 s
190 .
191 t
192 x
193 t
194
195 c
196 o
197 n
198 t
199 e
200 n
201 t
202 '
203 ]

Tipps

[
'
A
l
w
a
y
s
i
n
c
l
u
d
e
a
S
i
t
e
m
a
p
d
i
r
e
c
t
i
v
e
f
o
r
b
e
t
t
e
r
c
r
a
w
l
i
n
g
'
,
'
U
s
e
C
r
a
w
l
-
d
e
l
a
y
s
p
a
r
i
n
g
l
y
—
m
o
s
t
m
a
j
o
r
c
r
a
w
l
e
r
s
i
g
n
o
r
e
i
t
'
,
'
T
e
s
t
y
o
u
r
r
o
b
o
t
s
.
t
x
t
w
i
t
h
G
o
o
g
l
e
S
e
a
r
c
h
C
o
n
s
o
l
e
'
,
'
T
h
e
A
I
c
r
a
w
l
e
r
b
l
o
c
k
l
i
s
t
i
n
c
l
u
d
e
s
G
P
T
B
o
t
,
C
C
B
o
t
,
a
n
d
o
t
h
e
r
s
'
]

Categories