This commit is contained in:
parent
f61efc7a51
commit
9aa761455f
1 changed files with 2 additions and 0 deletions
|
|
@ -36,6 +36,8 @@ User-agent: Factset_spyderbot
|
||||||
User-agent: FirecrawlAgent
|
User-agent: FirecrawlAgent
|
||||||
User-agent: FriendlyCrawler
|
User-agent: FriendlyCrawler
|
||||||
User-agent: Gemini-Deep-Research
|
User-agent: Gemini-Deep-Research
|
||||||
|
User-agent: Google-CloudVertexBot
|
||||||
|
User-agent: GoogleOther
|
||||||
User-agent: GPTBot
|
User-agent: GPTBot
|
||||||
User-agent: iaskspider/2.0
|
User-agent: iaskspider/2.0
|
||||||
User-agent: ICC-Crawler
|
User-agent: ICC-Crawler
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue