（2022年最新）统计目前市面上国内外各大搜索引擎的UserAgent

如果各位有使用防火墙，例如cloudflare，可以根据搜索引擎蜘蛛的UA使用防火墙规则放行搜索引擎的爬虫，减少对网站收录的影响

百度蜘蛛/Baidu Spider

Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko)Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0;+http://www.baidu.com/search/spider.html)

Mozilla/5.0 (iPhone;CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko)Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0;+http://www.baidu.com/search/spider.html) 

Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)

Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)

Baiduspider-image+(+http://www.baidu.com/search/spider.htm)

360蜘蛛/360 spider

Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.34 (KHTML, like Gecko) Version/11.0 Mobile/15A5341f Safari/604.1;360Spider

Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider

Mozilla/5.0 (Linux; U; Android 4.0.2; en-us; Galaxy Nexus Build/ICL53F) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30; 360Spider

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider

神马蜘蛛

YisouSpider

Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.81 YisouSpider/5.0 Safari/537.36

今日头条蜘蛛

Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181;Bytespider;https://zhanzhang.toutiao.com/

搜狗蜘蛛

Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

Sogou Pic Spider/3.0 (+http : / / www.sogou.com/docs/help/webmasters.htm#07)

Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25(compatible; Sogou web spider/4.0; +http://www.sogou.com/docs/help/webmasters.htm#07)

搜搜蜘蛛（不知道是不是假的，我的日志里出现了）

Sosospider+(+http://help.soso.com/webspider.htm)

花瓣搜索蜘蛛（华为的）

Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)

Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot)

(PetalBot;+https://webmaster.petalsearch.com/site/petalbot)

必应蜘蛛/Bing Bot

Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b

Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

msnbot/1.0 ( http://search.msn.com/msnbot.htm)

谷歌爬虫/Google Bot

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot-Image/1.0

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36

Yandex Bot

Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexMedia/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexCatalog/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexDirect/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexImageResizer/2.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexFavicons/1.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexAntivirus/2.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexVideo/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexBlogs/0.99; robot; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexZakladki/3.0; +http://yandex.com/bots)

Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexAccessibilityBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106

Mozilla/5.0 (compatible; YandexWebmaster/2.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexPagechecker/2.0; +http://yandex.com/bots)

Yandex/1.01.001 (compatible; Win16; I)

Yandex/1.01.001 (compatible; Win16; P)

YandexSomething/1.0

Yandex/1.01.001 (compatible; Win16; H)

Yandex/1.01.001 (compatible; Win16; m)

DuckDuckGo蜘蛛

Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com)

Yahoo雅虎搜索引擎/Yahoo Slurp

Mozilla/5.0 (compatible; Yahoo Ad monitoring; https://help.yahoo.com/kb/yahoo-ad-monitoring-SLN24857.html) tands-prod-eng.hlfs-prod---sieve.hlfs-desktop/

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)

Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/5.0 (Yahoo-MMCrawler/4.0; mailto:[email protected])

Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html)

Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) NOT Firefox/3.5

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp

YahooCacheSystem

Mozilla/5.0 (YahooYSMcm/3.0.0; http://help.yahoo.com)

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/

Applebot

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1)

BLEXBot（可以不要，没啥用，分析SEO用）

Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)

欧洲搜索引擎Qwant的爬虫/Qwantify

Mozilla/5.0 (compatible; Qwantify/1.0; +https://www.qwant.com/)

SemrushBot（可以不要，没啥用，分析SEO用）

Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)

欧洲搜索引擎Exabot爬虫

Mozilla/5.0+(compatible;+Exabot/3.0;++http//www.exabot.com/go/robot)

DotBot（可以不要，没啥用，分析SEO用）

Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, [email protected])

越南搜索引擎COCCOC/coccocbot-web蜘蛛

Mozilla/5.0+(compatible;+coccocbot-web/1.0;++http //help coccoc com/searchengine)

搜索引擎YaCy蜘蛛/yacybot

yacybot (i386 Linux 2.6.28-gentoo-r5; java 1.5.0_18; Europe/en) http://yacy.net/bot.html

搜索引擎SeznamBot

Mozilla/5.0 (compatible; SeznamBot/3.2; +http://napoveda.seznam.cz/en/seznambot-intro/)

俄罗斯搜索引擎Mail.Ru/Mail.Ru bot

Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +https://help.mail.ru/webmaster/indexing/robots)

Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Fast/2.0; +https://help.mail.ru/webmaster/indexing/robots)

Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Img/2.0; +https://help.mail.ru/webmaster/indexing/robots)

Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Video/2.0; +https://help.mail.ru/webmaster/indexing/robots)

Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Robots/2.0; +https://help.mail.ru/webmaster/indexing/robots)

Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Mail/2.0; +https://help.mail.ru/webmaster/indexing/robots)

Mozilla/5.0 (compatible; Mail.RU/2.0)

Mail.RU/2.0

Mozilla/5.0 (compatible; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)

Mojeek 搜索引擎/MojeekBot

Mozilla/5.0 (compatible; MojeekBot/0.10; +https://www.mojeek.com/bot.html)

搜索引擎Seekport蜘蛛/Seekport Crawler

Mozilla/5.0 (compatible; Seekport Crawler; http://seekport.com/)

搜索引擎Gigablast蜘蛛/Gigabot

Gigabot (https://www.gigablast.com/)

InfoTiger搜索引擎蜘蛛/InfoTigerBot

Mozilla/5.0 (compatible; InfoTigerBot/1.9; +https://infotiger.com/bot)

有道已死，不用再看

可能不全，欢迎补充