check_robots
Check a pasted robots.txt
Match a robots.txt file the caller pastes against Botcard's AI crawler tokens, using RFC 9309. Each row is allow or disallow plus that token's vendor stance. This tool does not fetch any website. Do not send url, host, target, website, robots_url, or fetch.
idspathrobots_txt必填