{"id":18161,"date":"2024-05-24T15:36:53","date_gmt":"2024-05-24T15:36:53","guid":{"rendered":"https:\/\/hostvento.com\/kb\/web-hosting\/how-to-block-bots-with-robot-txt-file-via-cpanel\/"},"modified":"2024-06-17T03:36:55","modified_gmt":"2024-06-17T03:36:55","slug":"how-to-block-bots-with-robot-txt-file-via-cpanel","status":"publish","type":"docs","link":"https:\/\/www.hostvento.com\/kb\/docs\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/","title":{"rendered":"How to Block Bots with robot.txt File via cPanel?"},"content":{"rendered":"\n<p>Bot traffic is non-user traffic that can seriously impair the functionality of your website. Protecting your website against abusive bot traffic is crucial, and there are a few ways to achieve this. This tutorial will teach you how to use your cPanel account&#8217;s robot.txt file to stop bots.<\/p>\n\n\n\n<p>Robot.txt is a text file that you may place in the root directory of your website to allow search engine bots to crawl and index it. It seems that it also determines which files the bots may access, thus configuring it is crucial. This is also the reason why you may use the robot.txt file to prevent these bots from crawling your website in search engines.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">To Block Bots Using Robot.txt File<\/h2>\n\n\n\n<p>1. Log in to your cPanel account.<\/p>\n\n\n\n<p>2. Go to the&nbsp;<strong>FILES<\/strong>&nbsp;section and click on&nbsp;<strong>File Manager<\/strong>.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.hostvento.com\/kb\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file-manager.png\" alt=\"\" class=\"wp-image-11012\"\/><\/figure>\n<\/div>\n\n\n<p>The File Manager interface will open. You will have to create the&nbsp;robot.txt file.<\/p>\n\n\n\n<p>3. Click the&nbsp;<strong>+File<\/strong>&nbsp;button on the top left corner of the interface.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.hostvento.com\/kb\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_file_manager_add_file.jpg\" alt=\"\" class=\"wp-image-12577\"\/><\/figure>\n<\/div>\n\n\n<p>4. Enter the file name as<strong>&nbsp;robot.txt<\/strong>&nbsp;and click the&nbsp;<strong>Create new file<\/strong>&nbsp;button.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.hostvento.com\/kb\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_robot_txt.png\" alt=\"\" class=\"wp-image-12578\"\/><\/figure>\n<\/div>\n\n\n<p>5. Locate and right-click on the&nbsp;<strong>robot.txt<\/strong>&nbsp;file. Select the&nbsp;<strong>Edit<\/strong>&nbsp;option.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.hostvento.com\/kb\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_edit_robot_txt_censored.jpg\" alt=\"\" class=\"wp-image-12580\"\/><\/figure>\n<\/div>\n\n\n<p>The file will open for you to enter\/edit its information.<\/p>\n\n\n\n<p>6. Copy this code in the file.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">RewriteEngine On\nRewriteCond %{HTTP_USER_AGENT} Baiduspider [NC]\nRewriteRule .* \u2013 [F,L]<\/pre>\n\n\n\n<p>This code will block the bot from the Baidu search engine.<\/p>\n\n\n\n<p>7. Click the&nbsp;<strong>Save Changes<\/strong>&nbsp;button and&nbsp;<strong>Close<\/strong>&nbsp;the file.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.hostvento.com\/kb\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_robot_txt_save.jpg\" alt=\"\" class=\"wp-image-12581\"\/><\/figure>\n<\/div>\n\n\n<p><br>With this robot.txt file, the Baidu search engine cannot use bots on your website.<\/p>\n\n\n\n<p>Note<\/p>\n\n\n\n<p>You can also add the bot crawler of any other search engine to the code preventing it from indexing your website.<\/p>\n\n\n\n<p>In short, you can even block bots from other search engines with the robot.txt file by editing it via cPanel File Manager.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Bot traffic is non-user traffic that can seriously impair the functionality of your website. Protecting your website against abusive bot traffic is crucial, and there are a few ways to achieve this. This tutorial will teach you how to use your cPanel account&#8217;s robot.txt file to stop bots. Robot.txt is a text file that you [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":23377,"menu_order":1233,"comment_status":"closed","ping_status":"closed","template":"","doc_tag":[],"class_list":["post-18161","docs","type-docs","status-publish","hentry","no-post-thumbnail"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/docs\/18161","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/docs"}],"about":[{"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/types\/docs"}],"author":[{"embeddable":true,"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/comments?post=18161"}],"version-history":[{"count":1,"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/docs\/18161\/revisions"}],"predecessor-version":[{"id":18162,"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/docs\/18161\/revisions\/18162"}],"up":[{"embeddable":true,"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/docs\/23377"}],"wp:attachment":[{"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/media?parent=18161"}],"wp:term":[{"taxonomy":"doc_tag","embeddable":true,"href":"https:\/\/www.hostvento.com\/kb\/wp-json\/wp\/v2\/doc_tag?post=18161"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}