Old robots.txt file not changing, can't update to the current robots.txt

While my wordpress website was in production, I created a robots.txt file to disallow everything. When the site was ready, I deleted the robots.txt file through cpanel and never thought much about it. Recently, I realized that the website was not showing up on google search results, upon further investigation, i realized that the old robots.txt file was still there (even though I can't locate the file in my root folder).

http://youngauthors.my/robots.txt

I inserted another robots.txt file in my root folder, this time with the allow option instead of disallow. But it doesn't update and still blocks google from accessing the site.

Topic robots.txt Wordpress google

Category Web


Just delete file robots.txt. WordPress is generating virtual robots.txt file, and Yoast SEO plugin (and other plugins too) allows you to customize it. But, if you have an actual robots.txt file, it will prevent virtual one to work.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.