Should /wp-includes/js/* folder be blocked in robots.txt file for a WordPress website?

In my WordPress website. I am disallowing /wp-includes/js/* folder in the robots file. Is that bad? I use SEMRush to audit my site and it shows an error that a resource at the above location is being blocked. This is my robots.txt User-agent: * Allow: /wp-content/uploads/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /linkout/ Disallow: /recommended/ Disallow: /comments/feed/ Disallow: /trackback/ Disallow: /index.php Disallow: /xmlrpc.php Disallow: /readme.html Disallow: /refer/ Disallow: /search/ Allow: /wp-admin/admin-ajax.php Sitemap: https://www.example.com/post-sitemap.xml Sitemap: https://www.example.com/page-sitemap.xml Sitemap: https://www.example.com/video_portal-sitemap.xml Sitemap: https://www.example.com/press_release-sitemap.xml …
Category: Web

Missing XML tag in the Sitemap

Some posts on my blog were somehow non-indexed at "Google search console", has it something to do with enabling the LOADING GOOGLE FONTS LOCALLY option? To solve it, I resubmitted my sitemap generated by my SEO plugin(showing all the posts) again but the next day I found following message in the Google search console- The sitemap can be read but has errors Missing XML tag This required tag is missing. Please add it and resubmit. 1 instance Line 10 Parent …
Category: Web

change permalink structure with htaccess without SEO impact

I can read a lot of questions about this topic but with different interpretations. I'm going to change my permalink structure from http://www.example.com/%year%/%monthnum%/%day%/%postname%/ to http://www.example.com/%postname%/ So new posts will be natively in the new url format, about old posts I'm going to add just one row in the .htaccess file, following the suggestion coming from yoast.com RedirectMatch 301 ^/([0-9]{4})/([0-9]{2})/([0-9]{2})/(?!page/)(.+)$ https://www.example.com/$4 After few test in another small website, it seems to work. My questions are: Is finally the "best way" to …
Category: Web

WordPress site using Yoast SEO being blocked from indexing by robots.txt

Morning all! I've just built a website for a client, using WordPress and the Yoast SEO plugin (free version). https://www.fly2help.org/ Whilst they're really happy with the website, I've noticed an issue when trying to submit it to search engines, such as Google, for indexing. It is continually being blocked, with an error like this: Crawl allowed? error No: blocked by robots.txt I'm not 100% sure why this is the case. I've checked the robots.txt file and there's nothing in it …
Category: Web

Sub Domain's Pages return 404

Need some help, I created these websites http://mccsedu.org http://parents.mccsedu.org http://faculty.mccsedu.org http://students.mccsedu.org The main site is working fine, but the pages of the those subdomains are not working. it always returns 404, you can try visiting those sites and click on any posts or pages..
Category: Web

Add image tags to native WordPress XML Sitemaps

I would like to add image tags into my sitemap posts: <image:image> <image:loc> <![CDATA[ https://example.com/image.jpg ]]> </image:loc> <image:caption> <![CDATA[ Image caption ]]> </image:caption> <image:title> <![CDATA[ Image Title ]]> </image:title> </image:image> Is there any filter that allows to add these tags into a native WP sitemap? Thank you
Category: Web

how can I check if I have sitemap?

maybe it's a very stupid question but I'm just getting into wordpress and I installed a plugin Google XML Sitemaps and activated it, but I'm not quite sure that it really provides me with sitemap.xml how can I check that? when I go to mywebsite.com/sitemap.xml I can see site map information, so I guess it works?
Category: Web

Wordpress Sitemap including pages that aren't there

I dont know if this is a severe cache, or a bug of sorts, I've installed multiple "seo" plugins to try and fix this, trawled the file system, cleaned the database, I don't know what is generating this sitemap and why it includes links to things that aren't there! https://eastcoasttyreandauto.com.au/sitemap.xml Example: https://eastcoasttyreandauto.com.au/articles/662043552335510/3138icps06200/ezdrdmh/114838124 Is there any tricks to figure out what is controlling this sitemap?
Category: Web

How to remove Wordpress Virtual Robots.txt that keeps overriding my root robots.txt?

So here is the issue. I am trying to remove the virtual robots.txt file in a wordpress site. I realized this was generated with a plugin (xml sitemap plugin) prior to me working on the site. Here is what I have done. I generated a new robots.txt file, uploaded and ensure it was on the root. Unfortunately, it doesn't show up, it is as if the virtual one is overriding the actual root robots.txt. I can't even use a speed …
Category: Web

Multilingual XML sitemaps when using qTranslate-X

I am taking over a multilingual website that a previous developer left unfinished, and I found that he had used qTranslate-X as a localization plugin, in order to create handle the multilingual part of it. The problem I find now is how to create a XML sitemap that has all the pages in their different language versions: Yoast SEO can generate sitemaps, but it doesn't work with qTranslate-X. The plugin they recommend in order to make both play nice together …
Category: Web

how to add urls dynamically into the sitemap

I basically want to add such urls to my existing sitemap index. I'm currently using WP SEO by yoast, but I'm open to trying out alternative plugins. My urls are currently like this... e.g www.mysite.com/used-cars/location/paris/model/bmw-m3 & www.mysite.com/used-cars/model/bmw-m3 I want to have all these kind of urls in a loop, then push them into a sitemap. How can I do this?
Category: Web

Sitemap_index.xml is not displayed

When I want to check out my /sitemap_index.xml i get: http://mysite/post-sitemap.xml 2017-02-04T21:52:21+04:00 http://mysite/page-sitemap.xml 2017-01-31T17:16:39+04:00 http://mysite/testimonials-widget-sitemap.xml 2017-02-03T06:08:41+04:00 What could be the problem. I am using Yoast Seo sitemap xml generator. The google google xml sitemaps is displaying the same.
Category: Web

WP-RestAPI and xml sitemap

Using the WP-RestAPI to pull a wordpress blog into an angularjs application. Works extremely well. The blog is using Yoast SEO for the for meta descriptions and keywords. I would like to use the yoast xml sitemap as well for submitting to google. However, because of route structure I need to save the sitemap to a different location than the root wordpress folder. I have also tried using the Google Sitemap Generator plugin. With neither plugin can I seem to …
Category: Web

Generate XML Sitemap for Blog on Magento Platform

Our store is on Magento which is installed at root. Our blog uses Wordpress and is installed at /wp of our domain and renders at /blog. I've tried just about every plugin but cannot generate an xml sitemap for the blog. Each one creates an HTML sitemap of the Magento store without any reference to blog or posts whatsoever. Our store is rissyroos.com. The blog location is rissyroos.com/blog. I've tried at least a dozen different plugins starting and now ending …
Category: Web

Anchors in WordPress not followed/ indexed in sitemap.xml

Have anchor tags on a WordPress page about 300 on a page which are not listed in sitemap.xml, these links are generated dynamically. How do I index these? Structure: A navigation link say Games when clicked --> Opens List of Games (Football, Basketball etc.) when clicked Football it opens --> Different versions of the games. These are not listed in the sitemap.xml when created from online sitemap generators. When clicked them manually in pages they open but doesn't appear in …
Category: Web

Pinging or sitemap plugin: which practice is better

As I have understand WordPress 4.0.1 has it's own "Update Service" enabled by default. The address indicated in my case is http://rpc.pingomatic.com/. So based on this thing I can state that my WordPress pack has ping service implemented and enabled. However there are several plugins that can automatically build and upload XML Sitemap to major search engines (Google, Bing, Yahoo...). For me it looks like those plugins is not really necessary. But if I really need my posts to be …
Category: Web

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.