PrestaShop robots.txt blocking Google: how to check and fix it quickly
When PrestaShop robots.txt blocks Google, the culprit is most often a `Disallow: /` line carried over from a test copy, or a rule that covers categories or images. Open yourdomain.com/robots.txt, find the lines starting with Disallow and compare them with your product addresses. Google usually reads a corrected file within a day.
Author: The TellMyShop team. A guide for PrestaShop store owners: the cause, the manual fix and the same work in a chat conversation.
Last updated: · 6 min read
How to check in a few minutes whether robots.txt blocks the store
The quickest way is in the browser and in Search Console. You don't need server access for this.
- Open yourdomain.com/robots.txt in a browser. You will see plain text with rules.
- Look for a
Disallow: /line with nothing after it. That one line closes the whole store to robots. - Check that no rules start with the names of your categories, the image folder (
/img/) or theme files (/themes/). Google needs the CSS and scripts to see the page the way a customer does. - In Search Console, open the robots.txt report. It shows when the file was last read and any errors. The report is described in Search Console help.
- Paste the address of one product into the URL inspection tool in Search Console. If you see a message about a robots.txt block, you have your confirmation.
If the file looks fine and the products still aren't in Google, the cause lies elsewhere. See the post on the “Crawled, currently not indexed” status.
Where does a bad robots.txt in a store come from?
A bad file rarely appears by itself. Usually it is left behind by work on the store whose results nobody checked afterwards.
| What happened | What it looks like in the file | What to do |
|---|---|---|
| The store was moved from a test copy | Disallow: / right at the top | remove that line or generate the file again |
| Someone blocked images or the theme | Disallow: /img/, Disallow: /themes/ | remove them, Google must see how the page looks |
| A rule catches too much | Disallow: /*category instead of a specific parameter | narrow the rule to the filter or sorting parameter |
| An SEO module writes its own file | rules you didn't add | check the settings of the module that manages robots.txt |
| The file was generated again | no sitemap line | add a Sitemap: line |
| A new language or store in multistore | rules for only one language folder | generate the file again after adding the language |
In its description of how it reads robots.txt, Google reminds you that the file controls robots' visits, not what goes into results. So blocking a page in robots.txt is not a way to hide it in Google.
How to fix robots.txt in PrestaShop
You have two routes: generate the file in the admin or fix it by hand. The choice depends on whether the file has your own rules in it.
Generating in the admin. PrestaShop creates robots.txt in the traffic and SEO settings, under Shop Parameters → Traffic & SEO. At the bottom of the SEO and URLs settings page there is a button to generate the file. The page is described in the PrestaShop documentation on SEO and URLs. The generated file blocks technical folders and pages, such as the cart and customer account, and leaves products and categories open.
Remember one thing: the button overwrites the whole file. If it had your sitemap address or your own rules for filters, copy them before generating and add them back afterwards.
Fixing by hand. When the file has rules you want to keep, you fix only the bad line. You edit the file over FTP or in your host's file manager. Make a backup before the change, because one typo in a rule can close the whole store.
An example of a file that lets robots into products and categories, but closes the cart and search:
User-agent: *
Disallow: /*controller=cart
Disallow: /*controller=order
Disallow: /*controller=search
Disallow: /*?order=
Allow: /themes/
Allow: /img/
Sitemap: https://yourdomain.com/1_index_sitemap.xmlThis is an example, not something to copy as is. Your generated file will be longer and matched to your store version. When you want to limit filter addresses, first read the post on faceted navigation and SEO. A practical list of what is usually blocked and what is not is also in the MyPresta guide to robots.txt in PrestaShop.
What to do after fixing the file
After the fix, ask Google to read the file again and check a few pages. Saving the file alone does not bring the store back into results straight away.
- In the robots.txt report in Search Console, request a new fetch of the file.
- Check the home page, one category and one product in the URL inspection tool.
- Make sure the file has a sitemap line. How to turn the sitemap on and submit it is covered in the post on the PrestaShop sitemap.
- Watch the pages report in Search Console over the following weeks. Addresses will come back gradually.
When the store was blocked for a long time, Google may have removed some pages from results. They will come back once robots visit them and judge them worth showing. Nobody can speed this up on Google's behalf, but a proper sitemap and internal links help.
How the chat checks and fixes robots.txt with a backup of the file
With the TellMyShop module you describe the problem in one sentence. Your chat (Claude or ChatGPT) reads the file, compares the rules with the addresses of products, categories and images, then checks a few pages the way Google sees them.
You: Since we moved the store, traffic from Google has dropped. Check whether robots.txt is blocking anything.
Your chat: The file has a rule at the top that closes the whole store. It is a typical leftover from a test copy. The sitemap line is also missing. I suggest this change:
Your chat: Changing the file needs the Service switch. Before saving I'll make a backup of the current file. After saving I'll check that the home page, the Lamps category and the Nora lantern product are allowed.
You: I've turned on Service. Save it.
Your chat: Saved. A copy of the old file is in the file backups. All three pages are now allowed for Google. In Search Console, request a new fetch of robots.txt.
Card status: waiting for your “yes”old version: Disallow: /
new version: Disallow: /*controller=cart
new version: Disallow: /*controller=order
new version: Sitemap: https://yourdomain.com/1_index_sitemap.xml
The category and product names in the example are illustrative.
In the free TellMyShop Free the chat opens the store's pages the way Google sees them, reads the SEO report and tells you what is wrong. It changes nothing. TellMyShop Pro reads the whole store, including files, so it reads robots.txt itself and prepares a fix in a preview. Saving the file needs the Service switch, which you turn on in the module panel for 3 hours. The switch can be turned on when there is a database backup from today. Before every file save the module makes a copy of it, so you can restore the previous version from the conversation. An example of such a request is in the use case fixing a file that blocks Google.
TellMyShop has no access to your Google account. You handle the robots.txt report and the request for a new fetch yourself in Search Console. A wider review of the store for Google is in the use case full store SEO audit and in the post on an SEO audit in PrestaShop. Pro prices are on the pricing page.
Questions and answers
Where is the robots.txt file in PrestaShop?
In the store's main folder, next to the index.php file. In a browser you open it at yourdomain.com/robots.txt. If there is no file, Google assumes it can visit the whole store.
Will the generate robots.txt button in the admin overwrite my changes?
Yes. PrestaShop creates the file from scratch and removes everything added by hand. Before you click, copy your own lines, for example the sitemap address, and add them back after generating.
Does robots.txt remove pages from Google?
No. The file tells robots where not to go, but a blocked page's address can still appear in results if links lead to it. To remove pages from results you use the noindex tag, and a page with that tag must not be blocked in robots.txt.
How quickly will Google come back to the store after a fix?
Google usually reads robots.txt again within a day. Revisiting and assessing product pages takes longer and depends on the size of the store. In Search Console you can ask for the file to be fetched again.
Will TellMyShop Free fix robots.txt?
No. The free TellMyShop Free saves nothing. It shows how Google sees the store's pages and points to the problem. A fix to the file, with a backup, is made by TellMyShop Pro once the Service switch is on.
PrestaShop is a registered trademark of PrestaShop SA. Claude is a trademark of Anthropic PBC. ChatGPT and OpenAI are trademarks of OpenAI. TellMyShop is not affiliated with these companies.