Hey Milosz

I think you could use the canonical link element in the head of your html of your duplicates to handle this issue. Here is a little description of it on wikipedia but it is for this exact issue.

https://en.wikipedia.org/wiki/Canonicallinkelement

Geoff

Hi Geoffry,

Thank you for your reply. Since we have duplicates indexed such as

http://www.churchill-knight.co.uk/ and https://www.churchill-knight.co.uk/ I agree we should have canonical tag on http pages.

Unfortunately my .NET developer told me that for the time being HTTP to HTTPS migration cannot be done and in additional putting canonical tags can crash some of our external resources. One of the example is our http://www.churchill-knight.co.uk/calculator/

Any thoughts? Looks quite specific.

Hi Geoff,

It's been a while now but things have gone further. We have now implemented canonical tags but there is another challenge.

All the URL are crawlable to the degree that some software can crawl pages with dynamic parameters ending up with 50K pages.

We have got less than 400 actual pages excluding all the categories and pages with dynamic characters.

Is there any way to exclude them from crawling?

Example

https://www.churchill-knight.co.uk/blog/category/limited%20company/?page=8

Any thoughts?

Our staging website also is crawlable

https://staging.churchill-knight.co.uk/new-to-contracting/what-is-ir35/ir35-and-the-public-sector/

Any thoughts

Thanks, Milosz

is working on a reply...