Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Christo 10 posts 80 karma points notactivated
    Aug 07, 2017 @ 09:16
    Christo
    0

    Urgent problem cmsPropertyData Over a billion records.

    Good day all

    I have inherited a Umbraco site and I am still new with the Umbraco.

    I have a problem with one of the tables in Umbraco or especially the cmsPropertyData table as it consists of 1 118 860 825 Records so. The problem is I cannot delete the Log file in SQL grows way too big and fails as soon as there is no more space available on the hard drive, I have tried a couple of solutions but I wonder if someone does have a solution to my problem.

    Kind Regards

  • Alex Skrypnyk 6175 posts 24186 karma points MVP 8x admin c-trib
    Aug 07, 2017 @ 09:18
    Alex Skrypnyk
    0

    Hi Christo

    What version of Umbraco are you using?

    The problem is that Umbraco stores previous versions of content items, so with time database grows.

    Thanks,

    Alex

  • Alex Skrypnyk 6175 posts 24186 karma points MVP 8x admin c-trib
    Aug 07, 2017 @ 09:20
    Alex Skrypnyk
    1

    Try to use this sql script for clearing database, and don't forget about backup -

    -- Umbraco Clear Old Document Versions To Decrease Database Size And Improve Performance
    -- http://borism.net/2008/12/16/fixing-a-large-cmspropertydata-table-in-umbraco/
    DECLARE @createdDate Datetime = DATEADD(m, -1, getdate())
    
    -- dump logs
    -- TRUNCATE TABLE umbracolog -- faster if log table is very big and you don't need anything
    DELETE FROM umbracolog WHERE Datestamp < @createdDate
    
    -- clean up old versions
    DELETE FROM cmsPropertyData WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE updateDate > @createdDate OR published = 1 OR newest = 1) AND
        contentNodeId IN (SELECT DISTINCT nodeID FROM cmsDocument)
    DELETE FROM cmsPreviewXml WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE updateDate > @createdDate OR published = 1 OR newest = 1) AND
        nodeId IN (SELECT DISTINCT nodeID FROM cmsDocument)
    DELETE FROM cmsContentVersion WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE updateDate > @createdDate OR published = 1 OR newest = 1) AND
        ContentId  IN (SELECT DISTINCT nodeID FROM cmsDocument)
    DELETE FROM cmsDocument WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE updateDate > @createdDate OR published = 1 OR newest = 1) AND
        nodeId IN (SELECT DISTINCT nodeID FROM cmsDocument)
    
    --http://blog.dampee.be/post/2015/11/28/Remove-Old-versions-of-Umbraco-Previews.aspx
    delete
    --select *
    from cmsPreviewXml
    where versionId in (
        select cmsPreviewXml.versionId
        from cmsPreviewXml join cmsDocument on cmsPreviewXml.versionId=cmsDocument.versionId
    where cmsDocument.newest <> 1)
    
  • Christo 10 posts 80 karma points notactivated
    Aug 07, 2017 @ 09:27
    Christo
    0

    Good day Alex

    I have tried this script and the problem is I cannot delete a billion records as the hard drive run out of space before even a 100000 has been deleted(Log file issue, growing log file), I would use this script as soon as the first clean up has been done. I am using Umbraco 6.2.2.

    Thanks for the reply

  • Nick Ryan 9 posts 79 karma points
    Aug 07, 2017 @ 15:25
    Nick Ryan
    1

    Without much thought on the matter (on my front), have you tried changing the MS-SQL recovery model?

    Details from Microsoft are here: https://technet.microsoft.com/en-us/library/ms190203(v=sql.105).aspx

    You'll need the BULKLOGGED mode for this operation. Switch to it, purge the transaction log to empty and then start deleting data.

    With a backup of course.

    Thinking of a backup, if you don't have space on the live system an alternative would be to perform a full DB backup, transfer the backup files to a less constrained system, restore the DB, perform the maintenance, backup the smaller database and then, here's the slightly scary bit - delete the database entirely (check all files are gone as well) and restore the smaller database in its place.

  • Collyn Philleo 2 posts 97 karma points
    Aug 07, 2017 @ 16:43
    Collyn Philleo
    2

    I've used this script in the past to remove extra content versions. It loops through and uses transactions, which prevents the log file getting too large. As with any script as a disclaimer, please backup the database first.

    /* This script will delete extraneous content versions */
    delete from cmsPreviewXml where VersionId in (select versionId from cmsDocument where newest = 0)
    delete from cmsContentVersion where VersionId in (select versionId from cmsDocument where newest = 0)
    delete from cmsDocument where newest = 0
    
    
    
    DECLARE @Deleted_Rows INT;
    SET @Deleted_Rows = 1;
    
    WHILE (@Deleted_Rows > 0)
      BEGIN
    
       BEGIN TRANSACTION
    
         delete top(1000) from cmsPropertyData where VersionId in (select versionId from cmsDocument where newest = 0)
    
       SET @Deleted_Rows = @@ROWCOUNT;
    
       COMMIT TRANSACTION
       CHECKPOINT -- for simple recovery model
    END
    
  • Colin Wiseman 54 posts 196 karma points
    Oct 09, 2022 @ 07:58
    Colin Wiseman
    0

    I know this is 5 years old but I used this script as part of getting rid of 60 million rows on a test database. But you have an error.

    delete from cmsDocument where newest = 0
    

    should be after the delete from cmsPropertyData as you have deleted all from the cmsDocument where newest is 0, thus when you come to

    (select versionid from cmsDocument where newest = 0)
    

    in the next statement there will be nothing to delete, cmsPropertyData will now have orphaned versions and everything won't be tidy :D

Please Sign in or register to post replies

Write your reply to:

Draft