Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Amir Khan 1287 posts 2744 karma points
    Jan 13, 2014 @ 18:40
    Amir Khan
    0

    Umbraco 4.7.1 Database Filling up

    Hi, I've had the databases for two of my Umbraco 4.7.1 sites grow to an enourmous (500MB) size for no apparent reason. Has anyone expereinced this

    -Amir

  • Dan Lister 416 posts 1974 karma points c-trib
    Jan 14, 2014 @ 09:25
    Dan Lister
    0

    Yeah thats quite common I'd say. Especially if a lot of editing has previously taken place. The database can quite easily fill up with document versions, log entries and user login records. I often use the below SQL script to tidy up databases every now and again:

    TRUNCATE TABLE umbracoLog
    TRUNCATE TABLE umbracoUserLogins
    
    -- Delete all related nodes and table contents
    DELETE FROM cmsPreviewXml WHERE nodeId IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    DELETE FROM cmsContentVersion WHERE contentId IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    DELETE FROM cmsDocument WHERE nodeId IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    DELETE FROM cmsContentXML WHERE nodeId IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    DELETE FROM cmsContent WHERE nodeId IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    DELETE FROM cmsPropertyData WHERE contentNodeId IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    
    -- Delete XML nodes
    DELETE FROM umbracoDomains WHERE domainRootStructureID IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    DELETE FROM umbracoUser2NodePermission WHERE nodeId IN (SELECT id FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20)
    DELETE FROM umbracoNode WHERE path LIKE '%-20%' AND id != -20
    
    DELETE FROM cmsPropertyData WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE published = 1 OR newest = 1) AND
        contentNodeId IN (SELECT DISTINCT nodeID FROM cmsDocument)
    DELETE FROM cmsPreviewXml WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE published = 1 OR newest = 1) AND
        nodeId IN (SELECT DISTINCT nodeID FROM cmsDocument)
    DELETE FROM cmsContentVersion WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE published = 1 OR newest = 1) AND
        ContentId  IN (SELECT DISTINCT nodeID FROM cmsDocument)
    DELETE FROM cmsDocument WHERE
        versionId NOT IN (SELECT versionId FROM cmsDocument WHERE published = 1 OR newest = 1) AND
        nodeId IN (SELECT DISTINCT nodeID FROM cmsDocument)
    
    -- Reindex tables  
    DBCC DBREINDEX (cmsPropertyData)
    DBCC DBREINDEX (cmsPreviewXml)
    DBCC DBREINDEX (cmsContentVersion)
    DBCC DBREINDEX (cmsDocument)
    DBCC DBREINDEX (cmsContentXml)
    DBCC DBREINDEX (umbracoDomains)
    DBCC DBREINDEX (umbracoUser2NodePermission)
    DBCC DBREINDEX (umbracoNode)
    DBCC DBREINDEX (cmsContent)
  • Mike Chambers 636 posts 1253 karma points c-trib
    Jan 15, 2014 @ 10:23
    Mike Chambers
    0

    F.A.L.M housekeeping can be user for a gui for this [http://our.umbraco.org/projects/backoffice-extensions/falm-housekeeping]

    Did you turn of debugging in the web.config for your live site, this would reduce logging?

    Also you can turn off logging in config/umbracosettings.config if memory serves...

    Just be aware that truncating the log or deleting versions will not reduce the file allocation size of SQL.. you need to run shrink commands against your db and then reinitialse indexes to avoid any fragmentation... If you google you'll find that its not ideal practise to go about shrinking databases...

    Also you'll loose your audit trail information if you are usuing that in anger...

    a script that we have used in the past.. taken from a source on the net.. so I can't take credit for it... but make sure you do backup your sql db in case of any issues :-)

    truncate table umbracoLog
    
    exec sp_spaceused @updateusage = 'true'
    
    dbcc SHRINKDATABASE ([YOUR DATABASE NAME])
    
    -- Check the index fragmentation again
    SELECT OBJECT_NAME(OBJECT_ID) as [object_name], index_id,index_type_desc,index_level,
    avg_fragmentation_in_percent,avg_page_space_used_in_percent,page_count, fragment_count
    FROM sys.dm_db_index_physical_stats
    (DB_ID(N'YOUR DATABASE NAME'), NULL, NULL, NULL , NULL)
    ORDER BY avg_fragmentation_in_percent DESC
    
    --Description : This script reorganizes and rebuilds the index if the fragmentation level is higher the given threshold
    -- You can define the threshold for reorganize as well as for rebuild and script will work accordingly 
    -- INPUTS : @fillfactor - While rebuilding index what would be FILLFACTOR for new index 
    -- @FragmentationThresholdForReorganizeTableLowerLimit - Fragmentation Level lower threshold to check for reorganizing the table, if the fragmentation is higher than this level, it will be considered for reorganize
    -- @@FragmentationThresholdForRebuildTableLowerLimit - Fragmentation Level lower threshold to check for rebuilding the table, if the fragmentation is higher than this level, it will be considered for rebuild
    -- NOTES : PRINT statements are all queued up and don't show up until the entire script is printed. However, there is an alternative to PRINTing messages. 
    -- You can raise an error that isn't really an error (code of 0) and you'll get the same effect--message will be printed immediately. 
    DECLARE @cmd NVARCHAR(1000)  
    DECLARE @Table VARCHAR(255)  
    DECLARE @SchemaName VARCHAR(255) 
    DECLARE @IndexName VARCHAR(255) 
    DECLARE @AvgFragmentationInPercent DECIMAL 
    DECLARE @fillfactor INT  
    DECLARE @FragmentationThresholdForReorganizeTableLowerLimit VARCHAR(10) 
    DECLARE @FragmentationThresholdForRebuildTableLowerLimit VARCHAR(10) 
    DECLARE @Message VARCHAR(1000) 
    
    SET NOCOUNT ON 
    
    --You can specify your customized value for reorganize and rebuild indexes, the default values 
    --of 10 and 30 means index will be reorgnized if the fragmentation level is more than equal to 10  
    --and less than 30, if the fragmentation level is more than equal to 30 then index will be rebuilt 
    SET @fillfactor = 90  
    SET @FragmentationThresholdForReorganizeTableLowerLimit = '10.0' -- Percent 
    SET @FragmentationThresholdForRebuildTableLowerLimit = '30.0' -- Percent 
    
    BEGIN TRY 
    
    -- ensure the temporary table does not exist 
    IF (SELECT OBJECT_ID('tempdb..#FramentedTableList')) IS NOT NULL 
    DROP TABLE #FramentedTableList; 
    
    SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Retrieving indexes with high fragmentation from ' + DB_NAME() + ' database.'
    RAISERROR(@Message, 0, 1) WITH NOWAIT 
    
    SELECT OBJECT_NAME(IPS.OBJECT_ID) AS [TableName], avg_fragmentation_in_percent, SI.name [IndexName],  
    schema_name(ST.schema_id) AS [SchemaName], 0 AS IsProcessed INTO #FramentedTableList 
    FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL , NULL) IPS 
    JOIN sys.tables ST WITH (nolock) ON IPS.OBJECT_ID = ST.OBJECT_ID 
    JOIN sys.indexes SI WITH (nolock) ON IPS.OBJECT_ID = SI.OBJECT_ID AND IPS.index_id = SI.index_id 
    WHERE ST.is_ms_shipped = 0 AND SI.name IS NOT NULL 
    AND avg_fragmentation_in_percent >= CONVERT(DECIMAL, @FragmentationThresholdForReorganizeTableLowerLimit) 
    ORDER BY avg_fragmentation_in_percent DESC 
    
    SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Retrieved indexes with high fragmentation from ' + DB_NAME() + ' database.'
    
    RAISERROR(@Message, 0, 1) WITH NOWAIT 
    RAISERROR('', 0, 1) WITH NOWAIT 
    
    WHILE EXISTS ( SELECT 1 FROM #FramentedTableList WHERE IsProcessed = 0 ) 
    BEGIN 
    
      SELECT TOP 1 @Table = TableName, @AvgFragmentationInPercent = avg_fragmentation_in_percent,  
      @SchemaName = SchemaName, @IndexName = IndexName 
      FROM #FramentedTableList 
      WHERE IsProcessed = 0 
    
      --Reorganizing the index 
      IF((@AvgFragmentationInPercent >= @FragmentationThresholdForReorganizeTableLowerLimit) AND (@AvgFragmentationInPercent < @FragmentationThresholdForRebuildTableLowerLimit))
      BEGIN 
        SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Reorganizing Index for [' + @Table + '] which has avg_fragmentation_in_percent = ' + CONVERT(VARCHAR, @AvgFragmentationInPercent) + '.'
        RAISERROR(@Message, 0, 1) WITH NOWAIT 
        SET @cmd = 'ALTER INDEX ' + @IndexName + ' ON [' + RTRIM(LTRIM(@SchemaName)) + '].[' + RTRIM(LTRIM(@Table)) + '] REORGANIZE'  
        EXEC (@cmd) 
        --PRINT @cmd  
        SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Reorganize Index completed successfully for [' + @Table + '].' 
        RAISERROR(@Message, 0, 1) WITH NOWAIT 
        RAISERROR('', 0, 1) WITH NOWAIT 
      END 
      --Rebuilding the index 
      ELSE IF (@AvgFragmentationInPercent >= @FragmentationThresholdForRebuildTableLowerLimit ) 
      BEGIN 
        SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Rebuilding Index for [' + @Table + '] which has avg_fragmentation_in_percent = ' + CONVERT(VARCHAR, @AvgFragmentationInPercent) + '.'
        RAISERROR(@Message, 0, 1) WITH NOWAIT 
        SET @cmd = 'ALTER INDEX ' + @IndexName + ' ON [' + RTRIM(LTRIM(@SchemaName)) + '].[' + RTRIM(LTRIM(@Table)) + '] REBUILD WITH (FILLFACTOR = ' + CONVERT(VARCHAR(3),@fillfactor) + ', STATISTICS_NORECOMPUTE = OFF)' 
        EXEC (@cmd) 
        --PRINT @cmd 
        SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Rebuild Index completed successfully for [' + @Table + '].'
        RAISERROR(@Message, 0, 1) WITH NOWAIT 
        RAISERROR('', 0, 1) WITH NOWAIT 
      END 
    
      UPDATE #FramentedTableList 
      SET IsProcessed = 1  
      WHERE TableName = @Table 
      AND IndexName = @IndexName 
    END 
    
    DROP TABLE #FramentedTableList  
    
    END TRY 
    
    BEGIN CATCH 
      PRINT 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' There is some run time exception.' 
      PRINT 'ERROR CODE : ' + CONVERT(VARCHAR, ERROR_NUMBER())  
     -- PRINT 'ERROR MESSAGE : ' + ERROR_MESSAGE
    END CATCH
    
    
    -- Check the index fragmentation again
    SELECT OBJECT_NAME(OBJECT_ID) as [object_name], index_id,index_type_desc,index_level,
    avg_fragmentation_in_percent,avg_page_space_used_in_percent,page_count, fragment_count
    FROM sys.dm_db_index_physical_stats
    (DB_ID(N'YOUR DATABASE NAME'), NULL, NULL, NULL , NULL)
    ORDER BY avg_fragmentation_in_percent DESC
  • Mike Chambers 636 posts 1253 karma points c-trib
    Jan 15, 2014 @ 10:27
    Mike Chambers
    0

    Forgot to say that moving to v6+ has much better housekeeping inbuilt on the logging side of things, as far as I'm aware...  which should mitigate these huge sql growth issues.

  • Amir Khan 1287 posts 2744 karma points
    Jan 31, 2014 @ 20:35
    Amir Khan
    0

    Thank you both for your help! So generally you think just don't shrink it and disable logging to prevent future growth?

Please Sign in or register to post replies

Write your reply to:

Draft