Category Archives: Db2 Built In Tables

Automated DB2 Reorganisation, Runstats and Rebinds – Version 2

A while back I did the first version of this code (can be found here). Over time I have been running this code on our production servers, it started out by working fine but sometimes it would over run and interfere with the morning batch, so a different solution was needed. In a previous article I discussed if it was better to let the included automated DB2 functionality take care of the maintenance of tables etc, or to create your own process that uses included stored procedures to identify the tables that need reorganising.

So this new version of the script will only work between certain times and only do offline reorganisations, but is still possible to just reorganise a single partition of a range partitioned table. The reason for the time restriction is to take a leaf from the included automated scripts having an offline maintenance window, and to stop the scripts that I have created before overrunning into the morning batch. The previous version of the reorganisation script attempted to be to “clever” and do an online reorg of non partitioned tables and an offline reorg of the partitions of the range partitioned tables. The problem with this is that capturing when the online reorgs have finished (as they are asynchronous), so that the table can have it statistics run so that it is not identified again by the SYSPROC.REORGCHK_TB_STATS stored procedure. Equally another issue is that you would have to reorganise the index’s on the tables that you have on-line reorganised as they would not have been done, where as an offline reorganisation also does the indexes at the same time.

So I made the decision to do all the reorganisations offline, followed by a runstats and a rebind. The main controlling stored procedure looks like:

CREATE PROCEDURE DB_MAIN.RUN_ALL_AUTOMATED_MAINTENANCE(IN MAINT_SCHEMA VARCHAR(255), IN REORG_FINISH_TIME TIME, IN RUNSTATS_FINISH_TIME TIME, IN DAY_TO_REMOVE INTEGER)
LANGUAGE SQL
BEGIN
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --This procedure is the wrapper for all the rest to tidy it up a little bit.
 --It will only run the reorgs tille the time specified, then will just finish the one
 --that it is on once the time has expired.
 --Similar thing for the runstats so that it does not impact on the running of the
 --morning loads.
 --Rebind the procedures so that they get new packages based on the updated statistics
 --from the reorg and runstats.
 --All Reorg done off line as this is what DB2 does.
 --MAINT_SCHEMA = The schema you wish to be looked at
 --REORG_FINISH_TIME = The time you wish the reorgs to run until
 --RUNSTATS_FINISH_TIME = The time you wish runstats to run till
 --DAY_TO_REMOVE = The number of day back you wish staging tables to be emptied from
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------

 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --Reorg the tables
 CALL DB_MAIN.RUN_AUTOMATED_TABLE_REORG(MAINT_SCHEMA, REORG_FINISH_TIME, DAY_TO_REMOVE);
----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --Runstat the tables that have been reorged
 CALL DB_MAIN.RUN_AUTOMATED_TABLE_RUNSTATS(MAINT_SCHEMA, RUNSTATS_FINISH_TIME,DAY_TO_REMOVE);
----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --Rebind the stored procedures to take advantage of the potentially new plans
 CALL DB_MAIN.RUN_AUTOMATED_REBIND_PROCEDURES(MAINT_SCHEMA);

END

This is now a three stage operation, the first two stages have time limits and so they will carry out new operations until this time limit is breached. What you have to realise here is that if the end time is 18:00:00 then it will start work right up until 17:59:59, this means if it picks up a particularly large reorganisation task at this last second then it will run till it has finished.

Some of the code especially the runstats stuff is quite a lot like the previous version just with a change for the time. As I cant upload a single .zip file as apparently it will be a security risk, and apparently a .sql file is also a risk please find a number of .doc files a the bottom of the article. Please just change the file extension and then you will be able to access them. I would very interested in having feedback from anyone who uses this code to see how you get on with it.

DISCLAIMER: As stated at the top of the blog use this code in your production systems at your own peril. I have tested and know it works on my systems, please test and check it works on yours properly as reorganising tables can potentially dangerous.

FILES WITH CODE IN:

OverallRunnerStoredProcedure

ReorganiseTablesStoredProcedures

ReorganiseTableTables

ReorganiseTableViews

RunstatsTableTables

RunstatsTableViews

RunstatsTableStoredProcedures

RebindSchemaStoredProcedure

DB2 Move Table Partitions Automatically

One of the hot topics in databases at the moment is temperature based data. This is basically where you put the data that is most often access on fast disks, and data that is accessed least often is put on slower disks. To accomplish this is pretty easy as you can set the containers for an “archive” tablespace to a different disk / directory mount point. This does similar things to my two previous posts on attaching table partitions and detaching table partitions

The problem comes when you already have data in a table and the table partition already is part of a tablespace, you cant just alter the partition to the new tablespace. You have to identify the table partition to move, detach it recreate it in the new tablespace and then reload the data. You can do this manually, but who has time to do this, I have written code that will allow you to move the tablespace of the partition quite easily, and can be part of an automated process. The stored procedure call looks a little like this:

CALL DB_MAIN.MOVE_PARTITION('INSURANCE', 'TRANSACTIONS', 10, '/home/db2inst1/detach-archive/', 'INSURANCE_ARCH_TS')

The stored procedure that I have created deals with all of this and creates “safe” copies of the data, and records what it has done and where it currently is, although there is no functionality to stop and start the process. The stored procedure and the table it uses can be found in the file at the bottom of the page, I will go through briefly below what it does.

Given the parameters passed in then the procedure works out if there are any viable partitions to move. Once it has done that it will detach the partition in question and will store it away in the location given in the parameters. As the stored procedure moves faster than DB2 can detach the data you need to use SYSIBMADM.SNAPUTIL a view that will show the progress of the utility, the stored procedures can’t continue till this is done. Once the data is detached and reorg of the indexes will get rid of the last vestiges of the partition and you can run the command to attach the new one in the archive tablespace. Then it reloads the data that was extracted into the new partition in the new tablespace.

When this finishes then you are left with a row in the DB_MAIN.MOVE_PARTITION (assuming you choose to keep it in that schema) and an IXF of the extracted data, that you can write of somewhere as part of a backup strategy.

DISCLAIMER: As stated at the top of the blog use this code in your production systems at your own peril. I have tested and know it works on my systems, please test and check it works on yours properly as moving table data partitions can potentially dangerous. The file is a .doc only as that’s the only way I could get it uploaded onto wordpress, it should open fine like that, or knock the .doc off and it will open in your favourite text editor

FILE WITH CODE IN: DB2-move-data-partitions-sps-dcp

DB2 Attach Table Partitions Automatically

This follows on from my last post on detaching DB2 table partitions that can be found here. After you have detached a partition, more than likely you are going to need to attach one. Using the code here you can do this automatically for range partitioned tables that work on ranges based on dates, but it could easily be adapted to work on any number of different data type ranges.

Attaching a partition like detaching is an entirely manual process after the initial creation of the table. The code I have created allows you too just call a simple command below and it will add the next partition for you without having to look at what the next range is. In this case it is the next 3 months.

CALL DB_MAIN.ATTACH_PARTITION('INSURANCE','TRANSACTIONS','INSURANCE_TS')

So the code is pretty simple it is just a wrapper around the commands that you would normally have to run to add a partition, plus with the added bonus it works out what the next range is and records the work that it has done.

First it works out what is the maximum partition so that it can create the new one:

CREATE PROCEDURE DB_MAIN.GET_MAX_PARTITION(IN TABLESCHEMA VARCHAR(255), IN TABLENAME VARCHAR(255), OUT PARTOBJID INT)
LANGUAGE SQL
BEGIN
    --Decalre Vars for use
        DECLARE Task_problem condition FOR SQLSTATE '99999';

            SET PARTOBJID = (SELECT PARTITIONOBJECTID
                            FROM SYSIBM.SYSDATAPARTITIONS
                            WHERE LTRIM(RTRIM(TABNAME)) = TABLENAME
                                AND LTRIM(RTRIM(TABSCHEMA)) = TABLESCHEMA
                            ORDER BY DATAPARTITIONNAME DESC
                            FETCH FIRST ROW ONLY);                    

    --Return the
        RETURN PARTOBJID;
END

This is then used to work out what the next partition values for high and low should be. Next is the main procedure itself, it uses a calendar table the standard attach command with inclusive start and end values and an index reorganisation at the end. The reorganisation is needed so that the new partition is recognised by the indexes. All the code and definition for the calendar table are in the file available at the bottom of this post.

CREATE PROCEDURE DB_MAIN.ATTACH_PARTITION(IN TABLESCHEMA VARCHAR(255), IN TABLENAME VARCHAR(255), IN NEWTABLESPACE VARCHAR(20))
LANGUAGE SQL
BEGIN
--Declare vars for use in SP
DECLARE Task_problem condition FOR SQLSTATE '99999';
DECLARE AttActPartNo INT;
DECLARE AttCurHighVal DATE;
DECLARE AttQuarterVal CHAR;
DECLARE AttLastDateQu DATE;
DECLARE AttTableSpace varchar(20);
DECLARE AttSQL Varchar(500) DEFAULT 'No Dont do it';
DECLARE YearAttLastDateQu VARCHAR(4);
DECLARE ReorgString VARCHAR(500);

--Get the Latest Partition in Existance for the table
	CALL DB_MAIN.GET_MAX_PARTITION(TABLESCHEMA,TABLENAME,AttActPartNo);

--Create the new Lower value for the Partition and the Quarter
--Get the current High value
	SET AttCurHighVal = (SELECT DATE(SUBSTR(HIGHVALUE,10,2) || '/' || SUBSTR(HIGHVALUE,7,2) || '/' || SUBSTR(HIGHVALUE,2,4)) + 1 DAY
		             FROM SYSIBM.SYSDATAPARTITIONS
		             WHERE PARTITIONOBJECTID = AttActPartNo
                                AND LTRIM(RTRIM(TABNAME)) = TABLENAME
                                AND LTRIM(RTRIM(TABSCHEMA)) = TABLESCHEMA);

--Get the Quarter val
        SET AttQuarterVal = (SELECT CHAR(Quarter_of_year)
                             FROM GLOBAL.CALENDAR
                             WHERE CALENDAR_DATE = AttCurHighVal);

--Get the last date in month
	SET AttLastDateQu = (SELECT Last_Date_of_Quarter
                             FROM GLOBAL.CALENDAR
                             WHERE CALENDAR_DATE = AttCurHighVal);

--Set the year for the partition
	SET YearAttLastDateQu = CHAR(YEAR(AttLastDateQu));

--Build the SQL to attach a new section
	SET AttSQL = 'ALTER TABLE ' || TABLESCHEMA || '.' || TABLENAME || ' ADD PARTITION ';
        SET AttSQL = AttSQL || TABLENAME || '_' || YearAttLastDateQu || '_Q' || AttQuarterVal;
	SET AttSQL = AttSQL || ' STARTING FROM (''' || CAST(AttCurHighVal as varchar(10)) || ''')';
        SET AttSQL = AttSQL || ' ENDING AT (''' ||  CAST(AttLastDateQu as varchar(10)) || ''')';
        SET AttSQL = AttSQL || ' IN ' || NEWTABLESPACE; 

	IF(AttSQL <> '')THEN
	--Insert into the logging table
		INSERT INTO DB_MAIN.ATTACHPARTITIONS(
			TABLESCHEMA,
			TABLENAME,
			ATTACHDATE,
			ATTACHTABLESCHEMA,
			ATTACHTABLENAME,
			ATTACHCODE
		)
		VALUES(
			TABLESCHEMA,
			TABLENAME,
			CURRENT DATE,
			TABLESCHEMA,
			TABLENAME,
			AttSQL
		);

        COMMIT;

--Execute the code
	PREPARE S2 FROM AttSQL;
        EXECUTE S2;
--Reorg the table
  --Create the string
        SET ReorgString = 'REORG INDEXES ALL FOR TABLE '  || TABLESCHEMA || '.' || TABLENAME || ' ALLOW NO ACCESS CLEANUP ONLY';

  --Run the command
        CALL SYSPROC.ADMIN_CMD(ReorgString);
END IF;
END
GO

DISCLAIMER: As stated at the top of the blog use this code in your production systems at your own peril. I have tested and know it works on my systems, please test and check it works on yours properly as attaching partitions can potentially dangerous. The file is a .doc only as that’s the only way I could get it uploaded onto wordpress, it should open fine like that, or knock the .doc off and it will open in your favourite text editor

FILE WITH CODE IN: DB2 Attach Table Partitions Tables SPs