Category Archives: Reorg Index

Automated DB2 Reorganisation, Runstats and Rebinds – Version 2

A while back I did the first version of this code (can be found here). Over time I have been running this code on our production servers, it started out by working fine but sometimes it would over run and interfere with the morning batch, so a different solution was needed. In a previous article I discussed if it was better to let the included automated DB2 functionality take care of the maintenance of tables etc, or to create your own process that uses included stored procedures to identify the tables that need reorganising.

So this new version of the script will only work between certain times and only do offline reorganisations, but is still possible to just reorganise a single partition of a range partitioned table. The reason for the time restriction is to take a leaf from the included automated scripts having an offline maintenance window, and to stop the scripts that I have created before overrunning into the morning batch. The previous version of the reorganisation script attempted to be to “clever” and do an online reorg of non partitioned tables and an offline reorg of the partitions of the range partitioned tables. The problem with this is that capturing when the online reorgs have finished (as they are asynchronous), so that the table can have it statistics run so that it is not identified again by the SYSPROC.REORGCHK_TB_STATS stored procedure. Equally another issue is that you would have to reorganise the index’s on the tables that you have on-line reorganised as they would not have been done, where as an offline reorganisation also does the indexes at the same time.

So I made the decision to do all the reorganisations offline, followed by a runstats and a rebind. The main controlling stored procedure looks like:

CREATE PROCEDURE DB_MAIN.RUN_ALL_AUTOMATED_MAINTENANCE(IN MAINT_SCHEMA VARCHAR(255), IN REORG_FINISH_TIME TIME, IN RUNSTATS_FINISH_TIME TIME, IN DAY_TO_REMOVE INTEGER)
LANGUAGE SQL
BEGIN
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --This procedure is the wrapper for all the rest to tidy it up a little bit.
 --It will only run the reorgs tille the time specified, then will just finish the one
 --that it is on once the time has expired.
 --Similar thing for the runstats so that it does not impact on the running of the
 --morning loads.
 --Rebind the procedures so that they get new packages based on the updated statistics
 --from the reorg and runstats.
 --All Reorg done off line as this is what DB2 does.
 --MAINT_SCHEMA = The schema you wish to be looked at
 --REORG_FINISH_TIME = The time you wish the reorgs to run until
 --RUNSTATS_FINISH_TIME = The time you wish runstats to run till
 --DAY_TO_REMOVE = The number of day back you wish staging tables to be emptied from
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------

 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --Reorg the tables
 CALL DB_MAIN.RUN_AUTOMATED_TABLE_REORG(MAINT_SCHEMA, REORG_FINISH_TIME, DAY_TO_REMOVE);
----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --Runstat the tables that have been reorged
 CALL DB_MAIN.RUN_AUTOMATED_TABLE_RUNSTATS(MAINT_SCHEMA, RUNSTATS_FINISH_TIME,DAY_TO_REMOVE);
----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 --Rebind the stored procedures to take advantage of the potentially new plans
 CALL DB_MAIN.RUN_AUTOMATED_REBIND_PROCEDURES(MAINT_SCHEMA);

END

This is now a three stage operation, the first two stages have time limits and so they will carry out new operations until this time limit is breached. What you have to realise here is that if the end time is 18:00:00 then it will start work right up until 17:59:59, this means if it picks up a particularly large reorganisation task at this last second then it will run till it has finished.

Some of the code especially the runstats stuff is quite a lot like the previous version just with a change for the time. As I cant upload a single .zip file as apparently it will be a security risk, and apparently a .sql file is also a risk please find a number of .doc files a the bottom of the article. Please just change the file extension and then you will be able to access them. I would very interested in having feedback from anyone who uses this code to see how you get on with it.

DISCLAIMER: As stated at the top of the blog use this code in your production systems at your own peril. I have tested and know it works on my systems, please test and check it works on yours properly as reorganising tables can potentially dangerous.

FILES WITH CODE IN:

OverallRunnerStoredProcedure

ReorganiseTablesStoredProcedures

ReorganiseTableTables

ReorganiseTableViews

RunstatsTableTables

RunstatsTableViews

RunstatsTableStoredProcedures

RebindSchemaStoredProcedure

IDUG – EMEA – 17th – Final day

First of all can I apologise for the spelling and poor english in some of my posts from IDUG EMEA, I have no excuse other than they were generally wrote late at night and with a few beers inside me. With that out the way lets get on with what I did today. Surprise of the day was seeing a lady in the restaurant having cucumber, chocolate cake and scrambled egg at the same time, but hey if she enjoyed it fair enough.

The talks that I went to today:

IO, IO its off to disk we go – Scott Hayes

There was bit of repetition of the index talk that Scott gave yesterday as the two are really closely related, and seeing him dancing on a video this morning was quite entertaining too (I wonder if that was his wife, or does she know?). Between all the talks on performance monitoring and index and IO tuning I have been to at IDUG EMEA I and my junior are going to have loads to do for several weeks. From this I learnt:

  1. REORGCHK – Does a runstats every time it is called. I am going to have to give this one a try next time I run it and check the col in the TABLES table. ITs not that I dont believe you Scott, its just it works so quickly to do that.
  2. SSD disks are better for random IO or even though you might not be able to afford to buy enough SSD to fit a whole database on it why not just the some of the database that is used the most often!

Database I/O in the brave new world – Aamer Sachedina

This was the second IO talk I went to in the day but it was completely different to Scotts, Aamer looked at it from a more hardware point of view as opposed to the database point of view. It was interesting as the hardware side was always something that I have wanted to know more about and this gave me a good foundation, and some questions to ask my storage manager when I get back. I learnt that:

  1. Thin provisioning does not give you space at all and it is more like sudo space allocated, which can lead to a whole heap of trouble. Will be asking some questions when I get home
  2. If you are using thin provisioning then there are some special db2 registry variables that you need to set (db2set)
  3. Soon we will be getting Fiber channel over copper at the low levels of the SAN stack!

Understanding and tuning page cleaning – Kelly Schlamb

Another talk on improving the IO on my DB2 databases, I am going to be investigating these things as a matter of urgency once I get back to work, if not before as I am itching to improve and learn.  This talk was mainly to do with the differences between settings that you need between having DB2_USE_ALTERNATE_PAGE_CLEANING ON or OFF.

After the conference finished I went for some cheeky sight seeing with Colin a DB2er that I met while over here and Iqbal from Triton consulting and one of the DB2Geeks. We got the Prague Metro into the center of town and got to see some of the sights. We had no idea where we were going, but I think by pure accident we saw most of the sights, or at least things that a lot of other tourists took photos of; so they must be sights right?

Second surprise of the day was meeting @db2fred in the local restaurant to the hotel, that was not the surprise, the fact that he knew who I was before I had even opened my mouth absolutely threw me. Good to put a face to twitter name. So have a nice journey home tomorrow morning Fred.

Tomorrow is the last full day I have in Prague before I fly home early doors on Saturday, because of the money that coming to IDUG as the student of Iqbal (under the mentor scheme) saved me it enabled to sign up for Scott Hayes – Rocket Science: DB2 LUW Performance analysis and Tuning Workshop, which I am hoping will give me even more areas to work on the database and teach me even more about the correct set up. Again I can’t say Thank you enough for doing this for me enabling me to be able to take part in IDUG, Triton Consulting and Iqbal Goralwalla.

IDUG – EMEA – 16th – Day Three

Today was another long day, but was ended by an excellent dinner put on by IBM to thank its customers, with ostrich leg and proper sushi so now we know where all out licensing fee goes!! The talks that I attended did not teach me as much as I had hoped, but I did learn something in each of them though so not a total waste of time.

A DBA’s guide to using TSA – Fredric Engelen

This covered the basics of HADR and then went on to cover how you set up the TSA to take over the HADR, and did not cover the TSM that I hoped it would that I will be implementing soon at Holiday extras. Learn’t:

  1. db2rfpen – Will let force the rollforward of the primary database.

Managing DB2 Performance in an Heterogeneous environment - Jim Wankowski

This covered the differences and similarities between DB2 LUW and DB2 z/OS. Although it was informative I feel the title was not correct for the session and should have been different. I learnt:

  1. When a Sort happens on a VARCHAR column then the column is expanded to its full length – I may ask this question to Scott Hayes when I do his Rocket Science Seminar on Friday

Deep Dive into DB2 LUW offline table and index reorg – Saeid Mohseni

This session was very good, if you are a frequent reader of my blog then you will know that I am trying to get a straight answer to my questions on Reorganisation and Runstats in DB2 and so I got confirmed and learnt:

  1. DB2 reorgs need the current runstats on the table to be correct to give the correct results for the reorganisation identifying stored procedure.
  2. You can parallel run a reorg on a partitioned table index as long as the first, and subsequent runnings do not allow reads.

Data Warehousing – SIG

This was a little disappointing as it did not have an agenda so was unstructured, and I would have liked to have had a little more information on how it was going to be run. It was informative and if any one has heard of “Data Vaulting” then there is a lady from the Netherlands that would really like to know.

Back to the fifties . . . . . 50 fabulous ways for forecasting failures, flaws and finding flubber – Alexander Kopac

This was an excellent talk and there is a lot to try out when I get back home and enough work to keep us going for week probably. The presenter dressed up as a wizard and the bits of SQL he has given in the slides will hopefully make the DB2 team at HX wizards.One main thing to remember is:

  1. KISS – Keep It Simple Stupid

Useful but widely unknown DB2 Functions – Michael Tiefenbacher

Second talk from this guy and if I did not already know, used or have blogged about all the things that he presented this would have been an extreamly useful and I really should have read the Agenda better before getting in there.
And to the final talk of the day:

DB2 LUW Index design, best practice and case studies – Scott Hayes

This was a very good talk and used in conjunction with Alexander’s information I think will build a framework for reviewing indexes and designs at HX. I learnt that:
  1. I need to read up on CLUSTERED indexes
  2. Single column indexes are not good, even though it is the recommended by IBM
  3. You need a good problem statement to come up with a good solution – Can be applied to everything in life.
Tomorrow is the last day of the conference and so it finishes pretty early and so I might get some sight seeing done in the afternoon, but before that I plan on attending:

Thursday, November 17, 2011

08:30 AM – 09:30 AM
Session 15
09:45 AM – 10:45 AM
Session 16
11:15 AM – 12:15 PM
Session 17
12:30 PM – 01:30 PM
Thursday DB2 Panel
So have a good night and see you all in the morning.