Wednesday, July 17, 2013

RESOURCE SEMAPHORE, QUERY MEMORY and PERFORMANCE

Memory has always been my favourite topic when comes to troubleshooting performance in SQL Server :)

I have been contributing and posting few articles on Buffer and Data pages in cache and how it works and all that! But this time I would like to share what exactly query memory is
and how and where to look at, when query performance is suffering!

Let me try to explain in easy language, what happens when query enters the door of SQL Server RDBMS:

1. SQL Server receives query

2. SQL Server creates compiled plan before it creates execution plan

3. SQL server when create compiled plan evaluate the need of two memory parameters

·       Required memory( Min memory need to run sort and hash operations)
·       Additional memory(need to store rows in memory)

4. Server calculates how much memory query need. Formula is
(Required memory * degree of parallelism) + Additional memory

5. If above needed memory exceeds the query memory limit then server reduce the memory to fit the need

6. Now this memory which is able to fit a.k.a requested memory need is fulfilled by RESOURCE SEMAPHORE

7. If RESOURCE_SEMAPHORE can not satisfy this memory need then query is put under RESOURCE_SEMAPHORE wait type

8. And cycle goes on..

If you see high RESOURCE_SEMAPHORE wait types, it is an indicator that query need more requested memory.

This opens up the case to optimize the query!

Tuesday, May 21, 2013

Memory, Memory and Memory

It’s not been long when I was troubleshooting sql server memory related performance issue and it came back again for another environment

I thought why not write it down something about memory, which can help you and me remember clearly about memory related troubleshooting :) :)
 To easily understand this, I am trying to put notes in bullet points.
First two important questions you should ask yourself are:

·        Is this external memory pressure? In simple terms is it Physical RAM A.KA. Windows OS memory pressure?
·        Is this internal memory pressure a.k.a pressure on VAS allocated to SQL Server by windows OS?
Two answer this, it is important to determine where the pressure is coming from?
Next question comes:
·        How quickly I can determine that? and using what tool?
 Answer is using sys.dm_os_ring_buffers:)
 ·        What exactly this DMO is?
This DMO tells us about if any alerts are fired or recorded within the system
If you select result set from this DMO, you will get lot of ring buffers. The one and only which can point to memory pressure is called "RING_BUFFER_RESOURCE_MONITOR"
If you look at the xml record for RING_BUFFER_RESOURCE_MONITOR, keep an eye on node called "<Notification>"
This node will give you insight on the reason for low memory pressure.
 So before we start looking at performance counters like buffer cache hit ratio, page life expectancy, target server memory etc etc, why not query sys.dm_os_ring_buffers and see what it has to tell about any recording of memory!

Monday, April 22, 2013

My SSIS performance is slow! What can I do apart from sneaking into TSQL behind each package!

Hi Guys, it’s been while I posted something interesting but lately I experienced the scenario dealing with SSIS performance tuning and I thought it’s a good chance to share some knowledge with you!
I am sure, as a DBA's we come across scenarios where we get complaints like SSIS ETL data loading exceeds allowable time threshold etc etc
I am also sure, we try to leverage things like sneaking into queries running behind each step or turn on performance counters on source and destination sql server
But there is something else, we can look into and that is SSIS buffer usage!!
I would like to split this article into two major parts
1. Transformation
I would prefer not to just say to my client that SSIS design is not efficient. Rather I would say how efficient transformation behaviour for each tree in the SSIS package is :)!

Let’s take one step further, where we would like to collect how SSIS plays with its buffer when executing the threads in each tree. I am highlighting each sentence as a bullet point for better understanding!
·        We all are familiar with Buffer, but here I am focusing more on SSIS buffer instead of SQL Server DB engine buffer.
·        SSIS takes advantage of their in- memory structure (Buffer) in a same way as SQL engine does. Basically to avoid loading data all the time from the source, It controls the data using buffer as part of the transformation from source and destination
·        Behind SSIS, we have logic of execution trees and each tree has its own in memory structure a.k.a buffer to play with.
·        There can be multiple threads running for each tree.
·        Goal of design should be to avoid overhead of data into new buffer every time!

2. Buffer Sizing
To utilise buffer to its maximum potential, SSIS makes certain decisions, which are based on 4 input parameters:
Estimated Row Size: SSIS computes this value based on metadata
DefaultMaxBufferRows: setting which can be change (10,000 records)
DefaultMaxBufferSize: setting which can be change (10 MB)
MinBufferSize: setting which can be change (65536 bytes)

If Estimated Row Size * DefaultMaxBufferRows > MaxBufferSize
{
     SSIS reduces total number of rows which can be store in buffer
}
Else
{
SSIS increases the total number of rows which can be store in buffer
}
So next time, If you turn on package logging with the BufferSizeTuning property enabled, you can get better picture of where and how buffer size changes and you can adjust the values of DefaultMaxBufferRows and DefaultMaxBufferSize to optimize the number of rows in buffer

Sunday, March 3, 2013

Make DBA life easier with powershell - Get File size and percentage of space used

As a SQL dba, we all manage disk spaces. Whether it is associated with data drive, log drive or even TempDB drive.

In one particular scenario, it can be bit tricky or rather I would say bit time consuming to get the correct insight on what is taking most spaces in your volume.

This scenario is where you are hosting several databases and their files on the same drive or mount points and each database files are managed in different file group and folders.

One way to know is by manually logging on to the SSMS and actually sees the size of each database files and location.

But the way, I am going to suggest you here is to use to our great PowerShell script. All you have to do is simply remote desktop and open/execute PowerShell script.

For example: to get all the files under c:\temp location and their space usage (in %), you can execute below script:

$path = "c:\temp"
$totalfilesize = [int]((Get-ChildItem $path | Measure-Object -Sum Length).Sum / 1kb)
Get-ChildItem $path  | Select-Object Name, CreationTime,  @{Name="kb";Expression={$_.Length / 1KB}}, `
@{Name="Percent";Expression={(($_.Length / 1KB)/$totalfilesize )*100}}| sort Percent -descending;

This is very handy especially when you are managing lot of low disk space alerts.

Tuesday, December 18, 2012

SQL Server audit and Lock request time out period exceeded. (.Net SqlClient Data Provider) error


Have you ever noticed "PREEMPTIVE_XE_CALLBACKEXECUTE" wait type on one of your SPID which is not letting you perform any operation at SQL Server instance???

I have seen this wait type  when using SQL Server auditing and it seems that transaction is stuck and blocking meta deta. The exclusive lock was placed on sysclsobj in master database

If you try to see corresponding OS thread(KPID), you would notice that thread  actually belongs to sql server agent task.

Below are some additional findings :

 SPID for server audit is blocking all other instance level operation
 Stopping and restarting sql server agent job does nothing
 "PREEMPTIVE_XE_CALLBACKEXECUTE" reflects that sql thread is waiting on OS level resource
 Eventually there is no wait resource

This is a bug in SQL Server 2008 R2 SP1 and is fixed in CU 6 : ): )
Puneet

Sunday, December 9, 2012

Update statistics some key points!

As a DBA, we all know that update stats is one of the key, which let SQL optimizer to decide the best (best means fast :)) way to retrieve the query results.
Now we also know that the best practice is to not just let statistics get updated automatically but also to place some kind of manual plan.
Although, you can choose a best time to manually update the statistics on every server in your farm, but realistically, you don’t have control over the time of modification of data into the tables as its control by application/users/business.
So what you can do your best is to make sure that statistics are updated most of the time!
You can leverage "sys.dm_db_stats_properties" to monitor the stats on your objects, especially large objects.
For smaller to medium table, it’s not a big concern as auto stats with manual plans can take care of most of the situation but with Large size tables(millions of rows), it can be bit tricky!
I would not rely on default automatic update of statistics for large objects.
Tables with millions of rows mean, you have to wait longer before it reaches 20% of threshold value.
Thankfully, SQL Server 2008 R2 released trace flag 2371, which controls the update dynamically.
What this mean is larger the number of rows exposed to modification, smaller the threshold get set and quicker the automatic update can happen.
Having said all this, make sure you test this trace flag  properly for your application and then make a decision.

Tuesday, September 25, 2012

Memory architecture changed in SQL Server 2012

So far, as a DBA'S we were always  trained to configure memory on 64 bit OS and follow the things like lock pagesin memory, setting max and min memory etc

With the release of SQL Server 2012, some of the things need consideration as memory structure has changed a bit ( for good ofcourse :) )

In SQL Server 2012, now CLR and MPA are now part of the buffer pool itself.

What this means?????

This means requests for CLR allocation and multi-page allocation(MPA) will be handled by buffer pool and is not outside like it used to be in sql server 2008 R2 and previous versions.

Now when we are configuring max server and min server memory, we have to keep in mind that allocations of pages greater than 8KB, like that for CLR and MPA are also controlled by max and min settings.

So these settings need to be revised, whenever you are upgrading to sql server 2012.

sql server 2012  now will account for more memory allocations(even pages > 8kb) from buffer.

Also for the 32 bit SQL, AWE feature is deprecated.