Showing posts with label index. Show all posts
Showing posts with label index. Show all posts

Monday, March 12, 2012

processing cube with partitions

Hi,

I have a cube with 2 partitions. Are these 2 partitions being processed everytime I have a ProcessData and Process Index? Coz i worry that if it's just processing 1 partition every time, i might have new data in one extraction that should reside in Partition 1 and another set of data (also in the same extraction) that reside on Partition 2.

cherriesh

If you are issuing your ProcessData and ProcessIndex commands against the cube, then it will cascade down and process all the partitions in that cube.|||

ic... but is there an option which partition i will just process?

cherriesh

|||Sure, you can process just one partition at a time (or multiple in parallel). There are multiple ways to do this. You can right click on a partition in SSMS and process it or you can build an XMLA command to process one or more partitions, you can do it from the partitions screen in BIDS.|||

is it a standard to just process only a partition of a cube every time (mine is partitioned based on This Year and Previous Years...) or by using Process Data and Process Index instead (meaning, process all)? What do you usually do?

|||

Usually you try to only process the smallest amount of data possible. I have done both extremes, sometimes I have to process the entire cube everytime, other times I only do a recent partition. It depends on the nature of the data and the source systems.

Note: ProcessData and ProcessIndex can be done at a partition level too. In fact when you do a process operation at the cube level, all that happens is that SSAS performs the specified processing command on each partition in the cube.

|||

Hi,

How to do it in the partition level?

thanks a lot!

cherriesh

|||

I don't have access to an Adventure Works database at the moment, so I don't know if the object Id's below are valid, but the basic structure of the XMLA command to process a partition would look like the following.

<Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">

<Parallel>

<Process xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance

xmlnsBig Smiledl2=http://schemas.microsoft.com/analysisservices/2003/engine/2

xmlnsBig Smiledl2_2=http://schemas.microsoft.com/analysisservices/2003/engine/2/2>

<Object>

<DatabaseID>Adventure Works DW</DatabaseID>

<CubeID>Adventure Works</CubeID>

<MeasureGroupID>Sales</MeasureGroupID>

<PartitionID>Sales 2003</PartitionID>

</Object>

<Type>ProcessFull</Type>

<WriteBackTableCreation>UseExisting</WriteBackTableCreation>

</Process>

</Parallel>

</Batch>

You can also navigate to the partition in the object browser in SSMS, right click on one and process it and you should be able to use the processing task in SSIS to do just a single partition. There are probably a couple of other methods of doing this too, essentially the all end up sending an XMLA command to the SSAS server. In my example I am using ProcessFull, but ProcessData and ProcessIndex will work at the partition level.

processing cube with partitions

Hi,

I have a cube with 2 partitions. Are these 2 partitions being processed everytime I have a ProcessData and Process Index? Coz i worry that if it's just processing 1 partition every time, i might have new data in one extraction that should reside in Partition 1 and another set of data (also in the same extraction) that reside on Partition 2.

cherriesh

If you are issuing your ProcessData and ProcessIndex commands against the cube, then it will cascade down and process all the partitions in that cube.|||

ic... but is there an option which partition i will just process?

cherriesh

|||Sure, you can process just one partition at a time (or multiple in parallel). There are multiple ways to do this. You can right click on a partition in SSMS and process it or you can build an XMLA command to process one or more partitions, you can do it from the partitions screen in BIDS.|||

is it a standard to just process only a partition of a cube every time (mine is partitioned based on This Year and Previous Years...) or by using Process Data and Process Index instead (meaning, process all)? What do you usually do?

|||

Usually you try to only process the smallest amount of data possible. I have done both extremes, sometimes I have to process the entire cube everytime, other times I only do a recent partition. It depends on the nature of the data and the source systems.

Note: ProcessData and ProcessIndex can be done at a partition level too. In fact when you do a process operation at the cube level, all that happens is that SSAS performs the specified processing command on each partition in the cube.

|||

Hi,

How to do it in the partition level?

thanks a lot!

cherriesh

|||

I don't have access to an Adventure Works database at the moment, so I don't know if the object Id's below are valid, but the basic structure of the XMLA command to process a partition would look like the following.

<Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">

<Parallel>

<Process xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance

xmlnsBig Smiledl2=http://schemas.microsoft.com/analysisservices/2003/engine/2

xmlnsBig Smiledl2_2=http://schemas.microsoft.com/analysisservices/2003/engine/2/2>

<Object>

<DatabaseID>Adventure Works DW</DatabaseID>

<CubeID>Adventure Works</CubeID>

<MeasureGroupID>Sales</MeasureGroupID>

<PartitionID>Sales 2003</PartitionID>

</Object>

<Type>ProcessFull</Type>

<WriteBackTableCreation>UseExisting</WriteBackTableCreation>

</Process>

</Parallel>

</Batch>

You can also navigate to the partition in the object browser in SSMS, right click on one and process it and you should be able to use the processing task in SSIS to do just a single partition. There are probably a couple of other methods of doing this too, essentially the all end up sending an XMLA command to the SSAS server. In my example I am using ProcessFull, but ProcessData and ProcessIndex will work at the partition level.

Saturday, February 25, 2012

Process Cube using Integration Services

Hi,

I want to process my cube using Process Data and Process Index instead of the Process Full. However, after configuring the 2 Analysis Services Processing Tasks (one for process data and the other for process index) and were executed sequentially (process data first then process index), I got this error:

Errors in the metadata manager. The process type specified for the CASES cube is not valid since it is not processed

Have I done the right thing?

The reason why I prefer using the Process Data and then Process Index, it's because it is much faster than the latter.

cherriesh

It's interesting that you're seeing an error, because 'Process Full' is nothing more than 'Process Data' and then 'Process Index' under the covers.

I'll admit that I was skeptical that executing the two sequentially was faster for you than process full, but I found an Analysis Services best practices whitepaper that confirmed this could happen.

http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/ssaspbpr.mspx

Are you sure you are using 'Process Data' on the cube object itself and not a measure group or partition?

Also, you may want to post this question in the Analysis Services forum.

|||

Hi,

I did the process data for both the dimensions and cube. and did also process index for the two.

am i right there?

cherriesh

|||

I think that might be your issue. If it processes a dimension after the cube that would explain the error message you received on the step to process the index. Try doing process data for just the cube (it should do the dimensions as part of this).

Process Cube

Hi,

I want to process my cube using Process Data and Process Index instead of the Process Full. However, after configuring the 2 Analysis Services Processing Tasks (one for process data and the other for process index) and were executed sequentially (process data first then process index), I got this error:

Errors in the metadata manager. The process type specified for the CASES cube is not valid since it is not processed

Have I done the right thing?

The reason why I prefer using the Process Data and then Process Index, it's because it is much faster than the latter.

I'm not sure though what Process Index is for. As far as I'm concerned, I have done partitions (I used performance gains reaches ) but I don't remember to explicitly declare indexes.

Please help...

cherriesh

Hi,

In theory what you did to me seems as if it should work. Have you tried doing it outside SSIS? So in managment studio, process data and then index, might be a problem with the cube not actually processing the data step properly.

but here is a description of what the different process types do

http://technet.microsoft.com/en-us/library/ms174774.aspx

Hope that helps

Matt

|||

Hi,

The ProcessFull is working fine. I dunno how to do this outside SSIS. Am I right to have Process Data for both Dimension and Cube and to have another Analysis Services Process Task for Process Index for both (Dimension and cube)

cherriesh

|||

Hi,

I found a better link:

http://msdn2.microsoft.com/en-us/library/ms345142.aspx

For the dimensions:

A process full will reprocess the dimension completely

Never done a process index on a dimension, but i would assume process data then process index.

Once the dimensions are processed then process the cube

Process Full

Process data and process index (two seperate tasks I am guessing)

You could try

(task 1) Is to process the structure of the cube (processes all the dimensions)

(task 2) Process full of cube

OR

task 1, process data of dims

task 2, process indexes

task 3, process data of cube

task 4, process indexes of cube

OR

task 1, process structure of cube

task 2, process data of cube

task 3, process indexes of cube

Sure there are lots of different ways.

With reference to aleternative ways of processing:

http://technet.microsoft.com/en-us/library/aa337509.aspx

Hope that helps,

Matt