Old ladyHow Big Is Your Package?
Size up and convert your DTS package to SSIS. If you have the biggest DTS package, win a gift card and certificate of a world record package size.  
November 2008 - Posts - Brian Knight

Pragmatic Works

Welcome to Pragmatic Works Sign in | Join | Help
in Search

Brian Knight

November 2008 - Posts

  • How to use a Script Transformation

    Using a Script transformation is a handy way to extend what transforms are available to you. For example, picture social security data that goes into your Script transformation unencrypted and then out encrypted. Typically, you can also use this type of custom script to extend what a Derived Column task can do. If you have the choice between the two though, always choose a Derived Column.

    A good extension of a Derived Column transformation into a Script transformation would be where you need to replace certain text with other text. If you have only two options, you could use the REPLACE expression inside a Derived Column transform. When you go above two, the Replace expression becomes less elegant. So, in comes the Script transform.

    After connecting the Script transform to the data flow, go to the Input Column task for the transform and check the columns that will be sent into the script as an input. Next, go to the Inputs and Outputs tab and click Add Output. Name the columns that you wish to output. I’m going to call this one OutputValue for this example. Next, click Design Script.

    In this script you can see that first we localize the variable. Because I can’t predict whether the user inputted mixed, lower or upper case, I use the UCASE function to upper case the input value for comparison purposes. The word InputValue in Row.InputValue would be replaced with whatever you checked in the input screen. Then, the main thing to remember is Row.OutputValue will set the OutputValue output we set earlier will be set to the new value. So this code will take the input of whatever the color is and translate it to the hex value. Then, the OutputValue output will be presented to the pipeline and can be consumed by the destination or the next transform in the pipeline.

    Imports System
    Imports System.Data
    Imports System.Math
    Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
    Imports Microsoft.SqlServer.Dts.Runtime.Wrapper

    Public Class ScriptMain
    Inherits UserComponent

    Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
    Dim HexValue As String

    Select Case UCase(Row.InputValue)
    Case "RED"
          HexValue = "#FF0000"
       Case "YELLOW"
          HexValue = "#FFFF00"
       Case "GREEN"
    HexValue = "#008000"
    End Select

    Row.OutputValue = HexValue
    End Sub
    End Class

  • Parsing the OnPipelineRowsSent

    The OnPipelineRowsSent event functions in the data flow to log how many rows go between step to step in the pipeline (or from input to output). Once you have a data flow task, you can right-click in the design pane and select Logging. Configure the logging provider and now notice in detail you have OnPipelineRowsSent. This will now create a record for each transform (source or destination) that the data goes through to show how many rows were sent through the pipeline. It writes this though in a single message column pipe delimitted. So, you'll need to write some sort of view or routine to parse out the critical data you need. Here's the query that's from one of the SSIS report packs to parsse this into a readable format that I use:

    SELECT     source, sourceid, executionid, ssrs.ParsePipeline(message, 1) AS PathID, ssrs.ParsePipeline(message, 2) AS PathIDName,
                          ssrs.ParsePipeline(message, 3) AS ComponentID, ssrs.ParsePipeline(message, 4) AS ComponentIDName, ssrs.ParsePipeline(message, 5) AS InputID,
                          ssrs.ParsePipeline(message, 6) AS InputIDName, CONVERT(int, ssrs.ParsePipeline(message, 7)) AS RowsSent, starttime
    FROM         dbo.sysdtslog90
    WHERE     (event LIKE 'onpipelinerowssent%')

    Here's the function needed to do the parsing:

    set ANSI_NULLS ON
    set QUOTED_IDENTIFIER ON
    GO
    ----------------------------------------------------------------------------
    --! CREATE FUNCTION (for OnPipeLineRowsSent report)
    -- function to parse OnpipelineRowsSent log entry from SSIS sysdtslog90 table
    -- Function parses one log entry at a time so a view is also needed (below)
    -- to call\use the function for each log entry.
    ----------------------------------------------------------------------------

    CREATE function [ssrs].[ParsePipeline] (@message varchar(8000), @which int)

    --should be created in the DB with desired sysdtslog90 table

    returns varchar(200)

    as begin

    --@which defines which value is desired

    -- 1= PathID
    -- 2= declare PathIDName
    -- 3= declare ComponentID
    -- 4= declare ComponentIDName
    -- 5= declare InputID
    -- 6= declare InputIDName
    -- 7= declare rowssent
    declare @sourcemessage varchar(600)
    declare @where as integer
    declare @mycounter integer
    If @which < 1 or @which > 7 return null
    set @mycounter=0

    --catch older versions of the messages that lacked the extra parameters

    if patindex('%: :%', @message) = 0 return null

    --chop the initial wordy stuff out
    set @sourcemessage = right(@message, len(@message) - patindex('%: :%', @message) - 3)

    --loop through occurances of : until we get to the desired one
    set @where = 99
    while @where <> 0 begin
    set @mycounter = @mycounter+1
    set @where = patindex('%:%',@sourcemessage)
    If @mycounter = 7 return @sourcemessage
    if @mycounter = @which return(left(@sourcemessage, @where - 1))
    set @sourcemessage = right(@sourcemessage, (len(@sourcemessage) - @where))

    end --while

    --should not execute this but a return is required as the last statement
    return @sourcemessage
    end --function

    Posted Nov 28 2008, 10:13 PM by Brian Knight with no comments
    Filed under:
  • Setting Variables in the Script Task in SSIS

    One of the key reasons that you use the script task is to change the value of a variable at runtime. There is a lot of real-world scenarios that you would use this for. If you're reading this you have already probably thought of a few. To set a variable in the script task, there are two main methods you can use.

    Method 1

    This method involves using the LockOneForWrite method in the VariableDispenser class. The advantage to this method is it allows for you to read and write to the variables at runtime without having to use the ReadOnlyVariables and ReadWriteVariables options in the Script task. The price of that though is that you have to write quite a bit more code. In the following example, I'm going to open the AlertAdmin variable for writing and then set it to the boolean value of True.

    Public Sub Main()
    Dim vars As Variables
       Dts.VariableDispenser.LockOneForWrite("AlertAdmin", vars)
       vars("AlertAdmin").Value = True
       Dts.TaskResult = Dts.Results.Success
    End Sub

    Method 2

    The second method is typically what I would recommend using just for simplicity. In this method, you would open the Script task and before clicking on Design Script, you must ReadOnlyVariables and ReadWriteVariables options. If you have more than one variable you wish to be available in the Script task, you can seperate them with commas. If you do not set this, you will not see an error at design time but at run time, you'll receive the error shown below:

    The element cannot be found in a collection. This error happens when you try to retrieve an element from a collection on a container during execution of the package and the element is not there.

    at Microsoft.SqlServer.Dts.Runtime.Variables.get_Item(Object index)

    at ScriptTask_8693feb81f2d4b7b8a26ae87a8a5f960.ScriptMain.Main() in dts://Scripts/ScriptTask_8693feb81f2d4b7b8a26ae87a8a5f960/ScriptMain:line 19

    With the variables now being passed in, you can perform the same type of action as I showed in the earlier script in a single line (the second line shown below). The locking is done by the Script task UI.

    Public Sub Main()
       Dts.Variables("AlertAdmin").Value = True
       Dts.TaskResult = Dts.Results.Success
    End Sub

    The ReadOnlyVariables and ReadWriteVariables options are available to you in the Expressions tab too if the variable name that you wish to pass in is unknown or dynamic. There are other ways to set the variables of course. Most tasks have hooks into the variables like the Execute SQL task. Hope this helps!

  • Templates in SSIS

    Templates in SSIS provide a great way to create standards across your company or enterprise. For example, you may want to enforce a standard annotation set of notes at the top of each package or have each package come with a standard set of connections or error handlers to speed up development. To use a template, first create a package just as you would want to see it as a template. Add all the connections, tasks, comments or log providers. After the package meets your needs, copy it to the following directory (of course replace %Program Files%) : %ProgramFiles%\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies\ProjectItems\DataTransformationProject\DataTransformationItems

    For a developer to consume the template, he must only right-click at the project-level node in BIDS and select New Item. You'll see the package you copied over there and once selected, all your information you created earlier is ported over.

    Posted Nov 27 2008, 08:41 PM by Brian Knight with no comments
    Filed under:
  • PASS Followup

    Thanks to all who came to one of my four sessions at PASS. I had a blast speaking with you guys in the hallways and at lunches. For those who are looking for the code from one of the SSIS sessions, you can use the following zip file, which combines a good chunk of the demos.

    http://www.pragmaticworks.com/bootcamp.zip

    Thanks again and congrats to Andy Warren and Douglas McDowell for joining the PASS board along with the Lynda.

More Posts
Powered by Community Server (Non-Commercial Edition), by Telligent Systems