[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Row Operations using OID suppression and aggregate row objects




Something that I forgot to add, but which should have been included in 
the text that I sent out earlier regarding this alternative approach to
Row Operations is a footnote as to where this concept originated (to give
credit where credit is due). The text that I have written below is
discussing an approach that was presented to the EOS working group 
by Dr. Jeff Case at the 50th IETF meeting in Minneapolis. 


			Sincerely,

			Sandra McLeod




> *********  Row Operations using OID suppression and aggregate row objects
> 
> Goals of OID Suppression for performing Row Operations 
> 
>   OID Suppression techniques provide a mechanism for simpler, more
>   efficient row-based operations through the definition and use of
>   aggregate row type objects.  OID Suppression can be used to address the
>   need for more efficient bulk data retrieval, increased efficiency for
>   performing large atomic configuration operations, and simplified
>   row-based SNMP operations. OID Suppression as described in this memo
>   meets these needs in the following ways:
>  
>     - OID suppression reduces the amount of redundant information in SNMP
>       PDUs, allowing a greater number of varbinds to be encoded in a
>       smaller amount of space within a single SNMP PDU. This provides for
>       more efficient bulk PDUs and allows for larger atomic row
>       operations.
>  
>    - The OID suppression technique discussed in this memo introduces a new
>      aggregate row object type which provides the ability to represent and
>      operate on a conceptual table row as a single logical unit, which is
>      often how they are managed and communicated by management
>      applications.
> 
> Terminology 
> 
>   The purpose of this memo is to discuss the benefits of OID Suppression
>   and its application to row-based operations. However, it is important
>   for clarification purposes to discuss terminology and, in particular,
>   the difference between OID Suppression and OID Compression.  These two
>   techniques are similar in that they both attempt to reduce the amount of
>   space required to encode variable bindings in SNMP PDUs by reducing
>   redundant information in the name portions of the variable bindings. The
>   goal of both these techniques is to provide for more efficient bulk data
>   retrieval and to address current problems of passing large amounts of
>   configuration information to a command responder in a single atomic SNMP
>   Set Request. However, OID Suppression and OID Compression use different
>   approaches to achieve both of these goals and can be applied either
>   independently or in combination.
> 
>   OID Suppression reduces the amount of redundant information in the
>   Object Identifiers in a series of variable binding names by reducing the
>   actual number of OIDs that must be encoded in the SNMP PDU. OID
>   suppression is particularly well suited for row-based operations (row
>   retrieval or configuration) where all of the Object Identifiers for a
>   row of objects share the same base table OID and instance information.
>   When using OID Suppression techniques the base table OID and instance
>   information can be provided once for all objects in the same row, with
>   only a single column identifier (either implicitly or explicitly
>   provided) required to represent each column in the row.
>  
>   OID Compression, on the other hand, reduces the amount of redundant
>   information in a series of variable binding names by reducing the size
>   of each OID without actually omitting any OIDs in the encoded SNMP PDU.
>   This technique can potentially be used to compress any collection of
>   variable binding names, not just row objects, in the same SNMP PDU. This
>   makes OID compression well suited for operations such as MIB walks, bulk
>   data retrieval, and large configuration operations where the variable
>   bindings include both scalars and tabular objects and a large percentage
>   of the Object Identifiers in the varbind list have some common base set
>   of subidentifiers.
>  
>   The rest of this memo is limited to the scope of OID Suppression and its
>   application to row operations.  OID Compression is discussed in more
>   detail in the Internet Draft draft-ietf-eos-oidcompression-00.txt.
> 
> 
> OID Suppression Through Aggregate Row Objects
> 
>   OID Suppression can be used to provide more efficient retrieval and
>   configuration of conceptual rows in a table by reducing the amount of
>   information required to represent a row of objects. Within a row in a
>   table, the OIDs of each of the objects in the row contain a substantial
>   amount of redundant information since each of these OIDs contain the
>   same table name and instance information.  The only variation among the
>   OIDs for a group of objects in the same row of a table is the single
>   column subidentifier in the OID which uniquely identifies that specific
>   column in the table.
> 
>   OID Suppression, as described in this memo, reduces the amount of
>   redundant information required to represent a row in a table by allowing
>   the table name and instance information to be provided only once for the
>   entire row instead of requiring this information to be provided N times
>   for N objects in a row.  This is achieved through the definition of a
>   new aggregate row type object. The aggregate row object allows multiple
>   columns from the same conceptual row to be represented as a single
>   atomic unit with only a single OID required to identify this row in the
>   table.  The OID of this aggregate row type object would include the
>   table name and instance information for this row. The value of the
>   aggregate row object would actually consist of a sequence of values
>   for the columns in this row.
>  
>   In general, an aggregate row object with N columns would have the
>   following format :
> 
>        <tablename>.2.<instance>  =  (val1, val2, ..., valN)
> 
>   The OID for this aggregate row object would include the table name and
>   instance information.  The value of the aggregate object would actually
>   be a sequence of values for the columns in this row of the table.
> 
>   In the format specified above, the use of the subidentifier '2'
>   separating the tablename from the instance information is required to be
>   able to uniquely and unambiguously distinguish aggregate row objects
>   from single columnar objects since it is anticipated that aggregate row
>   objects may be included in SNMP PDUs that also contain non-aggregate
>   type objects as well. OIDs for single columnar objects will always begin
>   with the OID prefix <tablename>.1 whereas OIDs for aggregate rows will
>   always begin with the OID prefix <tablename>.2.
>  
>   The definition and use of the aggregate row object type would allow a
>   significant amount of OID suppression to be achieved by removing the
>   redundant OID information that is normally required to represent a
>   series of objects in a row.  With the use of aggregate row objects, the
>   tablename and instance information for a row of objects would need to be
>   provided only once per row instead of N times per row for N objects in a
>   row. This means that the individual representation of each row would be
>   more efficient and therefore operations to retrieve and configure rows
>   using aggregate row objects would be more efficient overall.
> 
> 
> Row Operations Using Aggregate Row Objects
>   
>   OID Suppression through the use of aggregate row objects not only
>   provides the benefit of more efficient retrieval and configuration of
>   rows, it also simplifies row-based operations by allowing rows to be
>   represented and operated on as a single logical unit. This is useful to
>   management applications which tend to manage tables by conceptual rows
>   and which often treat objects in the same conceptual row as part of a
>   larger atomic unit for operational purposes.
> 
>   One of the ways that aggregate row objects and more efficient row
>   operations would be beneficial is in the case of row-based configuration
>   operations where it is desirable for manager applications to be able to
>   configure an entire conceptual row with a single atomic SNMP
>   SetRequest.  For large tables this can be a problem due to the size of
>   the required encoding for an entire row and may require that the row be
>   configured in multiple partial SetRequest operations.  This adds
>   complexity to the agent by requiring that they implement complex state
>   mechanisms such as RowStatus in their MIB tables. However, the use of
>   OID Suppression, and in particular, aggregate row objects, means that
>   row operations are substantially more efficient, allowing a greater
>   number of row objects to be encoded in a smaller amount of space within
>   a single SNMP PDU.  This would make it easier to configure larger tables
>   in a single atomic SNMP SetRequest.
>  
>   The use of aggregate row objects would also help to simplify the
>   implementation of row-based operations by imposing some ordering to the 
>   objects in each row that is encoded as an aggregate row object in an
>   SNMP PDU. Aggregate row objectgs that provide values for the row in
>   order by column would be particularly beneficial to agents implementing
>   tables with interdependency requirements between values of objects in
>   the same row of a table.  
> 
> 
> Implicit versus Explicit column identification
> 
>   Aggregate row objects can be defined with either implicit or explicit
>   column identification in the value sequence. Implicit column
>   identification relies on positional context in order to map a sequence
>   of values to their corresponding column in a table. Explicit column
>   identification requires an explicit column identifier to be specified
>   for each value in the aggregate object.   It is foreseeable that both of
>   these approaches will be useful under different circumstances or
>   possibly in combination with the other.
> 
>   Implicit column identification would require that a value be specified
>   for each object in the row.  The first value in the sequence would
>   correspond to the first column in the table. The second value in the
>   sequence would correspond to the second column, and so on.  This
>   approach requires less encoding for each value as it would not require
>   that a column identifier be explicitly specified for each corresponding
>   value. However, in the case where instances of some of the columns were
>   missing, NULL placeholders would be required in order to maintain a
>   one-to-one mapping between the sequence of values and the table columns.
> 
>   Explicit column identification would require that a column identifier be
>   explicitly specified for each columnar value in the aggregate row
>   objects value sequence. This approach would require additional encoding
>   for the column identifiers but could be beneficial in the case of tables
>   with a large number of missing columns and could also be useful for
>   performing operations on a subset of the columns in a table.
>  
>  
> OID Suppression Example
> 
>  
>   To illustrate the application of OID suppression to a variable bindings
>   list consider a row of objects in the snmpNotifyTable :
> 
>   snmpNotifyTable.1.2.101.110.116.114.121.49 = trapMgrs  --snmpNotifyTag
>   snmpNotifyTable.1.3.101.110.116.114.121.49 = 1        --snmpNotifyType
>   snmpNotifyTable.1.4.101.110.116.114.121.49 = 3  -- snmpNotifyStorageType
>   snmpNotifyTable.1.5.101.110.116.114.121.49 = 1   --snmpNotifyRowStatus
>  
>   The variable bindings list above is presented without the use of OID
>   Suppression. If OID Suppression is applied to this variable bindings
>   list, the result would be a single aggregate row object. Below is the
>   aggregate row object for this row in the snmpNotifyTable with implicit
>   column identification :
> 
>  
>    snmpNotifyTable.2.101.110.116.114.121.49 =    ( NULL,trapMgrs,1,3,1) 
>  
>   In this example above the 4 objects in the row of the snmpNotifyTable
>   have been combined into a single aggregate row object. The name portion
>   of this variable binding is of the format <tablename>.2.<instance>.  The
>   value portion of the aggregate row object is actually a sequence of
>   values. This sequence of values consists of a value for each of the
>   columns in the snmpNotifyTable. Note that the first column is NULL as
>   this is the index column that is defined as not-accessible.  The NULL
>   value acts as a placeholder where holes or inaccessible columns in the
>   table exist.
> 
>   It is easy to see that the resulting aggregate row object is 
>   considerably more compact with the redundant OID information removed.
>   The aggregate row object is more efficient and would require fewer bytes
>   to encode than the original varbind list above.
>  
>   If explicit column identification was applied instead of implicit column
>   identification, then the resulting aggregate row object would look
>   something like this :
>  
>      snmpNotifyTable.2.101.110.116.114.121.49 =  (   (col2,trapMgrs),
>                                                            (col3, 1),  
>                                                            (col4, 3),
>                                                            (col5, 1)   )
>  
>   Note that with explicit column identification the columns that are
>   inaccessible or which have missing values for this row can be omitted
>   from the value sequence. This is not possible with implicit column
>   identification since the position of each value in the sequence is used
>   to determine which column that value corresponds to.
>  
>  
>  
> Encoding Aggregate Row Objects 
>  
>   The ASN.1 variable binding notation requires some changes to accommodate
>   aggregate objects.  It is expected that the new aggregate row object
>   will only be valid for use in a new set of SNMP PDUs that are to be
>   defined to address the need for simpler, more efficient row operations.
>   The following is a proposal for modifying the variable binding notation
>   for these new, yet-to-be-defined PDUS to allow the varbinds in these
>   PDUs to include both aggregate and non-aggregate objects. In the
>   following proposal, there are no changes to the VarBindList or
>   ObjectSyntax definitions.
>  
>     VarBind ::=
>         SEQUENCE {
>             name
>                 ObjectName,
>             CHOICE {
>                 value
>                     ObjectSyntaxNonAggregate,
>                 value
>                     ObjectSyntaxAggregate
>             }
>         }
>  
>     ObjectSyntaxNonAggregate ::=
>         CHOICE {
>             ObjectSyntax,            -- as in [RFC2578]
>             ObjectSyntaxExtension,   -- new scalar types
>             ObjectSyntaxNullType     -- NULL and exceptions
>         }
>  
>     ObjectSyntaxExtension ::=
>         CHOICE {
>             -- Nothing for now
>             -- Eventually there will be new 64 bit types, these
>             -- are being defined elsewhere
>         } 
>  
>  
>     --  ObjectSyntaxNullType separated for cleanliness.  There is
>     --  no protocol requirement for this.
>  
>     ObjectSyntaxNullType ::=
>         CHOICE {
>             unSpecified         -- in retrieval requests
>                     NULL,       -- exceptions in responses
>             noSuchObject[0]
>                     IMPLICIT NULL,
>             noSuchInstance[1]
>                     IMPLICIT NULL,
>             endOfMibView[2]                              
>                     IMPLICIT NULL                        
>             nonInstantiatedRowObject[3]   -- plugs holes in rows
>                     IMPLICIT NULL
>         }
>  
>     ObjectSyntaxAggregate ::=
>         CHOICE {
>           implicitAggregateValue
>                 ImplicitAggregate,
>           explicitAggregateValue
>                 ExplicitAggregate
>          }
>  
>     ImplicitAggregate ::=
>         [Application 12] IMPLICIT SEQUENCE OF
>              ObjectSyntaxNonAggregate
>  
>     ExplicitAggregate ::= 
>         [Application 13] IMPLICIT SEQUENCE OF
>             EAFragment
>  
>     EAFragment ::=
>         SEQUENCE {
>             eAFragmentNamePart
>                 INTEGER (0..4294967295),
>             eAFragmentValuePart
>                 ObjectSyntaxNonAggregate
>         }   
>                 
> 
> 
>   Note that, if no aggregate types are used and no 
>   nonInstantiatedRowObject is used, that this notation produces results
>   equivalent to that of [RFC1905]. That is to say, if no row operations
>   are used, then the encoding on the wire is unchanged by this notation.
>  
> 
> 
> OID Suppression Encoding Cost Comparison
> 
>   The following is a comparison of cost for encoding a variable bindings
>   list with and without OID Suppression. Using the following example that
>   was provided previously in this text with the snmpNotifyTable objects :
> 
>   snmpNotifyTable.1.2.101.110.116.114.121.49 = trapMgrs  --snmpNotifyTag
>   snmpNotifyTable.1.3.101.110.116.114.121.49 = 1        --snmpNotifyType
>   snmpNotifyTable.1.4.101.110.116.114.121.49 = 3  -- snmpNotifyStorageType
>   snmpNotifyTable.1.5.101.110.116.114.121.49 = 1   --snmpNotifyRowStatus
> 
> 
>   the size of the encoded variable binding list for the row shown above
>   both with and without OID Suppression is as follows (see further below
>   for actual encodings) :
> 
>     
>      Without OID Suppression :			               101 bytes    
>      Using OID Suppression and Implicit Column Identification : 44 bytes
>      Using OID Suppression and Explicit Column Identification : 62 bytes
> 
> 
>   It is obvious that using OID Suppression provides a significant decrease
>   in the encoded size of the variable bindings list and therefore provides
>   a much more efficient representation of this row in the snmpNotifyTable.
>   In this example, OID Suppression provides a 56% decrease in size when
>   using implicit column identification and a 40% decrease in size when
>   explicit column identification is used.
>                     
>   The following is an example of the variable binding list above
>   encoded with and without OID Suppression. The first encoding example is
>   provided without any use of OID Suppression. The two subsequent encoding
>   examples illustrate the encoding of the aggregate row object using OID
>   Suppression first with implicit and then with explicit column
>   identification.                     
>                     
> 
>  
> Encoding of snmpNotifyTable row  without OID Suppression
> ----------------------------------------------------------------------------
>         
> 30      UNIVERSAL [16] SEQUENCE OF constructor(VarBindList)
> 63      length =  99
> 
>  
> 	   -- varbind for snmpNotifyTag
>    30      UNIVERSAL [16] SEQUENCE OF constructor (VarBind)
>    1c      length = 28
>       06      UNIVERSAL [6] OBJECT IDENTIFIER (ObjectName)
>       10      length = 16
>       2b 06 01 06 03 0d 01 01 01 02 65 6e 74 72 79 31
> 		    =  1.3.6.1.6.1.3.13.1.1.1.2.101.110.116.114.121.49
> 		    =  snmpNotifyTag. 101.110.116.114.121.49
>       04       UNIVERSAL [4] OCTET STRING (ObjectName)
>       08       length = 8
>       74 72 61 70 4d 67 72 73      = trapMgrs
> 
> 	   -- varbind for snmpNotifyType
>   30       UNIVERSAL [16] SEQUENCE OF constructor (VarBind)
>   15       length =  21  
>      06        UNIVERSAL [6] OBJECT IDENTIFIER (ObjectName)
>      10        length = 16
>      2b 06 01 06 03 0d 01 01 01 03 65 6e 74 72 79 31
> 		    =  1.3.6.1.6.1.3.13.1.1.1.3.101.110.116.114.121.49
> 		    =  snmpNotifyType. 101.110.116.114.121.49 
>      02        UNIVERSAL [2] INTEGER  (ObjectSyntax)
>      01        length = 1
>      01       = 1
> 
> 	   -- varbind for snmpNotifyStorageType
>   30       UNIVERSAL [16] SEQUENCE OF constructor (VarBind)
>   15       length = 21
>      06        UNIVERSAL [6] OBJECT IDENTIFIER (ObjectName)
>      10        length = 16
>      2b 06 01 06 03 0d 01 01 01 04 65 6e 74 72 79 31
>                     =  1.3.6.1.6.1.3.13.1.1.1.4.101.110.116.114.121.49
>                     =  snmpNotifyStorageType. 101.110.116.114.121.49
>      02         UNIVERSAL [2] INTEGER  (ObjectSyntax)
>      01         length = 1
>      03         = 3 
> 
> 	   -- varbind for snmpNotifyRowStatus
>   30       UNIVERSAL [16] SEQUENCE OF constructor (VarBind)
>   15       length = 21
>      06         UNIVERSAL [6] OBJECT IDENTIFIER (ObjectName)
>      10         length = 16
>      2b 06 01 06 03 0d 01 01 01 05 65 6e 74 72 79 31
>                     = 1.3.6.1.6.1.3.13.1.1.1.5.101.110.116.114.121.49
>                     = snmpNotifyRowStatus.101.110.116.114.121.49
>      02         UNIVERSAL [2] INTEGER  (ObjectSyntax)
>      01         length = 1
>      01         = 1
>                     
>  
>  
>  
>  
>  
> Encoding of snmpNotifyTable row  using OID Suppression and
> Implicit Column Identification
> ----------------------------------------------------------------------------
>  
> 30     UNIVERSAL[16] SEQUENCE OF constructor(VarBindList)
> 28     length = 42
>         
>   30      UNIVERSAL [16] SEQUENCE OF constructor (VarBind)
>   26      length =40
>      06      UNIVERSAL [6] OBJECT IDENTIFIER (ObjectName)
>      0f      length = 15
>      2b 06 01 06 03 0d 01 01 02 65 6e 74 72 79 31
>                    = 1.3.6.1.6.1.3.13.1.1.2.101.110.116.114.121.49
>                    = snmpNotifyTable.2.101.110.116.114.121.49
>      4c      APPLICATION [12] SEQUENCE  OF ObjectSyntaxNonAggregate
> 						(ImplicitAggregate)
>      15      length = 21     
>                    
> 
> 		-- column 1 --> snmpNotifyName (non-accessible index)
>          83     nonInstantiatedRowObject[3]       -- placeholder 
>          00     length = 0			  
>      
> 		-- column 2 --> snmpNotifyTag
>          04     UNIVERSAL [4] OCTET STRING (ObjectSyntaxNonAggregate)
>          08     length = 8
>          74 72 61 70 4d 67 72 73  = trapMgrs        ---value of column2
> 
> 		-- column 3 --> snmpNotifyType
>          02     UNIVERSAL [2] INTEGER  (ObjectSyntaxNonAggregate)
>          01     length = 1
>          01      = 1                                ---value of column3
> 
> 		-- column 4 --> snmpNotifyStorageType
>  	 02     UNIVERSAL [2] INTEGER  (ObjectSyntaxNonAggregate)
>          01     length = 1 
>          03     = 3                                 ---value of column4
> 
>                 -- column 5 --> snmpNotifyRowStatus
>          02     UNIVERSAL [2] INTEGER  (ObjectSyntaxNonAggregate)
>          01     length = 1
>          01     = 1                                 ---value of column5  
> 	
> 
> 
>                                 
> Encoding of snmpNotifyTable row  using OID Suppression and
> Explicit Column Identification  
> ----------------------------------------------------------------------------
>            
> 30   UNIVERSAL[16] SEQUENCE OF constructor(VarBindList)
> 3c   length = 60
> 
>   30    UNIVERSAL [16] SEQUENCE OF constructor (VarBind)
>   3a    length =58
>       06      UNIVERSAL [6] OBJECT IDENTIFIER (ObjectName)
>       0f      length = 15
>       2b 06 01 06 03 0d 01 01 02 65 6e 74 72 79 31
>                    = 1.3.6.1.6.1.3.13.1.1.2.101.110.116.114.121.49
>                    = snmpNotifyTable.2.101.110.116.114.121.49 
>      
>       4e      APPLICATION [13] SEQUENCE  OF ObjectSyntaxNonAggregate
>                                              (ExplicitAggregate)     
>       27      length = 39
>    
>   
>                   -- column 2 --> snmpNotifyTag
>          30       APPLICATION [16] SEQUENCE OF  EaFragment
>          0c       length = 13
>               02        UNIVERSAL [2] INTEGER  (eaFragmentNamePart)
>               01        length = 1
>               02        = 2     -- column 2 identifier
>               04        UNIVERSAL [4] OCTET STRING
>  						(eaFragmentValuePart) 
>               08        length = 8
>               74 72 61 70 4d 67 72 73        = trapMgrs
>                    
> 
>                   -- column 3 --> snmpNotifyType
>          30        UNIVERSAL [16] SEQUENCE OF constructor
>          06        length = 06
>               02        UNIVERSAL [2] INTEGER  (eaFragmentNamePart)
>               01        length = 1
>               03        = 3     ---column 3 identifier
>               02        UNIVERSAL [2] INTEGER  (eaFragmentValuePart)
>               01        length = 1
>               01        = 1    ---column 3 value  
> 
>                   -- column 4 --> snmpNotifyStorageType
>          30        UNIVERSAL [16] SEQUENCE OF constructor
>          06        length = 06
>               02        UNIVERSAL [2] INTEGER  (eaFragmentNamePart)
>               01        length = 1
>               03        = 3    ---column 4 identifier
>               02        UNIVERSAL [2] INTEGER  (eaFragmentValuePart)
>               01        length = 1
>               01        = 1    ---column 4 value       
> 
>                   -- column 5 --> snmpNotifyRowStatus
>          30        UNIVERSAL [16] SEQUENCE OF constructor
>          06        length = 06
>               02        UNIVERSAL [2] INTEGER  (eaFragmentNamePart)
>               01        length = 1
>               02        = 5    ---column 5 identifier
>               02        UNIVERSAL [2] INTEGER  (eaFragmentValuePart)
>               01        length = 1
>               01        = 1    ---column 5 value
>          
> 
> 
> 
> Additional Considerations
>  
> It is expected that the aggregate row objects would only be used in new
> PDUs that were defined specifically for the purpose of providing more
> efficient row operations to SNMP. It may be that the aggregate row objects
> would provide enough efficiency to address current problems with bulk data
> retrieval and large configuration operations so that a solution requiring
> the definition of PDUs that are specific to certain types of row
> operations are unnecessary.  Or, it may be that the different types of
> row-operation PDUs are still necessary but that these new PDUs can take
> advantage of the aggregate row objects to simply the representation of
> rows in these new PDUs.  In either case, it seems that this is something
> that should be researched further.
>  
> 
>  
> 
> 
> ****************************************************************************
> Sandra McLeod <mcleod@snmp.com>
> SNMP Research International         |   voice: +1 865 579-3311  
> 3001 Kimberlin Heights Road         |   fax: +1 865 573-9197 
> Knoxville, TN  37920                |   WWW:  http://www.snmp.com
> 
>  
> 
> 
> 
> 
> 
> 
>