XML and Database
XML and Database
July2325,2007
Taipei,Taiwan
Introductionto
pureXMLinDB29forz/OS
GuogenZhanggzhang@us.ibm.com
SteveChenyschen@us.ibm.com
July25,2007
DB2forz/OSTechnicalForum
Agenda
WhyXMLandXMLDatabases
ComparingpureXMLwithexistingapproaches
pureXMLfeatures
XMLdatatype,DDL,DML
XMLQueryLanguagesandAPI
Indexingandaccessmethods
Schemasupport
Utilities
Performance
Tools
Summary
DB2forz/OSTechnicalForum
AnXMLPurchaseOrder
<?xmlversion="1.0"encoding="UTF8"?>
<ipo:purchaseOrder
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"
xmlns:ipo="http://www.example.com/IPO"orderDate="19991201">
<shipToexportCode="1"xsi:type="ipo:UKAddress">
<name>HelenZoe</name>
<street>47EdenStreet</street>
<city>Cambridge</city>
<postcode>CB11JR</postcode>
</shipTo>
<billToxsi:type="ipo:USAddress">
<name>RobertSmith</name>
<street>8OakAvenue</street>
<city>OldTown</city>
<state>PA</state>
<zip>95819</zip>
</billTo>
<?xmlversion="1.0"encoding="UTF8"?>
<ipo:purchaseOrder
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"
xmlns:ipo="http://www.example.com/IPO"orderDate="19991201">
<shipToexportCode="1"xsi:type="ipo:UKAddress">
<name>HelenZoe</name>
<street>47EdenStreet</street>
<city>Cambridge</city>
<postcode>CB11JR</postcode>
</shipTo>
<billToxsi:type="ipo:USAddress">
<name>RobertSmith</name>
<street>8OakAvenue</street>
<city>OldTown</city>
<state>PA</state>
<zip>95819</zip>
</billTo>
<items>
<itempartNum="833AA">
<productName>Lapisnecklace</productName>
<quantity>1</quantity>
<USPrice>99.95</USPrice>
<comment>Wantthisforthe
holidays!</comment>
<shipDate>19991205</shipDate>
</item>
<itempartNum="926AA">
<productName>BabyMonitor</productName>
<quantity>1</quantity>
<USPrice>39.98</USPrice>
<shipDate>19991221</shipDate>
</item>
</items>
</ipo:purchaseOrder>
<items>
<itempartNum="833AA">
<productName>Lapisnecklace</productName>
<quantity>1</quantity>
<USPrice>99.95</USPrice>
<comment>Wantthisforthe
holidays!</comment>
<shipDate>19991205</shipDate>
</item>
<itempartNum="926AA">
<productName>BabyMonitor</productName>
<quantity>1</quantity>
<USPrice>39.98</USPrice>
<shipDate>19991221</shipDate>
</item>
</items>
</ipo:purchaseOrder>
DB2forz/OSTechnicalForum
XMLDataModel(XDM)
Doc
Elm
NS Att Elm Elm
Elm
ipo:purchaseorder
xmlns:ipo
shipTo billTo
orderDate
NS
xmlns:xsi
Elm
items
Att ElmElm
Elm
Elm
item
item
Att ElmElm
Elm
. .
.
Detailsignored
Sevenkindsofnodes
Nodeidentity
Documentorder
Sequence
02
02.02
02.02.02 02.02.08
DB2forz/OSTechnicalForum
XMLCharacteristics
XMLcanrepresentflexiblestructureddataintext
Nesting
Repeating
Selfdescribing
XMLisauniversallanguagetorepresenteBusinessdata
andtransactions.
Platformindependent,andUnicodecompliant
Easytounderstandandeasytoprocess(withtherighttools)
DB2forz/OSTechnicalForum
WhyHybridXMLDatabases?
BusinessesneedtomanageXMLdataw/ACIDproperties,auditingand
regulatorycompliance,togetherwithrelationaldata.
XMLcanbeusedasapowerfuldatamodel,withpowerfuldeclarative
querylanguage.
ManaginglargevolumesofXMLdataisaDBproblem
allthesamereasonsasforrelationaldata!
Integration
IntegratenewXMLdatawithexistingrelationaldata
Publish(relational)dataasXML
Databasesupportforwebapplications,SOA,webservices(SOAP)
DB2forz/OSTechnicalForum
pureXMLinDB29
SQLXMLdatatypeandnativestorage
DesignedspecificallyforXML
SupportsXMLhierarchicalstructurestorage
Nativeoperationsandlanguages:XPath,SQL/XML,(XQueryin
thefuture)
Nottransformingintorelational
Notusingobjectsornestedtables
NotusingLOBs
Integratedwithrelationalengine,withalltheutilitiesandtools
support
DB2forz/OSTechnicalForum
WhatYouCanDowithpureXML
CreatetableswithXMLcolumns
InsertXMLdata,optionally
validatedagainstschemas
CreateindexesonXMLdata
EfficientlysearchXMLdata
ExtractXMLdata
DecomposeXMLdatainto
relationaldata
ConstructXMLdocumentsfrom
relationalandXMLdata
Alltheutilitiesandtoolssupportfor
XML
XML
DOC
XMLColumn
XML
Index
XML
DB2forz/OSTechnicalForum
XMLEnabledDatabases:TwoMain
Options
XML
DOC
Extract
selected
elements/attr.
SideTables
CLOB/Varchar
XMLDOC
XMLDOC
XMLDOC
XML
DOC
Varcharorclob
column
Fixed
Mapping
Shredder
(regulartablesfor
fasterlookup)
(regularrelational
tables)
Decompositon
Shredding
DB2forz/OSTechnicalForum
XMLDataProcessingbeforepureXML
Fixed
Mapping
Shredder
(regularrelationaltables)
Storing/Shredding
XML
Composer
Composing
XML
Complex
Logic
Decompositon/Composition
Rapid
Changes
Painful
Changes
Painful
Schema
Changes
Painful
Data
Migration
Painful
Changes
DB2forz/OSTechnicalForum
DB2pureXMLAdvantages
DirectlystoreXML,no
decomp/comp,normalize/de
normalize
Eliminatesdatabaseschema
evolutionbottleneck
Declarativelanguage,reduce
complexity,dramaticallyimprove
applicationdevelopment
productivity
Nativeprocessing,high
performance
Unparalleledreliability,
availability,scalability
XML
DOC
XMLColumn
XML
Index
XML
DB2s hierarchicalstorage:
XMLtypeasXML
Upto10
times
DB2forz/OSTechnicalForum
UsageScenarios
DirectlyprocessingXML
UNIFI,ACORD,FIXML,FpML,MIMSO,XBRL,
DJXDM,HRXML,HL7,ARTS,HIPAA,NewsML,XForms
Insurancepolicy,contract,purchaseorder,emailsetc
Versatileschemas
Sparseattributevalues(nullv.s.absence)
Objectpersistence(singlecolumnv.s.manytables)
Migrationfromlegacydatamodel(network,hierarchical,
relational)
Generatingwebpages:XHTML
Provide/consumeWebServices(SOAP),supportSOA
DB2forz/OSTechnicalForum
Example:TaxForms
Application
Processing&validatingtaxreturns,payments,refunds
CorporateTax,PersonalIncomeTax(PIT),SalesTax
Objectives
MoveTaxprocessingofflegacysystems
Movetoamoreflexible,automated,extensibleframework
Reducecost&laborforimplementingtaxformchanges
Increaseperformance.Improvestraightthroughprocessingfromfilingto
refund/payment
Typicalcurrentenvironment
Processingusingmanualand/orlegacysystems
ThisisanexampleofusageforOnlineFormsprocessingingeneral
DB2forz/OSTechnicalForum
Usuallyhundredsthousands
ofdifferenttaxforms
SchemaDiversity
Typicallynoteveryfieldina
formisused
SparseData
Manyformschangeevery
year
SchemaEvolution
AcaseforXML!
TaxForms
DB2forz/OSTechnicalForum
Solution1:Eachformhasadifferentsetoffields
(schema)
ThousandsofTablesi.e.oneperform?
Considerednotfeasible
Toomanytablestomaintain
Relationalschemawoulddeteriorate
overtime
Notsufficientlyflexibleandextensible
Solution2:Singletablewhoserows
canstoreanyform
100sofgenericcolumnsOuch!
TypicalCurrentUsage:RelationalDatabase
DB2forz/OSTechnicalForum
XML:Avoidssparsity.Properdatalabeling.2columns,not
1000.Transformable.Extensible.Simplifiesmapping.
GenericcolumnsXML
Currentrelationalstorage,
inefficient,anonymous
columns,requirescomplex
mappingsintheapplication
NewXMLformat:
InputStreamfin=newFileInputStream(file)
pstmt.setBinaryStream(2,fin,flen)
pstmt.execute()
Statements=connection.createStatement()
ResultSetrs=s.executeQuery("selectponumber,xmlpofrompurchaseOrders")
while(rs.next()){
intpo_no=rs.getInt("ponumber")
Stringspo=rs.getString(2)
System.out.println(spo)//uninterpretedflatxmltext
}
Orusecom.ibm.db2.jcc.DB2Xmlinterface
DB2forz/OSTechnicalForum
FETCHCONTINUEforXMLandLOB
NosizeassociatedwithXMLvalues
Hardtoallocatelargememory
ShortcomingswithLOBLocator
NewFETCHCONTINUEstatements:(oneoftwoways)
DECLARECURSOR1CURSORFORSELECTC2FROMT1
OPENCURSOR1
FETCHWITHCONTINUECURSOR1into:clobhv
if(sqlcode>=0)&sqlcode<>100
Loopiftruncationoccursuntillob/xmlcomplete(totallength)
FETCHCURRENTCONTINUECURSOR1into:clobhv
Consume:clobhvcontent
endloop
AnotherwayistouseFETCHINTODESCRIPTOR:SQLDA
DB2forz/OSTechnicalForum
XMLDataExchangeinDRDA
SupportforDRDAXMLdataaccess
DB2forz/OSV9(ODBCdriversupportisprovided)
DB2UniversalDatabaseforLinux,Unix,andWindowsVersion9.1(includingCLIsupport).
DB2UniversalDriver(JDBC,SQLJ)V3.1
TransportisalsoinserializedXMLformat
EncodingeitherwithinXMLvalueitself(DRDAinternallyencoded)orinDRDAdescriptor
(DRDAexternallyencoded)
SendinginputdatafromDRDARequestertoServer
BLOBashostvariable
DataisinDRDAinternallyencodedstringvalue
DBCLOB,CLOBashostvariables
DataisintheDRDAexternallyencodedstringvalue,i.e.CCSIDinDRDAdescriptor
ServersideconvertsXMLstringvaluetoUTF8
SendingoutputdatafromDRDAServertoRequester
AlwayswithUTF8DRDAexternallyencoded
RequestersideconvertsXMLstringvalueintoapplicationhostvariableCCSID
Separatesend/receiveforXMLdataifremoteresultsetcontainsXMLdata
DB2forz/OSTechnicalForum
XMLIndexes
XPathvalueindex:indexvaluesof
elementsorattributesinsidea
document.
Indexentriesinclude:
(keyvalue,DocID,NodeID,RIDx)
Supportstring(VARCHAR)or
numeric(DECFLOAT)keytype
CREATEINDEXON
PurchaseOrders(XMLPO)Generate
KeysUsingXMLPATTERN
/purchaseOrder/items/item/desc
asSQLVARCHAR(100)
<?xmlversion="1.0"?>
<purchaseOrder orderDate="19991020">
<shipTo country="US">
<name>AliceSmith</name>
...
</shipTo>
<billTo country="US">
<name>RobertSmith</name>
...
</billTo>
<comment>Hurry,mylawnisgoingwild!</comment>
<items>
<item partNum="872AA">
<desc>Lawnmower</desc>
<quantity>1</quantity>
<USPrice>148.95</USPrice>
<comment>Confirmthisiselectric</comment>
</item>
<item partNum="926AA">
<desc>BabyMonitor</desc>
<quantity>1</quantity>
<USPrice>39.98</USPrice>
<shipDate>20030521</shipDate>
</item>
</items>
</purchaseOrder>
<?xmlversion="1.0"?>
<purchaseOrder orderDate="19991020">
<shipTo country="US">
<name>AliceSmith</name>
...
</shipTo>
<billTo country="US">
<name>RobertSmith</name>
...
</billTo>
<comment>Hurry,mylawnisgoingwild!</comment>
<items>
<item partNum="872AA">
<desc>Lawnmower</desc>
<quantity>1</quantity>
<USPrice>148.95</USPrice>
<comment>Confirmthisiselectric</comment>
</item>
<item partNum="926AA">
<desc>BabyMonitor</desc>
<quantity>1</quantity>
<USPrice>39.98</USPrice>
<shipDate>20030521</shipDate>
</item>
</items>
</purchaseOrder>
Thisindexcanbeusedforpredicate:
XMLEXISTS(/purchaseOrder/items/item[desc=BabyMonitor]passingXMLPO)
DB2forz/OSTechnicalForum
SomethingSpecialforXMLIndex
Thenumberofkeysforeachdocument(eachbaserow)
dependsonthedocumentandXMLPattern.
Foranumericindex,ifastringfromadocumentcannotbe
convertedintoanumber,itisignored.
<a><b>X</b><b>5</b></a>,XMLPattern/a/basSQLDecfloat.
Onlyoneentry5intheindex.
Forastring(VARCHAR(n))index,ifakeyvalueislongerthan
thelimit,INSERTorCREATEINDEXwillfail.
DB2forz/OSTechnicalForum
BeCarefulCreatingIndexesonNonleaf
Nodes
Indexingeverythingisnotsupported.SQLCODE20305ifakey
valuespansmultiplerowsintheinternalXMLtable.
Alwayssafetoindexleafnodes(<=1000bytes).
Indexingnonleadnodeswillresultinconcatenationofvalues
forthekey:
//name,thekeyvaluewillbeJohnJoewithstripwhitespace,
or
JohnJoe(w/LFs)withpreservewhitespace
Queryhastousethesame
stringtomatch:
/customer[name=JohnJoe]
<name>
<first>John</first>
<last>Joe</last>
</name>
<name>
<first>John</first>
<last>Joe</last>
</name>
DB2forz/OSTechnicalForum
NewAccessMethods
XMLExists(/Catalog/Categories/Product[RegPrice>100]
passingcatalog)withindexon
/Catalog/Categories/Product/RegPriceasSQLDECFLOAT
DocIDlistaccessDXunique
DocIDlistfromanXMLindex,
thenaccessthebasetable
andXMLtable.
XMLExists(/Catalog/Categories/Product[RegPrice>100and
Discount>0.1])
Withindexeson:
//RegPriceasSQLDECFLOATand
//DiscountasSQLDECFLOAT
DocIDANDing/ORing
DX/DI/DUunionorintersect
(unique)DocIDlistsfromXML
indexes,thenaccessthebase
tableandXMLtable.
Basealgorithm:givenadocument,scanandevaluateXPath DocScanR(QuickXScan)
Description AccessMethods
DB2forz/OSTechnicalForum
AnotherExampleforIndexesandQuery
CREATETABLEACORD.REQUEST(
IDBIGINTNOTNULLPRIMARYKEY,
REQUESTXMLXML,
RESPONSEXMLXML
)INDATABASEDBACORD
CREATEINDEXACORD.ACORDINDEX1ONACORD.REQUEST(REQUESTXML)
GENERATEKEYSUSINGXMLPATTERN
'declaredefaultelementnamespace"http://ACORD.org/Standards/Life/2"
/TXLife/TXLifeRequest/TransRefGUID'asSQLVARCHAR(24)
CREATEINDEXACORD.ACORDINDEX2ONACORD.REQUEST(REQUESTXML)
GENERATEKEYSUSINGXMLPATTERN
'declaredefaultelementnamespace"http://ACORD.org/Standards/Life/2"
/TXLife/TXLifeRequest/OLifE/Holding/Policy/@id'ASSQLVARCHAR(9)
DB2forz/OSTechnicalForum
AnotherExampleforIndexesandQuery
(conted)
++
|PLANNO|ACCESSTYPE|MATCHCOLS|ACCESSCREATOR|ACCESSNAME|MIXOPSEQ|
++
1_|1|M|0|| |0|
2_|1|DX|1|ACORD|ACORDINDEX2|1|
3_|1|DX|1|ACORD|ACORDINDEX1|2|
4_|1|DI|0|| |3|
++
Findparticipantinformation
aboutapolicy.
SELECT
XMLQuery('declaredefaultelementnamespace"http://ACORD.org/Standards/Life/2"
/TXLife/TXLifeRequest/OLifE/Holding/Policy/Life/Coverage/LifeParticipant'
PASSINGR.REQUESTXML),
XMLQuery('declaredefaultelementnamespace"http://ACORD.org/Standards/Life/2"
/TXLife/TXLifeRequest/OLifE/Party
[@id=
/TXLife/TXLifeRequest/OLifE/
Holding/Policy/Life/Coverage/
LifeParticipant/@PartyID]'
PASSINGR.REQUESTXML)
FROMACORD.REQUESTR
WHEREXMLExists('declaredefaultelementnamespace
"http://ACORD.org/Standards/Life/2"
/TXLife/TXLifeRequest[TransRefGUID="20041217141016000012"]/
OLifE[Holding/Policy/@id="POLICY12"]'
PASSINGR.REQUESTXML)
DB2forz/OSTechnicalForum
ASEPAQueryExample
AssumetableINTERBANKDD(CollectionMsgXML,StatusMsgXML)
CollectionMsg:pacs.003.001.01
StatusMsg:pacs.002.001.02
FindinformationofDirectDebitTransactionsthatwerecreatedonorafter20060628and
havebeenrejected,andreturnthereasonalso.
SELECTXMLQuery(declaredefaultelementnamespace
"urn:iso:std:iso:20022:tech:xsd:pacs.003.001.01"
/Document/pacs.003.001.01/DrctDbtTxInf'PASSINGCollectionMsg),
XMLQuery(declaredefaultelementnamespace
"urn:iso:std:iso:20022:tech:xsd:pacs.002.001.02"
/Document/pacs.002.001.02/TxInfAndSts/StsRsnInf/StsRsn'PASSINGStatusMsg)
FROMINTERBANKDD
WHEREXMLEXISTS('declaredefaultelementnamespace
"urn:iso:std:iso:20022:tech:xsd:pacs.002.001.02"
/Document/pacs.002.001.02[OrgnlGrpInfAndSts/OrgnlCreDtTm>="200606
28"]/TxInfAndSts[TxSts="RJCT"]'PASSINGSTATUSMsg)
DB2forz/OSTechnicalForum
XMLSchemaSupport
XMLSchemaaddsconstraintsonXMLdata.
RegisteraschemainXMLSchemaRepository(XSR)
Externalnames
targetnamespace:e.g.,"http://www.ibm.com/software/catalog"
schemalocation:e.g.,
"http://www.ibm.com/schemas/software/catalog.xsd"
SQLidentifierusedtoreferenceschemasinSQL
uniqueidentifierinDB,e.g.,SYSXSR.ORDERSCHEMA
Whereareschemasused?
SYSFUN.DSN_XMLValidateinSQL(UDFforXMLValidate)
Decomposition
DB2forz/OSTechnicalForum
RegisteringanXMLSchema(Stored
Procedures)
XSR_REGISTER(rschema,name,schemalocation,xsd,docproperty)
XSR_ADDSCHEMADOC(rschema,name,schemalocation,xsd,docproperty)
XSR_COMPLETE(rschema,name,schemaproperties,isUsedForDecomp)
XSR_REMOVE(rschema,name)
Parameters:
rschema nullorSYSXSR
identifier SQLname(VARCHAR(128))
schemalocationVARCHAR(1000)
xsd XMLschemadocument(BLOB(30M))
docproperty BLOB(5M),maybeusedbytools
schemaproperties sameasdocproperties
isUsedForDecomp INTEGER,1yes,0no.
JavaDriver(JCC)providesasetofAPIsforschemaregistration
DB2forz/OSTechnicalForum
XSR_REGISTER(SYSXSR','ORDERSCHEMA',
'http://www.n1.com/order.xsd',:xsd,:docproperty)
XSR_ADDSCHEMADOC(SYSXSR','ORDERSCHEMA',
'http://www.n1.com/lineitem.xsd',:xsd,:docproperty)
XSR_ADDSCHEMADOC(SYSXSR','ORDERSCHEMA',
'http://www.n1.com/parts.xsd',:xsd,:docproperty)
XSR_COMPLETE(SYSXSR','ORDERSCHEMA',:schemaproperty,0)
Orderschema
order.xsd
Order
Lineitem
lineitem.xsd parts.xsd
import
include
Namespace: Namespace:
Example:RegisteringanXMLSchema
DB2forz/OSTechnicalForum
UsingXMLSchema
Schemavalidation(typeannotationnotkept)
INSERTintoPurchaseOrders
VALUES('200300001',CURRENTDATE,'A',
SYSFUN.DSN_XMLValidate(:lobPO,SYSXSR.ORDERSCHEMA))
Annotatedschemabaseddecompositionstoreusingtables.
(XDBDECOMPXMLstoredproc)
E.g.orderID>PORDER.ORDERID
<attributename="orderID"type=xs:string
db2xdb:rowSet=PORDER
db2xdb:column=ORDERID/>
annotations
DB2forz/OSTechnicalForum
Utilities
EnhancedtohandlenewXML
type,XMLtablespaces,and
XMLindexes
CHECKDATA
CHECKINDEX
COPYINDEX
COPYTABLESPACE
COPYTOCOPY
LISTDEF
LOAD
MERGECOPY
QUIESCETABLESPACESET
REALTIMESTATISTICS
REBUILDINDEX
RECOVERINDEX
RECOVERTABLESPACE
REORGINDEX
REORGTABLESPACE
REPORTTABLESPACESET
UNLOAD
BasicRUNSTATS
DB2forz/OSTechnicalForum
DBAConsiderations
DatabaseadministratorsshouldtreatXMLdatabaseobjectsastheydoinLOB
databaseobjects.
LikeLOBobjects,XMLobjectscontaindatastoredoutsidethebasetablespace.
XMLtablespacesandindexspacesmustbeconsistentalsowiththeirrelatedbasetable
AlltheutilitieseithersupportortolerateXML,withsomespecificXMLkeyword
support(nextchart).
Inaddition,XMLhasmoreconsiderations,suchastablespacesizelimit,
compression,andindexes.
DB2forz/OSTechnicalForum
GetXMLDataInwithLOAD
ToloadXMLdirectlyfrominputrecord,specifyXMLasthefieldtype.
LOADDATAINDDN(INFILE)LOGNORESUME(NO)FORMAT
DELIMITED
INTOTABLEPURCHASEORDERS
LOADDATAINDDN(INFILE)LOGNORESUME(NO)
XMLPOPOSITION(20)
XMLPRESERVEWHITESPACE
INTOTABLEPURCHASEORDERS
ToloadXMLfromafile,specifyCHARorVARCHARalongwitheither
BLOBF,CLOBForDBCLOBF.
SchemavalidationnotsupportedforLOAD.
XMLcompressiontakeseffectafterfirstREORG,notoninitialLOAD.
SameforFREEPAGE,PCTFREE.
DB2forz/OSTechnicalForum
GetXMLDataOutwithUNLOAD
TounloadXMLdatadirectlytooutputrecord,specifyXMLas
thefieldtype.
nondelimitedformat:a2bytelengthwillprecedethevalueofthe
XML.
Fordelimitedoutput,nolengthfieldispresent.
TounloadXMLdatatoaseparatefile:
SpecifyCHAR(n)/VARCHAR(n)BLOBF,CLOBForDBCLOBFfor
filenames
UsethetemplatecontrolstatementtocreatetheXMLoutputfile
andfilename
DB2forz/OSTechnicalForum
OperationandRecovery
Torecoverbasetablespace,takeimagecopiesofallrelatedobjects
UseREPORTTABLESPACESETtoobtainalistofrelatedobjects
UseQUIESCETABLESPACESETtoquiesceallobjectsintherelatedset
UseSQLSELECTtoquerytheSYSIBM.SYSXMLRELStableforrelationshipsbetween
basetablespacesandXMLtablespaces
COPYTOCOPYmaybeusedtoreplicateimagecopiesofXMLobjects.
MERGECOPYmaybeusedtomergeincrementalcopiesofXMLtablespaces.
PointinTimerecovery(RECOVERTOCOPY,TORBA,TOLOGPOINT)
Allrelatedobjects,includingXMLobjectsmustberecoveredtoaconsistentpointintime
CHECKutilitiestovalidatebasetablespaceswithXMLcolumns,XMLindexesandrelated
XMLtablespaces.
Ifthereisanavailabilityissuewithoneobjectintherelatedset,availabilityoftheothers
maybeimpacted.
DB2forz/OSTechnicalForum
DiagnosingProblemrelatedtoXMLobjects
IdentifyXMLtablesandtheirrelatedobjects
RunREPORTTABLESPACESET
Validatethattheauxiliaryindexisconsistentwiththeunderlyingtablespaces
RunCHECKINDEXonallindexes,DocID,NodeIDandXMLvalueindexes
ValidatethelogicalconnectionbetweenthebasetableandXMLtable.
RunCHECKDATAagainstthebasetablespace.
UseRepairtodiagnoseproblemrelatedtobasetablespaceswithXMLcolumnsandtheir
DocIDindex
UseREPAIRLOCATEKEYtolocatearowusingDocIDkeyintheDocIDindex
UseRepairtodiagnoseproblemrelatedtoXMLtablespacesandtheirNodeIDindexor
XMLValueIndex
UseREPAIRLOCATERIDtolocatearowusingaRID.
DB2forz/OSTechnicalForum
PerformanceMonitoringandTuning
SinceXMLnativesupportisbuiltontopofregulartablespacestructure,there
arenospecialchangesinDB2PerformanceExperttosupportXMLother
thanminorpoints suchasnewXMLlocks.
XMLperformanceproblemcanbeanalyzedthroughaccountingtracesand
performancetraces.
ThereisanewLOADMODULEforXML:DSNNXML
XMLindexeshavethesameconsiderationasotherindexes.
TheREORGutilityshouldbeusedtomaintainorderandfreespace.
RunRUNSTATSforstatisticstohelppickXMLindexes.
DB2forz/OSTechnicalForum
TableSpaceSizeConsideration
XMLstorageisabout1:1originaldocsizewithoutcompression
andwithstripwhitespace.
AnXMLtablespacealwaysuse16KBpages.
Fornonrangepartitionedbasetablespaces,PBGtablespaceis
usedforXML.
Rangepartitionedbasetablespaces:XMLpartitioningfollows
basetablepartitioning.
Thenumberofrowstofitintoarelationalpartitionislimitedby
thenumberofdocumentstofitintoanXMLpartition.
Forexample,4Kdocsize,400GBpartitioncanroughlystore100M
documents(or75Mtobesafe).
DB2forz/OSTechnicalForum
Compression
XMLTablespacecompression
InheritfrombasetableCompressYES
Significantdiskstoragesavings,especiallywithpreserve
whitespaceoption(70%)
CPUcostsimilartorelational
SignificantCPUimpactifyouselectmanydocument(DocScan)
InitialLOADwilltriggerbasetablecompressionbutnotforXML
tablspace
CompressionhappensatnextREORG
DB2forz/OSTechnicalForum
XMLIndexing
Eachindexadds1520%CPUtimetothebasic
INSERTcost.Createindexesthatareonlyneeded.
SpecifyfullpathforindexXMLpatterns,avoidwild
card,ordescendantaxis
Rebuildindexisrecommendedovercreateindex
CodeXPathconditionsthatwillmatchindexpatterns.
DB2forz/OSTechnicalForum
IndexExploitation
Indexesarecriticalforqueryperformance.
IndexesarenotusedfortheXMLQueryfuntion.
IndexesareusedforXMLExistsandXMLTable.
/po/items/item[price>123.5]canmatchXMLPattern
/po/items/item/price
/po[billTo/city=London]/items/item[price>123.5]matches
twoindexes:
/po/billTo/city and
/po/items/item/price
(also//cityand//pricenotrecommended)
DocScan
Useindex
DB2forz/OSTechnicalForum
PerformanceandScalability
XMLstorageisverycompact,anditleveragesmature
optimizedstorageinfrastructure(regulartablespaces).
Nextgenerationparsers:z/OSXMLSystemServicesand
XLXPC.
HighlyefficientXPathstreamingalgorithm
Supportpartitionedtablespacesanddatasharing.
Initialsweetspot:alargenumberofsmalldocuments.
DB2forz/OSTechnicalForum
StorageforUNIFIMessages
0
20
40
60
80
100
120
140
160
180
S
t
o
r
a
g
e
(
K
B
)
(
S
e
c
)
Elapsed CPU
3.9millions10K
docsperhouror
1100docs/sec
DB2forz/OSTechnicalForum
InsertXMLwithindexes
InsertElapsedandCPU
100% 100%
111%
116%
125%
138%
0%
20%
40%
60%
80%
100%
120%
140%
160%
Elapsed CPU
XML
w/1index
w/2indexes
DB2forz/OSTechnicalForum
InsertPerformance
InsertElapsedandCPU
100% 100%
111%
116%
67%
87%
0%
20%
40%
60%
80%
100%
120%
140%
Elapsed CPU
XML
XMLw/index
CLOB
(averageof1Kto10Mdocumentinsertperformance)
DB2forz/OSTechnicalForum
FetchPerformance(Batch)
MeasurementinMarch2007,z9DS8300,Singlethread,DocsinEBCDIC
0
10
20
30
40
50
60
70
80
1Kx1000000 10Kx100000 100Kx10000 1Mx1000
DocsizexNumber
T
i
m
e
(
s
e
c
)
Elapsed CPU
9.3millions10K
docsperhouror
2580docs/sec
DB2forz/OSTechnicalForum
XMLIndexExploitation
DB2forz/OSTechnicalForum
Insertv.s.Validationv.s.Decomposition
z9109,3x1.7GHz,12GBCS,ESSM800
0.62s 0.26s 2.02s 0.55s Shred23
three2kdocs
x1000
5.6s 10.33s 2.21s 0.97s 3.37s 1.89s Qcapture4k
docx3000
times
13.98s 43.51s 13.98s 8.37s 43.51s 35.90s Custacc
10,000docs4
19K
Decomp
CPU
Decomp
elapsed
Valid.
CPU
Insert
CPU
Valid.
elapsed
Insert
elapsed
Testcases
DB2forz/OSTechnicalForum
SEPASampleXMLDocuments
A:Interbankdirectdebitcollection(pacs.003.001.01)
B:ReturnorRefund,interbankpaymentreturn(pacs.004.001.01)
C:RejectPaymentstatusreport(pacs.002.001.02)
D:Interbankreversal(pacs.007.001.01)
E:CustomerDirectDebitInitiation(pain.008.001.01)
F:Customertobankpaymentreversal(pain.007.001.01)
G:Banktocustomerreject(pain.002.001.02)
H:CustomerCredittransferinitiation(pain.001.001.02)
I:FItoFICustomercredittransfer(pacs.008.001.01)
DB2forz/OSTechnicalForum
SEPA/UNIFIXMLSampleInsertPerformance
0
20
40
60
80
100
120
140
A
v
g
T
i
m
e
(
S
e
c
)
A B C D E F G H I
50,000Documents
ET/Ins
CPU/Ins
ET/Val
CPU/Val
ET:ElapsedTime,CPU:CPUTime,Ins:INSERTonly,Val:w/Validation
DB2forz/OSTechnicalForum
SEPA/UNIFIXMLSampleInsertThroughput
0
1
2
3
4
5
6
7
8
9
T
h
r
o
u
g
h
p
u
t
(
M
i
l
l
i
o
n
s
/
H
o
u
r
)
A B C D E F G H I
Documents
Insert
Validate
DB2forz/OSTechnicalForum
Tools
Toolchoices:
RationalDataArchitect
RationalApplicationDeveloper
DeveloperWorkbench(DWB)
.NET
QMF
SPUFI
Schemaregistration,validation
Annotationfordecomposition
MappingrelationaltoXMLschemaforXMLgeneration
DB2forz/OSTechnicalForum
XMLFeaturesinV9Summary
FirstclassXMLtype,nativestorageofXQuery DataModel
(XDM)
CompleteSQL/XMLconstructorfunctions
XMLPARSEandXMLSERIALIZE
XMLindexes
OtherSQL/XMLfunctionswithXPath
XMLEXISTS,XMLQUERY,XMLTABLE
XMLSchemarepository,ValidationUDF,anddecomposition
DRDA(distributedsupport)andapplicationinterfaces
Utilitiesandtools
DB2forz/OSTechnicalForum
SomeRestrictionsandLimits
Update:wholedocumentreplacement
Largestdocumentsize:~2GB
Typeannotationnotkeptinstorageaftervalidation.
XMLindexes:
Numericandstringtypes
Stringindexkeylength:1000bytes
Keysdonotspanrecords
XMLEXISTSisindexable,butstage2,alwaysreevaluatedby
DocScanafterindexaccess.
Rangepartitionedtablespaces:XMLpartitioningfollowsbasetable
partitioning
Triggers:XMLcolumnscannotbetransitionvars
Storedprocedures:noXMLtypearguments
DB2forz/OSTechnicalForum
SystemConfigurations
BasicXMLparsingrequiresz/OSXMLSS:z/OS1.8orz/OS1.7
withAPAROA16303
XMLschemasrequiresIBM31bitSDKforz/OS,Java2
TechnologyEdition,V5(5655N98),SDK5.
Zparmsforstorage:XMLVALAandXMLVALS
Default:200MBand10GB.
BufferpoolforXMLtables(defaultBP16K0),authorization.
DEFAULTBUFFERPOOLFORUSERXMLDATA===>BP16K0
BP16K0BP16K9
DB2forz/OSTechnicalForum
XMLNormalizationExample
Manager
Employee
Department
Project
Skill
1:1
1:n
n:n
n:n
n:n
1. (D,M,S*,(E,P*,S*)*)*,S*,P*
2. (M,S*,D,(E,S*,P*)*)*,P*,S*
3. (P,(E,S*,M,S*,D)*)*,S*
4. (S,(M,D,(E,P*)*)*
5.
DB2forz/OSTechnicalForum
RepresentingN:NRelationship
N3 E3
N2 E2
N1 E1
PN2 P2
PN1 P1
E3 P2
E2 P2
E3 P3
E2 P1
E1 P1
Employee
Project
PE
<projects>
<projectno=P1>
<name>PN1<name>
<team>
<member>E1</member>
<member>E2</member>
</team>
</project>
<projectno=P2>
.
</project>
</projects>
<employees>
<employeeno=E1>
<name>N1</name>
</employee>
<employeeno=E2>
<name>N2</name>
</employee>
<employees>
Divideinto
multipledocs
tostore
Project<>Employee
DB2forz/OSTechnicalForum
IndexingN:NRelationship
N3 E3
N2 E2
N1 E1
PN2 P2
PN1 P1
E3 P2
E2 P2
E3 P3
E2 P1
E1 P1
Employee
Project
PE
<projects>
<projectno=P1>
<name>PN1<name>
<team>
<member>E1</member>
<member>E2</member>
</team>
</project>
<projectno=P2>
.
</project>
</projects>
<employees>
<employeeno=E1>
<name>N1</name>
</employee>
<employeeno=E2>
<name>N2</name>
</employee>
<employees>
/employee/@no
/project/@no /project/team/member
Divideinto
multipledocs
tostore
Project<>Employee
(Pdetails&P>E)
(E>P)
(Edetails)
DB2forz/OSTechnicalForum
XMLRelatedLocks
DB2forz/OSTechnicalForum
Summary
WhyXMLandXMLdatabases
Usagescenarios
DB2pureXMLfeatures
Performance
Usageguide