Notes

Outline

Slide 1

Slide 2

What is Replication Multiple copies of data at different sites Increased availability Manual data replication implementations Export/Import CREATE TABLE AS SELECT FROM REMOTETABLE COPY

Oracle9i Features for Information Sharing Features Introduced In Prior Releases Data Guard – Physical Standby Database Advanced Queuing -- Message Queuing Advanced Replication -- Replication of Data Change Data Capture -- DW Loading Features available in Oracle9iR2 Oracle Streams -- a comprehensive information sharing solution Data Guard -- Logical Standby Database based on Oracle Streams

Oracle Advanced Replication

Oracle9i Replication Technology Oracle9i provides built-in technology to create and manage replicated environments Integrated, no add-ons, no special commands Managed with Oracle Enterprise Manager Advanced data replication technology Bi-directional, all copies potentially updatable Automatic conflict detection and resolution Tables and supporting objects Full copies or subsets DDL (schema changes) as well as DML (transactions) Continuous or on demand replication

Architectural Overview Single master replication

Architectural Overview Multiple master replication

Master vs. Materialized View Replication

Advanced Replication Key Features Near real-time replication (multimaster) Parallel data propagation Multiple, pre-defined conflict resolution methods Mass deployment (materialized views) subquery materialized views deployment templates multitier materialized views Oracle Enterprise Manager for configuration and administration Specialized options: procedural and synchronous replication

Near Real-time Replication Benefits: availability, scalability, failover Uses: Telesales, support Requirements: Efficient data capture and storage Efficient data propagation Continuous data propagation Automatic resolution of conflicting updates

Architectural Overview Multimaster

Replication Objects Database object replication to multiple servers The following database objects can be replicated Tables, Indexes, Views, Synonyms, Triggers Packages, Procedures, Functions Advanced Data Types User-Defined Types, Indextypes Tables with column objects, object tables Nested Tables, Varrays

Efficient Data Capture and Storage Committed changes are added to queue for later propagation to remote sites Enqueued using advanced queueing mechanism Captured and applied using internal C code Minimum data needed to apply change is captured

Slide 15

Slide 16

Efficient Data Propagation Queued changes are pushed to remote sites in parallel for improved performance Single parallel stream Maintains transactional consistency Automatically detects transaction dependencies

Parallel Data Propagation Dependencies Transaction “B” is dependent on transaction “A” if “B” accesses data “A” has updated Dependency detection is dynamic and light-weight Ordering Dependent transactions are propagated in dependency order All other transactions are propagated in parallel

Continuous Propagation Changes can be continuously propagated or at a fixed interval, fixed time, or on demand Different intervals can be used for each location Different intervals can be used for each group Dynamic Views to monitor propagation activity and throughput

Automatic Conflict Resolution Automatic conflict detection with user-selectable conflict resolution routines latest timestamp, earliest timestamp, maximum or minimum value, overwrite, priority group, discard, site priority, average, or additive User-definable resolution routines Detection and resolution based on column groups

Mass Deployment Replication Benefits: disconnected, updatable Uses: field sales, field service Requirements: Easily define unique subsets Easily deploy to 100’s of sites Support mobile users refresh on demand

Architectural Overview Materialized Views

Managing Advanced Replication

Specialized Options Synchronous Replication always up to date, no conflicts slower response, network dependent Procedural Replication faster for batch processing must be serialized, best done in off hours useful for purging

Replication Summary Full and subset Near realtime or on demand Graphical administration tool Sophisticated Functionality Efficient data capture and storage Parallel data propagation Automatic conflict detection and resolution Subquery subsetting Deployment templates Multitier materialized views Specialized options

Oracle Streams

Oracle Streams – Unified Messaging and Data Movement

Oracle Streams A new solution for information sharing Provides a unified architecture for all information sharing solutions uniquely flexible replication message queuing data warehouse loading event management and notification The foundation of Data Guard Logical Standby Database

Streams Basic Elements Three basic elements in each database Capture Staging Consumption (apply)

Multi-Database Streams  A stream can contain multiple elements from multiple databases Events flow between staging areas

Capture Streams captures events Implicitly:  log-based capture of DML and DDL Explicitly:  Direct enqueue of user messages Captured events are published in the staging area

Log-Based Change Capture Low overhead, low latency change capture Changes to the database are written to the online redo log Oracle Streams can extract changes from the log as it is written (mining the active log) Changes are formatted as a Logical Change Record (LCR), a SQL like representation of the change

Direct Enqueue User applications can explicitly enqueue user messages into the staging area Multiple open interfaces supported: JMS, C, PLSQL, SOAP (XML/HTTP), XML/SMTP Allows applications to communicate at a higher level Allows users to introduce events into the stream from non-Oracle systems

Staging Streams publishes captured events into a staging area Implemented as a queue Supports for new self-describing type “any” datatype allows a single staging area to hold any type of data All events, LCRs and user-messages, can be staged in the same queue Messages remain in staging area until consumed by all subscribers

Staging Area Propagation Other staging areas can subscribe to events in same database in a remote database Events can be routed through a series of staging areas

Transformations Transformations can be performed as events enter the staging area as events leave the staging area as events propagate between staging areas Transformation examples change format, data type, column name, table name

Consumption Staged events are consumed by subscribers Implicitly:  Apply Process Default Apply User-Defined Apply Explictly:  Application dequeue via open interfaces JMS, C, PLSQL, SOAP (XML/HTTP), XML/SMTP

Default Apply The default apply engine will directly apply the DML or DDL represented in the LCR apply to local Oracle table apply via DB Link to non-Oracle table Automatic conflict detection with optional resolution unresolved conflicts placed in exception queue Parallel apply maximizes concurrency

User-defined Apply User-written custom apply functions Written in PL/SQL, Java, C, C++ Uses: custom transformations column subsetting normalizing or denormalizing data populating related fields or tables

Rule-based Subscription Consumers subscribe to published events Content-based subscriptions limit delivered events to those meeting the subscription criteria Rules govern capture, staging, and consumption

Directed Networks Propagation independent of Apply Rules-based subscription determine if event is locally applied London applies UK only WAN Friendly Send once, fan out NY-->London, London-->Milan, London-->Paris

Heterogeneous Support Oracle to non-Oracle apply via gateway Apply process on Oracle node applies change Non-Oracle to Oracle change capture supported via explicit enqueue of LCRs Message Gateways MQ Series Tibco

Streams Deployments Streams can be deployed to meet a variety of information sharing requirements Replication Data Warehouse Loading Event Notification Message Queuing Data Guard Logical Standby Database

Replication Streams asynchronously maintains multiple copies of objects via automatic apply Identical objects Related via a transformation or function Streams automatically captures, propagates,  and applies DML and DDL changes Detects and optionally resolves conflicts Supports flexible data movement and subsetting Gateways and APIs for heterogeneous support Compatible with Materialized Views

Replication Benefits: No quiesce for DDL Lower overhead on production system Reduced network traffic Flexible configurations

Data Warehouse Loading Streams can load data warehouse staging areas and Operational Data Stores Updates captured from a production system Messages and business events from a process flow Supports continuous or batch loading Automatically transforms data to appropriate format and schema during Operational Data Store load

Data Warehouse Loading Benefits: low overhead automatic transformation near real-time loading of operation data stores

Event Notification Streams can notify subscribers that events of interest have occurred Pager notification of flight delays (Orbitz) Notification of price drops (CNET Shopper) Notification to sales manager of Gold Customer purchase (CRM App) Streams can evaluate DML events and send notifications to applications that send emails, page users, etc Users get information they want

Event Notification Benefits: scalable reduced custom development

Message Queuing Streams can be deployed as an enhanced database integrated message queuing solution Point-to-point messaging, publish and subscribe Single data, security and transactional model for database and message queuing operations Centrally managed and multi-consumer queues to simplify configuration Content-based subscriptions, internet access Automatic dequeue to server-run user function Automatic transform DML/DDL into messages

Message Queuing Benefits: Reduced development costs Easy database integration Single development, operational, security model Reliability and integrity of database

Data Guard Logical Standby Database Multiple copies of data protects from human and data errors,  and disasters Special case of replication Entire database (by default) One direction only Streams supports reporting from standby as updates are applied Data Guard adds higher level interface, tailored GUIs and broker

Data Guard Logical Standby Database Benefits: Open while protecting data Support near real-time reporting from standby Protects from physical corruptions Additional indexes and materialized views

Other Oracle9i Information Sharing Features Advanced Replication provides compatible replication with Oracle 8, 8i, and 9i databases Migration path to Oracle Streams in future release Advanced Queuing Compatible with Oracle Streams Most functionality offered in Streams API’s retained for compatibility Migration path to Oracle Streams in future release

Other Oracle9i Information Sharing Features Data Guard Physical Standby Database Uses media recovery mechanism to apply changes to database Creates an exact copy of the production database block-for-block copy Same version of Oracle, same hardware/software architecture Supports very high transaction workloads Will coexist with Data Guard Logical Standby Database

Summary Oracle Streams unifies all enterprise information into a single Stream Unifies database, messaging, replication, publish/subscribe APIs and capabilities Allows deployment of a variety of solutions Provides a single, unified solution to the problem of Information Sharing

Slide 57