The Oracle Database 12c R2: Administration Workshop Ed 3 course is designed to provide you with a firm foundation in administration of an Oracle Database. In this course you will gain a conceptual ...Full description
The Oracle Database 12c R2: Administration Workshop Ed 3 course is designed to provide you with a firm foundation in administration of an Oracle Database. In this course you will gain a conceptual ...
oracleDescripción completa
oracle
SYD FIELDFull description
Administração
slidesFull description
Recrutamento e Seleção
Off Shore Drilling
asdsd
odi saDescripción completa
Descripción: asdsd
Formato ODI
IFADescripción completa
GFYKDUTGDescripción completa
ODI ELECTRICOSDescripción completa
ODI presentación
Odu for the Yearly Reading 2014 of Ile IfeFull description
Descripción completa
Training
Descripción completa
Oracle Data Integrator Overview Eddie Ho,Senior Sales Consulting Manager, DIS Prashant Kompella, Senior Sales Consultant
Oracle Data Integrator Overview Eddie Ho,Senior Sales Consulting Manager, DIS Prashant Kompella, Senior Sales Consultant
Best for Oracle: Merge, Spatial, MultiTable Insert, Optimizer Hints and more
Legacy
Applications
Real-Time: CDC with GoldenGate Applications: E-Business Suite, Siebel, PeopleSoft, JD Edwards Enterprise One, JD Edwards World, SAP ERP and SAP BW Heterogeneous: Optimizations for all major RDBMS: IBM DB2, Microsoft SQL
Databases
Unstructured
JMS
Oracle Data Integrator in Data Integration Solutions
Reverse-Engineer Develop Projects Release Scenarios Java - Any Platform
Monitor sessions View Reports
Submit Jobs
Repository Read sessions Write reports
Any ISO-92 RDBMS Agent
Handles schedules Orchestrate sessions Java - Any Platform
Return Code
Execute Jobs
Information System
Lightweight Distributed
Architecture
ODI 12c Mapping Declarative and Flow combined
Easy for moderate use cases Shows relationships of source tables Abstracts out physical implementation from logical map Quick to define and refactor maps Universal for big and complex use cases Extensible model for any data
Any component can map result to multiple downstream components
Split component allows to split input logically based on expressions
Reusable Mappings
Capture common logic (sources, targets, transformations)
Reuse logic in mappings
Step by Step Debugger Overview
Provide control to the user on the Session Execution Flow
Pause, Resume, Run to next Step, etc.
Define breakpoints on Steps in Packages
Toggle breakpoints in Sessions
Query data through Agent (gives access to uncommitted data)
Types of ODI Knowledge Modules There are six different types of knowledge modules:
KM Type c a f r s e e t n I s l e d o M
LKM IKM CKM RKM JKM
Description
Loading
Assembles data from source datastores to the staging area.
Integration
Uses a given strategy to populate the target datastore from the staging area.
Check Reverseengineering Journalizing
Checks data in a datastore or during an integration process. Retrieves the structure of a data model from a database. Only needed for customized reverse-engineering. Sets up a system for Changed Data Capture to reduce the amount of data that needs to be processed.
E-LT Architecture with Data Integrator Comprehensive, Lightweight Data Integration
Key Architecture Benefits: 100% Java, Open APIs, very fast E-LT – E-LT Architecture uses DBMS CPUs, not the Application CPUs – Open APIs, Open Metadata, Open XML Knowledge Modules
Apps
Business Intelligence ODI Agent ODI Agent may be deployed in any part of the architecture
JKM A
LKM
IKM D
B
File
ODI natively integrates with all major Data Warehouses
C$_0 LKM
C$_1
I$
E$ (Errors)
Today’s hands-on lab environment
Hands-On Lab Scenario 1
Orders Application: An application for tracking
customer orders, hosted in Oracle and used as a source.
l a c i g o L
Parameters (File): Flat files (ASCII) issued from
the production system containing a list of sales representatives and the segmentation of ages into age ranges. Sales Administration: The administration or
tracking of sales, hosted in another supplied database. This data warehouse is populated with our transformations.
l a c i s y h P
Orders Application
Lookup Sales Administration
Parameters (Sales Source)
Join
2
3
Hands-On Lab Scenario What you are about to see ODI Studio
ODI Model & Logical/Physical Schema across Environments Project 1
Project 2
Project 1
Project 2
Model A
Model B
Model C
Model A
Model B
Model C
LS X
LS Y
LS Z
LS X
LS Y
LS Z
Logical/Physical Views for Productivity Improvement
“ETL” jobs are developed on logical views - Portable across environments
Logical switching to different
ODI Tight Binding vs Loose Binding
Types of ODI Knowledge Modules There are six different types of knowledge modules:
KM Type c a f r s e e t n I s l e d o M
LKM IKM CKM RKM JKM
Description
Loading
Assembles data from source datastores to the staging area.
Integration
Uses a given strategy to populate the target datastore from the staging area.
Check Reverseengineering Journalizing
Checks data in a datastore or during an integration process. Retrieves the structure of a data model from a database. Only needed for customized reverse-engineering. Sets up a system for Changed Data Capture to reduce the amount of data that needs to be processed.
Which KMs for What Flow?
When processing happens between two data servers, a data transfer KM is required. – Before integration (Source Staging Area)
Requires an LKM, which is always multi-technology. E.g. SQL Server to Oracle
– At integration (Staging Area Target)
Requires a multi-technology IKM
When processing happens within a data server, it is entirely performed by the server. – A single-technology IKM is required.
Case #1 Using the Target as the Staging Area Target (Oracle) Source (MS SQL)
Staging Area
ORDERS
LKM_1 ORDER_LINES
LKM SQL to Oracle
LKM_2 PRODUCT File
LKM File to Oracle (SQLLDR)
IKM_1
C$_0
IKM_1 C$_1
I$_SALES
IKM Oracle Incremental Update
SALES
IKM Oracle Incremental Update
Case #2 Using a third server as the Staging Area Sunopsis Memory Engine IKM SQL to SQL Append
Source (MS SQL)
Staging Area
ORDERS
IKM_1 LKM_1
SALES
C$_0
ORDER_LINES
IKM_1
LKM SQL to SQL
PRODUCT File
LKM_2 LKM File to SQL
C$_1
Target (Oracle)
I$_SALES
IKM SQL to SQL Append
Case #3 Using the Source as the Staging Area
Source (MS SQL) ORDERS
IKM_1 ORDER_LINES
Staging Area C$_0
IKM SQL to SQL Append
IKM SQL to SQL Append
IKM_1 C$_1
LKM_1
IKM SQL to SQL Append
I$_SALES
IKM_1
Target (Oracle) SALES
Which KMs for What Flow?
Four possible arrangements: Source
Loading phase
Staging area
Integration phase
Multi-tech LKM
Multi-tech IKM
Multi-tech LKM
Single-tech IKM
(No LKM needed) Staging area on source
(No LKM needed)
Staging area on target
Multi-tech IKM Single-tech IKM
Source, staging area and target in same location
Target
ODI Check Knowledge Module (CKM) Static Control – Check Data in Table against Constraints
ODI Check Knowledge Module (CKM) Static Control – Check Transformed data Before Loading Target
Records in error table can be fixed and merged back to next ETL batch
ODI CDC Framework Overview (JKM) Downstream Data Consumers Application Table XXX
Data mart 01
CDC Journal Table J$
Primary Key columns
window_id
123
44848
124
44848
Subscriber Datamart01 Datamart02
Data mart 02
DML Changes
CDC Journal View JV$D
Comparison of 2 ODI CDC Frameworks CDC Journal Table J$
DML on table (Classic)
Primary Key columns
window_id
123
Empty
Insert by per-row table trigger
CDC Journal Table J$
GoldenGate replicat
Insert by OGG
Primary Key columns
window_id
123
From SEQ+RBA of trail file
3 Simple Steps to Create ODI CDC Infrastructure
1. Create a new subscriber at ODI model
2. Add each table needed to CDC
Each Table in ODI Mapping Needs 2 Parameters
How ODI Handles Delete Records at Source
JV$D
.JRN_FLAG = ‘I’ – Insert/update at source
JV$D
.JRN_FLAG = ‘D’ – Delete
It’s up to data model design for synchronizing changes such as delete records from ODS by ODI
ODI Job Packaging for CDC Framework Process changed records in marked “window”
Light-weight check of data changes (independent of change volume)
Lock/Extend - Mark journal records
Unlock/Purge - Purge processed journal records
Enterprise Data Quality: Governed Data Quality Data Stewards
Data Analysts
Data Stakeholders
EDQ Profiling Example
Build-out Full DQ Process
Key Feature: Pre-built processors • Comprehensive DQ Functionality with a Single User Interface and Repository
EDQ for DW/BI 1. 2. 3. 4.
ODI or GoldenGate move data into DW ODI maps data into Staging tables EDQ ‘cleanses’ data ODI moves from staging into production tables
Golden Gate
Data in DW is ‘Fit for Purpose’ BI/Analytics Source of truth
Replica source tables
ODI ODI
BI/analytics
Staging tables
ODI
Data Sources
EDQ Parsing, Cleansing,
Production Tables
Single source of truth for reference
Latest ODI Updates (12.1.3)
ODI 12.1.3 New Features
FIPS Compliance
XML Driver Enhancements (e.g. Recursion)
JSON Support Hadoop Sqoop/Hbase Integration
Multi-Threaded Target Table Load in ODI Engine
Improved Control for Scenario/Load Plan Concurrent Execution
Create New Mode/Topology Objects (without building topology from physical/logical to model)
More details can be found from viewlet http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/odi/odi_12c/odi_12.1.3_nf/presentation.
New Features in ODI 12.1.2 Improved User Experience and Productivity Core Features for Better Performance Enterprise Deployment Improved Manageability Deeper Oracle Stack
• Declarative Flow-Based User Interface • Reusable Mappings • Step By Step Debugger • Parallel Target Table Load • In-Session Parallelism • Low Overhead Session • Integration with Common Admin Model • Integration with OPSS Enterprise Roles
• Integration with Enterprise Manager Cloud Control
Provide a migration path to OWB users Integrate OWB and ODI runtime for phased migration
How
OWB repositories can now be managed in Topology OWB Jobs can be executed and monitored in ODI OWB to ODI migration utility available (http://docs.oracle.com/middleware/1213/od
Management Pack for Oracle Data Integrator Key Features
Discovery of ODI components
SLA Management
Agents: Standalone and JEE Repositories Source and Target databases Alerts creation Notifications: email, SNMP, etc.
Monitoring
Availability Monitoring ODI homepage to quickly assess infrastructure status Performance Monitoring
Management Pack for Oracle Data Integrator Key Features
Performance Monitoring (continued)
Sessions execution statistics
Historical metrics
SQL Drill Down (requires Oracle DB Diagnostics and Tuning packs licenses)
Access database execution statistics
ODI Big Data – SQOOP Load
Fast Load between databases and Hadoop
Database can be source or target
HDFS, Hive, or HBase supported
Parallelized load using a data partitioning scheme
Oracle Data Integrator Parallel JDBC Queries
Source Database
SQOOP
HDFS
SQOOP
HDFS
SQOOP
HDFS
In-Session Parallelism Overview ODI 11g
Goal
Allow the execution of tasks and steps in parallel rather than sequentially
SrcSet1 SrcSet2 SrcSet3 Integration
Total Execution Time
How
Introduction of a new Runtime concept: Execution Units In 12.1.2 loading of C$ tables can be executed in parallel
ODI 12.1.2 SrcSet1 SrcSet2 SrcSet3
Integration
Multi-Threaded Target Load (12.1.3)
Data Server settings:
Fast Load through parallel JDBC connections Parallel Load Threads
Configurable per Data Server
JDBC Source
ODI Agent
Target Database
Enterprise Role Integration Introduction
Goal
Tighter integration with LDAP security
How
Introduction of ODI Roles ODI Roles can be mapped to Enterprise Roles Enterprise Role inherits ODI Role privileges
Oracle Data Integration
Oracle Warehouse Builder Product Update
OWB Support Policies Customer Downloadable OWB Support Policies: http://www http://ww w.oracle.com/us/sup .oracle.com/us/support/library/ port/library/lifetime-support lifetime-support-technology-0691 -technology-069183.pdf 83.pdf Release
Premier Support Ends
GA Date
Extended Support Ends
Sustaining Support Ends
OWB 9.2
Jul 2003
Oct 2007
Not Available
Oct 2010
OWB 10.1
Apr 2004
Jul 2007
Not Available
Jul 2010
OWB 10.2
Jul 2005
Jul 2010
Jul 2013
Indefinite
OWB 11.1
Aug 2007
Aug 2012
Aug 2015
Indefinite
OWB 11.2
Sep 2009
Current Dates: Jan 2015
Current Dates: Jan 2018
Indefinite
Future Dates: Same as DB12
Future Dates: Same as DB12
For more Information about Support Levels:
Best Practices for OWB to ODI Migrations
ODI 12c supports a phased migration approach
Execute and monitor existing OWB 11gR2 11gR2 jobs from ODI 12c Use the migration utility to convert existing OWB 11gR2 11gR2 objects into their ODI equivalents Start new developments using ODI 12c
Understand that OWB and ODI are different ETL tools
There is no “free lunch” when migrating – a – a technical migration between different tools will never be 100% perfect or predictable
Invoking OWB 11.2 Processes from ODI 12c
New ‘OWB Runtime Repository’ technology available in Topology
OWB processes can be invoked using the OdiStartOwbJob tool in Packages
OWB audit information is available in Operator, Operator, ODI Console and Enterprise Manager
