Tuesday, May 24, 2022

Connecting Oracle Analytics Cloud to an Oracle cloud database with a private endpoint

This blog is one example on how to connect OAC to an Oracle Cloud database with a private endpoint  using OAC's private access channel. The basic steps are:

  1. Use Oracle Cloud Infrastructure Console to configure a private access channel within your OAC instance.
  2. From your OAC instance create a data connection to the private database.
Refer to the documentation for all of the detail regarding setting this up, in particular the prerequisites. 

I used the following OCI Services:
  • Oracle Analytics Cloud (OAC) with the Enterprise Analytics feature.
  • Exadata Cloud Service (ExaCS) with an 19c pluggable database. 

Step 1 -- Creating private access channel

Using the OCI Console access your OAC instance, navigate to the Resources section, click Private Access Channel, and then click Configure Private Access Channel.


The private access channel above and the ExaCS were in the same private subnet which makes it easier.  Private access channel requires four IP addresses, two IP addresses are required for network traffic egress, one IP address for the private access channel, and one reserved for future use. 

After about 30 minutes you can click on the created private access channel and note the ip addresses it is using within the subnet. 

Step 2 -- Creating private access channel


OAC can't access private data sources on an Oracle Database that uses a Single Client Access Name (SCAN). If you want to connect Oracle Analytics Cloud to an Oracle Database that uses a SCAN, use one of the following methods to set up the connection in Oracle Analytics Cloud: connect directly to the Oracle Database nodes, instead of SCAN or configure an Oracle Connection Manager in front of SCAN.

From your OAC Home Page, click Create, and then click Connection,  and Oracle Database.

When using ExaCS the default connection strings all use SCAN.  Below are three examples connecting to a PDB on ExaCS that uses the compute node names instead. 


Basic connection to a single compute node

Advanced with long connection string
(DESCRIPTION=(ENABLE=BROKEN)
(ADDRESS_LIST= (LOAD_BALANCE=on)(FAILOVER=ON)
(ADDRESS=(PROTOCOL=tcp)(HOST=exacsphx1-8abcd1.sub1111112222.exacsvcnphx1.oraclevcn.com)(PORT=1521))
(ADDRESS=(PROTOCOL=tcp)(HOST=exacsphx1-8abcd2.sub111111222.exacsvcnphx1.oraclevcn.com)(PORT=1521)))
(CONNECT_DATA=(SERVICE_NAME=GFSwing_SCALE100.paas.oracle.com)))

Advanced with easy connection string
exacsphx1-8abcd2.sub111111222.exacsvcnphx1.oraclevcn.com:1521/GFSwing_SCALE100.paas.oracle.com


Thanks.

Wednesday, February 24, 2021

Setting up ExaCLI with ExaCS

What is needed:

First,  on your ExaCS:
  1. Get your cluster name as OS user grid via the command "crsctl get cluster name"

  2. Get IP address of a storage cell via "cat /etc/oracle/cell/network-config/cellip.ora"

  3. Test that you can login via ExaCLI as OS user oracle.   The exacli user id = "cloud_user_" + your cluster name from above and the password is what was used when provisioning the ExaCS.

    exacli -l cloud_user_cl-akhkpexa-11f -c 192.168.136.6




Now using Enterprise Manager:
  1. Create named credential using the userid and password tested above as EM user sysman


Now using EMCLI:
  1. Login as OS user oracle via the command "emcli login -username=sysman"


  2. Create a property file similar to below, substituting your values for each line

    configMap.targetName=ExaCS_PHX
    configMap.region=us-phoenix-1
    configMap.tenancy=natdsepltfrmanalyticshrd1
    configMap.serviceType=ExaCS
    configMap.monitorAgentUrl.0=https://exacsphx1-8mxxq1.sub10261618181.exacsvcnphx1.oraclevcn.com:3872/emd/main/

    credMap.cellCredSet=SYSMAN:EXACLI

    host.name.0=exacsphx1-8mxxq1.sub10261618181.exacsvcnphx1.oraclevcn.com
    host.name.1=exacsphx1-8mxxq2.sub10261618181.exacsvcnphx1.oraclevcn.com



  3. Run the following emcli command by specifying the property file

Finally go back to Enterprise Manager:
  1. Check the job submitted previously and verify it was successful via EM Home Page -> Enterprise icon -> Provisioning and Patching -> Procedure Activity


  2. Your exadata storage cell targets should now be discovered and able to see the cell metrics.


  3. Note that it takes about ~15 minutes before you will see the "Exadata" target in the menu.


Tuesday, August 25, 2020

Create non-cdb into existing home on Exadata Cloud Service

Here is an example json config which creates a new non-cdb database using an existing database home.

 

/var/opt/oracle/dbaasapi/dbaasapi -i createdb2.json

 

 

[root@exacsphx-z0pvn-pehrf1 dbinput]# cat createdb2.json

{

    "object": "db",

    "action": "start",

    "operation": "createdb",

    "params": {

        "nodelist": "",

        "cdb": "no",

        "ohome_name": "OraHome101_12102_dbbp200114_0",

        "bp": "JAN2020",

        "dbname": "gregfz2",

        "edition": "EE_EP",

        "version": "12.1.0.2",

        "adminPassword": "YourSysPasswordHere",

        "charset": "AL32UTF8",

        "ncharset": "AL16UTF16",

        "backupDestination": "NONE"

    },

    "outputfile": "/home/oracle/gregf/dbinput/createdb2.out",

    "FLAGS": ""

}

 

Monday, August 24, 2020

Create non-cdb on Exadata Cloud Service

 Please review MOS Note:  Creating non-CDB databases using Oracle Database 12c on the Exadata Cloud Service (Doc ID 2528257.1)

 

My environment:

  • ExaCS quarter rack in Phoenix. 
 

Step 1:  You may need to update your cloud tooling

Step 2:  You may need to download the non-cdb images

  • Sudo su -
  • dbaascli cswlib list
  • dbaascli cswlib download --version 19000 --bp JAN2020 --cdb no

Step 3:  Create  you non-cdb database

  • /var/opt/oracle/dbaasapi/dbaasapi -i createdb.json

 

[root@exacsphx-z0pvn-pehrf1 dbinput]# cat createdb.json

{

    "object": "db",

    "action": "start",

    "operation": "createdb",

    "params": {

        "nodelist": "",

        "cdb": "no",

        "bp": "JAN2020",

        "dbname": "gregfz1",

        "edition": "EE_EP",

        "version": "19.0.0.0",

        "adminPassword": "YourSysPasswordHere",

        "charset": "AL32UTF8",

        "ncharset": "AL16UTF16",

        "backupDestination": "NONE"

    },

    "outputfile": "/home/oracle/gregf/dbinput/createdb.out",

    "FLAGS": ""

}

Wednesday, August 12, 2020

JSON Search Index on Autonomous Database

 Just a quick test to validate JSON Search Index on ADB.   My environment:

  • ADB Shared 19c

 Below are the commands to create a simple table, insert a row, create the search index and validate the execution plan.

 

create table departments_json (
  department_id   integer not null primary key,
  department_data blob not null
);

alter table departments_json
  add constraint dept_data_json
  check ( department_data is json );
 
insert into departments_json
  values ( 110, utl_raw.cast_to_raw ( '{
  "department": "Accounting",
  "employees": [
    {
      "name": "Higgins, Shelley",
      "job": "Accounting Manager",
      "hireDate": "2002-06-07T00:00:00"
    },
    {
      "name": "Gietz, William",
      "job": "Public Accountant",
      "hireDate": "2002-06-07T00:00:00"
    }
  ]
}' ));

create search index dept_json_i on departments_json ( department_data ) for json;
 
select * from   departments_json d where  json_textcontains ( department_data, '$', 'Public' );

explain plan for select * from   departments_json d where  json_textcontains ( department_data, '$', 'Public' );

SELECT * FROM table(dbms_xplan.display); 

 

Looking at the plan we can see the search index was used.

 

Example came from this helpful blog article. 


Tuesday, August 11, 2020

Mounting OCI Object Storage Bucket as a File System on Exadata Cloud Service

My environment:

  • ExaCS running Oracle Linux 7.7

Step 1:  Install s3fs-fuse rpm

yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/s/s3fs-fuse-1.86-2.el7.x86_64.rpm --nogpgcheck

Step 2A:  Create file with credentials in the format access key:secret key

[root@vm1 bucket1]# cat /root/.passwd-s3fs
a4dzaa8049cbb6e1300b0a9170ff35ddebd50bfc:fYtbmNmp++lomv4z11YX4PuqR9gPbVEe4ZXB00TKUBA=

Step 2B: Change file permissions to 600 

chmod 600 /root/.passwd-s3fs

Step 3: Create directory for mount

mkdir /bucket1

Step 4:   Run the s3fs command

s3fs gregf1 /bucket1 -o endpoint=us-phoenix-1  -o passwd_file=/root/.passwd-s3fs -o url="https://yourtenancyhere.compat.objectstorage.us-phoenix-1.oraclecloud.com" -o nomultipart -o use_path_request_style

Example:

[root@exacsx7phx-023jw2 ~]# s3fs gregf1 /bucket1 -o endpoint=us-phoenix-1  -o passwd_file=/root/.passwd-s3fs -o url="https://yourtenancynamehere.compat.objectstorage.us-phoenix-1.oraclecloud.com" -o nomultipart -o use_path_request_style
[root@exacsx7phx-023jw2 ~]# df
Filesystem                      1K-blocks       Used    Available Use% Mounted on
devtmpfs                        371297736          0    371297736   0% /dev
tmpfs                           742617088    1262064    741355024   1% /dev/shm
tmpfs                           371308876       3044    371305832   1% /run
tmpfs                           371308876          0    371308876   0% /sys/fs/cgroup
/dev/mapper/VGExaDb-LVDbSys1     24639868   11840312     11524884  51% /
/dev/xvda1                         499656      52724       420720  12% /boot
/dev/xvdi                      1135204408  112486384    965029960  11% /u02
/dev/mapper/VGExaDb-LVDbOra1    154687468    6198624    140608140   5% /u01
/dev/xvdb                        51475068   13514280     35322964  28% /u01/app/19.0.0.0/grid
tmpfs                            74261776          0     74261776   0% /run/user/0
/dev/asm/acfsvol01-131          838860800  139691116    699169684  17% /acfs01
/dev/asm/upload_vol-131        8388608000 4577187480   3811420520  55% /scratch
tmpfs                            74261776          0     74261776   0% /run/user/2000
s3fs                         274877906944          0 274877906944   0% /bucket1
tmpfs                            74261776          0     74261776   0% /run/user/1001
[root@exacsx7phx-023jw2 ~]# ls /bucket1
fish121.dmp  fish.dmp  FISH_EXPDAT.DMP

 

Step 5:  Make mount permanent

 

Additional Information:




Monday, August 10, 2020

Mounting OCI Object Storage Bucket as a File System on Oracle Linux VM

My environment:

  • VM on OCI running Oracle Linux 7.8

Step 1:  Install s3fs-fuse rpm

yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/s/s3fs-fuse-1.86-2.el7.x86_64.rpm --nogpgcheck

Step 2A:  Create file with credentials in the format access key:secret key

[root@vm1 bucket1]# cat /root/.passwd-s3fs
a4dzaa8049cbb6e1300b0a9170ff35ddebd50bfc:fYtbmNmp++lomv4z11YX4PuqR9gPbVEe4ZXB00TKUBA=

Step 2B: Change file permissions to 600 

chmod 600 /root/.passwd-s3fs

Step 3: Create directory for mount

mkdir /bucket1

Step 4:   Run the s3fs command

s3fs gregf1 /bucket1 -o endpoint=us-phoenix-1  -o passwd_file=/root/.passwd-s3fs -o url="https://yourtenancyhere.compat.objectstorage.us-phoenix-1.oraclecloud.com" -o nomultipart -o use_path_request_style


Step 5:  Make mount permanent

 

Additional Information:





Tuesday, July 14, 2020

Snapshots with Exadata Cloud

Currently the OCI cloud tooling does not support cloning or snapshots however you can do these manually.   Below are a couple of examples of using SQL to create PDB snapshots on the Exadata Cloud Service or Exadata Cloud at Customer services.





Example #1 -- Creating PDB Snapshot using Full Test Masters
Scenario is to create a test master full copy of production, developers then create snapshots from the test master

Show pdbs

/* Create Full test master PDB - note this would usually be a remote clone*/
create pluggable database prod1tm1 from prod1 keystore identified by "WELcome__2019";

show pdbs;

/* Need to open Test Master read write before opening it read only */
alter pluggable database prod1tm1 open   instances=all;
alter pluggable database prod1tm1 close immediate instances=all;
alter pluggable database prod1tm1 open read only instances=all;  /* read only test master pdb */

/* Create  First PDB snapshot from Test Master */

create pluggable database tm1snap1 from prod1tm1 tempfile reuse create_file_dest='+SPRC1' snapshot copy keystore identified by "WELcome__2019";

alter pluggable database tm1snap1 open instances=all;
alter session set container=tm1snap1;

create table panda as select * from dba_users;
select * from panda;

/* Create second snapshot from same test master */
create pluggable database tm1snap2 from prod1tm1 tempfile reuse create_file_dest='+SPRC1' snapshot copy keystore identified by "WELcome__2019";
alter pluggable database tm1snap2 open instances=all;

alter session set container=cdb$root;
Show pdbs

/* query shows parent of snapshots */
column name format a20
select CON_ID, NAME, OPEN_MODE, SNAPSHOT_PARENT_CON_ID from v$pdbs;

alter pluggable database tm1snap1 close immediate instances=all;
alter pluggable database tm1snap2 close immediate instances=all;
alter pluggable database prod1tm1 close immediate instances=all;
drop pluggable database tm1snap1 including datafiles;
drop pluggable database tm1snap2 including datafiles;
drop pluggable database prod1tm1 including datafiles;



Example #2 -- Creating PDB Snapshots using Sparse  Test Masters
Scenario is to create one test master full copy of production, then create  sparse test masters updated each night from prod using GG, developers then create snapshots from the test masters.  Provide a new copy of production each night for the developers to create snaps from, full test master for Monday , sparse test masters Tuesday thru Friday.
Note: Using the same name for the test master each night so that the GG configuration doesn't have to change

/* create full copy of test master from prod, usually this would be a remote clone */
create pluggable database prod2tm from prod2 keystore identified by "WELcome__2019";
alter pluggable database prod2tm open  instances=all;

/* setup GG , catch up test master with production */

/* stop GG  when it is time to create a new test master*/

Alter pluggable database prod2tm close immediate instances=all;
alter pluggable database prod2tm unplug into '/home/oracle/snapshot/prod2tm_monday.xml' encrypt using "zzz";
drop pluggable database prod2tm keep datafiles;

/* create full copy of  test master*/
create pluggable database prod2tm_monday using '/home/oracle/snapshot/prod2tm_monday.xml' nocopy keystore identified by "WELcome__2019" decrypt using "zzz";

/* open full copy of test master read only,    need to open read/write first then read only */
alter pluggable database prod2tm_monday open instances=all;
alter pluggable database prod2tm_monday close immediate instances=all;
alter pluggable database prod2tm_monday  open read only instances=all;

/* create next days test master as a sparse */
create pluggable database prod2tm from prod2tm_monday tempfile reuse create_file_dest='+SPRC1' snapshot copy keystore identified by "WELcome__2019";
alter pluggable database prod2tm open  instances=all;

/* sync test master to prod*/
/* Restart GG */

/* developer creates sparse pdb to use */
create pluggable database prod2tm_monday_greg from prod2tm_monday tempfile reuse create_file_dest='+SPRC1' snapshot copy keystore identified by "WELcome__2019";
alter pluggable database prod2tm_Monday_greg open instances=all;

show pdbs
column name format a20
select CON_ID, NAME, OPEN_MODE, SNAPSHOT_PARENT_CON_ID from v$pdbs;

/* repeat process for each day's  sparse test master */

alter pluggable database prod2tm close immediate instances=all;
alter pluggable database prod2tm_monday close immediate instances=all;
alter pluggable database prod2tm_Monday_greg close immediate instances=all;
drop pluggable database prod2tm including datafiles;
drop pluggable database prod2tm_monday including datafiles;
drop pluggable database prod2tm_Monday_greg including datafiles;

rm /home/oracle/snapshot/*.xml


ASM

sqlplus / as sysasm
ALTER DISKGROUP DATAC1 SET ATTRIBUTE 'ACCESS_CONTROL.ENABLED' = 'TRUE';

Saturday, July 11, 2020

Querying External Data with Autonomous Database

Below is an example of querying an externaltext file with a comma delimiter on OCI object storage using an Autonomous Database.  This example is from the documentation which had a couple of typos that are corrected here.

I am using an ADB on Dedicated Exadata Infrastructure.

channels.txt file:
1,Direct Sales,Direct
2,Tele Sales,Direct
3,Catalog,Indirect
4,Internet,Indirect
5,Partners,Others


BEGIN
  DBMS_CREDENTIAL.CREATE_CREDENTIAL(
    credential_name => 'DEF_CRED_NAME',
    username => 'your OCI user name here',
    password => 'your token here'
  );
END;
/

select owner, credential_name, username, enabled from dba_credentials;

BEGIN
   DBMS_CLOUD.CREATE_EXTERNAL_TABLE(
    table_name =>'CHANNELS_EXT',
    credential_name =>'DEF_CRED_NAME',
    file_uri_list =>'https://objectstorage.us-ashburn-1.oraclecloud.com/n/youtenancynamehere/b/sampledata/o/channels.txt',
    format => json_object('delimiter' value ','),
    column_list => 'CHANNEL_ID NUMBER,
    CHANNEL_DESC VARCHAR2(20),
    CHANNEL_CLASS VARCHAR2(20)' );
END;
/

SELECT count(*) FROM channels_ext;
select * from channels_ext;
desc channels_ext;

Thursday, July 9, 2020

Creating an ACFS filesystem on Exadata Cloud Service

Below is an example on how to create an ACFS filesystem on an Exadata Cloud Service quarter rack.  This example creates a 2TB ACFS in the RECO1 disk group and is mounted at /scratch.



[opc@exacsphx-xyzn1 ~]$ sudo -s
[root@exacsphx-xyzn1 opc]# su - grid
Last login: Thu Oct 17 09:38:15 EDT 2019
[grid@exacsphx-xyzn1 ~]$ asmcmd
ASMCMD> volcreate -G RECOC1 -s 2000G upload_vol
ASMCMD> volinfo -G RECOC1 upload_vol
Diskgroup Name: RECOC1

         Volume Name: UPLOAD_VOL
         Volume Device: /dev/asm/upload_vol-58
         State: ENABLED
         Size (MB): 2048000
         Resize Unit (MB): 64
         Redundancy: HIGH
         Stripe Columns: 8
         Stripe Width (K): 1024
         Usage:
         Mountpath:

ASMCMD> exit
[grid@exacsphx-xyzn1 ~]$ logout

[root@exacsphx-xyzn1 opc]# /sbin/mkfs -t acfs /dev/asm/upload_vol-58
mkfs.acfs: version                   = 19.0.0.0.0
mkfs.acfs: on-disk version           = 46.0
mkfs.acfs: volume                    = /dev/asm/upload_vol-58
mkfs.acfs: volume size               = 2147483648000  (   1.95 TB )
mkfs.acfs: Format complete.
[root@exacsphx-xyzn1 opc]# /u01/app/19.0.0.0/grid/bin/srvctl add filesystem -d /dev/asm/upload_vol-58 -g RECOC1 -v upload_vol -m /scratch
[root@exacsphx-xyzn1 opc]# /u01/app/19.0.0.0/grid/bin/srvctl start filesystem -d /dev/asm/upload_vol-58

[root@exacsphx-xyzn1 opc]# df -h
Filesystem                    Size  Used Avail Use% Mounted on
devtmpfs                      355G     0  355G   0% /dev
tmpfs                         709G  1.5G  707G   1% /dev/shm
tmpfs                         355G  3.0M  355G   1% /run
tmpfs                         355G     0  355G   0% /sys/fs/cgroup
/dev/mapper/VGExaDb-LVDbSys1   24G   11G   12G  47% /
/dev/xvda1                    488M   82M  381M  18% /boot
/dev/xvdi                     1.1T   69G  959G   7% /u02
/dev/mapper/VGExaDb-LVDbOra1   20G  7.4G   12G  40% /u01
/dev/xvdb                      50G  9.3G   38G  20% /u01/app/19.0.0.0/grid
/dev/asm/acfsvol01-58         800G   39G  762G   5% /acfs01
tmpfs                          71G     0   71G   0% /run/user/2000
tmpfs                          71G     0   71G   0% /run/user/0
/dev/asm/upload_vol-58        2.0T  4.6G  2.0T   1% /scratch
[root@exacsphx-xyzn1 opc]#