[{"content":"Oracle Spatial Studio is a bit like a music studio. It is a place to explore things and get new ideas in the process. With this tool you can explore data on maps and discover new connections between various datasets. You don\u0026rsquo;t need a lot of data before a viualization becomes useful, especially for spatial data.\nFirst session with Oracle Spatial Studio With Spatial Studio it is easy to load, verify and visualize spatial data. Data you want to put on a map are often distributed in special format like GeoJSON and ESRI spatial files. I am not aware of any other simple tool that makes it easier to load this kind of data into the Oracle database.\nThis is a gentle introduction to get started. More advanced use will follow.\nFirst time you log into Spatial Studio you will see a screen like this:\nI have some data in an Oracle database, so I create a connection to it from the Connections page. You\u0026rsquo;ll see that the connection I created for the repository is listed:\nIn this case the table is in an Autonomous Database:\nAfter clicking Next you must choose authentication type, I\u0026rsquo;m using the first, \u0026ldquo;Wallet-based*:\nDrag the wallet into the area:\nAdd the connection details:\nNow with a connection in place I create a dataset:\nHere I select Database table/view from the connection I just created. You have other options here like loading data from files or from a URL.\nI have one table with longitude and latitude in.\nThe icon besides the table shows there is an issue with it:\nClick on the icon and offer you to fix it:\nI clicked the Create Latitude/Longitude Index. The data comes from one of my silly APEX apps where my position is fetched from the mobile and stored in two columns, latitude and longitude1. Spatial Studio got it right at first attempt so I can just click OK:\nBy the way, next time I try to add a datasource from the same connection, the table will have another icon beside it:\nIf you do something similar check with SQL Developer or similar tool what has been created. My table now has a function-based domain index that uses a new function created in my schema called SGTECH_PTF. It takes longitude and latitude as arguments as retuns a point with datatype sdo_point_type. Very simple and easy.\nAlmost there; we can now start on a new project. Hit the tab for projects and create a new project:\nThen add the dataset from the Add Dataset button, select the only dataset we have so far, and then OK:\nTo show it on the map, click on the table name and drag it into the map.\nI changed the default color so the points are more visible here, and zoomed in a bit:\nI see some places from last trip to France, lots of stops from charging my car, the rest is mostly conferences outside Norway. POUG, UKOUG, RoOUG, HrOUG, SwissOUC 🤩\nThat wasn\u0026rsquo;t really difficult. Showing points on a map is oddly satisfying.\nYou can change some settings for the layer by clicking on Settings:\nand then add Tooltip and Info window\nZoom in and hover the mouse over one of the points to get a tooltip:\nProbably the best place in Europe to meet other Oracle nerds:\nThere is a lot of interesting data out there. Probably the easiest way to get started is to look for data in GeoJSON format. For instance in this Github repository I found all the municipalities (kommuner) and counties (fylker) in Norway in GeoJSON. The largest file for all the municipalities is around 10MB. I uploaded the file from Dataset tab as shown earlier and selecting Local file, add the file and hit the Create button.\nAfter a few seconds of preprocessing I selected the connection where I want to store the table:\nAfter clicking the Submit-button Spatial Studio starts to upload the file in a background job (useful for large datasets so you don\u0026rsquo;t have to wait). After a few seconds it appears in the datasets list, but with a yellow warning sign indicating that something needs to be fixed:\nThat easy to fix, click the link and select the ID column as key and then validate:\nAfter clicking the Apply-button I verified that Spatial Studio has added an unique index for the ID-column and added a spatial index to the GEOM column. Again, quite some work I don\u0026rsquo;t have to worry about. This is how my muncipality is rendered after adding it to the project:\nWith this I hope you understand that Spatial Studio is pretty easy to use and useful to display data on maps. But there are lots of other features to explore, see upcoming posts.\nYou don`t need an extra app for this. You can take a picture with your phone instead and extract the EXIF metadata that will contain position and timestamp. With some Python code and an AI model you can extract features for additional analysis.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2026/01/oracle-spatial-studio-first-session/","summary":"\u003cp\u003eOracle Spatial Studio is a bit like a music studio. It is a place to explore things and get new ideas in the process. With this tool you can explore data on maps and discover new connections between various datasets. You don\u0026rsquo;t need a lot of data before a viualization becomes useful, especially for spatial data.\u003c/p\u003e\n\u003ch1 id=\"first-session-with-oracle-spatial-studio\"\u003eFirst session with Oracle Spatial Studio\u003c/h1\u003e\n\u003cp\u003eWith Spatial Studio it is easy to load, verify and visualize spatial data. Data you want to put on a map are often distributed in special format like GeoJSON and ESRI spatial files. I am not aware of any other simple tool that makes it easier to load this kind of data into the Oracle database.\u003c/p\u003e","title":"Oracle Spatial Studio - First session"},{"content":"This is part 0 in a series on Oracle Spatial Studio. There are a few ways to run Oracle Spatial Studio, this post shows you how to run Oracle Spatial Studio from your PC and from a stack in Oracle Cloud.\nGet started with Oracle Spatial Studio - Installation It has been a while\u0026hellip; I have looked into Oracle Spatial Studio and decided to blog about it.\nRun it from your PC Also known as on-prem. You can choose between:\nDeploy it on Oracle WebLogic Server Deploy it on Apache Tomcat Use Quick Start I am using the last one; easy to get started and experiment on your own: you simply download a zip-file, unzip it and start it from one script; details to follow. For this to work you need Java JDK 11 or newer.\nI usually go to https://otn.oracle.com/ to find what I need. That link now redirects to https://www.oracle.com/downloads/ . Look for Developer Downloads and Developer Tools.\nClick on the link for Spatial Studio, and it takes you to this page:\nHere you find link to Documentation as well as link to download. You may just as well bookmark the link to the documentation now.\nWhen you click on that link you will be redirected to edelivery.oracle.com and you need to login before you get this page:\nAs marked on the screen capture, I am using 25.2.0 Quick Start. You only need this file. Download it by clicking on the file name (not the Download button at the bottom right).\nUnzip the file in a place of your choice. You should now have a directory called Oracle_Spatial_Studio. Inside you\u0026rsquo;ll find start.sh and start.bat for Linux/MacOS and Windows respectively. I\u0026rsquo;m using a Mac and simply started it with ./start.sh. This will start Spatial Studio and listen with https on port 4040. If that port is taken on your machine you can change it in the file conf/server.properties. For instance to run it on port 81911 you must change the port number on line 6 in the file to:\njetty.ssl.port=8191 This is also where you enable http by setting httpEnabled=true. The installation comes with an unsecure certificate, something your browser will inform you about when you go to the default link https://localhost:4040. Login with username admin and password welcome12:\nBy the way, these steps are documented in 2.2.1 in the manual. After first login you will get this page:\nSpatial Studio needs an Oracle database to store the metadata. In this case I am using a local Oracle database, FREE version, running in podman as explained here , but any Oracle database should work (I haven\u0026rsquo;t tried all the versions). You can also use an Oracle Autonomous Database. Section 2.4 explains how to create the user for the metadata. Here is what I did:\ncreate user SPATIAL_STUDIO identified by \u0026#34;notExactlyThisPassword\u0026#34; default tablespace users quota unlimited on users; grant create procedure, create sequence, create session, create synonym, create table, create trigger, create type, create view to SPATIAL_STUDIO; On 26ai granting the role db_developer_role also works.\nContinue in the browser and add connection info for the this user:\nAfter you hit next it connects and creates the repository and you may see messages like this:\nThis takes just a few seconds and I hardly managed to capture a screenshot before it disappeared.\nThe Spatial Studio is now ready to use.\nRun Spatial Studio from OCI If you have an account in the Oracle Cloud, free tier or paid subscription, you may find it easier to just fire up a stack in the marketplace.\nAfter you\u0026rsquo;ve logged into cloud.oracle.com, click on the hamburger menu in the top-left corner, scroll down, click Marketplace and finally on All Applications as shown here:\nOn next page, just search for Spatial Studio, choose the one with shortest title:\nClick the button with Launch Stack on it. On the first page for creating the stack, you may give it a name before you hit Next.\nI use the second option on the drop-down list of Server Availability Domain3:\nFurther down you need to set up networking, it is mostly a sequence of obvious selections from drop-down menus (unless this is the first time you create anything in OCI and need to setup all the networking):\nYou also need to add your ssh key (not shown here), and setup database configuration for the metadata that Spatial Studio uses:\nThe user must exist in the database you choose. As in previous section it can be created like this:\ncreate user studio_repo identified by \u0026#34;fromYourPasswordManager\u0026#34; quota unlimited on data; grant create procedure, create sequence, create session, create synonym, create table, create trigger, create type, create view to studio_repo; After Next you\u0026rsquo;ll see a review of the configuration, hit Create at the bottom of the page. It will then take you to the Resource Manager and show you the job that it is running to create the stack.\nAfter a few minutes it should turn green and you can download the Terraform configuration which may become useful later:\nClick on the Application information tab to get the Spatial Studio HTTPS URL. Click on it to start Spatial Studio, you may get a warning regarding an unsecure certificate. Login with the admin user created earlier. Spatial Studio is now ready for your exploration.\nNot a totally random choice on my side; this is also the fifth Mersenne prime number \u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThe README.txt file included explains how to change the password before you start the server. In order to change the password for admin to say, SpecialSpatial42 you run this command: ./moduser.sh admin SpecialSpatial42 The last line in the output looks like this: admin: CRYPT:ad0LbH3EOmaNk,valid-user Copy the part ad0LbH3EOmaNk and replace the equivalent encrypted password in the file jetty-home/security/jetty-realm-mt.properties. After the change the line looks like this: admin: CRYPT:ad0LbH3EOmaNk,valid-user,SGTech_SystemAdmin. As you can see from the file there are three other users with different roles, either as spatial developer or normal user with read-only access.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThe reason being that this domain seems to have more capacity.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2026/01/oracle-spatial-studio-installation/","summary":"\u003cp\u003eThis is part 0 in a series on Oracle Spatial Studio. There are a few ways to run Oracle Spatial Studio, this post shows you how to run Oracle Spatial Studio from your PC and from a stack in Oracle Cloud.\u003c/p\u003e\n\u003ch1 id=\"get-started-with-oracle-spatial-studio---installation\"\u003eGet started with Oracle Spatial Studio - Installation\u003c/h1\u003e\n\u003cp\u003eIt has been a while\u0026hellip; I have looked into Oracle Spatial Studio and decided to blog about it.\u003c/p\u003e\n\u003ch2 id=\"run-it-from-your-pc\"\u003eRun it from your PC\u003c/h2\u003e\n\u003cp\u003eAlso known as on-prem. You can choose between:\u003c/p\u003e","title":"Oracle Spatial Studio - Installation"},{"content":" I wrote about UUIDs in the Oracle database two months back in this post , but it turns out that UUID v4 is so\u0026hellip; 2005 .\nYou probably need UUID v7 Head over to How UUIDv7 makes your (database) life easier to understand why you want v7.\nAs mentioned in the article there is no native support for v7 in Oracle, neither a function to generate it or a proper datatype for UUIDs like PostgreSQL has. But the article shows an implementation of v7 generation in a PL/SQL function called generate_uuid_v7 that I used in some testing.\nUUID from JavaScript Now that we have a solution for this for any version of Oracle after version 6 when PL/SQL was introduced we don\u0026rsquo;t need to look further. But this is an example where JavaScript in the database (since version 21c) is useful. JavaScript libraries for generating all kinds of UUIDs can be found rather easily, and with a few lines of code they can be called from PL/SQL.\nIn this example I am using a library uuidv7 found at npmjs.org:\nnpm install uuidv7 Save this into a file called e.g. uuidv7.js:\nimport { uuidv7, uuidv7obj } from \u0026#34;uuidv7\u0026#34;; export function uuid_v7() { return uuidv7(); } export function uuid_v7_raw() { return uuidv7obj().bytes ; } The following command builds a bundle that we\u0026rsquo;ll load into Oracle:\nnpx esbuild uuidv7.js --bundle --outfile=uuidv7-bundle.js --format=esm SQLcl has a command to load MLE bundles into Oracle:\nmle create-module -filename uuidv7-bundle.js -module-name uuidv7 -language javascript -replace The PL/SQL call specifications look like this:\nCREATE OR REPLACE FUNCTION uuid_v7 RETURN VARCHAR2 AS MLE MODULE uuidv7 SIGNATURE \u0026#39;uuid_v7\u0026#39;; / CREATE OR REPLACE FUNCTION uuid_v7_raw RETURN RAW AS MLE MODULE uuidv7 SIGNATURE \u0026#39;uuid_v7_raw\u0026#39;; / Compared to the ready made PL/SQL function generate_uuid_v7 this have more steps, but if you have an implementation in JavaScript somewhere, this is a reasonable way to have utility functions available in the database. Just have a look at validator.js and imagine how to implement all those validation functions in PL/SQL (if you can find a PL/SQL developer).\nPerformance Back to the UUIDs. With native support for v4 in Oracle we expect it to be much faster than calling a PL/SQL function and perhaps even worse with a function based on MLE JavaScript. But does it matter?\nLet us do a simple test. First create a table with 1 million rows using the new builtin UUID function:\ncreate table uuid_v4 as select uuid() uuid, dbms_random.string(\u0026#39;a\u0026#39;,42) foo from dual connect by level \u0026lt;= 1e6; On my Mac mini with M1 CPU and Oracle 23.26 FREE edition this takes 21.5 seconds1.\nSame test but with the PL/SQL function generate_uuid_v7() from the referenced article:\ncreate table uuid_v7_plsql as select generate_uuid_v7() uuid, dbms_random.string(\u0026#39;a\u0026#39;,42) foo from dual connect by level \u0026lt;= 1e6; This one takes 30.3 seconds.\nAnd finally the JavaScript version uuid_v7_raw():\ncreate table uuid_v7_js as select uuid_v7_raw() uuid, dbms_random.string(\u0026#39;a\u0026#39;,42) foo from dual connect by level \u0026lt;= 1e6; This one take 38.20 seconds\nThe JavaScript version takes 77% more than the new uuid() function. But does it matter? It is less than 17 microseconds extra pr row. I doubt that the end-user will notice, or that this has any significance unless you generate a whole lot of UUIDs.\nI did some investigating around the indexes on the UUID column and compared it with an index on a sequence based key. The clustering factor increased a lot with the UUIDs, but this may be different in a production system, perhaps on RAC where each node has a cache for the sequence. Also the number of leaf blocks in the indexes on UUIDs increased substantial.\nEarlier I looked into some testing using Swingbench; my plan was to compare performance with sequence based IDs vs UUIDs. With Swingbench you typically generate data in a schema before you run the test. Somewhere in the code there is an initial functional test where a select statement assumes the existence of row in a specific table that has an ID=1, which is safe because the data generation is deterministic since sequence is always created with a start value of 1. But you can\u0026rsquo;t make such assumptions with UUIDs because the UUID will not be deterministic.\nI ran each test three times and used the average.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2025/12/uuid-v7-in-oracle-database/","summary":"\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/guids_in_database.png\" width=\"400\"/\u003e \n\u003c/figure\u003e\n\n\u003cp\u003eI wrote about UUIDs in the Oracle database two months back \u003ca href=\"https://enesi.no/2025/10/uuids-in-oracle-database/\"\u003ein this post\u003c/a\u003e\n, but it turns out that UUID v4 is so\u0026hellip; \u003ca href=\"https://datatracker.ietf.org/doc/html/rfc4122\" target=\"_blank\" rel=\"noopener\"\u003e2005\u003c/a\u003e\n.\u003c/p\u003e\n\u003ch2 id=\"you-probably-need-uuid-v7\"\u003eYou probably need UUID v7\u003c/h2\u003e\n\u003cp\u003eHead over to \u003ca href=\"https://medium.com/@jasminfluri/how-uuidv7-makes-your-database-life-easier-5eee3d0ff9e2\" target=\"_blank\" rel=\"noopener\"\u003eHow UUIDv7 makes your (database) life easier\u003c/a\u003e\n to understand why you want v7.\u003c/p\u003e\n\u003cp\u003eAs mentioned in the article there is no native support for v7 in Oracle, neither a function to generate it or a proper datatype for UUIDs like PostgreSQL has. But the article shows an implementation of v7 generation in a PL/SQL function called \u003ccode\u003egenerate_uuid_v7\u003c/code\u003e that I used in some testing.\u003c/p\u003e","title":"UUID v7 in Oracle Database"},{"content":"The problem I just got a new iPhone, I have had the old one for four years and I hope the new one will last even longer; moving all those 2FA apps and all the rest is a big hassle. Pro tip: Don\u0026rsquo;t reset or give away your old phone right after the new one seems to be working. You will need the old one for much longer than you think.\nEspecially the Oracle Mobile Authenticator was a big hassle. Not the app itself (it was copied automatically with the rest), but finding the page somewhere where I could add the new phone to my accounts wasn\u0026rsquo;t easy, so here it is until next time.\nI have 4 different accounts at Oracle and each of them requires me to use 2FA. One is for regular login that I use when I download software, register for webinars, etc, three of them are cloud accounts at OCI.\nAdd a new phone to your Oracle profile account For your regular Oracle SSO account go to this page and login, if you have 2FA enabled already you will need your old phone to approve the login from the app, or enter the passcode. From here you can add a new Mobile App:\nYou will be asked to scan a QR code as usual.\nAfter you have done so, set the new one to default. Don\u0026rsquo;t delete the old one yet, you never know if you have to reset your new phone or something. Create a reminder for yourself somewhere to remove it later.\nAdd a new phone to your cloud account Login to your cloud account as usual. Once connected got to User Settings in the Profile menu at top right corner:\nOn the next page click on the Security tab. From the Actions menu select Add mobile app:\nAs usual you just scan the QR code and the rest is easy. Change the default in the menu after the new app is added.\n","permalink":"https://enesi.no/2025/12/moving-oracle-mobile-authenticator-to-a-new-phone/","summary":"\u003ch2 id=\"the-problem\"\u003eThe problem\u003c/h2\u003e\n\u003cp\u003eI just got a new iPhone, I have had the old one for four years and I hope the new one will last even longer; moving all those 2FA apps and all the rest is a big hassle. Pro tip: Don\u0026rsquo;t reset or give away your old phone right after the new one seems to be working. You will need the old one for much longer than you think.\u003c/p\u003e","title":"Moving Oracle Mobile Authenticator to a new phone"},{"content":"This is an update of previous post on How to install OML4Py Client on Ubuntu , this time Oracle Linux 9.\nInstall OML4Py on Oracle Linux The goal here is to have OML4Py 2.1 available in order to export embedding models to ONNX format. To follow this procedure you need to have a free Oracle Account, you will be offered to create one when you try to access https://edelivery.oracle.com/ later.\nThis installation is divided in parts:\nInstall a specific version of Python Install Oracle Instant Client Download OML4Py and install specific Python packages it requires Install OML4Py One advice for future releases of OML4Py is to download it first (see below how) and inspect a file to verify what specific Python version OML4Py actually requires, so you don\u0026rsquo;t waste time installing the wrong Python version, look for line requires Python ...in file client/OML4PInstallShared.pm As of today the documentation in chapter 3 says it requires 3.12.0, but on version 2.1_03142025 (see oml4py.ver) it requires 3.12.3, this is also reflected in table 4-1 in the current manual.\nFor any testing against Oracle AI Database, use the latest Free version unless you have access to another edition with 26ai.\nDownload, compile and install Python 3.12.6 mkdir -p $HOME/Python cd $HOME/Python curl -O https://www.python.org/ftp/python/3.12.6/Python-3.12.6.tgz tar xvf Python-3.12.6.tgz cd Python-3.12.6 Install packages that OML4Py needs plus libaio that Instantclient requires:\nsudo dnf install perl-Env libffi-devel openssl openssl-devel tk-devel xz-devel zlib-devel bzip2-devel readline-devel libuuid-devel ncurses-devel libaio Configure, compile, and install:\n./configure --enable-shared --prefix=$HOME/Python/Python-3.12.6 --enable-optimizations make clean; make make altinstall Add links\ncd $HOME/Python/Python-3.12.6/bin/ ln -s python3.12 python3 ln -s python3.12 python Download Instant Client Go to https://www.oracle.com/database/technologies/instant-client/linux-x86-64-downloads.html and find the latest version on 26ai, the first line Basic Package (ZIP)\nThe link at the time this post was written: https://download.oracle.com/otn_software/linux/instantclient/2326000/instantclient-basic-linux.x64-23.26.0.0.0.zip mkdir -p $HOME/Oracle cd $HOME/Oracle curl -O https://download.oracle.com/otn_software/linux/instantclient/2326000/instantclient-basic-linux.x64-23.26.0.0.0.zip unzip instantclient-basic-linux.x64-23.26.0.0.0.zip rm instantclient-basic-linux.x64-23.26.0.0.0.zip Create a file with environment variables to be used later\ncd $HOME/Python mkdir OML4Py cd OML4Py vi env_vars Put this in a file, the filename env_vars is used here:\nPYTHONHOME=$HOME/Python/Python-3.12.6 PATH=$PYTHONHOME/bin:$PATH LD_LIBRARY_PATH=$PYTHONHOME/lib:$HOME/Oracle/instantclient_23_26:$LD_LIBRARY_PATH export PYTHONHOME PATH LD_LIBRARY_PATH This file will be sourced by executing . env_vars later.\nDownload OML4Py Go to https://www.oracle.com/database/technologies/oml4py-downloads.html Click the link for OML4Py 2.1.0, it will take you to edelivery.oracle.com and ask you to login. Accept the Oracle License Agreement and click the link for V1048628-01.zip. Transfer to the machine where you are installing OML4Py if different. I have put it in $HOME/tmp\ncd $HOME/Python/OML4Py unzip $HOME/tmp/V1048628-01.zip Source the file created earlier, create a virtual environment, and upgrade pip\n. $HOME/Python/OML4Py/env_vars python3 -m venv venv . venv/bin/activate pip install --upgrade pip Add this to a new file called requirements.txt\npandas==2.2.2 setuptools==70.0.0 scipy==1.14.0 matplotlib==3.8.4 oracledb==2.4.1 scikit-learn==1.5.1 numpy==2.0.1 onnxruntime==1.20.0 onnxruntime-extensions==0.12.0 onnx==1.17.0 torch==2.6.0 transformers==4.49.0 sentencepiece==0.2.0 Install the packages with\npip install -r requirements.txt Install the client package with:\npip3 install client/oml-2.1-cp312-cp312-linux_x86_64.whl Finally install OML4Py with\nperl -Iclient client/client.pl When running this it is normal to wait a few seconds on the line Checking OML4P version \u0026hellip; It is starting Python and tries to load the oml-library.\nAnswer yes and complete the installation. This is from a successful installation:\nOracle Machine Learning for Python 2.1 Client. Copyright (c) 2018, 2025 Oracle and/or its affiliates. All rights reserved. Checking platform .................. Pass Checking Python .................... Pass Checking dependencies .............. Pass Checking OML4P version ............. Pass Current configuration Python Version ................... 3.12.6 PYTHONHOME ....................... /home/opc/Python/OML4Py/venv Existing OML4P module version .... 2.1 Operation ........................ Install/Upgrade Proceed? [yes] Found existing installation: oml 2.1 Uninstalling oml-2.1: Successfully uninstalled oml-2.1 Processing ./client/oml-2.1-cp312-cp312-linux_x86_64.whl Installing collected packages: oml Successfully installed oml-2.1 Done Try the new installation and check the version and path for the package in Python. Run this in Python:\nimport oml oml.__version__ oml.__path__ I am going to use this to export available pretrained models to ONNX format, this code lists the models available:\nfrom oml.utils import ONNXPipeline, ONNXPipelineConfig for i in ONNXPipelineConfig.show_preconfigured(): print(i) This shows the available templates:\nONNXPipelineConfig.show_templates() This exports one of the models to a file:\npipeline = ONNXPipeline(model_name=\u0026#34;sentence-transformers/all-mpnet-base-v2\u0026#34;) pipeline.export2file(\u0026#34;all_mpnet_base_v2\u0026#34;,output_dir=\u0026#34;.\u0026#34;) The first time you run this it will download a bunch of files and place them under $HOME/.cache/OML. They allocate around 420 MB. After this you should have a file called all_mpnet_base_v2.onnx in your current directory.\nIf you are reading this because you will import the model to an Oracle AI Database you can now transfer the file to a directory on the database server. In the database you will define a directory and load the model to the database as explained in Chapter 3, SQL Quick Start Using a Vector Embedding Model Uploaded into the Database in the AI Vector Search User\u0026rsquo;s Guide .\nThis blogpost will be mentioned in my presentation at the DOAG 2025 Conference and at UKOUG Conference Discovery 2025 , see agenda .\n","permalink":"https://enesi.no/2025/11/oml4py-2.1-on-linux/","summary":"\u003cp\u003eThis is an update of previous post on \u003ca href=\"https://enesi.no/2024/05/oml4py-client-on-ubuntu/\"\u003eHow to install OML4Py Client on Ubuntu\u003c/a\u003e\n, this time Oracle Linux 9.\u003c/p\u003e\n\u003ch1 id=\"install-oml4py-on-oracle-linux\"\u003eInstall OML4Py on Oracle Linux\u003c/h1\u003e\n\u003cp\u003eThe goal here is to have OML4Py 2.1 available in order to export embedding models to ONNX format. To follow this procedure you need to have a free Oracle Account, you will be offered to create one when you try to access \u003ca href=\"https://edelivery.oracle.com/\" target=\"_blank\" rel=\"noopener\"\u003ehttps://edelivery.oracle.com/\u003c/a\u003e\n later.\u003c/p\u003e\n\u003cp\u003eThis installation is divided in parts:\u003c/p\u003e","title":"OML4Py 2.1 on Linux"},{"content":"Update: I wrote another post about UUID v7 with some alternative solutions.\nUUIDs are useful, especially when you expose data in REST APIs, but there are cases where you may want to stick to the good old sequence-based primary key.\nUUIDs in Oracle Database #JoelKallmanDay UUID, short for universally unique identifier, are also known as globally unique identifier (GUID). You can use the function sys_guid() in the Oracle database to generate it:\nselect sys_guid() guid from dual connect by level \u0026lt; 10; The result, however, is not what we are used to see these days:\nGUID ___________________________________ 40536A604DFD9FF2E0637700000A85F6 40536A604DFE9FF2E0637700000A85F6 40536A604DFF9FF2E0637700000A85F6 40536A604E009FF2E0637700000A85F6 40536A604E019FF2E0637700000A85F6 40536A604E029FF2E0637700000A85F6 40536A604E039FF2E0637700000A85F6 40536A604E049FF2E0637700000A85F6 40536A604E059FF2E0637700000A85F6 It generates them in a monotonic order and it is difficult to see that they are unique.\nIn 23.9 the function uuid() was introduced:\nselect uuid() uidV4 from dual connect by level \u0026lt; 10; UIDV4 ___________________________________ 7211D5EB2A8D4F2ABF97350CA430E426 E2E0F762E7E84F7BBF4D4F5275109EBA 4428304718E54F4CBFB3ECA83EA35993 FE5190FE9B584F5BBF601BBF08B08C57 5303087A00764FCDBF927F3E5AE95986 CD4202C7BD664FDABF223336A5BC2F02 51A814ABB52D4F19BF0BD938EBF05D1F 2D2571341D704F01BFF9B080F933AACC 6459A47343694F69BFA155C620E91FD7 It looks more random, but not exactly how we are used to see them displayed. Oracle added another function to format them nicely:\nselect raw_to_uuid(uuid()) uidV4_formatted from dual connect by level \u0026lt; 10; UIDV4_FORMATTED _______________________________________ b2471554-f9e1-4f26-bf2a-1d7b04ced765 d69694dd-b695-4fbf-bf8e-c4e553485b98 63c579bd-e310-4f92-bf39-4d85e0fee379 001fc2fa-02ec-4fc1-bfe2-6786606d3ea1 28bee96b-c21c-4f4d-bf01-eee5724d69af e6b2473c-9dbc-4f70-bf40-4ac033bc4af7 7f289ebd-5c21-4fbd-bf40-dfd59a0f2f8b 1375aa43-c862-4f24-bff7-8516e2567299 227c3b6c-4c5a-4f53-bf56-225483e04a9c That\u0026rsquo;s better. There is also the reverse function of this:\nselect uuid_to_raw(\u0026#39;28bee96b-c21c-4f4d-bf01-eee5724d69af\u0026#39;) uid_raw ; UID_RAW ___________________________________ 28BEE96BC21C4F4DBF01EEE5724D69AF This function returns a RAW(16) if the input passes the function is_uuid():\nselect \u0026#39;1375aa43-c862-4f24-bff7-8516e2567299\u0026#39; foo , is_uuid(\u0026#39;1375aa43-c862-4f24-bff7-8516e2567299\u0026#39;) OK union select \u0026#39;fubar\u0026#39;, is_uuid(\u0026#39;fubar\u0026#39;); FOO OK _______________________________________ ________ 1375aa43-c862-4f24-bff7-8516e2567299 true fubar false (If your client returns 1 and 0 instead of true/false, then upgrade your client. SQLcl 24.2.0 returns 1, latest version 25.2 works fine).\nSo we now have:\nuuid() returns RAW(16) raw_to_uuid() formats UUID in RAW(16) and returns VARCHAR2. uuid_to_raw() converts UUID the way we are used to see them to RAW(16), provided that the input is valid. is_uuid() returns a boolean value after validation of input. For is_uuid() and uuid_to_raw() the input may be enclosed by {} and contain hyphens, see the four different text formats in documentation .\nI think the naming of these functions are slightly confusing. uuid_raw() in stead of just uuid() would have been less confusing. Try select is_uuid(uuid()); to get an illustration1.\nMore details regarding UUIDs can be found in RFC 9562 and on Wikipedia .\nUUIDs in your API Say you have data about your large wine collection exposed through a REST API. Then you may have an endpoint that returns details for one specific wine: /wines/{wine_id} If that wine_id is your sequence based primary key, that is, an integer, people will soon figure out that they can download all your data by just increasing that number for each fetch. After a simple loop they have all your data. With an UUID they will need extreme luck to get a hit (and your monitoring system may block the client causing massive amounts of 404s).\nYou can easily add UUIDs to existing tables, or perhaps a mapping table is better. Then you can use UUIDs for publication and use the normal keys for internal SQL. I have seen in newer system that the primary and foreign keys are all based on UUIDs, but for large data sets that may have a downside. Connor McDonald looked at this in his blogpost Gooey GUIDs from 2018 (and recently updated). His concern is mostly the CPU time it takes to generate the GUIDs with data loaded in bulk (e.g. data warehouses), but I wonder if storage space may become an issue as well.\nIn a case a few weeks ago I came across tables with many million rows where all the keys were based on UUID v4. The keys were stored as formatted UUD V4 strings, not in RAW(16) as output from the old sys_guid() or the new uuid(). For simple lookup that worked great, but for reporting and other bulk operation it was not optimal. My first thought that these would add a lot of overhead on disk space, so I decided to do some testing.\nSome tests on a 100 million rows dataset I am using a dataset X-Wines 2. You can download the full dataset from their Google Drive folder . I used FREE edition version 23.9 for these tests.\nAfter load3 I have two tables, wines and ratings, where the latter has a reference to the former, but the constraints must be added after load:\nalter table wines add constraint wines_pk primary key(wineid); alter table ratings add constraint ratings_wines_fk foreign key(wineid) references wines; alter table ratings_pk add constraint ratings_pk primary key(ratingid); One index added to the rating table to see what Oracle uses and stats collected in a lazy way:\ncreate index ratings_wineid on ratings(wineid); exec dbms_stats.gather_schema_stats(user); This query finds the top 20 most rated wines (the having clause is not necessary for the result, it got in there after some evolved tests, and makes the CBO pick up the index.):\nselect winename,count(*) ratings_cnt from wines join ratings using (wineid) group by winename having count(*) \u0026gt; 100 order by 2 desc fetch first 20 rows only; Here is the explain plan for the query:\nPlan hash value: 1429720065 ---------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | ---------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | | | | 31986 (100)| | |* 1 | COUNT STOPKEY | | | | | | | | 2 | VIEW | | 3063 | 427K| | 31986 (3)| 00:00:02 | |* 3 | SORT ORDER BY STOPKEY | | 3063 | 122K| | 31986 (3)| 00:00:02 | |* 4 | HASH GROUP BY | | 3063 | 122K| | 31986 (3)| 00:00:02 | |* 5 | HASH JOIN | | 20M| 818M| 4720K| 31446 (1)| 00:00:02 | | 6 | VIEW | VW_GBF_8 | 100K| 3538K| | 1365 (1)| 00:00:01 | | 7 | TABLE ACCESS FULL | WINES | 100K| 2948K| | 1365 (1)| 00:00:01 | | 8 | INDEX FAST FULL SCAN| RATINGS_WINEID | 21M| 100M| | 12867 (1)| 00:00:01 | ---------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter(ROWNUM\u0026lt;=20) 3 - filter(ROWNUM\u0026lt;=20) 4 - filter(SUM(\u0026#34;ITEM_2\u0026#34;)\u0026gt;100) 5 - access(\u0026#34;ITEM_1\u0026#34;=\u0026#34;RATINGS\u0026#34;.\u0026#34;WINEID\u0026#34;) In order to analyze the case where the UUIDs are stored as formatted string I decided to create two new tables (and a temporary one) with UUIDs V4 stored as varchar2(36):\n-- drop table wines_guid_tmp purge ; create table wines_guid_tmp as SELECT raw_to_uuid(uuid()) wine_guid, wineid, WINENAME, TYPE, ELABORATE, GRAPES, HARMONIZE, ABV, BODY, ACIDITY, CODE, COUNTRY, REGIONID, REGIONNAME, WINERYID, WINERYNAME, WEBSITE, VINTAGES FROM WINES; -- drop table ratings_guid purge; create table ratings_guid as SELECT raw_to_uuid(uuid()) rating_guid, USERID, w.wine_guid, VINTAGE, RATING, DATE$ FROM RATINGS r join wines_guid_tmp w using (wineid); -- drop table wines_guid purge ; create table wines_guid as SELECT wine_guid, WINENAME, TYPE, ELABORATE, GRAPES, HARMONIZE, ABV, BODY, ACIDITY, CODE, COUNTRY, REGIONID, REGIONNAME, WINERYID, WINERYNAME, WEBSITE, VINTAGES FROM WINES_GUID_TMP; drop table wines_guid_tmp purge; alter table wines_guid add constraint wines_guid_pk primary key (wine_guid); alter table ratings_guid add constraint ratings_guid_pk primary key (rating_guid); alter table ratings_guid add constraint ratings_guid_fk foreign key (wine_guid) references wines_guid; create index ratings_guid_wine_guid on ratings_guid(wine_guid); exec dbms_stats.gather_schema_stats(user); Repeating the query on the new tables:\nselect winename,count(*) ratings_cnt from wines_guid join ratings_guid using (wine_guid) group by winename having count(*) \u0026gt; 100 order by 2 desc fetch first 20 rows only; The plan looks familiar:\nPlan hash value: 2314587929 ------------------------------------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | | | | 89878 (100)| | |* 1 | COUNT STOPKEY | | | | | | | | 2 | VIEW | | 3063 | 427K| | 89878 (1)| 00:00:04 | |* 3 | SORT ORDER BY STOPKEY | | 3063 | 314K| | 89878 (1)| 00:00:04 | |* 4 | HASH GROUP BY | | 3063 | 314K| | 89878 (1)| 00:00:04 | |* 5 | HASH JOIN | | 21M| 2104M| 7864K| 89335 (1)| 00:00:04 | | 6 | VIEW | VW_GBF_8 | 100K| 6683K| | 1510 (1)| 00:00:01 | | 7 | TABLE ACCESS FULL | WINES_GUID | 100K| 6093K| | 1510 (1)| 00:00:01 | | 8 | INDEX FAST FULL SCAN| RATINGS_GUID_WINE_GUID | 21M| 741M| | 38678 (1)| 00:00:02 | ------------------------------------------------------------------------------------------------------------ Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter(ROWNUM\u0026lt;=20) 3 - filter(ROWNUM\u0026lt;=20) 4 - filter(SUM(\u0026#34;ITEM_2\u0026#34;)\u0026gt;100) 5 - access(\u0026#34;ITEM_1\u0026#34;=\u0026#34;RATINGS_GUID\u0026#34;.\u0026#34;WINE_GUID\u0026#34;) The response doesn\u0026rsquo;t look bad, 3.69 seconds compared to 3.02 seconds for the first query, but it is a 21% increase in response time (but perhaps good enough?). Anyway, have a look at line 8 in the execution plans, the amount of data read for the INDEX FAST FULL SCAN operation was many times higher for the last query; 741M vs 100M. You can get an idea when you look at the size of the indexes on disk:\nselect segment_name,sum(bytes)/1e6 mb from user_segments where segment_name in (\u0026#39;RATINGS_WINEID\u0026#39;,\u0026#39;RATINGS_GUID_WINE_GUID\u0026#39;) group by segment_name; SEGMENT_NAME MB _________________________ _____________ RATINGS_WINEID 392.167424 RATINGS_GUID_WINE_GUID 1179.648 With the HAVING clause removed the plans change to:\nPlan hash value: 3266154295 --------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | --------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | | | | 56823 (100)| | |* 1 | COUNT STOPKEY | | | | | | | | 2 | VIEW | | 61248 | 8553K| | 56823 (2)| 00:00:03 | |* 3 | SORT ORDER BY STOPKEY | | 61248 | 2871K| 5568K| 56823 (2)| 00:00:03 | | 4 | HASH GROUP BY | | 61248 | 2871K| 5568K| 56823 (2)| 00:00:03 | |* 5 | HASH JOIN | | 100K| 4717K| 2968K| 54129 (2)| 00:00:03 | | 6 | VIEW | VW_GBF_8 | 101K| 1776K| | 52418 (2)| 00:00:03 | | 7 | HASH GROUP BY | | 101K| 493K| 241M| 52418 (2)| 00:00:03 | | 8 | TABLE ACCESS FULL| RATINGS | 21M| 100M| | 29368 (1)| 00:00:02 | | 9 | TABLE ACCESS FULL | WINES | 100K| 2948K| | 1365 (1)| 00:00:01 | --------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter(ROWNUM\u0026lt;=20) 3 - filter(ROWNUM\u0026lt;=20) 5 - access(\u0026#34;WINES\u0026#34;.\u0026#34;WINEID\u0026#34;=\u0026#34;ITEM_1\u0026#34;) and\nPlan hash value: 2283342047 ------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | | | | 159K(100)| | |* 1 | COUNT STOPKEY | | | | | | | | 2 | VIEW | | 61248 | 8553K| | 159K (1)| 00:00:07 | |* 3 | SORT ORDER BY STOPKEY | | 61248 | 6699K| 11M| 159K (1)| 00:00:07 | | 4 | HASH GROUP BY | | 61248 | 6699K| 11M| 159K (1)| 00:00:07 | |* 5 | HASH JOIN | | 100K| 10M| 6080K| 153K (1)| 00:00:07 | | 6 | VIEW | VW_GBF_8 | 100K| 4896K| | 151K (1)| 00:00:06 | | 7 | HASH GROUP BY | | 100K| 3623K| 887M| 151K (1)| 00:00:06 | | 8 | TABLE ACCESS FULL| RATINGS_GUID | 21M| 741M| | 79377 (1)| 00:00:04 | | 9 | TABLE ACCESS FULL | WINES_GUID | 100K| 6093K| | 1510 (1)| 00:00:01 | ------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter(ROWNUM\u0026lt;=20) 3 - filter(ROWNUM\u0026lt;=20) 5 - access(\u0026#34;WINES_GUID\u0026#34;.\u0026#34;WINE_GUID\u0026#34;=\u0026#34;ITEM_1\u0026#34;) The response time was around 50% worse for the second plan (The Time column in the plan is not reliable, the timings I used here come from set timing on in SQLcl.)\nIf you need to use GUIDs as keys, then storing them as RAW(16) will give you better numbers (Repeat the test with uuid() only). In a new project, I would ask if we can use normal system generated IDs and have the UUID V4 stored as RAW(16) in an extra column. That would add to the total space allocated on disk, but certainly worth it if performance becomes an issue as the data volume increases.\nIt won\u0026rsquo;t even run, the outer function expects text, but the inner function returns RAW/BINARY and the database complains with ORA-22849: Type BINARY is not supported for this function or operator\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nde Azambuja, R.X.; Morais, A.J.; Filipe, V. X-Wines: A Wine Dataset for Recommender Systems and Machine Learning. Big Data Cogn. Comput. 2023, 7, 20. https://doi.org/10.3390/bdcc7010020 \u0026#160;\u0026#x21a9;\u0026#xfe0e;\nA few tips to load this dataset. Use SQLcl, but create the table without loading the data first: load wines XWines_Full_100K_wines.csv create Then modify columns that are too narrow since SQLcl doesn\u0026rsquo;t scan more than 5000 rows. (Enhancement request: allow us to scan much more than the upper limit of 5000 rows in set load scan_rows 5000): alter table wines modify (wineid number(10), winename varchar(300)); Then load the data with load wines XWines_Full_100K_wines.csv;. Likeways for the ratings: load ratings XWines_Full_21M_ratings.csv create ; alter table ratings modify ratingid number(10); load ratings XWines_Full_21M_ratings.csv ;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2025/10/uuids-in-oracle-database/","summary":"\u003cp\u003e\u003cstrong\u003eUpdate\u003c/strong\u003e: I wrote \u003ca href=\"https://enesi.no/2025/12/uuid-v7-in-oracle-database/\"\u003eanother post \u003c/a\u003e\n about UUID v7 with some alternative solutions.\u003c/p\u003e\n\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/guids_in_database.png\"/\u003e \n\u003c/figure\u003e\n\n\u003cp\u003eUUIDs are useful, especially when you expose data in REST APIs, but there are cases where you may want to stick to the good old sequence-based primary key.\u003c/p\u003e\n\u003ch1 id=\"uuids-in-oracle-database-joelkallmanday\"\u003eUUIDs in Oracle Database #JoelKallmanDay\u003c/h1\u003e\n\u003cp\u003eUUID, short for \u003cem\u003euniversally unique identifier\u003c/em\u003e, are also known as \u003cem\u003eglobally unique identifier\u003c/em\u003e (GUID). You can use the function \u003ccode\u003esys_guid()\u003c/code\u003e in the Oracle database to generate it:\u003c/p\u003e","title":"UUIDs in Oracle Database"},{"content":" Foursquare open sourced over 104 million points of interest (POIs) collected over several years. This is a large dataset perfect for testing and learning more about Spatial in the database. Oracle APEX has what you need to explore and display these places in your own browser.\nMeet Foursquare OS Places Foursquare used to be very popular. People where checking in everywhere with their mobile phones. In November last year Foursquare made these POIs available for download. You can read more about the dataset here, and you can explore the places with Foursquare Studio where the graphics above is taken from.\nThe files are delivered in Parquet format from an S3 bucket at AWS. They are also available from Huggingface and Snowflake.\nIn Oracle Autonomous Database you can read parquet files via an external table, created with the DBMS_CLOUD package, example below.1\nYou can access the files directly from the S3 bucket, but I thought downloading them first is a good idea to avoid unnecessary load on their infrastructure. You need the aws client and an AWS account to download, look at the userguide on how to install the client. Foursquare has published updated versions of these files, in this post I am using the parquet files at s3://fsq-os-places-us-east-1/release/dt=2025-02-06/places/parquet/, the categories are stored in one file located at s3://fsq-os-places-us-east-1/release/dt=2025-02-06/categories/parquet/\nThis is how you list the files in the S3 bucket with the aws cli:\naws s3 ls s3://fsq-os-places-us-east-1/release/dt=2025-02-06/places/parquet/ 2025-02-06 17:40:35 110079156 places-00000.zstd.parquet 2025-02-06 17:40:35 117677765 places-00001.zstd.parquet : : 2025-02-06 17:41:47 108197266 places-00085.zstd.parquet 2025-02-06 17:41:59 122402515 places-00098.zstd.parquet 2025-02-06 17:42:00 108789011 places-00099.zstd.parquet External table on Parquet files I downloaded the files and then uploaded them to a bucket in OCI object storage. You can read more about object storage in this blogpost .\nWith all the files in one bucket in OCI I created an external table like this:\nbegin dbms_cloud.create_external_table ( table_name =\u0026gt;\u0026#39;foursquare_categories\u0026#39;, credential_name =\u0026gt;\u0026#39;OCI_CRED\u0026#39;, file_uri_list =\u0026gt;\u0026#39;https://frgurbagurbavdn.objectstorage.eu-frankfurt-1.oci.customer-oci.com/n/frgurbagurbavdn/b/Foursquare/o/categories.zstd.parquet\u0026#39;, format =\u0026gt; \u0026#39;{\u0026#34;type\u0026#34;: \u0026#34;parquet\u0026#34;, \u0026#34;schema\u0026#34;: \u0026#34;first\u0026#34;}\u0026#39; ); end; / The table above has all the categories. The POIs resides in 100 parquet files and can be accessed in one external table with this syntax, note the use of * to include all the files:\nbegin dbms_cloud.create_external_table ( table_name =\u0026gt;\u0026#39;foursquare\u0026#39;, credential_name =\u0026gt;\u0026#39;OCI_CRED\u0026#39;, file_uri_list =\u0026gt;\u0026#39;https://frgurbagurbavdn.objectstorage.eu-frankfurt-1.oci.customer-oci.com/n/frgurbagurbavdn/b/Foursquare/o/places-*.zstd.parquet\u0026#39;, format =\u0026gt; \u0026#39;{\u0026#34;type\u0026#34;: \u0026#34;parquet\u0026#34;, \u0026#34;schema\u0026#34;: \u0026#34;first\u0026#34;}\u0026#39; ); end; / I created the credential used above with the dbms_cloud.create_credential procedure, look at the post Use OCI object storage from PL/SQL for an explanation.\nThe main table has 26 columns including a reference to the categories table:\nNow with this in place I can look for places around me with a query like this:\nselect * from foursquare where country=\u0026#39;NO\u0026#39; and latitude is not null and longitude is not null and sdo_within_distance( sdo_geometry(2001,8307,sdo_point_type(longitude,latitude,null),null,null), sdo_geometry(2001,8307,sdo_point_type(8.758214057686658,58.4493290890655,null),null,null), \u0026#39;distance=1000\u0026#39;) = \u0026#39;TRUE\u0026#39;; But that is not really\u0026hellip; fast.\nA note on coordinate systems As is common with positions collected with GPS, Foursquare uses WGS84 (Wikipedia ). When you define a column with datatype SDO_GEOMETRY you usually specify the coordinate system aka spatial reference system with a specific SRID. It is a study in itself (and it might cure your insomnia) with a large number of different systems around the world2. Originally WGS84 was defined by EPSG with SRID 4326 (see Wikipedia ) and coordinates order (latitude,longitude), but later some computer scientists came along and preferred the order that reminded them about (x,y)3. Since early days, Oracle has used SRID 8307 for \u0026ldquo;Longitude / Latitude (WGS 84)\u0026rdquo;, but to add to the confusion they later added SRID 4326, and in the Oracle Spatial manual, section 6.2.4, an example has this comment beside SRID 4326: \u0026ldquo;\u0026ndash; SRID for WGS 84 longitude/latitude\u0026rdquo;. According to some sources SRID 4326 in Oracle is \u0026ldquo;EPSG SRID 4326 with axes reversed\u0026rdquo; (see this thread on Oracle Forums for an interesting discussion.) It\u0026rsquo;s like chili con carne for vegetarians (chili con carne sin carne?) I won\u0026rsquo;t say the Oracle Spatial manual is hysterical clear on these matters.\nOK, I may have gotten the history wrong here, I promise to update this if my friend in Hamburg corrects me.\nThe order longitude/latitude is what is used in many web applications and databases today, including APEX (more on this later). If your house is displayed in the middle of the ocean somewhere it is a sign that you inverted the order. (I found mine in Turkey the first time I tried maps in APEX.) I have decided to use SRID 8307 in order to not contribute more to the confusion, and since the reference table SDO_COORD_REF_SYS gives it the name Longitude / Latitude (WGS 84).\nBring it in Lets create a normal table so we can add a spatial index later:\ncreate table foursquare_geom as select f.*, sdo_geometry(2001 ,8307 ,sdo_point_type(f.longitude,f.latitude,null),null,null) geometry from foursquare f; The number 2001 is the kind of geometry (SDO_GTYPE) which is a point in two dimensions. I could have used the constant SDO_POINT2D, see Spatial Developer\u0026rsquo;s Guide, 2.2.1. Section 2.2 goes into detail about the SDO_GEOMETRY type.\nThis is just one way to create the spatial datatype SDO_GEOMETRY. The column GEOM in the table has the position in WKB format (see section 2.4 in the manual), and you can achieve the same result with:\ncreate table foursquare_geometry as select f.*, sdo_geometry(geom,8307) geometry from foursquare f; Spatial metadata Spatial is special and needs some metadata for the spatial column. This means updating a view USER_SDO_GEOM_METADATA (some triggers take care of updating the underlying tables). You either do this in SQL Developer as shown below:\nor you can just write the insert like this:\nInsert into user_sdo_geom_metadata (TABLE_NAME,COLUMN_NAME,DIMINFO,SRID) values (\u0026#39;FOURSQUARE_GEOM\u0026#39;,\u0026#39;GEOMETRY\u0026#39; ,MDSYS.SDO_DIM_ARRAY( MDSYS.SDO_DIM_ELEMENT(\u0026#39;longitude\u0026#39;, -180, 180, 0.000001) , MDSYS.SDO_DIM_ELEMENT(\u0026#39;latitude\u0026#39;, -90, 90, 0.000001)) ,8307); Depending on your patience, you may experiment with different levels of tolerance (0.000001 above.)\nSpatial index With the metadata in place you can create the index:\nCREATE INDEX GEOM_SI ON FOURSQUARE_GEOM(GEOMETRY) INDEXTYPE IS MDSYS.SPATIAL_INDEX_V2; Note the indextype. The indextype SPATIAL_INDEX_V2 is system-managed and Oracle recommends it over the previous type.\nA simplified version of the query I ran on the external table, but on the internal table with a spatial index:\nselect * from foursquare_geom where sdo_within_distance( geometry, sdo_geometry(2001,8307,sdo_point_type(8.75821,58.44932,\tnull),null,null), \u0026#39;distance=1000\u0026#39;) = \u0026#39;TRUE\u0026#39;; returns in \u0026ldquo;no time\u0026rdquo;, even without partitioning in place.\nMaps in APEX Creating maps in APEX is easy with the map region introduced some versions ago. But with so many points presenting them can be a challenge (there are over 160 in a radius of 1000 meter from me). First attempt just showed a completely overcrowded map. But the layer type Heat Map works great as you zoom out (up to a level):\nAnother layer of type Points is set to be visible for specific zoom levels where it makes sense to show individual points.\nVisually inspection of these points shows that some points are inaccurate (or the guy who went to Øyvinds pub4 was walking on water when he got the location fix.)\nFurther explorations I had some fun with simple analysis of the data, like how many points where created each year. The first was created at the end of 2003, and a simple group by query shows you the popularity over time (it has been declining).\nI did not go much into the details about the datatypes and the data model in Oracle Spatial. It takes a bit to get used to, but makes really sense when you work with it. There are plenty of features I want to explore further like Spatial Analysis and Mining (chapter 9 in the manual).\nAlso I did a minimal conversion when I created the internal table, e.g. the date columns should be converted with to_date(), and there is no need to preserve the BLOB column. The data set is also perfect for experimenting with partitioning.\nFor more details on parquet files and Oracle, see the article Working with Parquet files with Oracle Database. \u0026#160;\u0026#x21a9;\u0026#xfe0e;\nHave a look at this: select * from SDO_COORD_REF_SYS; Currently 6194 entries.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThe JSON document in the BBOX column was probably not defined by someone who worries about correct naming, example: {\u0026quot;xmin\u0026quot;:-121.75161480903625,\u0026quot;ymin\u0026quot;:38.539413226528,\u0026quot;xmax\u0026quot;:-121.75161480903625,\u0026quot;ymax\u0026quot;:38.539413226528}\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nYes, there was a pub with that name in Arendal, I went there a few years ago, I did not ask for a discount. Current status unknown.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2025/02/104-million-places-from-foursquare-in-oracle/","summary":"\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/Foursquareplaces.jpg\"/\u003e \n\u003c/figure\u003e\n\n\u003cp\u003eFoursquare open sourced over 104 million points of interest (POIs) collected over several years. This is a large dataset perfect for testing and learning more about Spatial in the database. Oracle APEX has what you need to explore and display these places in your own browser.\u003c/p\u003e\n\u003ch2 id=\"meet-foursquare-os-places\"\u003eMeet Foursquare OS Places\u003c/h2\u003e\n\u003cp\u003eFoursquare used to be very popular. People where checking in everywhere with their mobile phones. In November last year Foursquare made these POIs available for download. You can read more about \u003ca href=\"https://opensource.foursquare.com/os-places/\" target=\"_blank\" rel=\"noopener\"\u003ethe dataset here,\u003c/a\u003e\n and you can explore the places with \u003ca href=\"https://studio.foursquare.com/map/public/32a54c6d-81fb-47b6-a9c7-12fdea15f9a0\" target=\"_blank\" rel=\"noopener\"\u003eFoursquare Studio\u003c/a\u003e\n where the graphics above is taken from.\u003c/p\u003e","title":"104 million places from Foursquare in Oracle"},{"content":" Embed data in Python and store the vector in Oracle 23ai I wrote about Vector search in Oracle 23ai in Oracle Database 23ai and Vector Search. Part of the process was embedding chunks of data in the database and add the embeddings as a VECTOR data type in a new table. In this part I use Python to do the embedding outside the database and store the result as a VECTOR in the database using the oracledb module. The Python code does not call any external API, but performs the embedding on my computer using downloaded models from HuggingFace. Doing the embedding outside the database has some advantages:\nLoading of these models using DBMS_VECTOR.LOAD_ONNX_MODEL is not currently supported in FREE-version on Mac with Apple chip (it was not supported in 23.5, but now it fails with an internal error in 23.6). Embedding is compute-intensive, running outside the database reduces CPU load on the database server. Newer hardware or access to GPUs makes a big difference if your software supports it (like PyTorch with CUDA support). The FREE version of Oracle 23ai database will only use two CPUs. You can implement parallel processing according to your needs. I don\u0026rsquo;t have to load the ONNX model into the database for embedding. Regarding the last point, one disadvantage of not having the model in the database is that I cannot easily embed query text directly from SQL with the VECTOR_EMBEDDING function, and I would have to use some service external to the database for that. For a limited amount of text the cost using an external API should not be much. In my lab I have the same model loaded into the database as I use from Python.\nWhat follows is two ways of doing the embedding in Python:\nUsing LLM from https://llm.datasette.io/ Sentence Transformers from https://www.sbert.net/ Wine Reviews For this test I am using the a Wine Tasting dataset available at Kaggle. It has about 130.000 reviews of wine and is useful for testing vector search on the review texts combined with a relational query on the other columns (country, region, price, points, etc). The dataset is a bit old, the youngest wine is from 20171.\nLoading the CSV file In case you want to do the embedding in the database, or just have a look at the dataset the way it is, you can load the data from SQLcl first. For the external embedding in Python you can load the data and embed in one go, see further down.\nI started with the file winemag-data-130k-v2.csv from the zip file and loaded it with SQL Developer command line version, SQLcl, Download from oracle.com. I wrote about loading CSV files with SQLcl in Practical Guide to Load Large CSV Files to Oracle. Being practical means looking at the file and find an easy way to get the job done (loading the file). In this case some of the review texts have line endings as carriage return and new line (\\r\\n in Unix lingo), so I decided to add the | character at the end of each line that marked the end of each row with this one-liner in Perl:\nperl -pe 's/([^\\r])\\n/$1|\\n/g' winemag-data-130k-v2.csv \u0026gt; winereviews.csv Since the default for NLS_NUMERIC_CHARACTERS in my region is ,. I changed it for the session with:\nalter session set nls_numeric_characters='.,'; You probably won\u0026rsquo;t have to do that.\nThese commands change the end of line terminator to | and bumps up the number of rows to be scanned before creating the table:\nset loadformat csv term | set load scan 5000 Then I loaded it with:\nload winereviews winereviews.csv new Two rows failed (wrong format in file) which is acceptable in this test case.\nThe review for each wine is included in the description field. The longest has 820 characters. I plan to map each of the descriptions as one point in the 384 dimensional space. In other words, I am not going to slice the text into chunks as I did in the previous blogpost on the subject. From the description of the model at Huggingface input text longer than 256 word pieces will be truncated, but all reviews are inside this limit.2 That is why I add one column to the table for the embedding.\nWith the table in place I can add a new column for the embedding vector for each review3:\nalter table winereviews add review_embedding vector(384,FLOAT32); Why did I use 384 and FLOAT32? Have a look at the model card at Hugging Face.\nFor instructions on how to do the embedding in the database see my previous post about that. If you are a DBA you may worry about row-chaining, well, you can move the table after load.\nEmbedding from Python There are a few ways to do this, I have tried out a couple in Python.\nInstall LLM I have played with the CLI utility and Python library LLM from Simon Willison. This utility lets you interact with LLM models on your own machine, or if you prefer use external APIs from OpenAI, Mistral, Anthropic and Google. See page on Github and his LLM page for more information on this.\nI have Python 3.10.12 installed. Since I will insert the embeddings to Oracle I will also install the oracledb module as well. Setting up a virtual environment and install the modules:\npython3 -m venv venv . venv/bin/activate pip install --upgrade pip pip install llm oracledb The utility comes with a lot of features worth checking out, but here I only want to write simple code to do the embedding with models available here. You can check the models available out of the box with:\nllm models Only models from OpenAI are listed, so we\u0026rsquo;ll install the gpt4all plugin:\nllm install llm-gpt4all (For more information on GPT4All, see https://www.nomic.ai/gpt4all ). The models command now shows a much longer list, here is an excerpt of the first models from gpt4all.\ngpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM gpt4all: all-MiniLM-L6-v2 - SBert, 43.82MB download, needs 1GB RAM gpt4all: nomic-embed-text-v1 - Nomic Embed Text v1, 261.58MB download, needs 1GB RAM gpt4all: nomic-embed-text-v1 - Nomic Embed Text v1.5, 261.58MB download, needs 1GB RAM gpt4all: Llama-3 - Llama 3.2 1B Instruct, 737.21MB download, needs 2GB RAM gpt4all: qwen2-1_5b-instruct-q4_0 - Qwen2-1.5B-Instruct, 894.10MB download, needs 4GB RAM gpt4all: Llama-3 - Llama 3.2 3B Instruct, 1.79GB download, needs 4GB RAM We need another plugin as well in order to embed using local models:\nllm install llm-sentence-transformers Finally we need to install one embedding model:\nllm sentence-transformers register all-MiniLM-L6-v2 Load the wine reviews as JSON The zip file from Kaggle also has a file with the same dataset in JSON format. As I worked with the Python code I figured I could just insert the JSON from Python directly to the database since Oracle has good support for JSON. Loading the file into Python is easy with:\nimport json with open(\u0026#34;winemag-data-130k-v2.json\u0026#34;) as f: data = json.load(f) This is how I connect to the database and test the connection. Username, password and connection string (e.g. localhost/freepdb1) comes from the environment:\nimport oracledb import os un = os.environ.get(\u0026#39;PYTHON_USERNAME\u0026#39;) pw = os.environ.get(\u0026#39;PYTHON_PASSWORD\u0026#39;) cs = os.environ.get(\u0026#39;PYTHON_CONNECTSTRING\u0026#39;) with oracledb.connect(user=un, password=pw, dsn=cs) as connection: with connection.cursor() as cursor: sql = \u0026#34;select sysdate from dual\u0026#34; for r in cursor.execute(sql): print(r) Code to load JSON into table (simple, but not fastest):\nimport time connection = oracledb.connect(user=un, password=pw, dsn=cs) cursor = connection.cursor() started=time.time() for i in range(len(data)): cursor.execute(\u0026#34;insert into winereviews_stage(id,wine_data) values(:1,:2)\u0026#34;, [i,json.dumps(data[i])]) connection.commit() print(\u0026#34;Elapsed: {}\u0026#34;.format(str(time.time()-started))) Elapsed: 36.504868030548096 I cannot let the opportunity pass to show the speed-up with bulk-loading (See Chapter 8 Executing Batch Statements and Bulk Loading in the oracledb documentation.)\nWith bulkload:\nstarted=time.time() bulk_data=[(i,json.dumps(data[i])) for i in range(len(data))] cursor.executemany(\u0026#34;insert into winereviews_stage(id,wine_data) values(:1,:2)\u0026#34;, bulk_data) connection.commit() print(\u0026#34;Elapsed: {}\u0026#34;.format(str(time.time()-started))) Elapsed: 2.8074018955230713 The bulk loading takes less than 10% of the time (This with 23ai FREE version and Python both running locally on a Mac mini M1).\nOK, back to the main subject.\nThe staging table Here is the SQL to create the staging table with a column for the embedding vector:\ncreate table winereviews_stage( id integer constraint winereviews_stage_pk primary key, wine_data json, description_embedding vector(384,float32)); Embed with LLM from datasette.io This code is all it takes to embed and store the JSON together with an embedding of the description column into the database in one go:\nimport json import oracledb import os import llm import time import array un = os.environ.get(\u0026#39;PYTHON_USERNAME\u0026#39;) pw = os.environ.get(\u0026#39;PYTHON_PASSWORD\u0026#39;) cs = os.environ.get(\u0026#39;PYTHON_CONNECTSTRING\u0026#39;) # dbhost.example.com/orclpdb with open(\u0026#34;winemag-data-130k-v2.json\u0026#34;) as f: data = json.load(f) # Load the model embedding_model = llm.get_embedding_model(\u0026#34;sentence-transformers/all-MiniLM-L6-v2\u0026#34;) # Create connection connection = oracledb.connect(user=un, password=pw, dsn=cs) # Create cursor cursor = connection.cursor() started=time.time() # Use embed() from model to embed each description # embed() returns list, convert it to array of floats # List comprehension to create a list of tuples bulk_data=[(i,json.dumps(data[i]), array.array(\u0026#39;f\u0026#39;,embedding_model.embed(data[i][\u0026#34;description\u0026#34;]))) for i in range(len(data))] cursor.executemany(\u0026#34;\u0026#34;\u0026#34; insert into winereviews_stage(id,wine_data,DESCRIPTION_EMBEDDING) values(:1,:2,:3) \u0026#34;\u0026#34;\u0026#34;,bulk_data) connection.commit() print(\u0026#34;Elapsed: {}\u0026#34;.format(str(time.time()-started))) Only two methods are used from the LLM module; llm.get_embedding_model() loads the model into an object called embedding_model and the embed() embeds a string.\nEmbedding with Sentence Transformers from sbert.net As an alternative, and perhaps slightly easier to install, we can use Sentence Transformers a.k.a. SBERT. They have a large collection of pre-trained models and have their own Python package for embedding.\nCreate a new directory somewhere and a new virtual environment with required packages:\npython3 -m venv venv . venv/bin/activate pip install --upgrade pip pip install oracledb sentence-transformers The code does not change much:\nimport json import oracledb import os import time import array from sentence_transformers import SentenceTransformer model = SentenceTransformer(\u0026#34;all-MiniLM-L6-v2\u0026#34;) un = os.environ.get(\u0026#39;PYTHON_USERNAME\u0026#39;) pw = os.environ.get(\u0026#39;PYTHON_PASSWORD\u0026#39;) cs = os.environ.get(\u0026#39;PYTHON_CONNECTSTRING\u0026#39;) # dbhost.example.com/orclpdb with open(\u0026#34;winemag-data-130k-v2.json\u0026#34;) as f: data = json.load(f) connection = oracledb.connect(user=un, password=pw, dsn=cs) cursor = connection.cursor() print(\u0026#34;Truncating table\u0026#34;) cursor.execute(\u0026#34;truncate table winereviews_stage\u0026#34;) print(\u0026#34;Bulk insert\u0026#34;) started=time.time() bulk_data=[(i,json.dumps(data[i]), array.array(\u0026#39;f\u0026#39;,model.encode(data[i][\u0026#34;description\u0026#34;]))) for i in range(len(data))] cursor.executemany(\u0026#34;insert into winereviews_stage(id,wine_data,DESCRIPTION_EMBEDDING) values(:1,:2,:3)\u0026#34;,bulk_data) connection.commit() print(\u0026#34;Elapsed: {}\u0026#34;.format(str(time.time()-started))) In short, one line to load the model and the method encode() to embed the string. The encode() returns a numpy.ndarray type, but the conversion to an array uses the same constructor from the array package.\nAbout inserting vectors with oracledb The documentation for the oracledb package is quite good and explains how to store VECTOR values from Python, see Chapter 14 in the python-oracledb documentation. Relational data with JSON_TABLE We can convert the JSON stuff in the WINE_DATA field with the JSON_TABLE function in SQL, and create a new table that includes the vector column that was populated from Python:\ncreate table winereviews_embedded as select jt.*, w.description_embedding from winereviews_stage w, json_table(w.wine_data columns title, country,region_1,region_2,designation,price,points,variety, taster_name,description) jt; but this creates all columns with VARCHAR2(4000). This adds the types I want:\ncreate table winereviews_embedded as select jt.*, w.description_embedding from winereviews_stage w, json_table( w.wine_data columns ( title varchar(200), country varchar2(50), region_1 varchar2(50), region_2 varchar2(50), designation varchar2(50), price number, points number, variety varchar2(50), taster_name varchar2(100), description varchar(2000))) jt; NESTED is a shortcut that works in this case:\ncreate table winereviews_embedded as select jt.*, w.description_embedding from winereviews_stage w NESTED wine_data columns ( title varchar(200), country varchar2(50), region_1 varchar2(50), region_2 varchar2(50), designation varchar2(50), price number, points number, variety varchar2(50), taster_name varchar2(100), description varchar(2000)) jt; More research needed I have a few ideas on what I can do with these data. After all 2017 is not that old for a good Bordeaux, perhaps I can find one of my favorites by a normal relational search, and then use vector search to identify more wine similar to my taste. Probably I should try out different models and compare the results on familiar wines before I start drinking based on the first vector search.\nComparing embedding done in the database with external embedding is also interesting. I have seen bugs earlier, regarding vector distance (like cosine distance above 1).\nThere is a lot to explore here with these arrays vectors ;-)\nSimilar posts PyTorch to vectorize images, REST APIs to POST to Oracle PyTorch and the transformers package to embed images in Python and store it Oracle through a REST API offered by ORDS. Using Cohere to Generate Vector Embeddings Same wine reviews dataset used, embedding using an API at Cohere. Using OpenAI to Generate Vector Embeddings does the same with an API at OpenAI. The dataset does not contain a separate column for production year, but since the title usually includes the vintage you can find the youngest with this SQL: select max(regexp_substr(title,'[12][0-9]{3}')) year from WINEREVIEWS where regexp_like (title,'[12][0-9]{3}').\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAccording to https://platform.openai.com/tokenizer the average for English text is about 100 tokens for 75 words. The maximum word count in a review (description) is 138: select max(REGEXP_COUNT(description,'(\\w)\\W') ) word_count from winereviews; or about 185 tokens.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nSee Embedding Vectors vs. Vector Embeddings for Tanel Poder\u0026rsquo;s take on the correct writing. I think he is right and will try to use embedding vector from now on.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2024/12/external-embedding-in-oracle-23ai/","summary":"\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/man_with_pc_in_forest.jpg\"\n         alt=\"Man with a PC in the forres\"/\u003e \n\u003c/figure\u003e\n\n\u003ch1 id=\"embed-data-in-python-and-store-the-vector-in-oracle-23ai\"\u003eEmbed data in Python and store the vector in Oracle 23ai\u003c/h1\u003e\n\u003cp\u003eI wrote about Vector search in Oracle 23ai in \u003ca href=\"https://enesi.no/2024/05/oracle-database-23ai-and-vector-search/\"\u003eOracle Database 23ai and Vector Search.\u003c/a\u003e\n Part of the process was embedding chunks of data in the database and add the embeddings as a VECTOR data type in a new table. In this part I use Python to do the embedding outside the database and store the result as a VECTOR in the database using the \u003ccode\u003eoracledb\u003c/code\u003e module. The Python code does not call any external API, but performs the embedding on my computer using downloaded models from \u003ca href=\"https://huggingface.co/\" target=\"_blank\" rel=\"noopener\"\u003eHuggingFace.\u003c/a\u003e\n\u003c/p\u003e","title":"External embedding in Oracle 23ai"},{"content":"Oracle Flashback Time Travel aka Flashback Data Archive ensures you can track changes to the data and the structure of a table.\nFlashback Time Travel #JoelKallmanDay Flashback Time Travel previously called Total Recall lets you track all changes to the data in a table and even structural changes like added or removed columns. This means you can use it in various scenarios from detecting illegal state-transitions caused by bugs in application to compliance where full control of sensitive data is required.\nYou achieve this by adding tracking to the table(s) either during creation or later with alter table and the changes to the table will be stored in what is called a flashback data archive or just flashback archive.\nSetting it up First you should set up a separate tablespace:\ncreate bigfile tablespace flashback_archive_ts datafile size 1g autoextend on next 1g; With the SYSTEM or a user with the FLASHBACK ARCHIVE ADMINISTER system privilege you create a flashback archive with:\ncreate flashback archive default flashback_archive tablespace flashback_archive_ts quota 10G retention 6 month; This will create the default flashback archive in the database (you can only have one default, but many flashback archives). In this example it limits the quota to 10GB and keep the changes for 6 months (you can specify and integer and day, month or year). You can even add another tablespace with another quota to the same flashback archive later (think cheap storage with a bigger quota).\nIn order for a user to add tracking to a table using this flashback archive it needs the object privilege FLASHBACK ARCHIVE on this:\ngrant flashback archive on flashback_archive to scott; This is how you add flashback archive to an existing table:\nalter table cards flashback archive; This assumes that there is a default flashback archive in the database. Alternatively you can specify it:\nalter table cards flashback archive flashback_archive; Likewise, you can enable this when you create the table:\ncreate table xyz ( id number, zoo varchar2(4000)) flashback archive flashback_archive; Before I show how to do something useful, I should mention how to remove this. The syntax is straight forward:\nalter table cards no flashback archive; However, a user needs the system privilege FLASHBACK ARCHIVE ADMINISTER to do this! This kind of makes sense. A developer may conclude that a table should have flashback data archive enabled, but should not be able to remove it (temporarily). So we have some separation of duties here. Also, you cannot drop a table (and perform a few other DDL operations) with flashback archive enabled, so think twice before you add it.\nGoing back in time You can query the table as it were in the past by adding the as of timestamp clause, like this:\nselect count(*) from cards as of timestamp sysdate - 2/24; Note, you can actually do this without a flashback archive as long as you have enough undo data available in the database (check undo_retention parameter, maybe you should incease it to cover a whole business day if you have enough space), but with flashback data archive you can go back as much as specified by the retention in the flashback archive, or to the point in time it was enabled for the table. Also, a flashback query on a table that has been changed with an alter table statement at a later point will not work without a flashback archive.\nLooking at different versions of a row This is really useful when you need to look into unexpected data changes. Suppose something strange happened to a credit card, this fetches all versions of a row with one specific card_id the last 24 hours:\nselect card_number,customer_id,VERSIONS_OPERATION,VERSIONS_STARTTIME,VERSIONS_XID from card_details versions between timestamp sysdate - 1 and sysdate where card_id=1424242 order by versions_starttime; CARD_NUMBER CUSTOMER_ID VERSIONS_OPERATION VERSIONS_STARTTIME VERSIONS_XID ______________ ______________ _____________________ __________________________________ ___________________ 1136648200 865426 U 06-APR-24 07.03.07.000000000 AM 080021008B030000 1136648200 865426 U 06-APR-24 07.19.09.000000000 AM 05000D0061030000 1136648200 865427 U 06-APR-24 07.56.09.000000000 AM 020004001D030000 1136648200 865428 U 06-APR-24 08.08.06.000000000 AM 090015005F030000 1136648200 42 U 06-APR-24 05.01.11.000000000 PM 090009006C030000 1136648200 865426 A version query like this gives you new pseudocolumns:\nVERSIONS_OPERATION will show I, U, or D for Insert, Update, or Delete VERSIONS_STARTTIME will show when this version was created/started. VERSIONS_XID is the transaction id The transaction id (VERSIONS_XID) is what you need in order to investigate further in the audit trail.\nIf you have traditional audit enabled on a table this will show who did it:\nselect timestamp,os_username,userhost,username from user_audit_object where transactionid = hextoraw(\u0026#39;090009006C030000\u0026#39;); (You don\u0026rsquo;t have to use the hextoraw function above, but with it an index will be used and the query should run faster.)\nIn the new unified audit the query is:\nselect os_username,userhost,EVENT_TIMESTAMP,sql_text from unified_audit_trail where transaction_id = hextoraw(\u0026#39;090009006C030000\u0026#39;); Detecting illegal state changes Status columns are common to indicate the state of a process or some lifecycle flow. I have been asked to investigate why some rows have been seemingly reversed in a process. We could have solved that with just adding a trigger that aborts illegal state changes, but using flashback data archive we reported on the unexpected change and investigated further.\nSuppose you have a credit card that was cancelled by the owner, but then a rogue operator enabled it later:\nselect * from ( select card_id,is_valid, lag(is_valid) over (partition by card_id order by versions_starttime) prev_is_valid, versions_starttime, versions_operation from card_details versions between timestamp sysdate - 1 and sysdate ) where is_valid =\u0026#39;Y\u0026#39; and prev_is_valid=\u0026#39;N\u0026#39;; This rather simple analytical query in the inner query adds a column showing previous value for a column, the outer query filters on transitions that are not OK. With the pseudocolumn you can investigate further on what happened.\nDocumentation Flashback technology is documented in chapter 22 in the Database Development Guide (19c) . It has more details regarding the syntax, options and limitations. The SQL syntax is documented in the SQL Language Reference .\nConclusion I just touched a few areas where a flashback data archive is useful. It goes without saying that you need to configure this before you need it. It is not something you would enable for all tables, but for a few critical tables or temporarily in order to investigate a bug.\nFlashback Archive is just one of a many flashback technologies. I gave a presentation on all flashback technologies at POUG2024 last week. You should absolutely check out that conference for next year. The POUG conference is probably the best Oracle user group conference that exists. Try to get in touch with me if you want a copy of the presentation, the PDF-version has more details and code examples.\n","permalink":"https://enesi.no/2024/10/flashback-time-travel/","summary":"\u003cp\u003eOracle Flashback Time Travel aka Flashback Data Archive ensures you can track changes to the data and the structure of a table.\u003c/p\u003e\n\u003ch1 id=\"flashback-time-travel-joelkallmanday\"\u003eFlashback Time Travel #JoelKallmanDay\u003c/h1\u003e\n\u003cp\u003eFlashback Time Travel previously called \u003cem\u003eTotal Recall\u003c/em\u003e lets you track all changes to the data in a table and even structural changes like added or removed columns. This means you can use it in various scenarios from detecting illegal state-transitions caused by bugs in application to compliance where full control of sensitive data is required.\u003c/p\u003e","title":"Flashback Time Travel"},{"content":"Extend the boot device on OCI compute VM with Oracle Linux 9 I needed more space in my boot volume on a compute instance. The VM is using Oracle Linux 9.\nThis involves three steps:\nIncrease the boot volume from the cloud.oracle.com page Make the OS detect this change Grow the file system Increase the boot volume Log into your cloud account and find your compute instance from Compute \u0026#x27a1;\u0026#xfe0f; Instances \u0026#x27a1;\u0026#xfe0f; your instance from the list. In the menu on the left side called Resources click on the Boot volume link. Then click on the link for the boot volume. Then finally click the hot Edit button, and set the new volume size, and hit save. There is some information displayed along the way that may be useful.\nMake the OS detect the change In my case the documentation for Rescanning the Disk for a Block Volume or Boot Volume regarding step 2 did not work for me.\nInstead of running this command:\necho \u0026quot;1\u0026quot; | sudo tee /sys/class/block/\u0026lt;device_name\u0026gt;/device/rescan I ran this (as root, of course):\necho 1 \u0026gt; /sys/class/scsi_disk/2\\:0\\:0\\:1/device/rescan In your case the exact path may be different, there was only one alternative on my machine.\nThe first command from the manual will not work since there is no directory called device under /sys/class/block/dm-0.\nBy the way, lsblk is useful to see how these devices are connected (I removed some lines from the output):\n[root@ocilab ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 100G 0 disk ├─sda1 8:1 0 100M 0 part /boot/efi ├─sda2 8:2 0 2G 0 part /boot └─sda3 8:3 0 44.5G 0 part ├─ocivolume-root 252:0 0 29.5G 0 lvm / └─ocivolume-oled 252:1 0 15G 0 lvm /var/oled Also ls -l /dev/ocivolume shows the link to dm-0 in this case.\nGrow the filesystem Next step is to grow the filesystem, I decided to run it without -y as the manual suggests:\n[root@ocilab ~]# /usr/libexec/oci-growfs Volume Group: ocivolume Volume Path: /dev/ocivolume/root Mountpoint Data --------------- mountpoint: / source: /dev/mapper/ocivolume-root filesystem type: xfs source size: 29.4G type: lvm size: 29.5G physical devices: [\u0026#39;/dev/sda3\u0026#39;] physical volumes: [\u0026#39;/dev/sda\u0026#39;, \u0026#39;/dev/sda\u0026#39;] partition number: [\u0026#39;3\u0026#39;] volume group name: ocivolume volume group path: /dev/ocivolume/root Partition dry run expansion \u0026#34;/dev/sda3\u0026#34; succeeded. CHANGE: partition=3 start=4401152 old: size=93325312 end=97726463 new: size=205314015 end=209715166 Expanding partition /dev/sda3: Confirm? [y/N] y Partition expand expansion \u0026#34;/dev/sda3\u0026#34; succeeded. update-partition set to true FLOCK: try exec open fd 9, on failure exec exits this program FLOCK: /dev/sda: obtained exclusive lock resizing 3 on /dev/sda using resize_sfdisk_gpt 209715200 sectors of 512. total size=107374182400 bytes ## sfdisk --unit=S --dump /dev/sda label: gpt label-id: 03BB97DA-6F69-460E-A76C-82C8377A90CF device: /dev/sda unit: sectors first-lba: 34 last-lba: 97727250 sector-size: 512 /dev/sda1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=3AF7074E-C0BB-400D-8FC7-E9EC738AA53F, name=\u0026#34;EFI System Partition\u0026#34; /dev/sda2 : start= 206848, size= 4194304, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=14BE7023-6C02-4573-8891-9F639B9D936A /dev/sda3 : start= 4401152, size= 93325312, type=E6D6D379-F507-44C2-A23C-238F2A3DF928, uuid=E700F071-90A5-40BB-8132-52AF688193B7 padding 33 sectors for gpt secondary header max_end=209715166 tot=209715200 pt_end=97726463 pt_start=4401152 pt_size=93325312 resize of /dev/sda returned 0. FLOCK: /dev/sda: releasing exclusive lock CHANGED: partition=3 start=4401152 old: size=93325312 end=97726463 new: size=205314015 end=209715166 Extending /dev/sda3 succeeded. Device /dev/sda3 extended successfully. Logical volume /dev/ocivolume/root extended successfully. Successful operation is demonstrated with the command df -h.\nMy VM is now ready for more stuff, like running a container with the latest version of the FREE database as shown here ","permalink":"https://enesi.no/2024/09/extend-the-boot-device-on-oci-compute-vm-with-oracle-linux-9/","summary":"\u003ch1 id=\"extend-the-boot-device-on-oci-compute-vm-with-oracle-linux-9\"\u003eExtend the boot device on OCI compute VM with Oracle Linux 9\u003c/h1\u003e\n\u003cp\u003eI needed more space in my boot volume on a compute instance. The VM is using Oracle Linux 9.\u003c/p\u003e\n\u003cp\u003eThis involves three steps:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eIncrease the boot volume from the cloud.oracle.com page\u003c/li\u003e\n\u003cli\u003eMake the OS detect this change\u003c/li\u003e\n\u003cli\u003eGrow the file system\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"increase-the-boot-volume\"\u003eIncrease the boot volume\u003c/h2\u003e\n\u003cp\u003eLog into your cloud account and find your compute instance from \u003cem\u003eCompute\u003c/em\u003e \u0026#x27a1;\u0026#xfe0f; \u003cem\u003eInstances\u003c/em\u003e \u0026#x27a1;\u0026#xfe0f; your instance from the list. In the menu on the left side called \u003cem\u003eResources\u003c/em\u003e click on the \u003cem\u003eBoot volume\u003c/em\u003e link. Then click on the link for the boot volume. Then finally click the hot \u003cem\u003eEdit\u003c/em\u003e button, and set the new volume size, and hit save. There is some information displayed along the way that may be useful.\u003c/p\u003e","title":"Extend the boot device on OCI compute VM with Oracle Linux 9"},{"content":" Image by Mudassar Iqbal from Pixabay Sometimes you need to be practical and apply some tricks to get the job done instead of waiting for the perfect tool. With SQL Developer Command Line you can load CSV with millions of lines with not too much hassle.\nLoading large CSV files to Oracle database After a long summer break it is time to start blogging again. Today I loaded a CSV file containing all companies in Norway registered at The Brønnøysund Register Centre . It has over 1.1 million lines, more than you want to open in a GUI like SQL Developer. But SQL Developer has a command line version that is almost perfect for the job. You\u0026rsquo;ll find the download link and more information at https://www.oracle.com/database/sqldeveloper/ . This URL always gives you the latest version and from a Mac or Linux box you can just do:\ncurl -O \\ https://download.oracle.com/otn_software/java/sqldeveloper/sqlcl-latest.zip Unzip the file and start it with:\nsqlcl/bin/sql -L someuser@somehost/someservice. I forgot to mention; you need Java version 11 or higher.\nRemoving newlines SQLcl doesn\u0026rsquo;t like The file has of course comma as delimiter, and double quotes \u0026quot; as enclosure. But, in this file \u0026quot; is also used inside some of the fields. When that happens it is escaped by itself by using \u0026quot; twice. One example is the line for the company with organization id 911760347, one field contains \u0026quot;\u0026quot;\u0026quot;Knausen\u0026quot;\u0026quot; Vikan\u0026quot;. SQLcl handles that well by default setting for double in loadformat (see below).\nA more annoying problem is that some descriptive fields contains the newline character (line breaks), which SQLcl does not like (and from a short research I got the impression it is not the only tool that has this problem). If you have Oracle database installed you probably have Perl installed as well. I\u0026rsquo;ve done this on Linux and on my Mac, I\u0026rsquo;m sure you can do something similar in another language. This one-liner takes the file and replaces occurrences of newline (\\n) with a single space when it is inside a column value. I decided to create another file just in case:\nperl -pe 's/([^\u0026quot;])\\n/$1 /gs ; s/([^,\u0026quot;]\u0026quot;\u0026quot;)\\n/$1 /igs' \\ enheter_2024-08-21T04-20-40.551841944.csv \u0026gt; prep_enheter.csv If a line ends with something else than \u0026quot; or it ends with \u0026quot;\u0026quot; without a comma or another \u0026quot; in front the newline character \\n will be replaced with a space.\nUpdate 2024-08-26: Connor McDonald wrote a reply to my post on Twitter with this link to a question on AskTom . It gives you an idea how to preprocess the file before loading using a different end-of-line character/sequence.\nSuccess after three attempts is not bad With this out of the way you can start loading the new file, prep_enheter.csv in this case.\nSQLcl can scan the file and create the table for you before it loads the data into it. But you can also get it to just show you the DDL for the table without creating it and loading the data:\nload enheter prep_enheter.csv show_ddl Look at the SQL and you see that illegal characters in the header has been replaced; . with # etc. This is super useful since I don\u0026rsquo;t have to edit a 579MB file by hand (or messing around with vi, head and sed.)\nBy default SQLcl scans 100 rows before deciding on the DDL. In this case I want to set it to max which is 5000:\nset load scan_rows 5000 I wish I could set it much higher, some problems appeared after 22.000 lines.\nBy looking at the file we can see that the date format is yyyy-mm-dd, and ask SQLcl to use the same:\nset load date yyyy-mm-dd Number of lines pr batch is 50 in this version, I did not investigate the optimal number, but figured 1000 is a good number:\nset load batch_rows 1000 By defaults it commits every 10 batches, I did not change that number. These settings can all be combined in one command, of course:\nset load scan_rows 5000 date yyyy-mm-dd batch_rows 1000 You can check the current setting with:\nshow load show loadformat Paste the CREATE TABLE statement generated with the load enheter prep_enheter.csv show_ddl command into an editor and prepare yourself for a few rounds of changes. I\u0026rsquo;m calling the file cr_table.sql.\nThen do first attempt on a load with:\nload enheter prep_enheter.csv new This will create the table based on scanning 5000 rows and load data into it. It will probably fail after a little while with errors like ORA-12899: value too large for column. If SQLcl had scanned more rows it would probably have used wider columns. Anyway, it is easy to fix in the SQL, just find the column and crank up the column length. For the last column I replaced the VARCHAR2 type with CLOB1.\nA couple of other things, some date columns got the wrong datatype, change it to DATE. You may consider changing the datatype to VARCHAR2 for columns that only contains digits and therefore get the NUMBER data type when it is really something you are not going to do any calculations on, like ORGANISASJONSNUMMER; I changed it to VARCHAR2(11).\nAfter these changes, drop the table and create it again:\ndrop table enheter; @cr_table.sql Then load the file without creating the table first:\nload enheter prep_enheter.csv That is, no option after the file name. After I had resolved the problems with the newline character inside the field values I had only a 2-3 attempts before I managed to load the whole file. It takes a few minutes, or many, depending on the network, your database hardware, etc. Having an ExaCC helps.\nSince it commits regularly you can see the progress with a simple select count(*) from enheter; in another SQLcl session.\nIn case you want to redo the load without any changes to the table structure, use this option to have SQLcl truncate the table before load:\nset load truncate on At the source for this dataset there is a REST API to search for updates after a specific date, so I don\u0026rsquo;t have to download and ingest the whole file regularly.\nIt took almost five minutes to load on a 23ai FREE version, which is not bad, I think.\nselect count(*) from enheter where KONKURSDATO \u0026gt; sysdate - 30; COUNT(*) ___________ 228 228 bankruptcies last thirty days. This is an interesting dataset. I can find the number of bankruptcies in the last month, see active businesses by county, what dominates in each region, etc. I will use this dataset further to explore new features in Oracle 23ai. Similarity search and see how vector indexes behaves on larger dataset; I\u0026rsquo;ve heard a rumour about performance problems I need to check out.\nIn an Autonomous database in OCI extended character size is enabled by default and the limit for VARCHAR2 is 32767 size. You can check the parameter MAX_STRING_SIZE. STANDARD means the limit is 4000, EXTENDED means 32767. See article at oracle-base.com on how to enable it on your database.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2024/08/practical-guide-to-load-large-csv-files-to-oracle/","summary":"\u003cp\u003e\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/files_database.jpg\"/\u003e \n\u003c/figure\u003e\n Image by \u003ca href=\"https://pixabay.com/users/kreatikar-8562930/?utm_source=link-attribution\" target=\"_blank\" rel=\"noopener\"\u003eMudassar Iqbal from Pixabay\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eSometimes you need to be practical and apply some tricks to get the job done instead of waiting for the perfect tool. With SQL Developer Command Line you can load CSV with millions of lines with not too much hassle.\u003c/p\u003e\n\u003ch2 id=\"loading-large-csv-files-to-oracle-database\"\u003eLoading large CSV files to Oracle database\u003c/h2\u003e\n\u003cp\u003eAfter a long summer break it is time to start blogging again. Today I loaded a CSV file containing all companies in Norway registered at \u003ca href=\"https://www.brreg.no/en\" target=\"_blank\" rel=\"noopener\"\u003eThe Brønnøysund Register Centre\u003c/a\u003e\n. It has over 1.1 million lines, more than you want to open in a GUI like SQL Developer. But SQL Developer has a command line version that is almost perfect for the job. You\u0026rsquo;ll find the download link and more information at \u003ca href=\"https://www.oracle.com/database/sqldeveloper/\" target=\"_blank\" rel=\"noopener\"\u003ehttps://www.oracle.com/database/sqldeveloper/\u003c/a\u003e\n. \u003ca href=\"https://download.oracle.com/otn_software/java/sqldeveloper/sqlcl-latest.zip\" target=\"_blank\" rel=\"noopener\"\u003eThis URL\u003c/a\u003e\n always gives you the latest version and from a Mac or Linux box you can just do:\u003c/p\u003e","title":"Practical Guide to Load Large CSV Files to Oracle"},{"content":" Images by OpenClipart-Vectors from Pixabay Got too much data in the database? Offloading it to object storage is one option. Even better, get stuff in to the database and analyse it with SQL.\nAccessing OCI Object Storage from PL/SQL As mentioned in a previous post about OCI Object Storage I wanted to investigate how to access the OCI Object Storage from PL/SQL, in an Autonomous Database1.\nFirst step, looking into the documentation, it is not trivial where to start. There are good introductions on how to get started with different SDKs for Java, Python, TypeScript/JavaScript, .Net, Go and Ruby, but for the PL/SQL SDK it jumps right into a few examples as if they assume you are already set up with credentials in the database2. I did a quick search and found this: Manipulating the Oracle Cloud with PL/SQL which explains easily how to get started. Adrian shows in that article how to list out all the packages that make up the PL/SQL APIs for OCI, and most importantly how to get started with setting up necessary credentials for PL/SQL. This is what I did.\nAdding credentials to the database From my admin user I granted my own lab user execute privilege on the package dbms_cloud:\ngrant execute on dbms_cloud to lab_user; I generated a new API key pair, look in previous post on how to get there. dbms_cloud.create_credential has three versions documented, use this one:\nDBMS_CLOUD.CREATE_CREDENTIAL ( credential_name IN VARCHAR2, user_ocid IN VARCHAR2, tenancy_ocid IN VARCHAR2, private_key IN VARCHAR2, fingerprint IN VARCHAR2); The parameters are pretty obvious when you have the configuration that was shown when you generated a new key pair. The value for private_key is everything between -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----.\nYou can find all your credentials in the table user_credentials.\nDigging into the API documentation From here I went on to the API reference . The package here is called DBMS_CLOUD_OCI_OBS_OBJECT_STORAGE. There is a type hierarchy here that took some time to navigate. You can use the function Open Declaration in SQL Developer to see the package/type specification (right-click on the name and select third option from bottom on the menu, if it does not respond make sure the user has been granted execute on the package or type).\nAs an example look at the function to list buckets . It returns an object of type dbms_cloud_oci_obs_object_storage_list_buckets_response_t (listed under Object Storage Types in the left menu) that contains a response_body of type dbms_cloud_oci_object_storage_bucket_summary_tbl which of course is a table of dbms_cloud_oci_object_storage_bucket_summary_t , listed under Object Storage Common Types. Finally we find what we need (and some more):\nnamespace name compartment_id created_by time_created etag freeform_tags defined_tags In order to access the package and all the referenced type in my code the admin user must grant execute on them:\ngrant execute on dbms_cloud_oci_obs_object_storage to lab_user; grant execute on dbms_cloud_oci_obs_object_storage_list_buckets_response_t to lab_user; grant execute on dbms_cloud_oci_object_storage_bucket_summary_tbl to lab_user; grant execute on dbms_cloud_oci_obs_object_storage_get_object_response_t to lab_user; grant execute on dbms_cloud_oci_obs_object_storage_put_object_response_t to lab_user; Listing my buckets and objects This code lists my buckets:\ndeclare l_bucket_list_response dbms_cloud_oci_obs_object_storage_list_buckets_response_t; l_response_body dbms_cloud_oci_object_storage_bucket_summary_tbl; l_namespace varchar2(20) := \u0026#39;scrufrandom\u0026#39;; l_compartment_id varchar2(200) := \u0026#39;ocid1.tenancy.oc1..aaaaaaaan7morerandomcharactersdglk7ngvqmmr3dk4yo6ewjk2pa\u0026#39;; l_region varchar2(20) := \u0026#39;eu-frankfurt-1\u0026#39;; l_credential varchar2(20) := \u0026#39;LAB_CREDENTIAL\u0026#39;; begin l_bucket_list_response := DBMS_CLOUD_OCI_OBS_OBJECT_STORAGE.list_buckets ( namespace_name =\u0026gt; l_namespace, compartment_id =\u0026gt; l_compartment_id, region =\u0026gt; l_region, --endpoint can be used in stead of region credential_name =\u0026gt; l_credential) ; l_response_body := l_bucket_list_response.response_body; for i in 1..l_response_body.count loop dbms_output.put_line(l_response_body(i).name); end loop; end; / Similar code for listing objects in one named bucket:\ndeclare l_bucket_list_response dbms_cloud_oci_obs_object_storage_list_buckets_response_t; l_response_body dbms_cloud_oci_object_storage_bucket_summary_tbl; l_object_list dbms_cloud_oci_obs_object_storage_list_objects_response_t; l_namespace varchar2(20) := \u0026#39;scrufrandom\u0026#39;; l_compartment_id varchar2(200) := \u0026#39;ocid1.tenancy.oc1..aaaaaaaan7morerandomcharactersdglk7ngvqmmr3dk4yo6ewjk2pa\u0026#39;; l_region varchar2(20) := \u0026#39;eu-frankfurt-1\u0026#39;; l_credential varchar2(20) := \u0026#39;LAB_CREDENTIAL\u0026#39;; begin l_object_list := DBMS_CLOUD_OCI_OBS_OBJECT_STORAGE.list_objects ( namespace_name =\u0026gt; l_namespace, bucket_name =\u0026gt; \u0026#39;labbucket\u0026#39;, fields =\u0026gt; \u0026#39;name,size\u0026#39;, region =\u0026gt; l_region, credential_name =\u0026gt; l_credential); for i in 1..l_object_list.response_body.objects.count loop dbms_output.put_line(l_object_list.response_body.objects(i).name || \u0026#39; has \u0026#39; || l_object_list.response_body.objects(i).l_size || \u0026#39; bytes.\u0026#39;); end loop; end; / In this call I added parameter fields in order to get the l_size field, or else only the name would have been returned, see doc .\nReading objects into the database OK, I found my objects from last time. We can read an object into a blob with the function get_object that returns an object with type dbms_cloud_oci_obs_object_storage_get_object_response_t . The attribute response_body is a blob. This code is really simple. In order to store it in the database and look at it in SQL Developer I\u0026rsquo;ll create a table first:\ncreate table object_stage(id number generated always as identity, stuff blob); declare l_namespace varchar2(20) := \u0026#39;scrufrandom\u0026#39;; l_compartment_id varchar2(200) := \u0026#39;ocid1.tenancy.oc1..aaaaaaaan7morerandomcharactersdglk7ngvqmmr3dk4yo6ewjk2pa\u0026#39;; l_region varchar2(20) := \u0026#39;eu-frankfurt-1\u0026#39;; l_credential varchar2(20) := \u0026#39;LAB_CREDENTIAL\u0026#39;; l_bucket_name varchar2(20) := \u0026#39;labbucket\u0026#39;; l_object dbms_cloud_oci_obs_object_storage_get_object_response_t; begin l_object := DBMS_CLOUD_OCI_OBS_OBJECT_STORAGE.get_object ( namespace_name =\u0026gt; l_namespace, bucket_name =\u0026gt; l_bucket_name, object_name =\u0026gt; \u0026#39;cat.png\u0026#39;, region =\u0026gt; l_region, credential_name =\u0026gt; l_credential); insert into object_stage(stuff) values(l_object.response_body); commit; end; / select * from object_stage; In the output, double-click the pencil in the query result window, and in the pop-up window check View As: Image as shown here:\nThe cat is back, now in SQL Developer That was easy! After the initial hassle, actually getting the objects into a blob is almost trivial.\nWriting blobs to object storage I have a simple APEX app mentioned in a blogpost First Journey with SelectAI . I collect images with my mobile and store them as BLOBs in a table called EVENT. This code uses the put_object function to write all the images to an object in the same bucket as earlier. The function returns an object of type dbms_cloud_oci_obs_object_storage_put_object_response_t :\ndeclare l_namespace varchar2(20) := \u0026#39;scrufrandom\u0026#39;; l_compartment_id varchar2(200) := \u0026#39;ocid1.tenancy.oc1..aaaaaaaan7morerandomcharactersdglk7ngvqmmr3dk4yo6ewjk2pa\u0026#39;; l_region varchar2(20) := \u0026#39;eu-frankfurt-1\u0026#39;; l_credential varchar2(20) := \u0026#39;LAB_CREDENTIAL\u0026#39;; l_bucket_name varchar2(20) := \u0026#39;labbucket\u0026#39;; l_response dbms_cloud_oci_obs_object_storage_put_object_response_t; l_keys JSON_KEY_LIST; begin for i in (select replace(name,\u0026#39; \u0026#39;,\u0026#39;_\u0026#39;) || \u0026#39;.jpeg\u0026#39; file_name, image from event where image is not null) loop l_response := DBMS_CLOUD_OCI_OBS_OBJECT_STORAGE.put_object ( namespace_name =\u0026gt; l_namespace, bucket_name =\u0026gt; l_bucket_name, object_name =\u0026gt; i.file_name, content_length =\u0026gt; dbms_lob.getlength(i.image), put_object_body =\u0026gt; i.image, region =\u0026gt; l_region, credential_name =\u0026gt; l_credential); if l_response.status_code \u0026gt;= 400 then dbms_output.put_line(\u0026#39;Put object failed with status code \u0026#39; || l_response.status_code); l_keys := l_response.headers.get_keys; for k in 1..l_keys.count loop dbms_output.put_line(l_keys(k) || \u0026#39;:\u0026#39; || l_response.headers.get_string(k)); end loop; end if; end loop; end; / It took 35 seconds to write 256 files with an average size of 3.25 MB. The error code did not kick in, so I have not really seen the result of that. (I\u0026rsquo;m not much of a PL/SQL programmer, I just wrote it to impress a Dutch friend.) The easiest way to confirm that it actually worked is to check the object listing in the cloud console:\nI only had to mask one file name.\nConclusion Interacting with the Object Storage in the Oracle Cloud is quite easy once you get past the initial hassle. I wish the documentation was more developer friendly. A better Getting started section as Oracle has provided for the other SDKs would have helped.\nAnyway, now I know I can continue with more stuff in the APEX, PL/SQL, and AI world - I have some plans for these media files later on.\nSomething exciting to check out in this context is the new Multilingual Engine for JavaScript in Database 23ai.\nYou can install the required package dbms_cloud in a database on-prem by following this blogpost from Oracle \u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThe fact that the code examples for listing compartments is badly formatted does not help either.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2024/05/use-oci-object-storage-from-pl-sql/","summary":"\u003ctable\u003e\n\t\u003cthead\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003cth\u003e\u003c/th\u003e\n\t\t\t\t\t\u003cth\u003e\u003c/th\u003e\n\t\t\t\t\t\u003cth\u003e\u003c/th\u003e\n\t\t\t\u003c/tr\u003e\n\t\u003c/thead\u003e\n\t\u003ctbody\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003ctd\u003e\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/database_download.png\"/\u003e \n\u003c/figure\u003e\n\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003e\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003e\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/database_upload.png\"/\u003e \n\u003c/figure\u003e\n\u003c/td\u003e\n\t\t\t\u003c/tr\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003ctd\u003e\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003e\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003e\u003c/td\u003e\n\t\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eImages by \u003ca href=\"https://pixabay.com/users/openclipart-vectors-30363/?utm_source=link-attribution\u0026amp;utm_medium=referral\u0026amp;utm_campaign=image\u0026amp;utm_content=152940\" target=\"_blank\" rel=\"noopener\"\u003eOpenClipart-Vectors\u003c/a\u003e\n  from  \u003ca href=\"https://pixabay.com//?utm_source=link-attribution\u0026amp;utm_medium=referral\u0026amp;utm_campaign=image\u0026amp;utm_content=152940\" target=\"_blank\" rel=\"noopener\"\u003ePixabay\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eGot too much data in the database? Offloading it to object storage is one option. Even better, get stuff in to the database and analyse it with SQL.\u003c/p\u003e\n\u003ch2 id=\"accessing-oci-object-storage-from-plsql\"\u003eAccessing OCI Object Storage from PL/SQL\u003c/h2\u003e\n\u003cp\u003eAs mentioned in a previous post about \u003ca href=\"https://enesi.no/2024/05/stuffing-files-into-oci-object-storage/\"\u003eOCI Object Storage\u003c/a\u003e\n I wanted to investigate how to access the OCI Object Storage from PL/SQL, in an Autonomous Database\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e","title":"Use OCI object storage from PL/SQL"},{"content":" Oracle launched version 23ai of the database May 2. Immediately after the event we could pull the Free edition from the Oracle Container Registry and play with it.\nI blogged about how you can set up 23c Free in Your Own Oracle Database Lab in 1-2-3 218 days ago 1.\nThis command pulls the latest version:\npodman pull container-registry.oracle.com/database/free:latest Have a look in Your Own Oracle Database Lab in 1-2-3 for more details on how to create a new container with it. I created mine with:\npodman run -p 21521:1521 -v /tmp/models:/mnt --name ora23ai_free \\ -e ENABLE_ARCHIVELOG -e ORACLE_PWD=somepassword \\ container-registry.oracle.com/database/free:latest Vector Search I wanted to explore the new Vector Search and started with AI Vector Search User\u0026rsquo;s Guide . It has a fairly good introduction to what this is all about and explains:\nThe new VECTOR data type Vector Embeddings Similarity Search Vector Embedding Models Obviously, I am not an expert on this, but this field is something I want to work more with, and I believe this possibility to apply AI on data in the database without using external APIs has a lot of potential.\nSo I started with the User\u0026rsquo;s Guide and tried out some of the new stuff.\nI decided to look at textual data first. From the manual:\nFor textual data, sentence transformers transform words, sentences, or paragraphs into vector embeddings.\nYou can generate these embeddings outside the database and import them to the database later, but if you have a model in the ONNX format, then you can import it and generate those embeddings on your data right inside the database. That might not seem like a big difference, but for me as a DBA that means:\nI don\u0026rsquo;t have to offload a lot of data The data are kept safe (think database security, access, leakage, etc) I expect excellent performance on engineered systems with code running really close to the data Importing a pretrained model The database does not come with any embedding models, but we can load models in ONNX format; see https://onnx.ai for more information about the Open Neural Network Exchange (ONNX) format. It links to models that can be downloaded from https://github.com/onnx .\nModels that are in other formats (from Huggingface.co and others) can be converted with the Oracle Machine Learning for Python (OML4Py) Client. I tried some pretrained models in ONNX format, but struggled to load them, so I decided to look at OML4Py Client. That resulted in a separate post How to install OML4Py Client on Ubuntu (and delayed this post\u0026hellip;) That post also has links to documentation on how to install it on Oracle Linux.\nOML4Py Client comes with some preconfigured models that can be exported to a file:\nfrom oml.utils import EmbeddingModel, EmbeddingModelConfig EmbeddingModelConfig.show_preconfigured() The last function currently lists:\n[\u0026#39;sentence-transformers/all-mpnet-base-v2\u0026#39;, \u0026#39;sentence-transformers/all-MiniLM-L6-v2\u0026#39;, \u0026#39;sentence-transformers/multi-qa-MiniLM-L6-cos-v1\u0026#39;, \u0026#39;ProsusAI/finbert\u0026#39;, \u0026#39;medicalai/ClinicalBERT\u0026#39;, \u0026#39;sentence-transformers/distiluse-base-multilingual-cased-v2\u0026#39;, \u0026#39;sentence-transformers/all-MiniLM-L12-v2\u0026#39;, \u0026#39;BAAI/bge-small-en-v1.5\u0026#39;, \u0026#39;BAAI/bge-base-en-v1.5\u0026#39;, \u0026#39;taylorAI/bge-micro-v2\u0026#39;, \u0026#39;intfloat/e5-small-v2\u0026#39;, \u0026#39;intfloat/e5-base-v2\u0026#39;, \u0026#39;prajjwal1/bert-tiny\u0026#39;, \u0026#39;thenlper/gte-base\u0026#39;, \u0026#39;thenlper/gte-small\u0026#39;, \u0026#39;TaylorAI/gte-tiny\u0026#39;, \u0026#39;infgrad/stella-base-en-v2\u0026#39;, \u0026#39;sentence-transformers/paraphrase-multilingual-mpnet-base-v2\u0026#39;, \u0026#39;intfloat/multilingual-e5-base\u0026#39;, \u0026#39;intfloat/multilingual-e5-small\u0026#39;, \u0026#39;sentence-transformers/stsb-xlm-r-multilingual\u0026#39;] You can pick one from the list above and look them up at https://huggingface.co/ like this one: https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2 .\nYou can export one in two steps:\nem = EmbeddingModel(model_name=\u0026#34;sentence-transformers/all-MiniLM-L12-v2\u0026#34;) em.export2file(\u0026#34;all_MiniLM_L12_v2\u0026#34;,output_dir=\u0026#34;.\u0026#34;) The last function downloads a few files before the export completes. Filename will be like the string given + extension .onnx. Move that file to a directory on the database server where you can create a directory in the database for it:\ncreate directory dm_import as \u0026#39;/tmp/models\u0026#39;; create user oisene identified by \u0026#34;oisene\u0026#34; default tablespace users quota unlimited on users; grant db_developer_role to oisene; grant create mining model to oisene; grant read, write on directory dm_import to oisene; Connect to the new user (oisene) and start the import:\nbegin DBMS_VECTOR.LOAD_ONNX_MODEL( \u0026#39;DM_IMPORT\u0026#39;, \u0026#39;all_MiniLM_L12_v2.onnx\u0026#39;, \u0026#39;minilm_l12_v2\u0026#39;); end; / The last parameter is the name of the loaded model, and will be used later. After load we can check USER_MINING_MODELS:\nselect model_name, mining_function, algorithm,algorithm_type,creation_date from user_mining_models; MODEL_NAME MINING_FUNCTION ALGORITHM ALGORITHM_TYPE CREATION_DATE ________________ __________________ ____________ _________________ ________________ MINILM_L12_V2 EMBEDDING ONNX NATIVE 15-MAY-24 More info about the model in USER_MINING_MODEL_ATTRIBUTES:\nSELECT model_name, attribute_name, attribute_type, data_type, vector_info FROM user_mining_model_attributes ORDER BY ATTRIBUTE_NAME; MODEL_NAME ATTRIBUTE_NAME ATTRIBUTE_TYPE DATA_TYPE VECTOR_INFO ________________ _________________ _________________ ____________ ______________________ MINILM_L12_V2 DATA TEXT VARCHAR2 MINILM_L12_V2 ORA$ONNXTARGET VECTOR VECTOR VECTOR(384,FLOAT32) Loading of a model creates two views, in this case, DM$VMMINILM_L12_V2 and DM$VJMINILM_L12_V2 where the last part of the view name is the model name.\nTest Data There are plenty of data sets, but for this case a mix of structured and unstructured data is ideal. I downloaded beer review data from Kaggle , License CC BY 4.0 DEED .\nThe largest dataset has a file beer_reviews beer_profile_and_ratings.csv with 3197 rows in it, not exactly big data, but sufficient to demo the functionality. This being beer, some analysis before tasting may be worth it instead of starting from the top or just picking all the IPAs at random.\nI\u0026rsquo;m using SQLcl to load the data:\nload beer_reviews beer_profile_and_ratings.csv new This simple command scans the CSV file, creates a new table based on the scan, sets LOADFORMAT and LOAD properties, and finally loads the data. It even prints out the DDL statement for the table when it is done. (One quick tip, check your NLS settings with show parameter nls. If nls_territory is NORWAY or any country with different number format than US, then change it to AMERICAN with alter session set nls_territory='AMERICA'; This ensures that correct NUMBER datatype will be used.)\nI did rename a couple of columns after, to simplify SQL later.\nOne note about the dataset, some reviews have only the word \u0026ldquo;Notes:\u0026rdquo; in description. This SQL will count how many has real notes pr style:\nselect style,count(*), count(case when length(description)\u0026gt;10 then 1 else null end) cnt_descr from BEER_REVIEWS group by style order by 3 desc; I decided to remove those dummy values + some more changes:\ndelete from BEER_REVIEWS where description=\u0026#39;Notes:\u0026#39;; alter table beer_reviews add id number; update beer_reviews set id=rownum; alter table beer_reviews add constraint beer_reviews_pk primary key(id); alter table beer_reviews rename column X#NAME to NAME; alter table beer_reviews rename column BEER_NAME_#FULL# to BEER_NAME_FULL; Workflow At the end of the Overview chapter in User\u0026rsquo;s Guide there is a Oracle AI Vector Search Workflow that is useful to follow, but I didn\u0026rsquo;t entirely in this case. The five steps there maps to chapter 3 to 8.\nEmbeddings I decided to perform this inside the database.\nNew table to store the embeddings:\ncreate table beer_review_chunks( beer_review_id number, chunk_id number, chunk_data varchar2(4000), chunk_embedding vector); One (!) insert statement to embed all the descriptions:\ninsert into beer_review_chunks select br.id beer_review_id, et.embed_id chunk_id, et.embed_data chunk_data, to_vector(et.embed_vector) chunk_embedding from beer_reviews br, dbms_vector_chain.utl_to_embeddings( dbms_vector_chain.utl_to_chunks( dbms_vector_chain.utl_to_text(br.description), json(\u0026#39;{\u0026#34;normalize\u0026#34;:\u0026#34;all\u0026#34;}\u0026#39;)), json(\u0026#39;{\u0026#34;provider\u0026#34;:\u0026#34;database\u0026#34;, \u0026#34;model\u0026#34;:\u0026#34;MINILM_L12_V2\u0026#34;}\u0026#39;)) t, JSON_TABLE(t.column_value, \u0026#39;$[*]\u0026#39; COLUMNS (embed_id NUMBER PATH \u0026#39;$.embed_id\u0026#39;, embed_data VARCHAR2(4000) PATH \u0026#39;$.embed_data\u0026#39;, embed_vector CLOB PATH \u0026#39;$.embed_vector\u0026#39;)) et; commit; This actually took a few minutes on my old lab server.\nQuerying with Similarity Searches Yes, I skipped creating the vector indexes first.\nGenerating a vector for a search text:\nVARIABLE query_string VARCHAR2(1000) VARIABLE query_vector CLOB BEGIN :query_string := \u0026#39;IPA without fancy stuff\u0026#39;; SELECT vector_embedding(MINILM_L12_V2 using :query_string as data) into :query_vector; END; / PRINT query_vector Query for similar beers\nSELECT brc.beer_review_id, brc.chunk_id, br.name beer_name, brc.chunk_data FROM beer_review_chunks brc join beer_reviews br on brc.beer_review_id=br.id ORDER BY vector_distance(chunk_embedding , :query_vector, COSINE) FETCH FIRST 4 ROWS ONLY; You will have to scroll to the right to see all the text:\nBEER_REVIEW_ID CHUNK_ID BEER_NAME CHUNK_DATA _________________ ___________ _____________________________ _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 497 2 East India Pale Ale stone fruits and firm bitterness from IPA\u0026#39;s ancestral British home. Give our East IPA a try with some rich crab cakes or salmon, strike up a conversation with farmhouse cheddars, and find harmony alongside spicy dishes. East IPA\u0026#39;s blend of tradition and exuberance sets the standard for hop-driven deliciousness.\\t 504 1 All Day IPA Notes:The beer you\u0026#39;ve been waiting for. Keeps your taste satisfied while keeping your senses sharp. An all-day IPA naturally brewed with a complex array of malts, grains and hops. Balanced for optimal aromatics and a clean finish. The perfect reward for an honest day\u0026#39;s work and the ultimate companion to celebrate life\u0026#39;s simple pleasures.Formerly Endurance - All Day IPA.\\t 505 1 HopDevil Notes:Bold, spicy and menacingly delicious, this American-hopped IPA offers an aromatic punch and follows through with a lasting, full-bodied finish\\t 610 1 Lost Sailor India Pale Ale Notes:You don\u0026#39;t need a compass to find your way to our flagship India Pale Ale. This classic British-inspired IPA boasts a well-rounded malt profile, complex floral and citrus aromas, and generous Goldings dry hopping. Well balanced between malt sweetness and hop bitterness.\\t Let\u0026rsquo;s create a new query vector and repeat the query:\nBEGIN :query_string := \u0026#39;Pepper mint funky beer\u0026#39;; SELECT vector_embedding(MINILM_L12_V2 using :query_string as data) into :query_vector; END; / SELECT brc.beer_review_id, brc.chunk_id, br.name beer_name, brc.chunk_data FROM beer_review_chunks brc join beer_reviews br on brc.beer_review_id=br.id ORDER BY vector_distance(chunk_embedding , :query_vector, COSINE) FETCH FIRST 4 ROWS ONLY; BEER_REVIEW_ID CHUNK_ID BEER_NAME CHUNK_DATA _________________ ___________ ___________________________________________ __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 310 1 Billy\u0026#39;s Chilies Beer (Timberline Series) Notes:Our chili beer is made with five different kinds of fresh chili peppers. Rich pepper aroma greets you when the bottle is opened, followed by its satisfying beer taste and then subtle spiciness.\\t 455 1 Mexican Chocolate Stout Notes:Gold Medal Winner at Great American Beer Fest - Herb and Spice Category. A rich black ale with roasted malt flavors blended with bittersweet chocolate, our own blend of three Mexican chili peppers and cinnamon.\\t 312 1 Hot-Jala-Heim Notes:This style of beer is classified in the \u0026#34;Fruit/Vegetable\u0026#34; category as a \u0026#34;Chili beer\u0026#34;. The focus is obviously on the chili peppers used during the brewing process. Our version uses a mix of hot peppers, including jalapenos and anaheims (hence the name...Hot-Jala-Heim). It has a delightful \u0026#39;peppery\u0026#39; aroma as well as taste with a \u0026#39;mild\u0026#39; sensation of heat experienced after swallowing.\\t 335 1 Green Chili Beer Notes:This is the chili-heads favorite brew. A mild base beer is spiced up with Anaheim and Serrano peppers. This beer has a wonderful green chili aroma and a touch of heat in the finish. Have it with tomato juice as a \u0026#34;liquid enchilada\u0026#34;.\\t These two queries return really fast without a vector index, even on rather slow hardware. But then again, it is not really much data. Explain plan for the SQL above:\n------------------------------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 4 | 8376 | | 1176 (1)| 00:00:01 | |* 1 | COUNT STOPKEY | | | | | | | | 2 | VIEW | | 2043 | 4177K| | 1176 (1)| 00:00:01 | |* 3 | SORT ORDER BY STOPKEY| | 2043 | 4215K| 5456K| 1176 (1)| 00:00:01 | |* 4 | HASH JOIN | | 2043 | 4215K| | 272 (0)| 00:00:01 | | 5 | TABLE ACCESS FULL | BEER_REVIEWS | 1840 | 141K| | 68 (0)| 00:00:01 | | 6 | TABLE ACCESS FULL | BEER_REVIEW_CHUNKS | 2042 | 4056K| | 204 (0)| 00:00:01 | ------------------------------------------------------------------------------------------------------ Adding a Vector Index create vector index beer_review_chunks_v1 on beer_review_chunks(chunk_embedding) organization inmemory neighbor graph distance COSINE with target accuracy 95; That aborted with:\nORA-51961: The vector memory area is out of space\nI suspect this is due to the limitation of the Free edition, where the maximum size for the SGA is 2G. The sizing of the Vector Pool is documented in chapter 5 in the User\u0026rsquo;s Guide; that pool is part of the SGA.\nA parameter vector_memory_size has value 0 in my database. I tried to change it to a value as suggested in the manual, but another error was thrown:\nORA-51955: The Oracle Database Vector Memory size cannot be increased more than 0 MB for PDB ID 3.\nLast attempt was to increase it from cdb$root, no luck:\nORA-51950: The Oracle Database Vector Memory size cannot be increased.\nThe parameter is not yet documented in the Database Reference. I\u0026rsquo;ll have to dig into this at some point, I don\u0026rsquo;t expect to have this kind of problems when 23ai lands on the nearest Exadata.\nUpdate 2024-05-28: Simon Pane of Pythian (check out his posts ) sent me a solution. Run this from the root container:\nalter system set vector_memory_size=200m scope=spfile; shutdown immediate startup It may be I was too greedy and set it too high a value when I tried.\nORACLE instance started. Total System Global Area 1603726344 bytes Fixed Size\t5360648 bytes Variable Size\t369098752 bytes Database Buffers\t1006632960 bytes Redo Buffers\t4530176 bytes Vector Memory Area\t218103808 bytes Database mounted. Database opened. Note the third-last line.\nThe value was adjusted after startup:\nSQL\u0026gt; show parameter vector_memory_size NAME\tTYPE\tVALUE ------------------------------------ ----------- ------------------------------ vector_memory_size\tbig integer 208M With this change the error disappeared:\ncreate vector index beer_review_chunks_v1 on beer_review_chunks(chunk_embedding) organization inmemory neighbor graph distance COSINE with target accuracy 95; Vector INDEX created. I still have more to investigate regarding this index type. End of update\nConclusion I haven\u0026rsquo;t tried these beers and can\u0026rsquo;t evaluate the results, but the tests so far looks promising. It sure helps to understand more about the models and what is going on here. In the queries above I did only similarity search, but we can combine it with a relational query making the results even more precise. I expect a larger and more realistic data set will show this better.\nThere is a lot to explore and learn. I\u0026rsquo;m not bored yet.\nselect sysdate - date '2023-10-11';\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2024/05/oracle-database-23ai-and-vector-search/","summary":"\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/oracle23ai.jpeg\"\n         alt=\"Oracle database 23ai logo\" width=\"800\"/\u003e \n\u003c/figure\u003e\n\n\u003cp\u003eOracle launched version 23ai of the database May 2. Immediately after the event we could pull the Free edition from the Oracle Container Registry and play with it.\u003c/p\u003e\n\u003cp\u003eI blogged about how you can set up 23c Free in \u003ca href=\"https://enesi.no/2023/10/your-own-oracle-database-lab-in-1-2-3/\"\u003eYour Own Oracle Database Lab in 1-2-3\u003c/a\u003e\n 218 days ago \u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eThis command pulls the latest version:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003epodman pull container-registry.oracle.com/database/free:latest\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eHave a look in \u003ca href=\"https://enesi.no/2023/10/your-own-oracle-database-lab-in-1-2-3/\"\u003eYour Own Oracle Database Lab in 1-2-3\u003c/a\u003e\n  for more details on how to create a new container with it. I created mine with:\u003c/p\u003e","title":"Oracle Database 23ai and Vector Search"},{"content":"Update: See this post for an update for version 2.1 on Oracle Linux.\nHow to install OML4Py Client on Ubuntu 22.04 I needed the Oracle Machine Learning for Python (OML4Py) Client (not the Server) in order to convert a pretrained model so that I could import it into the newly released Oracle database 23ai.\nI wrote this from information I found in the User\u0026rsquo;s Guide and a lot of internet search. The PC I am using for this runs Ubuntu, so I adapted the installation for Ubuntu 22.04 (Jammy Jellyfish). After I completed the whole procedure I decided to verify it on a fresh Vagrant box:\nmkdir ubuntu2204 cd ubuntu2204 vagrant init ubuntu/jammy64 vagrant up vagrant ssh sudo apt-get update sudo apt-get upgrade Note, for some work later I needed to increase the memory available to the VM. I changed the Vagrantfile to include:\nconfig.vm.provider \u0026#34;virtualbox\u0026#34; do |vb| vb.memory = \u0026#34;4096\u0026#34; end These lines are commented out in the generated Vagrantfile.\nCompile Python 3.12 Some packages listed in the User\u0026rsquo;s Guide were already installed. I installed these:\nsudo apt install libffi-dev libbz2-dev libssl-dev \\ pkg-config libreadline-dev tk-dev unzip Download the source code for Python 3.12, configure and compile with:\nwget https://www.python.org/ftp/python/3.12.0/Python-3.12.0.tar.xz tar xvf Python-3.12.0.tar.xz export PREFIX=`pwd`/Python-3.12.0 cd $PREFIX ./configure --prefix=$PREFIX --enable-shared --enable-optimizations make clean make -j6 make altinstall The -j6 option to the second-last make command starts six compiler jobs in parallell. Compiling does take some time on old hardware, but this option requires more memory.\nI added the following in an environment file (called foo.env) in the directory above Python-3.12.0\nexport PREFIX=`pwd`/Python-3.12.0 export PYTHONHOME=$PREFIX export PATH=$PYTHONHOME/bin:$PATH export LD_LIBRARY_PATH=$PYTHONHOME/lib:$LD_LIBRARY_PATH and sourced it with: . ./foo.env.\nI added a link:\ncd $PYTHONHOME/bin ln -s python3.12 python3 Upgrade pip:\npython3 -m pip install --upgrade pip Python packages I had to install a bunch of packages (The Python community clearly does not mind breaking things so version numbers can be important, numpy must be removed and installed with a lower version):\npip3.12 install pandas==2.1.1 pip3.12 install scipy==1.11.3 pip3.12 install matplotlib==3.7.2 pip3.12 install oracledb==2.0.1 pip3.12 install threadpoolctl==3.1.0 pip3.12 install joblib==1.2.0 pip3.12 install setuptools pip3.12 install scipy==1.12.0 pip3.12 uninstall numpy pip3.12 install numpy==1.26.4 pip3.12 install scikit-learn==1.3 pip3.12 install onnx pip3.12 install onnxruntime_extensions pip3.12 install onnxruntime pip3.12 install transformers==4.38.1 pip3.12 install sentencepiece==0.2.0 pip3.12 install torch OML4Py Client Download the OML4Py Client . Unzip it somewhere with:\nunzip oml4py-client-linux-x86_64-2.0.zip It contains one folder client. Install it with:\nperl -Iclient client/client.pl If all the prerequisites are in place you should see something like this (you have to confirm yes at some point):\nOracle Machine Learning for Python 2.0 Client. Copyright (c) 2018, 2024 Oracle and/or its affiliates. All rights reserved. Checking platform .................. Pass Checking Python .................... Pass Checking dependencies .............. /home/vagrant/check_deps.py:2: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html from pkg_resources import WorkingSet, VersionConflict, DistributionNotFound Pass Checking OML4P version ............. Pass Current configuration Python Version ................... 3.12.0 PYTHONHOME ....................... /home/vagrant/Python-3.12.0 Existing OML4P module version .... None Operation ........................ Install/Upgrade Proceed? [yes] Processing ./client/oml-2.0-cp312-cp312-linux_x86_64.whl Installing collected packages: oml Successfully installed oml-2.0 Done Now, let\u0026rsquo;s see if it works. Execute python3 and run:\nimport oml It can actually take a few seconds to load.\nOne more check:\nfrom oml.utils import EmbeddingModel, EmbeddingModelConfig EmbeddingModelConfig.show_preconfigured() EmbeddingModelConfig.show_templates() In next post I will do the actual export of a model, load it into a Oracle Database 23ai Free and explore the Oracle AI Vector Search a bit.\n","permalink":"https://enesi.no/2024/05/oml4py-client-on-ubuntu/","summary":"\u003cp\u003e\u003cstrong\u003eUpdate:\u003c/strong\u003e See \u003ca href=\"https://enesi.no/2025/11/oml4py-2.1-on-linux/\"\u003ethis post\u003c/a\u003e\n for an update for version 2.1 on Oracle Linux.\u003c/p\u003e\n\u003ch1 id=\"how-to-install-oml4py-client-on-ubuntu-2204\"\u003eHow to install OML4Py Client on Ubuntu 22.04\u003c/h1\u003e\n\u003cp\u003eI needed  the Oracle Machine Learning for Python (OML4Py) \u003cem\u003eClient\u003c/em\u003e (not the Server) in order to convert a pretrained model so that I could import it into the newly released Oracle database 23ai.\u003c/p\u003e\n\u003cp\u003eI wrote this from information I found in the \u003ca href=\"https://docs.oracle.com/en/database/oracle/machine-learning/oml4py/2/mlugp/install-oml4py-client-linux-autonomous-database.html#GUID-4CF137D0-F70B-412A-81FB-8C0E32B7C73A\" target=\"_blank\" rel=\"noopener\"\u003eUser\u0026rsquo;s Guide\u003c/a\u003e\n and a lot of internet search. The PC I am using for this runs Ubuntu, so I adapted the installation for Ubuntu 22.04 (Jammy Jellyfish). After I completed the whole procedure I decided to verify it on a fresh Vagrant box:\u003c/p\u003e","title":"OML4Py Client on Ubuntu"},{"content":"Stuffing files away in the cloud might not be on your bucket list, but object storage in Oracle Cloud is quite useful in several situations.\nStoring files in OCI Object Storage Object storage can be used as you would use Amazon\u0026rsquo;s S3 through a special API provided in OCI. You can even store your RMAN database backups through a Swift API. But I will just upload and download a bunch of files.\nAs usual I am using the Oracle Cloud Free Tier. When I come up with a good business idea that requires more space and power I\u0026rsquo;ll upgrade to a paid account. This link at oracle.com shows you how to sign up if Oracle Free Tier is news for you. (I blogged about it in Norwegian almost three years ago.) The Free Tier gives me 20GB for object storage.\nMy plan is to explore the API from Python and APEX \u0026amp; PL/SQL. I got involved in a cool project and we figured out we need to store some images in the cloud, and retrieve it without much hassle.\nBut first, we need to get started.\nInstalling the command line interface If you have never used OCI before this section shows you how get started with the command line interface (CLI). But if you already has oci installed and configured then scroll down to next section.\nThe CLI is supported on Windows, Mac and a bunch of Linux versions. The Quickstart in the Developer Guide takes you straight to the list of environments and how to get started. I\u0026rsquo;m using my Mac for this demo:\nbrew update brew install oci-cli Simple test to see if the oci command is available:\noci --version oci --release-info The first returns a version number like 3.40.1 which is the version when I am writing this. The second command checks to see if there is a new version and the news it has for you. Next step is to create a configuration that the tool will use. This is the interactive way to do it:\noci setup config It will ask for a few things you probably don\u0026rsquo;t keep in your head, so you may just sign into your cloud account now, and click on the Profile icon:\nFirst it asks for the location of the configuration file. The default is OK, which in my case is /Users/oisene/.oci/config where /Users/oisene is my home directory. Next question is my user OCID which can be found by selecting My profile from the menu in previous picture. Then it asks for the tenancy OCID, which is found by selecting Tenancy: your_tenancy in the same menu. Third question is the region that I am using. You can see part of it in the last screen capture above; it is part of the navigation bar. But there are two locations in Frankfurt in the list (index 25 and 26), it is not obvious which one I am using. I am using Germany Central (Franktfurt), but is that eu-frankfurt-1 or eu-frankfurt-2? You can find out by clicking on the region name, and the Manage Regions as shown here:\nIt lists the regions including the region identifier for each. Mine is eu-frankfurt-1. It now asks me:\nDo you want to generate a new API Signing RSA key pair? (If you decline you will be asked to supply the path to an existing key.) [Y/n]:\nI had one, but I deleted it just to make sure that I started from scratch as may be your case. I accepted the defaults three times; Y, the location for the keys, and the name for my key. Since I am lazy I wrote N/A for passphrase to my keys; my key pair is probably not worth stealing, and I can always delete it later. At the end it prints out some helpful text with a link to a page that explains how you add this API Signing public key to the console . So, from the same menu as earlier, I choose My profile and in the menu on the right side I select API Keys and add the public key I just created:\nI then choose Paste a public key because it is easier to show the contents of the file in my terminal, copy and paste it into the text box than to search for the file and upload it. You can paste the whole content including the lines -----BEGIN PUBLIC KEY----- and -----END PUBLIC KEY-----.\nWhen you are done it will show you a configuration file preview, which is almost exactly what you have just created with the command above and stored in the config file. Sorry about that; next time you can go straight to this place and generate a new API key pair and it will generate the config-file for you. It only lacks the path to your key file, which is the file with your private key (oci_api_key.pem if you used the default value).\nOK, we are now ready to actually use this CLI. For instance I can list all my compartments with oci iam compartment list.\nFirst test of object storage The documentation for Object Storage can be found here . But you do get a lot of help with just executing oci -h or oci without any options, commands or arguments. The object storage commands start with os, and there is help available with oci os -h.\nMany of these commands require an option --compartment-id or (the shorter -c ). Since it is quite long I use a tip from the manual and define it in an environment variable C:\nexport C=ocid1.tenancy.... For now, I am going to put my stuff on the top compartment, whose id is the same as the OCID for the tenancy. If you list all your compartments with oci iam compartment list, it uses the OCID of the tenancy as default value for the -c option. And in case you already forgot, the OCID of your tenancy is now in your config file.\nFor instance to list all your buckets you execute:\noci os bucket list -c $C My list is empty, but I can create a new bucket with:\noci os bucket create -c $C --name labbucket It returns with a JSON document describing the newly created bucket. This commands uploads a file to this bucket:\noci os object put -bn labbucket --file dolphinframe827.jpg Yes, that is a picture from last vacation at Fernando de Noronha :\nBack to work (or labbing). I now have a memory from last vacation in the cloud. I can download it from anywhere by logging into cloud.oracle.com and navigate to my list of buckets as show here:\nWhen I click on my bucket I get a list of objects in it that I can select and download.\nSome terms Since they called it object storage it should not surprise us that they have more abstract terms. A few things are important:\nObject This is what you store, just about anything, that\u0026rsquo;s probably why they called it object. In my case here it is a file. Well, a file is also just a (binary) stream. From Python you can store whatever you can serialize to a byte stream.\nBucket You have seen it briefly in the example above, it is like the drawer, bag or general collection of stuff. This is where you throw stuff into. A bucket resides in a region. It is associated with a compartment as you saw in the command example above, i.e. you need to select which compartment it belongs to.\nNamespace A space for names? It is just another level of isolation, and simplifies some of the handling later. The manual says:\nA namespace is a logical entity that serves as a top-level container for all buckets and objects, \u0026hellip;\nNot abstract at all. Nobody has the same namespace as you do. The bucket names must be unique within one namespace, but you can replicate whatever you find in other tenants inside your namespace without risk of confusion.\nYou can get your namespace with this command:\noci os ns get You have one from the time your tenancy1 was created, it is a random string of letters. I have not seen any way to create other namespaces, so in practice, the namespace becomes just an alias for your tenancy used in the context of object storage. (You can execute the previous command with -c and specify another compartment, but it returns the same namespace in my case.) Also it does simplify management of objects between tenancies. When putting objects to a bucket you can specify namespace, but since we usually don\u0026rsquo;t, a call to oci os ns get is performed internally according to the manual.\nCompartment In your tenancy you have a root compartment, and you create more compartments below it as you need to organize stuff. When you create a bucket you need to select a compartment with option -c as demonstrated above.\nStorage tiers I would like to start exploring this stuff from Python, but I should mention that there are three different tiers for object storage:\nStandard - default tier. Optimal tier for frequently accessed data Infrequent Access - as the name implies, if you have data that is not accessed much, or the access time does not impact end users than this is more economical Archive - For data you don\u0026rsquo;t need access to unless you restore them. Objects here are not immediately available, just like stuff in an archive often is not readily available. This is the place for data you don\u0026rsquo;t dare the delete, but sure have no use for now and foreseeable future. You can specify default tier for a bucket, Standard or Archive (but not InfrequentAccess). If default is Standard the bucket may have a mix of Standard, Infrequent Access and Archive. But a bucket with default tier Archive cannot have objects on other storage tiers. This is a bit confusing. When you upload (put) an object you can specify all three, and for a bucket with default storage tier Standard all three will work.\nIf you have upgraded from Free Tier you may save some money by using Infrequent Access, assuming your access pattern allows for it. Auto-Tiering helps you reduce costs. It can automagically move objects in the Standard tier to Infrequent Access if the activity in your project drops. Auto tiering is off by default, but you can enable it with a parameter to the create command or update an existing bucket with:\noci os bucket update -bn labbucket --auto-tiering InfrequentAccess I\u0026rsquo;ll probably dig more into these details when I have a cloud bill to pay. For now it is all free for me.\nThe Python SDK There are at least 7 Software Development Kits (SDKs) you can use to access OCI. I\u0026rsquo;ll be using the Python SDK since most of the cool stuff happens in Python these days (not as cool as SQL, though). Brendan Tierney shows you in Python for OCI installation and configuration how to set up the Python library and configuration. You\u0026rsquo;ll probably enjoy the break and have a look at his first.\nBoth the information at the Python Package Index and the reference at docs.cloud.oracle.com are useful. If you are new to the SDK the first is a good place to start, if you want to explore the Object Storage API in particular then the reference at Oracle has code examples for six SDKs as well as the REST API (which all the SDKs are using) under each operation. Under each method description in the Python API reference there is a link to a code sample.\nFirst install the package:\npip install oci Then a small Python program to see that it works with DEFAULT profile:\nimport oci config = oci.config.from_file(\u0026#34;~/.oci/config\u0026#34;) identity = oci.identity.IdentityClient(config) user = identity.get_user(config[\u0026#34;user\u0026#34;]).data print(user) It returns 33 lines of JSON.\nUnlike the oci CLI command, the parameter namespace_name is mandatory as well as the compartment_id in list buckets for instance:\nos_client = oci.object_storage.ObjectStorageClient(config) ns=\u0026#34;frytqguihzrv\u0026#34; c=\u0026#34;ocid1.tenancy.oc1..aaaaaaaa.....\u0026#34; buckets = os_client.list_buckets(namespace_name=ns, compartment_id=c) print(buckets.data) New bucket for my Python stuff:\nnew_bucket_response = os_client.create_bucket( namespace_name=ns, create_bucket_details=oci.object_storage.models.CreateBucketDetails( name=\u0026#34;pythonstuff\u0026#34;, compartment_id=c, storage_tier=\u0026#34;Standard\u0026#34;)) new_bucket_response.status Look at the documentation for Object Storage in the Python SDK ; all the methods you need are methods under the ObjectStorageClient as demonstrated above.\nReading a file in from disk and put the binary stream to a bucket is easy with:\nwith open(\u0026#34;cat.png\u0026#34;,\u0026#34;rb\u0026#34;) as f: cat_data=f.read(); upload_response = os_client.put_object(namespace_name=ns, bucket_name=\u0026#34;pythonstuff\u0026#34;, object_name=\u0026#34;cat.png\u0026#34;, put_object_body=cat_data) upload_response.status There are a several optional parameters, I only used the mandatory ones, but this simple demo shows how easy it is. Cat pictures are popular, or else I could have used the storage_tier parameter to specify InfrequentAccess.\nListing objects in a bucket:\nobject_list = os_client.list_objects( namespace_name=ns, bucket_name=\u0026#34;labbucket\u0026#34;, fields=\u0026#34;name,size,timeModified\u0026#34;) print(object_list.data) Without the fields parameter above, it will only show the name. Get stuff back can be done with:\nget_response = os_client.get_object(namespace_name=ns, bucket_name=\u0026#34;labbucket\u0026#34;, object_name=\u0026#34;dolphinframe827.jpg\u0026#34; ) get_response.status with open(\u0026#34;dolphins_from_cloud.jpg\u0026#34;,\u0026#34;wb\u0026#34;) as f: f.write(get_response.data.content) This get_object function above returns an object of class oci.response.Response. So does list_buckets. The attribute data type depends on the request. For get_object the data has type oci._vendor.requests.models.Response. I did not find the documentation particularly strong at this point, but I explored the returned object in Python and guessed the method to use in order to fetch the content so I could store it to a file.\nI have a lot more to investigate here, both in Object Storage and other services in OCI. In next post I will look at how to access Object Storage from PL/SQL.\nOracle uses both tenancy and tenant in the documentation, so I\u0026rsquo;ll do the same, possibly not by the same logic.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2024/05/stuffing-files-into-oci-object-storage/","summary":"\u003cp\u003eStuffing files away in the cloud might not be on your bucket list, but object storage in Oracle Cloud is quite useful in several situations.\u003c/p\u003e\n\u003ch2 id=\"storing-files-in-oci-object-storage\"\u003eStoring files in OCI Object Storage\u003c/h2\u003e\n\u003cp\u003eObject storage can be used as you would use Amazon\u0026rsquo;s S3 through a special API provided in OCI. You can even store your RMAN database backups through a Swift API. But I will just upload and download a bunch of files.\u003c/p\u003e","title":"Stuffing files into OCI object storage"},{"content":"ORA-01835 day of week conflicts with Julian date - This means your weekday in input is wrong.\nORA-01835 This error message is also a bit confusing, especially if you were not using a Julian date at all; the error messages probably reflects how Oracle calculates dates.\nYou are likely to encounter this error when you are do testing or some how construct a string to be parsed by to_date or to_timestamp. It just means that you specified an incorrect weekday for a date. Like today, April 17, 2024 is a Wednesday.\nThis works:\nselect to_date(\u0026#39;Wed Apr 17 2024\u0026#39;,\u0026#39;Dy Mon dd yyyy\u0026#39;) from dual; Specifying wrong weekday leads to the error above:\nselect to_date(\u0026#39;Tue Apr 17 2024\u0026#39;,\u0026#39;Dy Mon dd yyyy\u0026#39;) from dual; 01835. 00000 - \u0026#34;day of week conflicts with Julian date\u0026#34; *Cause: A Julian date was specified with the day of the week, but the weekday did not correspond to the Julian date. If the day of the week was specified with a Julian date, it must be the same day of the week as the Julian date. *Action: Remove the day of the week value from the date specification or enter the correct day of the week for the Julian date. Of course you can just parse the date string without the weekday. I encountered this error while writing an adapter that parses timestamps from Enterprise Manager. When sending notifications to a webhook it uses a format nobody else does, and certainly not ISO-8601. My adapter parses the timestamp before it forwards the payload to another REST API. Of course my adapter had bugs, and while testing different date strings I changed the date, but not the weekday.\nFor other short explanations on DATE problems, see ORA-01861 and ORA-01821 ","permalink":"https://enesi.no/2024/04/ora-01835/","summary":"\u003cp\u003eORA-01835 \u003cem\u003eday of week conflicts with Julian date\u003c/em\u003e - This means your weekday in input is wrong.\u003c/p\u003e\n\u003ch2 id=\"ora-01835\"\u003eORA-01835\u003c/h2\u003e\n\u003cp\u003eThis error message is also a bit confusing, especially if you were not using a Julian date at all; the error messages probably reflects how Oracle calculates dates.\u003c/p\u003e\n\u003cp\u003eYou are likely to encounter this error when  you are do testing or some how construct a string to be parsed by \u003ccode\u003eto_date\u003c/code\u003e or \u003ccode\u003eto_timestamp\u003c/code\u003e. It just means that you specified an incorrect weekday for a date. Like today, April 17, 2024 is a Wednesday.\u003c/p\u003e","title":"ORA-01835"},{"content":"The error message ORA-01950 should be easy to solve, but perhaps you forgot the indexes?\nORA-01950 ORA-01950 no privileges on tablespace \u0026hellip; usually has a couple obvious solutions. If the tablespace in the error message is wrong like SYSTEM, the default tablespace must be changed for the user. Change it with something like this:\nalter user luser default tablespace users quota 1g on users; If the tablespace is correct, the user may be lacking a quota on the tablespace. The fix is almost the same:\nalter user luser quota 1g on users; The user then needs to move the table into the new tablespace. But insert may still fail, as this demo shows. It is based on a situation where I created a PDB with only the PDBADMIN user and forgot to add any additional tablespaces.\nCreate a new pluggable database:\ncreate pluggable database blogtest admin user pdbadmin identified by blogtest storage (maxsize 10g) tempfile reuse file_name_convert=(\u0026#39;pdbseed\u0026#39;,\u0026#39;blogtest\u0026#39;) create_file_dest=\u0026#39;/opt/oracle/oradata/ORCLCDB/\u0026#39;; alter pluggable database blogtest open; alter session set container=blogtest; grant resource to pdbadmin; Connect to the new database as PDBADMIN.\ncreate table foo ( id number generated always as identity, bar varchar2(42), constraint foo_pk primary key(id)); Note that Oracle does not complain yet 1. But when I try to insert a row:\ninsert into foo (bar) values (\u0026#39;failure\u0026#39;); I get:\nORA-01950: no privileges on tablespace \u0026lsquo;SYSTEM\u0026rsquo;\nOK, back to SYS (or SYSTEM):\ncreate tablespace users datafile \u0026#39;/opt/oracle/oradata/ORCLCDB/blogtest/users01.dbf\u0026#39; size 100M autoextend on next 1g; alter user pdbadmin default tablespace user quota unlimited on users; In the PDBADMIN session I move the table like this and retry the insert:\nalter table foo move tablespace users; insert into foo (bar) values (\u0026#39;failure\u0026#39;); Still the same error. Why? The table has a primary key constraint and Oracle created an index for it in the same tablespace, you can check it with:\nselect index_name,tablespace_name from user_indexes; The fix is:\nalter index foo_pk rebuild tablespace users; Way back Oracle always created one segment when you created a table and you would get the error message immediately, I think, it is a long time ago. This is controlled by the parameter deferred_segment_creation which is set to true by default. Check this 13 year old blogpost at oracle-base.com \u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://enesi.no/2024/04/ora-01950/","summary":"\u003cp\u003eThe error message ORA-01950 should be easy to solve, but perhaps you forgot the indexes?\u003c/p\u003e\n\u003ch2 id=\"ora-01950\"\u003eORA-01950\u003c/h2\u003e\n\u003cp\u003eORA-01950 \u003cem\u003eno privileges on tablespace \u0026hellip;\u003c/em\u003e usually has a couple obvious solutions. If the tablespace in the error message is wrong like SYSTEM, the default tablespace must be changed for the user. Change it with something like this:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003ealter\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003euser\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eluser\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003edefault\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003etablespace\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eusers\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003equota\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"k\"\u003eg\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eon\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eusers\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eIf the tablespace is correct, the user may be lacking a quota on the tablespace. The fix is almost the same:\u003c/p\u003e","title":"ORA-01950"},{"content":"Large Language Models (LLM) are everywhere now, now you can even get help with your SQL. A few months back, Oracle introduced a feature in the Autonomous Database where you can ask normal questions in order to query your data.\nFirst Journey with SelectAI Brendan Tierney at Oralytics has written two good articles, SelectAI – the beginning of a journey and SelectAI – Doing something useful on a cool feature in the Oracle Autonomous database; SelectAI. He shows how you in a few steps can query your data using natural language. I followed the steps in his posts, and in a few minutes got it to work in my own Autonomous Database that I am using in the Oracle Free Tier.\nThere are plenty of interesting data sets out there, but I wanted to explore some of my own data, because I obviously know them better and have other motivations to play with them.\nI have the simplest data model you can imagine and the data are collected with a made-in-2-minutes APEX app I have had on my mobile for a while.\nThe app allows me to record where I am with a NAME (Name or description) at different times (EVENT_TIME) with location stored in LATITUDE and LONGITUDE, and I may upload a picture into the IMAGE column. Obviously I had to make it slightly fancy and added a second table CATEGORY whose purpose is quite obvious.\nThe tables are stored in a schema called WKSP_EVENT and this is the code to create the profile.\nBEGIN --DBMS_CLOUD_AI.drop_profile(profile_name =\u0026gt; \u0026#39;OPEN_AI\u0026#39;); DBMS_CLOUD_AI.create_profile( profile_name =\u0026gt; \u0026#39;OPEN_AI\u0026#39;, attributes =\u0026gt; \u0026#39;{\u0026#34;provider\u0026#34;: \u0026#34;openai\u0026#34;, \u0026#34;credential_name\u0026#34;: \u0026#34;OPENAPI_CRED\u0026#34;, \u0026#34;object_list\u0026#34;: [{\u0026#34;owner\u0026#34;: \u0026#34;WKSP_EVENT\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;EVENT\u0026#34;}, {\u0026#34;owner\u0026#34;: \u0026#34;WKSP_EVENT\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;CATEGORY\u0026#34;}] }\u0026#39;); END; The rest of the commands are the same as shown in Brendan\u0026rsquo;s posts.\nSince I have a paid subscription to OpenAI I decided to use the openai provider. It failed at first attempt with HTTP code 429 and I figured out it was because my monthly subscription did not include API access. After adding $10 to my credit I was up and running.\nMy first attempt did not go well:\nselect ai how many event; When it fails it returns one row with an apology and some kind of suggestions with SQL, in this case:\n\u0026ldquo;Sorry, unfortunately a valid SELECT statement could not be generated for your natural language prompt. Here is some more information to help you further: To count the number of events in the \u0026ldquo;WKSP_EVENT\u0026rdquo; table, you can use the following Oracle SQL query:\nSELECT COUNT(*) AS event_count FROM WKSP_EVENT Looking at this SQL it is obvious what is missing. Peeking back at Brendan\u0026rsquo;s post I reckoned I should express myself better:\nselect ai how many events exists; In this case it returns with an answer. In order to see the SQL it generated I can get it by adding an action after the ai keyword as documented here and demonstrated in his second blog post.\nselect ai showsql how many events exists; It looked right to me. These two also worked fine:\nselect ai when is the earliest event; select ai how many events in each category; Time to try something more difficult. Two of my categories are Home and Conference. I asked:\nselect ai what is the average distance between events at home and events at conference; At one attempt it returned just one row and a column with NULL, later it failed. When I asked it to show the SQL using the same question it reported back:\nSELECT AVG(distance) AS average_distance FROM ( SELECT SQRT((e1.LONGITUDE - e2.LONGITUDE) * (e1.LONGITUDE - e2.LONGITUDE) + (e1.LATITUDE - e2.LATITUDE) * (e1.LATITUDE - e2.LATITUDE)) AS distance FROM WKSP_EVENT.EVENT e1 JOIN WKSP_EVENT.EVENT e2 ON e1.CATEGORY_ID = e2.CATEGORY_ID WHERE e1.NAME = \u0026#39;Home\u0026#39; AND e2.NAME = \u0026#39;Conference\u0026#39; ) subquery But the SQL has two errors; it uses NAME from EVENT rather than from CATEGORY, and the formula used is wrong (unless you believe the Earth is flat). I had now idea about the formula, but ChatGPT gave me a lesson on the Haversine formula\nMy expectations were perhaps too high for this tool.\nI work from home a lot and wondered what is the longest stretch I have stayed at home:\nselect ai what is the longest period I stayed at home; Didn\u0026rsquo;t work either, and it came up with what suggestions about what kind of tables and columns I needed to answer the question, what an insult.\nMy two tables did not have any comments, and I wondered if that would make any difference. So I added comments to the tables and the columns as best as I could. Then I recreated the profile with an extra attribute \u0026quot;comments\u0026quot;: true:\nBEGIN DBMS_CLOUD_AI.drop_profile(profile_name =\u0026gt; \u0026#39;OPEN_AI\u0026#39;); DBMS_CLOUD_AI.create_profile( profile_name =\u0026gt; \u0026#39;OPEN_AI\u0026#39;, attributes =\u0026gt; \u0026#39;{\u0026#34;provider\u0026#34;: \u0026#34;openai\u0026#34;, \u0026#34;credential_name\u0026#34;: \u0026#34;OPENAPI_CRED\u0026#34;, \u0026#34;comments\u0026#34;: true, \u0026#34;object_list\u0026#34;: [{\u0026#34;owner\u0026#34;: \u0026#34;WKSP_EVENT\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;EVENT\u0026#34;}, {\u0026#34;owner\u0026#34;: \u0026#34;WKSP_EVENT\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;CATEGORY\u0026#34;}] }\u0026#39;); END; / EXEC DBMS_CLOUD_AI.set_profile(\u0026#39;OPEN_AI\u0026#39;); But it did not make any difference, sigh.\nI tried to reformulate my question a bit, but no luck. At one point I think OpenAI got tired of SQL altogether and returned with:\nSorry, unfortunately a valid SELECT statement could not be generated for your natural language prompt. Here is some more information to help you further:\nThe longest period of events in the category of Home can vary depending on the specific context or perspective. However, some possible examples of long-lasting events in the category of Home could include:\nHomeownership: The period of time during which an individual or family owns and resides in a particular home. This can span several decades or even a lifetime.\nHome renovations: The process of making significant changes or improvements to a home. Depending on the scale and complexity of the renovations, this can take months or even years to complete.\nHome construction: The duration of time it takes to build a new home from scratch. This can range from several months to a couple of years, depending on the size and complexity of the project.\nHome restoration: The process of restoring an older or damaged home to its original condition. This can be a lengthy process, especially for historical homes or properties requiring extensive repairs.\nHome gardening: The ongoing cultivation and maintenance of a garden or outdoor space at home. This can be a continuous activity that spans multiple seasons or even years.\nIt\u0026rsquo;s important to note that the length of these events can vary greatly depending on individual circumstances, resources, and goals.\nAnyway, I noticed in the OpenAI dashboard at platform.openai.com that all requests to the API used the GPT-3.5 Turbo model. Reading the documentation is a good idea. You can specify which model to use like this:\nBEGIN DBMS_CLOUD_AI.drop_profile(profile_name =\u0026gt; \u0026#39;OPEN_AI2\u0026#39;); DBMS_CLOUD_AI.create_profile( profile_name =\u0026gt; \u0026#39;OPEN_AI2\u0026#39;, attributes =\u0026gt; \u0026#39;{\u0026#34;provider\u0026#34;: \u0026#34;openai\u0026#34;, \u0026#34;model\u0026#34;: \u0026#34;gpt-4\u0026#34;, \u0026#34;credential_name\u0026#34;: \u0026#34;OPENAPI_CRED2\u0026#34;, \u0026#34;object_list\u0026#34;: [{\u0026#34;owner\u0026#34;: \u0026#34;WKSP_EVENT\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;EVENT\u0026#34;}, {\u0026#34;owner\u0026#34;: \u0026#34;WKSP_EVENT\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;CATEGORY\u0026#34;}] }\u0026#39;); END; The available models are listed in the page I linked to above. OpenAI explains this:\nFor many basic tasks, the difference between GPT-4 and GPT-3.5 models is not significant. However, in more complex reasoning situations, GPT-4 is much more capable than any of our previous models.\nOne could think GPT-4 will make a difference in this situation. The new profile was activated with EXEC DBMS_CLOUD_AI.set_profile('OPEN_AI2');.\nNew attempt:\nselect ai showsql what is the longest chronological sequence of events with the same category; and new dissapointment:\nSorry, unfortunately a valid SELECT statement could not be generated for your natural language prompt. Here is some more information to help you further: To find the longest chronological sequence of events with the same category, you can use the LAG and LEAD functions to compare the current row with the previous and next rows. Then, use the ROW_NUMBER function to assign a unique number to each row in the sequence. Finally, use the MAX function to find the longest sequence.\nHere is the SQL query:\nWITH seq AS ( SELECT e1.ID, e1.EVENT_TIME, e1.CATEGORY_ID, CASE WHEN e1.CATEGORY_ID = LAG(e1.CATEGORY_ID) OVER (ORDER BY e1.EVENT_TIME) THEN 0 ELSE 1 END AS start_of_sequence, CASE WHEN e1.CATEGORY_ID = LEAD(e1.CATEGORY_ID) OVER (ORDER BY e1.EVENT_TIME) THEN 0 ELSE 1 END AS end_of_sequence FROM WKSP_EVENT.EVENT e1 ), seq2 AS ( SELECT s1.*, SUM(s1.start_of_sequence) OVER (ORDER BY s1.EVENT_TIME) AS sequence_number FROM seq s1 ), seq3 AS ( SELECT s2.sequence_number, COUNT(*) AS sequence_length, MIN(s2.EVENT_TIME) AS sequence_start, MAX(s2.EVENT_TIME) AS sequence_end, s2.CATEGORY_ID FROM seq2 s2 WHERE s2.start_of_sequence = 1 OR s2.end_of_sequence = 1 GROUP BY s2.sequence_number, s2.CATEGORY_ID ) SELECT MAX(s3.sequence_length) AS longest_sequence, s3.sequence_start, s3.sequence_end, c.NAME AS category_name FROM seq3 s3 JOIN WKSP_EVENT.CATEGORY c ON c.ID = s3.CATEGORY_ID ORDER BY longest_sequence DESC FETCH FIRST ROW ONLY; Certainly more fancy but still not correct, even after correcting the error in the main select.\nAt this point I think I have the following options:\nLearn to ask better (English) questions Wait for an improvement of SelectAI Lower my expectations Find more relevant data While I was writing 1. above I wondered, does it understand Norwegian?\nselect ai Hvor mange rader finnes det i event; It does! What about that sequence I tried earlier?\nselect ai Hva er den lengste kronologiske rekken av rader i event med samme kategori; And the SQL is:\nWITH event_sequence AS ( SELECT e.*, ROW_NUMBER() OVER (ORDER BY event_time) AS seq_num, ROW_NUMBER() OVER (PARTITION BY category_id ORDER BY event_time) AS seq_num_category FROM wksp_event.event e ) SELECT MIN(event_time) AS start_time, MAX(event_time) AS end_time, COUNT(*) AS sequence_length FROM event_sequence GROUP BY seq_num - seq_num_category, category_id ORDER BY sequence_length DESC FETCH FIRST ROW ONLY The result looks right from listing the events in order, but it took me a few seconds (not telling you how many) to understand the group by expression.\nI think this can actually become a nice feature.\nThere is more to explore in this feature, like varying the temperature attribute in the profile.\nBy the way, writing this post cost me $0.06 to OpenAI, and $0.00 to Oracle since I am using the Free Tier (See Your Own Oracle Database Lab in 1-2-3 and in Norwegian Gratis infrastruktur med Oracle Free Tier )\n","permalink":"https://enesi.no/2024/04/first-journey-with-selectai/","summary":"\u003cp\u003eLarge Language Models (LLM) are everywhere now, now you can even get help with your SQL. A few months back, Oracle introduced a feature in the Autonomous Database where you can ask \u003cem\u003enormal\u003c/em\u003e questions in order to query your data.\u003c/p\u003e\n\u003ch2 id=\"first-journey-with-selectai\"\u003eFirst Journey with SelectAI\u003c/h2\u003e\n\u003cp\u003eBrendan Tierney at Oralytics has written two good articles, \u003ca href=\"https://oralytics.com/2024/03/16/selectai-the-beginning-of-a-journey/\" target=\"_blank\" rel=\"noopener\"\u003eSelectAI – the beginning of a journey\u003c/a\u003e\n and \u003ca href=\"https://oralytics.com/2024/03/22/selectai-doing-something-useful/\" target=\"_blank\" rel=\"noopener\"\u003eSelectAI – Doing something useful\u003c/a\u003e\n on a cool feature in the Oracle Autonomous database; SelectAI. He shows how you in a few steps can query your data using natural language. I followed the steps in his posts, and in a few minutes got it to work in my own Autonomous Database that I am using in the Oracle Free Tier.\u003c/p\u003e","title":"First Journey with SelectAI"},{"content":"Your Own Oracle Database Lab in 1-2-3 #JoelKallmanDay You can\u0026rsquo;t release to production all the time. Sometimes you need to learn, try out stuff, and even have some fun. This post is about how you can get an Oracle database up and running in minutes (even in seconds) when you just need to do some experiments.\nThe fastest way to achieve that is using a prebuilt container from Oracle. This works nicely on Linux, Windows, and MacOS.\nInstall Podman Podman is fully compatible with Docker, on Linux you may even do alias docker=podman if you have old scripts using the docker command.\nDownload Podman from podman.io . I use Linux when I can, but given that most people have a Windows PC I decided to show you the procedure on a mini PC I have with Windows 10. I decided to only install the CLI version:\nRun the downloaded file to install Podman, and open a command window. The first time you need to init Podman:\npodman machine init It downloads a VM image:\nThen you can start it with:\npodman machine start As demonstrated in screenshot above, it prints out a warning to change a setting in case the containers will need root permissions. I think that is a good idea:\npodman machine set --rootful podman machine stop podman machine start Run a container With this in place we are now ready to download and start a container. Let us start with the Oracle 23c Free version. Go to Oracle Container Registry and click on Database:\nand from there click on free:\nThe pages gives you a few option. If you want to download the image only without actually running it you can use the command podman pull container-registry.oracle.com/database/free:latest, or you can just run it and it will download and start it in one go:\npodman run -p 1521:1521/tcp --name ora23cfree container-registry.oracle.com/database/free:latest The -p option means port 1521 from the container is available on the host (your PC), so we can connect to the database through the listener later. You are free to name it to whatever you want.\nThe first time you run this it will take some time to download the layers and in the beginning it will look like this:\nWhile you are waiting for the download to complete you can have a look on the other containers at https://container-registry.oracle.com/ . I\u0026rsquo;ll come back to those later. After a while it will start the container and print out messages like this:\nIf you include the option -d to the run command above it will run the container in the background. Since I did not specify it, I\u0026rsquo;ll just open another command window and run the following commands. We did not specify a password, but we can connect to the container and set one. But first verify that it is running:\npodman ps The following command will run bash in it with a terminal in interactive mode (that is what the -ti flag is for):\npodman exec -ti ora23cfree bash Since you don\u0026rsquo;t know the sys/system password you can set it with the next command (or you can connect to the database, see the following step):\nls ./setPassword.sh test42 Now you can connect to the database with SQL Developer or some other client from outside the container (that is, from your PC), but since we are here, we can connect directly to the CDB with sqlplus and test the new syntax:\nsqlplus / as sysdba select sysdate; show pdbs Connect with SQLcl For fast connection to an Oracle database nothing beats Oracle SQL Developer Command Line, SQLcl. You can download it from https://www.oracle.com/database/sqldeveloper/technologies/sqlcl/download/ or https://download.oracle.com/otn_software/java/sqldeveloper/sqlcl-latest.zip will always give you the latest version, no interaction required. It does need Java, at least version 11. Unzip the file into some place where you\u0026rsquo;ll find it later. Top level is the directory sqlcl. The following commands is executed on a Windows PC:\ncd sqlcl\\bin sql -L system/test42@/FREE In the connection string above both the host and port is left out and it will connect to default port 1521 on the same machine. I always use option -L with SQLcl; it will not reprompt for username and password if the initial connect does not succeed.\nSlightly more advanced Ctrl-C in the window where the container was started will kill it. You can start it again with podman start ora23cfree, but I want to delete it and start over with a few options as shown on the webpage at container-registry.oracle.com mentioned earlier. I want to preserve the datafiles in a volume and set the password for SYS, SYSTEM and PDBADMIN:\nmkdir c:\\oradata podman rm ora23cfree podman run -p 1521:1521/tcp --name ora23cfree -e ORACLE_PWD=secret#42 -v /mnt/c/oradata:/opt/oracle/oradata container-registry.oracle.com/database/free:latest Note the syntax for specifying the local directory: /mnt/c/oradata for the Windows directory c:\\oradata. The container startup takes longer this time since it copies the database files and does some other stuff before the database becomes available:\nThe message DATABASE IS READY TO USE! means just that:\nYou can now connect as before with SQLcl and the new password. This time connect to the PDB, and create a user:\nsql -L system/secret#42@/freepdb1 create user testuser identified by testuser42 default tablespace users quota unlimited on users; grant connect, resource to testuser; Disconnect with Ctrl-D and reconnect as the new user:\nsql -L testuser/testuser42@/freepdb1 Create a table\ncreate table foo as select level id, dbms_random.string('a',100) bar from dual connect by level \u0026lt; 1000; Disconnect again with Ctrl-D. Now we can stop the container, and restart it:\npodman stop ora23cfree podman rm ora23cfree podman run -d -p 1521:1521/tcp --name ora23cfree -e ORACLE_PWD=secret#42 -v /mnt/c/oradata:/opt/oracle/oradata container-registry.oracle.com/database/free:latest # Wait a few seconds for the container to start sql -L testuser/testuser42@\u0026amp;/freepdb1 select count(*) from foo; When you execute the stop command above you will notice messages from the alert log in the other window where the podman was run the first time. Even though I deleted the container the database is preserved since I ran the container with the same volume path as before. Also the startup time now is just a few seconds (I am running this on a small Core i5 PC with SSD, YMMV.) Also note that I added the -d option this time in order to continue to work from the same window. You can check the log (output from the alert log in this case) with podman logs -f ora23cfree, the -f options means follow; it will poll the log for more data continuously.\nFrom the testing above, we can see that if you are in hurry and don\u0026rsquo;t need to preserve the database after testing, then skip the volume option; it starts much faster. You can stop the container, start it again, and the data will be there until you delete the container with podman rm ora23cfree. When the volume options has been used, you can stop and remove the container, and next time you run it (create a new container with run command and volume option -v as above), it will start almost immediately and with the old data preserved. The main difference between these two options is the time it takes to run the container when the volume is empty (i.e. the database has never been created in the volume.)\nThis solution is perfect for extensive testing (integration tests, system tests, etc), or just try out stuff on your own.\nMore images As you saw Oracle provides a lot of different images you can try. You can even try Autonomous database on your PC without using the cloud. This was recently released by Oracle, and I plan to explore it in another post. I am already using the Autonomous database in the Oracle Free Tier, so it is more out of curiosity.\nPostscript Joel Kallman did a fantastic job for the Oracle community. It was all about sharing and encouraging others. Some years ago it used to be much more complicated when you wanted to learn and try out Oracle, but now we have access to many options like these containers, VM images, Oracle Free Tier, etc. If you manage to stay curious there is and will be a wealth of things to try out.\n","permalink":"https://enesi.no/2023/10/your-own-oracle-database-lab-in-1-2-3/","summary":"\u003ch2 id=\"your-own-oracle-database-lab-in-1-2-3-joelkallmanday\"\u003eYour Own Oracle Database Lab in 1-2-3 #JoelKallmanDay\u003c/h2\u003e\n\u003cp\u003eYou can\u0026rsquo;t release to production all the time. Sometimes you need to learn, try out stuff, and even have some fun. This post is about how you can get an Oracle database up and running in minutes (even in seconds) when you just need to do some experiments.\u003c/p\u003e\n\u003cp\u003eThe fastest way to achieve that is using a prebuilt container from Oracle. This works nicely on Linux, Windows, and MacOS.\u003c/p\u003e","title":"Your Own Oracle Database Lab in 1-2-3"},{"content":"Oracle REST Data Services (ORDS) offers a seamless installation experience in the root container. By making a simple configuration adjustment, all pluggable databases can effortlessly be addressed, empowering users with enhanced functionality and convenience.\nInstalling ORDS in the root container At one of my customers, Oracle Application Express (APEX) has been successfully deployed with Oracle REST Data Services (ORDS) running on Tomcat. However, ORDS was primarily utilized as a proxy for APEX in the database, with limited exploration of its broader capabilities. Typically, ORDS was installed within the pluggable database (PDB) as a natural choice.\nIn a recent project, I aimed to leverage ORDS extensively and take advantage of one of its bundled REST APIs known as the Pluggable Database Lifecycle Management API. To employ this API effectively, it was necessary to install ORDS in the root container, enabling the creation and deletion of pluggable databases.\nHowever, one crucial caveat surfaced during the process: ORDS doesn\u0026rsquo;t permit installation in the root container (CDB$ROOT) if it already exists in one or more PDBs. In such cases, uninstallation from all other containers is required before proceeding.\nImportant note: Uninstalling ORDS will remove any configuration for ORDS-enabled schemas and objects. It is imperative to create a comprehensive list of these schemas and objects (such as tables, views, functions, procedures, or packages) to ensure their re-enablement after installation. While interactively managing a small number of objects can be done via SQL Developer, it\u0026rsquo;s advisable to create scripts and store them in your repository for future reference.\nBased on my experience, the latest version of ORDS can be used to uninstall older versions. Simply execute the following command from the directory where you\u0026rsquo;ve extracted the latest version :\nbin/ords uninstall It will ask for the connection details for the PDB, and you have to execute this for each PDB where ORDS is installed.\nNow is time to install ORDS into CDB$ROOT. The following example does a silent install:\necho -e \u0026quot;YourSYSpassword\\nORDS_PUBLIC_USER_password42\u0026quot; | \\ bin/ords --config $HOME/ordsconfig install --admin-user SYS \\ --proxy-user --db-hostname your_host --db-port 1521 \\ --db-servicename db_service --feature-sdw true --password-stdin To ensure access to all PDBs, a crucial configuration step involves setting the db.serviceNameSuffix parameter to the same value as the db_domain database parameter. For detailed instructions, please refer to Section 7.3 of the installation guide .\nords --config $HOME/ordsconfig config set \u0026quot;db.serviceNameSuffix\u0026quot; \u0026quot;.example.com\u0026quot; Once configured, a PDB named BARDB, for instance, can be accessed through the URL /ords/bardb.\nNow you may have many application servers connecting either to the CDB$ROOT or to any of the PDBs. For instance if you install APEX in one PDB and want to deploy ORDS somewhere to route requests to it you will install ORDS as you did before. The only difference is that the password for user ORDS_PUBLIC_USER is the same for all PDBs since it is now a common user.\nPostscript I got some assistance from ChatGPT with parts of this text, mostly to see if it could improve the language. But it always comes up with words like leverage, empower, and imperative, not quite my style. I\u0026rsquo;ll stick with spelling check for future posts.\nI don\u0026rsquo;t recommend installing ORDS in the root container (CDB$ROOT) unless you are going to use the Pluggable Database Lifecycle Management API that comes with it. At this moment I am not aware of any other benefits. The downside with this installation is that you add a potential dependency between the containers. Testing so far gives me the impression that ORDS is flexible when it comes to versions; I have used lower versions of ORDS connected to an upgraded ORDS in the database.\n","permalink":"https://enesi.no/2023/10/moving-ords-from-pdb-to-cdb/","summary":"\u003cp\u003eOracle REST Data Services (ORDS) offers a seamless installation experience in the root container. By making a simple configuration adjustment, all pluggable databases can effortlessly be addressed, empowering users with enhanced functionality and convenience.\u003c/p\u003e\n\u003ch2 id=\"installing-ords-in-the-root-container\"\u003eInstalling ORDS in the root container\u003c/h2\u003e\n\u003cp\u003eAt one of my customers, Oracle Application Express (APEX) has been successfully deployed with Oracle REST Data Services (ORDS) running on Tomcat. However, ORDS was primarily utilized as a proxy for APEX in the database, with limited exploration of its broader capabilities. Typically, ORDS was installed within the pluggable database (PDB) as a natural choice.\u003c/p\u003e","title":"Moving ORDS from PDB to CDB"},{"content":"A Norwegian version of this post can be found here .\nAdd SRID 5973 to Oracle Spatial Oracle Spatial comes with over 6000 coordinate systems, but SRID 5973 is currently missing. However this is easy to add with a few insert-statements.\nRun this as user SYSTEM:\ninsert into sdo_datums ( DATUM_ID, DATUM_NAME, DATUM_TYPE, ELLIPSOID_ID, PRIME_MERIDIAN_ID, INFORMATION_SOURCE, DATA_SOURCE, SHIFT_X, SHIFT_Y, SHIFT_Z, ROTATE_X, ROTATE_Y, ROTATE_Z, SCALE_ADJUST, IS_LEGACY, LEGACY_CODE) values ( 1096, \u0026#39;Norway Normal Null 2000\u0026#39;, \u0026#39;VERTICAL\u0026#39;, null, null, \u0026#39;Norwegian Mapping Authority\u0026#39;, \u0026#39;OGP\u0026#39;, NULL, NULL, NULL, NULL, NULL, NULL, NULL, \u0026#39;FALSE\u0026#39;, NULL); INSERT INTO sdo_coord_ref_system ( srid, coord_ref_sys_name, coord_ref_sys_kind, coord_sys_id, datum_id, geog_crs_datum_id, source_geog_srid, projection_conv_id, cmpd_horiz_srid, cmpd_vert_srid, information_source, data_source, is_legacy, legacy_code, legacy_wktext, legacy_cs_bounds, is_valid, supports_sdo_geometry ) VALUES ( \u0026#39;5941\u0026#39;, \u0026#39;NN2000 height\u0026#39;, \u0026#39;VERTICAL\u0026#39;, \u0026#39;6499\u0026#39;, \u0026#39;5206\u0026#39;, \u0026#39;5206\u0026#39;, NULL, NULL, NULL, NULL, \u0026#39;Norwegian Mapping Authority.\u0026#39;, \u0026#39;EPSG\u0026#39;, \u0026#39;FALSE\u0026#39;, NULL, NULL, NULL, \u0026#39;TRUE\u0026#39;, \u0026#39;TRUE\u0026#39; ); INSERT INTO sdo_coord_ref_system ( srid, coord_ref_sys_name, coord_ref_sys_kind, coord_sys_id, datum_id, geog_crs_datum_id, source_geog_srid, projection_conv_id, cmpd_horiz_srid, cmpd_vert_srid, information_source, data_source, is_legacy, legacy_code, legacy_wktext, legacy_cs_bounds, is_valid, supports_sdo_geometry ) VALUES ( \u0026#39;5973\u0026#39;, \u0026#39;ETRS89 / UTM zone 33N + NN2000 height\u0026#39;, \u0026#39;COMPOUND\u0026#39;, NULL, NULL, NULL, NULL, NULL, \u0026#39;25833\u0026#39;, \u0026#39;5941\u0026#39;, \u0026#39;Norwegian Mapping Authority\u0026#39;, \u0026#39;EPSG\u0026#39;, \u0026#39;FALSE\u0026#39;, NULL, NULL, NULL, \u0026#39;TRUE\u0026#39;, \u0026#39;TRUE\u0026#39; ); commit; ","permalink":"https://enesi.no/2023/04/adding-an-srid-to-spatial/","summary":"\u003cp\u003eA Norwegian version of this post can be found \u003ca href=\"/no/2023/04/legg-til-referansesystem-5979-til-oracle-spatial/\"\u003ehere\u003c/a\u003e\n.\u003c/p\u003e\n\u003ch2 id=\"add-srid-5973-to-oracle-spatial\"\u003eAdd SRID 5973 to Oracle Spatial\u003c/h2\u003e\n\u003cp\u003eOracle Spatial comes with over 6000 coordinate systems, but SRID 5973 is currently missing. However this is easy to add with a few insert-statements.\u003c/p\u003e\n\u003cp\u003eRun this as user SYSTEM:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003einsert\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003einto\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003esdo_datums\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eDATUM_ID\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eDATUM_NAME\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eDATUM_TYPE\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eELLIPSOID_ID\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003ePRIME_MERIDIAN_ID\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eINFORMATION_SOURCE\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eDATA_SOURCE\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eSHIFT_X\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eSHIFT_Y\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eSHIFT_Z\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eROTATE_X\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eROTATE_Y\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eROTATE_Z\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eSCALE_ADJUST\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eIS_LEGACY\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eLEGACY_CODE\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003evalues\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"mi\"\u003e1096\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s1\"\u003e\u0026#39;Norway Normal Null 2000\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s1\"\u003e\u0026#39;VERTICAL\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003enull\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003enull\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s1\"\u003e\u0026#39;Norwegian Mapping Authority\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s1\"\u003e\u0026#39;OGP\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s1\"\u003e\u0026#39;FALSE\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eINSERT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eINTO\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003esdo_coord_ref_system\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003esrid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecoord_ref_sys_name\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecoord_ref_sys_kind\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecoord_sys_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003edatum_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003egeog_crs_datum_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003esource_geog_srid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eprojection_conv_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecmpd_horiz_srid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecmpd_vert_srid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003einformation_source\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003edata_source\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eis_legacy\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003elegacy_code\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003elegacy_wktext\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003elegacy_cs_bounds\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eis_valid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003esupports_sdo_geometry\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eVALUES\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;5941\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;NN2000 height\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;VERTICAL\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;6499\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;5206\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;5206\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;Norwegian Mapping Authority.\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;EPSG\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;FALSE\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;TRUE\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;TRUE\u0026#39;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eINSERT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eINTO\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003esdo_coord_ref_system\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003esrid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecoord_ref_sys_name\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecoord_ref_sys_kind\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecoord_sys_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003edatum_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003egeog_crs_datum_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003esource_geog_srid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eprojection_conv_id\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecmpd_horiz_srid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003ecmpd_vert_srid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003einformation_source\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003edata_source\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eis_legacy\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003elegacy_code\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003elegacy_wktext\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003elegacy_cs_bounds\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eis_valid\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003esupports_sdo_geometry\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eVALUES\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;5973\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;ETRS89 / UTM zone 33N + NN2000 height\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;COMPOUND\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;25833\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;5941\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;Norwegian Mapping Authority\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;EPSG\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;FALSE\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eNULL\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;TRUE\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;TRUE\u0026#39;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003ecommit\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e","title":"Adding an SRID to Spatial"},{"content":"REST: Don\u0026rsquo;t Let Them Wait I have been working with Oracle REST Data Services (ORDS) lately and have observed others struggling with third-party REST APIs from the sideline. Many things need to be done right, but speed is critical. Some systems are susceptible to latency, especially if the call to your API is just one piece of a more comprehensive workflow. Spare yourself from the hassle when people call and complain about timeouts.\nThe short answer is to do it asynchronously. Asynchronous means the client does not wait for you to finish processing the request. Instead, the code behind your API submits a job that runs independently in the database after validation and other necessary processing. The client can poll for the status of the request later. (The response to the POST request can include the self-link that simplifies this.)\nIf you work with ORDS, the solution is easy. Here is a simple example of how to do that:\nbegin DBMS_SCHEDULER.CREATE_JOB ( job_name =\u0026gt; \u0026#39;process_42\u0026#39;, job_type =\u0026gt; \u0026#39;PLSQL_BLOCK\u0026#39;, job_action =\u0026gt; \u0026#39;begin stuff.process(p_lotsofwork); end; \u0026#39;, enabled =\u0026gt; true); end; / First of all, the database user needs the CREATE JOB privilege. When submitted like this, the database will drop the job after completion, so there is no need to clean up later.\nOne important note is that a call to this procedure has an implicit commit, so you should put it at the end of your routine and after all validations, any previous work not committed will be after the call.\nHere is a link to the documentation (version 19c) for more details.\nSimple requests should return immediately. If the query takes too long, you need to optimize the query and achieve acceptable response times.\nIf you suspect that an SQL originating from ORDS takes too long in the database (and not elsewhere in the tech stack), you will easily find the session in the v$session view. The answer to this question on Stack Overflow shows how it works. (The guy who answered that has many good blog posts on ORDS and REST )\nMore advanced ways exist, like Advanced Queuing aka Transactional Event Queues in 21c, but a simple scheduler job will do in many use cases.\n","permalink":"https://enesi.no/2023/03/rest-dont-let-them-wait/","summary":"\u003ch2 id=\"rest-dont-let-them-wait\"\u003eREST: Don\u0026rsquo;t Let Them Wait\u003c/h2\u003e\n\u003cp\u003eI have been working with Oracle REST Data Services (ORDS) lately and have observed others struggling with third-party REST APIs from the sideline. Many things need to be done right, but speed is critical. Some systems are susceptible to latency, especially if the call to your API is just one piece of a more comprehensive workflow. Spare yourself from the hassle when people call and complain about timeouts.\u003c/p\u003e","title":"REST: Don't Let Them Wait"},{"content":"ORA-01821 Why does this SQL fail?\nSELECT TO_DATE (\u0026#39;10-Sep-02 14:10:10.123000\u0026#39;, \u0026#39;DD-Mon-RR HH24:MI:SS.FF\u0026#39;) FROM DUAL The example is from the 19c SQL Language Reference, with a slight change. The format is valid, but the thing is that the DATE data type does not support fractions of seconds. Therefore the last part of the format string is unknown to the TO_DATE-function.\nOne day I lost a few minutes because I stared at the format string too long before I noticed that I was using TO_DATE instead of TO_TIMESTAMP.\nThe data type for the column I was inserting data into was indeed DATE, but the string I had to convert had these fractions of seconds, which sidetracked me. I was too focused on transforming the entire string and getting the format right that I forgot to look at the bigger picture.\n","permalink":"https://enesi.no/2023/02/ora-01821/","summary":"\u003ch2 id=\"ora-01821\"\u003eORA-01821\u003c/h2\u003e\n\u003cp\u003eWhy does this SQL fail?\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eTO_DATE\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;10-Sep-02 14:10:10.123000\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;DD-Mon-RR HH24:MI:SS.FF\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eDUAL\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eThe example is from the 19c SQL Language Reference, with a slight change. The format is valid, but the thing is that the DATE data type does not support fractions of seconds. Therefore the last part of the format string is unknown to the TO_DATE-function.\u003c/p\u003e\n\u003cp\u003eOne day I lost a few minutes because I stared at the format string too long before I noticed that I was using TO_DATE instead of TO_TIMESTAMP.\u003c/p\u003e","title":"ORA-01821"},{"content":"Although you think you have verified that everything works after a switchover, you may have a hidden problem with the applications. You need to know how the applications will react if one of the network interfaces stops responding.\nDataguard: Switchover vs Failover I have been using Data Guard for many years but have never had to do an actual failover in production; it has always been a switchover for planned maintenance. The other day, we did a controlled power-off of the standby side some days after a switchover. We discovered that many applications started to hang after this power-off, although they worked fine when both the standby and primary sides were online.\nThat situation was the same as it would have been after a disaster. If you lose one side, there might not be a network interface that can answer the TCP connection requests from the application. (Obviously, you can lose the entire database on primary without losing the server, but I am concerned about the situation when the network to the database is gone.)\nYou can simulate this if you have an Oracle database running in Docker and shut down the network interface that Docker is using.\nI am using the SQL Developer Command Line (SQLcl) here. First, I time a stable connection to my database in Docker, running on a rusty server.\noisene@rio:~$ time sql -L oisene/somepassword@rio/orclpdb1 @test.sql SQLcl: Release 22.1 Production on Thu Feb 16 20:06:30 2023 Copyright (c) 1982, 2023, Oracle. All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 SYSDATE ____________ 16-FEB-23 Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 real\t0m4,285s user\t0m11,667s sys\t0m0,459s Over four seconds isn\u0026rsquo;t really fast, but this is in my lab. The script simply does a select sysdate from dual; and the exits.\nThe ifconfig command lists the interfaces and one is obviously the interface that Docker uses. I shut it down with ifconfig docker0 down and repeat the test:\noisene@rio:~$ time sql -L oisene/somepassword@rio/orclpdb1 @test.sql SQLcl: Release 22.1 Production on Thu Feb 16 20:08:06 2023 Copyright (c) 1982, 2023, Oracle. All rights reserved. USER = oisene URL = jdbc:oracle:thin:@rio/orclpdb1 Error Message = IO Error: The Network Adapter could not establish the connection (CONNECTION_ID=ackQG31RQL+lsACfkdIFWQ==) USER = oisene URL = jdbc:oracle:thin:@rio:1521/orclpdb1 Error Message = IO Error: The Network Adapter could not establish the connection (CONNECTION_ID=fZYH+FHXTMuhwV9+2gvhTA==) USER = oisene URL = jdbc:oracle:thin:@rio/orclpdb1:1521/rio/orclpdb1 Error Message = IO Error: Invalid connection string format, a valid format is: \u0026#34;//host[:port][/service_name]\u0026#34; (CONNECTION_ID=ALGju1nHQjSV3VAVDkmNVQ==) real\t4m23,605s user\t0m7,001s sys\t0m0,390s Four minutes and 23 seconds is way too much. I have done tests on other servers where it failed after 20 seconds, but that is also too long.\nThe solution is to add one parameter, but I also need to rewrite the connection string:\noisene@rio:~$ time sql -L oisene/somepassword@(description=(TRANSPORT_CONNECT_TIMEOUT=5sec)(address_list=(address=(protocol=tcp)(host=rio)(port=1521)))(connect_data=(service_name=orclpdb1))) SQLcl: Release 22.1 Production on Thu Feb 16 20:20:54 2023 Copyright (c) 1982, 2023, Oracle. All rights reserved. USER = oisene URL = jdbc:oracle:thin:@(description=(TRANSPORT_CONNECT_TIMEOUT=5sec)(address_list=(address=(protocol=tcp)(host=rio)(port=1521)))(connect_data=(service_name=orclpdb1))) Error Message = IO Error: The Network Adapter could not establish the connection (CONNECTION_ID=6dhk/ZWxQ8GkuCsKed6C6g==) real\t0m7,480s user\t0m7,030s sys\t0m0,333s The point here is that the situation where the Oracle Listener is up on both sides is different from the situation when nobody is listening on the server that used to be primary. This specific parameter limits how long the client/application will wait to establish the TCP connection with the Oracle Listener before it fails and proceeds to the following address.\nI did this test in my lab with Docker. You can, of course, simulate this by shutting down the interface on your standby server with a connection string that has failover configured, and with the address of the standby server first. Make sure that redo transport and apply start working again after you complete the test. And don\u0026rsquo;t wait too long, especially if it is production.\n","permalink":"https://enesi.no/2023/02/dataguard-switchover-vs-failover/","summary":"\u003cp\u003eAlthough you think you have verified that everything works after a switchover, you may have a hidden problem with the applications. You need to know how the applications will react if one of the network interfaces stops responding.\u003c/p\u003e\n\u003ch2 id=\"dataguard-switchover-vs-failover\"\u003eDataguard: Switchover vs Failover\u003c/h2\u003e\n\u003cp\u003eI have been using Data Guard for many years but have never had to do an actual failover in production;  it has always been a switchover for planned maintenance. The other day, we did a controlled power-off of the standby side some days after a switchover. We discovered that many applications started to hang after this power-off, although they worked fine when both the standby and primary sides were online.\u003c/p\u003e","title":"Dataguard: Switchover vs Failover"},{"content":"Be careful with identity columns.\nPrimary Key as Generated Identity From Oracle 12c you can have Oracle generate the primary key for you. As an example:\ncreate table customer ( id number generated always as identity, name varchar2(50), constraint customer_pk primary key (id)); Later you can remove this identity with an alter table command, but you can\u0026rsquo;t add it back!\nSQL\u0026gt; alter table customer modify (id drop identity); Table CUSTOMER altered. SQL\u0026gt; alter table customer modify (id number generated always as identity); Error starting at line : 1 in command - alter table customer modify (id number generated always as identity) Error report - ORA-30673: column to be modified is not an identity column 30673.0000 - \u0026#34;column to be modified is not an identity column\u0026#34; *Cause: An attempt was made to modify the identity properties of that is not an identity column. *Action: Modify the identity properties of an identity column. You can check the documentation for 21c , there is no other syntax to solve this issue.\nYou can drop the column and add it back, but dropping the primary key may be a problem if you have foreign keys in other tables referring to it.\nOne option is to use online table redefinition .\nBy the way, unless you specify more options than given in the example above, the system-generated sequence will have the same defaults as a normal sequence. You can change this as shown in this example (provided that the column is already created as an identity column):\nalter table customer modify (id number generated always as identity nocache order); I hope your design does not require the attributes set as shown above, but I have recently seen two systems that depended on it. One API was using the ID-column like a virtual clock; the ID-values had to be created in strictly increasing order and no gaps. If you know something about clusters and hot blocks you\u0026rsquo;ll know that does not scale well.\nThere is one option here I don\u0026rsquo;t understand the value of. You can define the system-generated sequence to be used when noe value is given for the ID-column. But unless you make sure that supplied ID-values are in another range it will likely clash with sequence:\ncreate table customer ( id number generated by default on null as identity, name varchar2(50), constraint customer_pk primary key (id)); SQL\u0026gt; insert into customer(id,name) values (1,\u0026#39;Fulano\u0026#39;); 1 row inserted. SQL\u0026gt; insert into customer(name) values (\u0026#39;Somebody\u0026#39;); Error starting at line : 1 in command - insert into customer(name) values (\u0026#39;Somebody\u0026#39;) Error report - ORA-00001: unique constraint (OISENE.CUSTOMER_PK) violated But here is the fix for that problem:\nalter table customer modify (id generated always as identity (start with limit value)); Table CUSTOMER altered. SQL\u0026gt; insert into customer(name) values (\u0026#39;Somebody\u0026#39;); 1 row inserted. SQL\u0026gt; select * from customer; ID NAME _____ ___________ 1 Fulano 2 Somebody This will change the sequence to start one above the maximum value for ID in the table, and any attempt to insert with a specified ID-value will fail:\ninsert into customer(id,name) values (42,\u0026#39;Someone else\u0026#39;) Error at Command Line : 1 Column : 22 Error report - SQL Error: ORA-32795: cannot insert into a generated always identity column 32795.0000 - \u0026#34;cannot insert into a generated always identity column\u0026#34; *Cause: An attempt was made to insert a value into an identity column created with GENERATED ALWAYS keywords. *Action: A generated always identity column cannot be directly inserted. Instead, the associated sequence generator must provide the value. Obviously this is what we want, to fail immediately in case of a coding error, not sometime in the future when the sequence reaches an already inserted value.\n","permalink":"https://enesi.no/2023/02/primary-key-as-generated-identity/","summary":"\u003cp\u003eBe careful with identity columns.\u003c/p\u003e\n\u003ch2 id=\"primary-key-as-generated-identity\"\u003ePrimary Key as Generated Identity\u003c/h2\u003e\n\u003cp\u003eFrom Oracle 12c you can have Oracle generate the primary key for you. As an example:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003ecreate\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003etable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecustomer\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e   \u003c/span\u003e\u003cspan class=\"n\"\u003eid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nb\"\u003enumber\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003egenerated\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ealways\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eas\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e   \u003c/span\u003e\u003cspan class=\"n\"\u003ename\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003evarchar2\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"mi\"\u003e50\u003c/span\u003e\u003cspan class=\"p\"\u003e),\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e   \u003c/span\u003e\u003cspan class=\"k\"\u003econstraint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecustomer_pk\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eprimary\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ekey\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003eid\u003c/span\u003e\u003cspan class=\"p\"\u003e));\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eLater you can remove this identity with an alter table command, but you can\u0026rsquo;t add it back!\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSQL\u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ealter\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003etable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecustomer\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003emodify\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003eid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003edrop\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eTable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eCUSTOMER\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ealtered\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSQL\u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ealter\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003etable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecustomer\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003emodify\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003eid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nb\"\u003enumber\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003egenerated\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ealways\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eas\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eError\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003estarting\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eat\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eline\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ein\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecommand\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003ealter\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003etable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecustomer\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003emodify\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003eid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nb\"\u003enumber\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003egenerated\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ealways\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eas\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eError\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ereport\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eORA\u003c/span\u003e\u003cspan class=\"o\"\u003e-\u003c/span\u003e\u003cspan class=\"mi\"\u003e30673\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ecolumn\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eto\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ebe\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003emodified\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eis\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enot\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ean\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ecolumn\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"mi\"\u003e30673\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"mi\"\u003e0000\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u003c/span\u003e\u003cspan class=\"w\"\u003e  \u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;column to be modified is not an identity column\u0026#34;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"n\"\u003eCause\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eAn\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eattempt\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ewas\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003emade\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eto\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003emodify\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ethe\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eproperties\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eof\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e           \u003c/span\u003e\u003cspan class=\"n\"\u003ethat\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eis\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enot\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ean\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ecolumn\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"n\"\u003eAction\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e   \u003c/span\u003e\u003cspan class=\"k\"\u003eModify\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ethe\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eproperties\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eof\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ean\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eidentity\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003ecolumn\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eYou can check the \u003ca href=\"https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/ALTER-TABLE.html#GUID-552E7373-BF93-477D-9DA3-B2C9386F2877\" target=\"_blank\" rel=\"noopener\"\u003edocumentation for 21c\u003c/a\u003e\n, there is no other syntax to solve this issue.\u003c/p\u003e","title":"Primary Key as Generated Identity"},{"content":"Is your SYSAUX tablespace way too large? If you think your SYSAUX tablespace is too big, it may be that AWR is leaving behind stuff that it should have deleted. Doc ID 2099998.1 on Oracle Support describes the problem. The database deletes snapshots that are older than the configured retention. But sometimes, if the process gets interrupted, it may leave some orphan data behind.\nEnsure you are correctly licensed before you start querying the AWR tables; it requires the diagnostic management pack (AFAIK, but I have not worked for years on an enterprise database without all the management packs licensed). You can use the script $ORACLE_HOME/rdbms/admin/awrinfo.sql to check how much space the various components of AWR occupy.\nThe article on Oracle Support lists three DELETE statements and six ALTER TABLE statements to fix the problem. The DELETE statements remove rows that do not belong to any snapshot\nDELETE FROM sys.WRH$_LATCH a WHERE NOT EXISTS (SELECT 1 FROM sys.wrm$_snapshot b WHERE b.snap_id = a.snap_id AND dbid=(SELECT dbid FROM v$database) AND b.dbid = a.dbid AND b.instance_number = a.instance_number); DELETE FROM sys.WRH$_SYSSTAT a WHERE NOT EXISTS (SELECT 1 FROM sys.wrm$_snapshot b WHERE b.snap_id = a.snap_id AND dbid=(SELECT dbid FROM v$database) AND b.dbid = a.dbid AND b.instance_number = a.instance_number) -- and rownum \u0026lt; 1e7 ; DELETE FROM sys.WRH$_PARAMETER a WHERE NOT EXISTS (SELECT 1 FROM sys.wrm$_snapshot b WHERE b.snap_id = a.snap_id AND dbid=(SELECT dbid FROM v$database) AND b.dbid = a.dbid AND b.instance_number = a.instance_number) -- and rownum \u0026lt; 1e7 ; COMMIT; On one occasion, I had to remove it in batches; that\u0026rsquo;s why and rownum \u0026lt; 1e7 is commented out.\nTo reclaim the space from the tables back to the tablespace, you must enable row movement and shrink the tables:\nALTER TABLE sys.WRH$_LATCH ENABLE ROW MOVEMENT; ALTER TABLE sys.WRH$_SYSSTAT ENABLE ROW MOVEMENT; ALTER TABLE sys.WRH$_PARAMETER ENABLE ROW MOVEMENT; ALTER TABLE sys.WRH$_LATCH SHRINK SPACE CASCADE; ALTER TABLE sys.WRH$_SYSSTAT SHRINK SPACE CASCADE; ALTER TABLE sys.WRH$_PARAMETER SHRINK SPACE CASCADE; From a pluggable database, you need to shrink the tables like this:\nbegin dbms_pdb.exec_as_oracle_script(\u0026#39;ALTER TABLE sys.WRH$_LATCH SHRINK SPACE CASCADE\u0026#39;); dbms_pdb.exec_as_oracle_script(\u0026#39;ALTER TABLE sys.WRH$_SYSSTAT SHRINK SPACE CASCADE\u0026#39;); dbms_pdb.exec_as_oracle_script(\u0026#39;ALTER TABLE sys.WRH$_PARAMETER SHRINK SPACE CASCADE\u0026#39;); end; / Lastly, if you want to reduce the size of the tablespace, you can reduce the data file size with the ALTER DATABASE command. You will probably not be able to reclaim all of it unless you start moving the tables around. Also, don\u0026rsquo;t forget that your time may be more valuable than disk space ;-)\n","permalink":"https://enesi.no/2023/02/is-your-sysaux-tablespace-way-too-large/","summary":"\u003ch2 id=\"is-your-sysaux-tablespace-way-too-large\"\u003eIs your SYSAUX tablespace way too large?\u003c/h2\u003e\n\u003cp\u003eIf you think your SYSAUX tablespace is too big, it may be that AWR is leaving behind stuff that it should have deleted. Doc ID 2099998.1 on Oracle Support describes the problem. The database deletes snapshots that are older than the configured retention. But sometimes, if the process gets interrupted, it may leave some orphan data behind.\u003c/p\u003e\n\u003cp\u003eEnsure you are correctly licensed before you start querying the AWR tables; it requires the diagnostic management pack (AFAIK, but I have not worked for years on an enterprise database without all the management packs licensed). You can use the script \u003ccode\u003e$ORACLE_HOME/rdbms/admin/awrinfo.sql\u003c/code\u003e to check how much space the various components of AWR occupy.\u003c/p\u003e","title":"Is your SYSAUX tablespace way too large?"},{"content":"The combination of a reverse proxy, ORDS in standalone and APEX Social Sign-In gave me a headache.\nORDS, APEX, and Social Sign-In If you have the following setup you may need an extra parameter in the global ORDS configuration:\nORDS running in standalone with HTTP (non-TLS) on port 8080 ORDS behind a reverse proxy with TLS terminated in reverse proxy APEX served by ORDS You want to use authentication scheme Social Sign-In and OpenID Connect (OIDC) or OAuth 2.0 In this case you need to add one line to the file global/settings.xml in the ORDS configuration directory:\n\u0026lt;entry key=\u0026#34;security.forceHTTPS\u0026#34;\u0026gt;true\u0026lt;/entry\u0026gt; As far as I can see, this parameter is not mentioned in the ORDS installation manual as of today, but on Oracle Support in Doc ID 2139195.1.\nOne symptom that this parameter is needed is that APEX works OK, but the redirect_uri that APEX sends as a parameter in the request to your OpenID Connect provider contains port 80 in an URI that starts with https://.\nYou can also check the environment in APEX as shown in Show CGI Environment in APEX If it shows HTTP_PORT=80, then the redirect_uri may be wrong.\nI don\u0026rsquo;t know if this is a bug in APEX; this has been tested with APEX 22.2 and ORDS 22.3.3.\n","permalink":"https://enesi.no/2022/11/ords-apex-and-social-sign-in/","summary":"\u003cp\u003eThe combination of a reverse proxy, ORDS in standalone and APEX \u003cem\u003eSocial Sign-In\u003c/em\u003e gave me a headache.\u003c/p\u003e\n\u003ch2 id=\"ords-apex-and-social-sign-in\"\u003eORDS, APEX, and Social Sign-In\u003c/h2\u003e\n\u003cp\u003eIf you have the following setup you may need an extra parameter in the global ORDS configuration:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eORDS running in standalone with HTTP (non-TLS) on port 8080\u003c/li\u003e\n\u003cli\u003eORDS behind a reverse proxy with TLS terminated in reverse proxy\u003c/li\u003e\n\u003cli\u003eAPEX served by ORDS\u003c/li\u003e\n\u003cli\u003eYou want to use authentication scheme Social Sign-In and OpenID Connect (OIDC) or OAuth 2.0\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn this case you need to add one line to the file \u003ccode\u003eglobal/settings.xml\u003c/code\u003e in the ORDS configuration directory:\u003c/p\u003e","title":"ORDS, APEX, and Social Sign-In"},{"content":"This simple tip may help you troubleshooting integrations in APEX and ORDS.\nShow CGI Environment in APEX I have been struggling with integration issues in APEX lately. Listing out the CGI environment helped to identify the probable cause.\nJust create a region of type Dynamic Content and add this code:\nbegin OWA_UTIL.PRINT_CGI_ENV; end; It will list values for HTTP_SERVER, HTTP_PORT, etc. If the configuration on your reverse proxy is wrong, it will likely show up here with values that will not work in the external world.\n","permalink":"https://enesi.no/2022/11/show-cgi-environment-in-apex/","summary":"\u003cp\u003eThis simple tip may help you troubleshooting integrations in APEX and ORDS.\u003c/p\u003e\n\u003ch2 id=\"show-cgi-environment-in-apex\"\u003eShow CGI Environment in APEX\u003c/h2\u003e\n\u003cp\u003eI have been struggling with integration issues in APEX lately. Listing out the CGI environment helped to identify the probable cause.\u003c/p\u003e\n\u003cp\u003eJust create a region of type Dynamic Content and add this code:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003ebegin\n   OWA_UTIL.PRINT_CGI_ENV;\nend;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eIt will list values for HTTP_SERVER, HTTP_PORT, etc. If the configuration on your reverse proxy is wrong, it will likely show up here with values that will not work in the external world.\u003c/p\u003e","title":"Show CGI Environment in APEX"},{"content":"A few tips on resolving ORA-29024 in the Oracle database. This error is thrown when Oracle cannot verify the SSL certificate.\nTroubleshooting ORA-29024: Certificate Validation Failure I have struggled with the ORA-29024 error a few times. Here are a few tips for sorting it out.\nStart fresh with a new wallet for SSL certificates only, and troubleshoot on a test database. Do not use the wallet that TDE uses or that has the database passwords. Add one certificate at a time. You can set the path to the new wallet as a parameter to various procedures like MAKE_REQUEST in APEX_WEB_SERVICE. In UTL_HTTP, you can set it with the procedure SET_WALLET.\nI recommend setting the wallet path explicitly in your code. If the file path is wrong, it will fail with another error, and you can be sure the database picks up the wallet you are debugging.\nWhen you get an error with the package UTL_HTTP, execute dbms_session.reset_package after changes to the wallet. It is my impression that Oracle caches some of the validations and this can cause confusion (see below).\nMake it work with a simple test, like using UTL_HTTP.REQUEST against a simple SSL-secured URL:\nselect UTL_HTTP.REQUEST('https://www.oracle.com/') FROM DUAL; curl is a valuable tool, use parameter -i to check the headers and look for redirects (301 Moved Permanently) to an address that may have another certificate.\nI have a hypothesis on how the database process the incoming certificates: If it is unknown, it will try to verify the parent certificate (upwards in the chain). When it gets to a certificate it has seen before, it is happy and returns OK before the whole chain is verified again. Once during testing at a customer, I had two certificates with the same intermediate parent certificate but different root certificates. One of those two root certificates was missing from the wallet. These things can drive you crazy. Because the next time (in another session), the order changed, and it tried to validate the intermediate certificate whose root certificate was missing. It failed with ORA-29024 even though the same URL worked earlier.\nI have seen both recommendations on adding only the root certificate or the whole chain except the last one. On 19c, it looks like only the top-level root certificate is necessary, and I would start with that and make it work with one case before moving on.\nSome sites, especially internal ones, use domain certificates (*.example.com). Set the parameter https_host to a string representing the hostname used in the certificate.\nIf something works that shouldn\u0026rsquo;t, you may have a picnic error . Perhaps the best tip I can give is this: Don\u0026rsquo;t try to fix a wallet with lots of stuff in it. Start over with an empty wallet and do one step at a time.\n","permalink":"https://enesi.no/2022/11/ora-20294/","summary":"\u003cp\u003eA few tips on resolving ORA-29024 in the Oracle database. This error is thrown when Oracle cannot verify the SSL certificate.\u003c/p\u003e\n\u003ch2 id=\"troubleshooting-ora-29024-certificate-validation-failure\"\u003eTroubleshooting ORA-29024: Certificate Validation Failure\u003c/h2\u003e\n\u003cp\u003eI have struggled with the ORA-29024 error a few times. Here are a few tips for sorting it out.\u003c/p\u003e\n\u003cp\u003eStart fresh with a new wallet for SSL certificates only, and troubleshoot on a test database. Do not use the wallet that TDE uses or that has the database passwords. Add one certificate at a time. You can set the path to the new wallet as a parameter to various procedures like \u003ccode\u003eMAKE_REQUEST\u003c/code\u003e in \u003ccode\u003eAPEX_WEB_SERVICE\u003c/code\u003e. In \u003ccode\u003eUTL_HTTP\u003c/code\u003e, you can set it with the procedure \u003ccode\u003eSET_WALLET\u003c/code\u003e.\u003c/p\u003e","title":"ORA-20294"},{"content":"If you connect to the database through SSH you may come across this message.\nError message Server key did not validate in SQL Developer I frequently use SSH to connect to the Oracle database through some jump server. It is straightforward to set up; I blogged about it over six years ago.\nToday my Windows PC had the BSoD , and after it came back, I had this error when I tried to connect to the database again:\nAn error occurred while opening the host connection. Server key did not validate.\nI googled and found something that helped me to investigate. One suggestion was to regenerate the SSH keys, but what fixed the problem for me was to delete the stored key in the known_hosts file. How that file got corrupted, or the key was wrong is a mystery. Since SSH in SQL Developer reinserts this line, it is safe to delete it. The path to the known_hosts file on Windows is %USERPROFILE%\\.ssh\\known_hosts.\n","permalink":"https://enesi.no/2022/10/error-server-key-did-not-validate-in-sql-developer/","summary":"\u003cp\u003eIf you connect to the database through SSH you may come across this message.\u003c/p\u003e\n\u003ch2 id=\"error-message-server-key-did-not-validate-in-sql-developer\"\u003eError message \u003cem\u003eServer key did not validate\u003c/em\u003e in SQL Developer\u003c/h2\u003e\n\u003cp\u003eI frequently use SSH to connect to the Oracle database through some jump server. It is straightforward to set up; \u003ca href=\"https://enesi.no/2016/02/sql-developer-and-ssh-for-dbas/\"\u003eI blogged about it\u003c/a\u003e\n over six years ago.\u003c/p\u003e\n\u003cp\u003eToday my Windows PC had the \u003ca href=\"https://en.wikipedia.org/wiki/Blue_screen_of_death\" target=\"_blank\" rel=\"noopener\"\u003eBSoD\u003c/a\u003e\n, and after it came back, I had this error when I tried to connect to the database again:\u003c/p\u003e","title":"Error 'Server key did not validate' in SQL Developer"},{"content":"I have an excellent career thanks to the Oracle database. For several reasons, I have met many people that support the community by sharing knowledge, and I have learned a lot along the way. One of those things is how to attack troubleshooting and optimization of slow applications. You may know \u0026ldquo;optimization\u0026rdquo; as tuning.\nThe Richness of Oracle Database Instrumentation #JoelKallmanDay As a consultant, I have helped in many development projects involving the Oracle database. The Java or C# developers frequently relate to the database with merely superficial knowledge. When they run into a problem, they ask Dr. Google for help. There are, however, some issues with this approach. What comes up on top of a Google search may not be the best solution and might be based on a system configuration or hardware setup quite different from the current project. A better approach is to record what is going on and measure what can be measured. \u0026ldquo;What can be measured\u0026rdquo; is not always much. If the developer were proactive, the application would have built-in code for logging and tracing. Quite often, though, the program is a black box that only throws an error message when it crashes.\nFrom the developer\u0026rsquo;s perspective, it is often somebody else\u0026rsquo;s fault when the application is slow, usually the network or the database. Usually, the burden of proof lies with the plaintiff. But it seems that the database and network administrators must deal with this. The DBA must collect the facts and analyze them to show what is happening. These situations are where my favorite feature in the Oracle database comes in: The rich instrumentation and the possibility to see what is going on and measure how long each operation takes. I can do that by enabling SQL trace, looking at the dynamic views for live observations, or looking at the workload data automatically collected in the database.\nProblems fall into two categories: slowness or failures with optional error messages. I attack performance problems by defining the start and end of the task and measuring what is taking time. When the database is not busy processing data on the CPU, it waits for something. Seeing what it is waiting for, how much time it spent waiting, and how my times this happened reveals a lot about the program and logic used. Not always is it a fault in the application. Perhaps the data model is not good, or even the database can have a suboptimal configuration. It helps me enormously to see what the database did during a slow performance or before the failure.\nSometimes the database is not doing much, and nothing can be improved there. But even this is useful to know that the problem is elsewhere. A trace on an operation will reveal excessive network roundtrips or offloading more data to the application than it can process in a reasonable time. The solution to both problems may be moving more of the processing into the database.\nMy best advice to a new DBA is to learn methods for troubleshooting and optimization. In the Oracle Database documentation set, the Database Performance Tuning Guide and SQL Tuning Guide are great places to start. And, of course, nobody explains this better than Cary Millsap at Method-R .\n","permalink":"https://enesi.no/2022/10/the-richness-of-oracle-database-instrumentation/","summary":"\u003cp\u003e\u003cstrong\u003eI have an excellent career thanks to the Oracle database. For several reasons, I have met many people that support the community by sharing knowledge, and I have learned a lot along the way. One of those things is how to attack troubleshooting and optimization of slow applications. You may know \u0026ldquo;optimization\u0026rdquo; as tuning.\u003c/strong\u003e\u003c/p\u003e\n\u003ch2 id=\"the-richness-of-oracle-database-instrumentation-joelkallmanday\"\u003eThe Richness of Oracle Database Instrumentation #JoelKallmanDay\u003c/h2\u003e\n\u003cp\u003eAs a consultant, I have helped in many development projects involving the Oracle database. The Java or C# developers frequently relate to the database with merely superficial knowledge. When they run into a problem, they ask Dr. Google for help. There are, however, some issues with this approach. What comes up on top of a Google search may not be the best solution and might be based on a system configuration or hardware setup quite different from the current project. A better approach is to record what is going on and measure what can be measured. \u0026ldquo;What can be measured\u0026rdquo; is not always much. If the developer were proactive, the application would have built-in code for logging and tracing. Quite often, though, the program is a black box that only throws an error message when it crashes.\u003c/p\u003e","title":"The Richness of Oracle Database Instrumentation"},{"content":"TL;DR: Your merge condition does not uniquely identify one row in the target table.\nORA-30926 and MERGE If we ignore bugs that may happen, ORA-30296 comes from a logical error in the merge condition. One row from the source set should match one and only one in the target table, if it exists, in the target. If it matches more than one, the merge statement cannot work as intended, and you need to check what uniquely identifies a row.\nThis problem does not happen when the merge condition is the same as the primary key of the target table. But if you are doing some data cleansing or importing where the data comes from a system where people stuffed data in any available field, you may have to be creative with your merge condition. If your merge statement is complex and the data is messy, it is easy to introduce this error.\nAlso, it is easy to forget that the first load will succeed because the target table is empty. (\u0026ldquo;Why does it fail now?\u0026rdquo;)\nIf you are in control of your data model, have an extra check to see if it has a logical flaw. Or it could be you need an additional predicate in your MERGE statement.\n","permalink":"https://enesi.no/2022/08/ora-30926-and-merge/","summary":"\u003cp\u003e\u003cstrong\u003eTL;DR: Your merge condition does not uniquely identify one row in the target table.\u003c/strong\u003e\u003c/p\u003e\n\u003ch2 id=\"ora-30926-and-merge\"\u003eORA-30926 and MERGE\u003c/h2\u003e\n\u003cp\u003eIf we ignore bugs that may happen, ORA-30296 comes from a logical error in the merge condition. One row from the source set should match one and only one in the target table, if it exists, in the target. If it matches more than one, the merge statement cannot work as intended, and you need to check what uniquely identifies a row.\u003c/p\u003e","title":"ORA-30926 and MERGE"},{"content":"What is the easiest way to create a table that can have at most one row? That question was asked on Twitter about two days before I intended to blog about my solution, a month ago.\nAnybody have a suggestion for the best way to ensure that an #oracledatabase table that already has one row can not have another row inserted into it?#sql #plsql\n\u0026mdash; SF on PL/SQL (@sfonplsql) March 2, 2022 My solution is practically the same as what @FranckPachot suggested , but that is not stopping me from blogging mine.\nNo triggers necessary If you can have just one favourite beer:\ncreate table beer (beer varchar2(100), brewery varchar2(42), just_one generated always as (1), constraint just_one_beer unique (just_one)); Add one row:\ninsert into beer (beer,brewery) values (\u0026#39;My best IPA\u0026#39;, \u0026#39;Dead Brothers Asylum Inc\u0026#39;); Try to add another, and it will fail with ORA-00001:\ninsert into beer (beer, brewery) values (\u0026#39;Heineken\u0026#39;,\u0026#39;Heineken Company\u0026#39;); ORA-00001: unique constraint (TESTUSER.JUST_ONE_BEER) violated Update is not a problem:\nupdate beer set beer=\u0026#39;Your best IPA\u0026#39;; And of course you can delete the row:\ndelete from beer; By the way, Live SQL by Oracle\tis perfect for such simple testing. You need a free account at Oracle to do this, but you can create one after you click Start Coding Now.\n","permalink":"https://enesi.no/2022/04/table-with-one-row/","summary":"\u003cp\u003eWhat is the easiest way to create a table that can have at most one row? That question was asked on Twitter about two days before I intended to blog about my solution, a month ago.\u003c/p\u003e\n\n\u003cblockquote class=\"twitter-tweet\"\u003e\u003cp lang=\"en\" dir=\"ltr\"\u003eAnybody have a suggestion for the best way to ensure that an \u003ca href=\"https://twitter.com/hashtag/oracledatabase?src=hash\u0026amp;ref_src=twsrc%5Etfw\"\u003e#oracledatabase\u003c/a\u003e table that already has one row can not have another row inserted into it?\u003ca href=\"https://twitter.com/hashtag/sql?src=hash\u0026amp;ref_src=twsrc%5Etfw\"\u003e#sql\u003c/a\u003e \u003ca href=\"https://twitter.com/hashtag/plsql?src=hash\u0026amp;ref_src=twsrc%5Etfw\"\u003e#plsql\u003c/a\u003e\u003c/p\u003e\u0026mdash; SF on PL/SQL (@sfonplsql) \u003ca href=\"https://twitter.com/sfonplsql/status/1499018875672932356?ref_src=twsrc%5Etfw\"\u003eMarch 2, 2022\u003c/a\u003e\u003c/blockquote\u003e \u003cscript async src=\"https://platform.twitter.com/widgets.js\" charset=\"utf-8\"\u003e\u003c/script\u003e\n\n\u003cp\u003eMy solution is practically the same as what  \u003ca href=\"https://twitter.com/FranckPachot\" target=\"_blank\" rel=\"noopener\"\u003e@FranckPachot\u003c/a\u003e\n \u003ca href=\"https://twitter.com/FranckPachot/status/1499022033686044684?s=20\u0026amp;t=nu0EeJUA8Y-y8z90s_MsVw\" target=\"_blank\" rel=\"noopener\"\u003esuggested\u003c/a\u003e\n, but that is not stopping me from blogging mine.\u003c/p\u003e","title":"Table with one row"},{"content":"Joel Kallman passed away earlier this year. He was one of the first creators of Oracle APEX and always dedicated to the user community. He left a lasting impression on all of us and this is just one post among many written to honour a great guy .\nFor anyone not familiar with Oracle APEX, check out https://apex.oracle.com/ .\nAPEX for Administrators #JoelKallmanDay It is my impression that few DBAs or other administators have much experience with software development. Perhaps that is why they don\u0026rsquo;t make any tools to help them manage their projects, but resort to general tools like Excel or an internal wiki. This leads to too much time spent on project management and fighting with a tool that is not fit for the task. This is not an issue with a small infrastructure, but if you need to plan downtime for hundreds of databases and their connected applications it can become a logistic nightmare.\nThe main goal here is to spend as little time as possible on managing your infrastructure. And we want to achieve this without creating another complex tool that requires a project in itself. Start simple, you can always add something later.\nNo project happens according to the initial plan. Activities needs to be rescheduled and people reassigned. With a simple data model and a browser interface on top you can update the project plan as you move on. You can delegate part of the planning to more people and even log changes as the project develops.\nFor instance, have you any idea how easy it is to display a calendar with scheduled activities? A few clicks in the web interface and a query that returns the date and the display text is all it takes. A link from the activity to an updatable form for the activity takes a few minutes or seconds to set up. If you need a demo of this, check out the productivity applications that come bundled with APEX.\nA simple status page is always useful. A chart based on a simple query takes a minute to create. Management loves charts that show the status in a second. Can you do that on your wiki or in your Excel sheet?\nYou probably have some lists or tables with the names of your databases, system owners, and other details. That can easily be imported to the database with SQL Developer (another free tool). Add a table to plan activities and create a few forms on the tables you are likely to update. It does not have to be perfect to be useful.\nMaybe you have data from different sources, they can be joined in your database and even cleaned up with some simple SQL. Data coming from Excel or some places where people can write free text may need a bit of cleaning, but I usually do this in an iterative manner to see how much work there is to get it right. Or you can delegate this; add reports and forms on the tables that need to be checked and ask the whole team to contribute.\nNot all applicatons need to live forever. But if you can manage a project using your own application, you may actually start enjoying the project more.\nA Great Motivator I rembember how Joel encouraged me when I gave presentations at user conferences, this is a typical tweet from him, almost 6 years ago:\nLarge audience of DBAs for #orclapex for the DBA, by @OyvindIsene at #ukoug_tech15. Impressive! pic.twitter.com/MyVQ8b7ZcS\n\u0026mdash; Joel R. Kallman (@joelkallman) December 7, 2015 The presentation I gave was probably not that great, but thanks to his friendliness and solid advices I kept going and above all kept working with APEX.\nThanks to the rest of the APEX team and the whole Oracle APEX community, this will not go away any time soon.\n","permalink":"https://enesi.no/2021/10/apex-for-administrators/","summary":"\u003cp\u003eJoel Kallman passed away earlier this year. He was one of the \u003ca href=\"https://en.wikipedia.org/wiki/Oracle_Application_Express#Background\" target=\"_blank\" rel=\"noopener\"\u003efirst creators\u003c/a\u003e\n of Oracle APEX and always dedicated to the user community. He left a lasting impression on all of us and this is just one post among \u003ca href=\"https://twitter.com/search?q=%23JoelKallmanDay\" target=\"_blank\" rel=\"noopener\"\u003emany written to honour a great guy\u003c/a\u003e\n.\u003c/p\u003e\n\u003cp\u003eFor anyone not familiar with Oracle APEX, check out \u003ca href=\"https://apex.oracle.com/\" target=\"_blank\" rel=\"noopener\"\u003ehttps://apex.oracle.com/\u003c/a\u003e\n.\u003c/p\u003e\n\u003ch2 id=\"apex-for-administrators-joelkallmanday\"\u003eAPEX for Administrators #JoelKallmanDay\u003c/h2\u003e\n\u003cp\u003eIt is my impression that few DBAs or other administators have much experience with software development. Perhaps that is why they don\u0026rsquo;t make any tools to help them manage their projects, but resort to general tools like Excel or an internal wiki. This leads to too much time spent on project management and fighting with a tool that is not fit for the task. This is not an issue with a small infrastructure, but if you need to plan downtime for hundreds of databases and their connected applications it can become a logistic nightmare.\u003c/p\u003e","title":"APEX for Administrators"},{"content":"UTL_HTTP from Autonomous Database The PL/SQL package UTL_HTTP in Oracle Autonomous Databases only allows HTTPS connections.\nTo call secure web services from the database it needs a wallet. The wallet must have the root and intermediate certificates for the remote site. If they are missing you will get the error:\nORA-29024: Certificate validation failure But how do you fix that if your database is an autonomous database, like the ADW, or ATP? Turns out, you don\u0026rsquo;t have to. The wallet that is already set up for you contains about 90 of the most commonly trusted root and intermediate SSL certificates. The only thing you have to do is execute:\nbegin utl_http.set_wallet(''); end; / That is, no arguments needed. As before, you need to add the remote host to the ACL, like this:\nbegin DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE( host =\u0026gt; 'github.com', ace =\u0026gt; xs$ace_type( privilege_list =\u0026gt; xs$name_list('connect','resolve'), principal_name =\u0026gt; 'FULANO', principal_type =\u0026gt; xs_acl.ptype_db)); commit; end; / This also means that you cannot access sites with self-signed certificates. Which is nice, that is always messsy. If a certian site still give you the error, try to access it in a browser and check the certificate chain.\n","permalink":"https://enesi.no/2021/03/utl_http-from-oracle-autonomous-database/","summary":"\u003ch2 id=\"utl_http-from-autonomous-database\"\u003eUTL_HTTP from Autonomous Database\u003c/h2\u003e\n\u003cp\u003eThe PL/SQL package UTL_HTTP in Oracle Autonomous Databases only allows HTTPS connections.\u003c/p\u003e\n\u003cp\u003eTo call secure web services from the database it needs a wallet. The wallet must have the root and intermediate certificates for the remote site. If they are missing  you will get the error:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eORA-29024: Certificate validation failure\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eBut how do you fix that if your database is an autonomous database, like the ADW, or ATP? Turns out, you don\u0026rsquo;t have to. The wallet that is already set up for you contains about 90 of the most commonly trusted root and intermediate SSL certificates. The only thing you have to do is execute:\u003c/p\u003e","title":"UTL_HTTP from Oracle Autonomous Database"},{"content":"The last two weeks were shorter than most weeks before.\nOpen for business And with that I mean that time flies when you are busy with something interesting.\nFinally, I am on my own, and it has been a pleasant experience so far. I have received many encouragements and advice from friends and contacts in the community: Today was my last day at Sysco/Bicon. I am starting in a new role, in my own company. Will do Oracle, development, tuning, data whatever. Feeling good.\n\u0026mdash; Øyvind Isene (@OyvindIsene) January 29, 2021 I have spent a lot of time to prepare myself for this, but getting 100% ready is impossible. So far it looks like I will be busy for a while and I enjoy my workdays where I can focus on one project at a time, and cut off distractions.\nI By the way, here is a good series on Becoming a DBA Consultant by @amitzil. It is quite thorough with things you should consider before you start.\n","permalink":"https://enesi.no/2021/02/alter-pluggable-database-enesi-open/","summary":"\u003cp\u003eThe last two weeks were shorter than most weeks before.\u003c/p\u003e\n\u003ch2 id=\"open-for-business\"\u003eOpen for business\u003c/h2\u003e\n\u003cp\u003eAnd with that I mean that time flies when you are busy with something interesting.\u003c/p\u003e\n\u003cp\u003eFinally, I am on my own, and it has been a pleasant experience so far. I have received many encouragements and advice from friends and contacts in the community: \u003cblockquote class=\"twitter-tweet\" data-dnt=\"true\"\u003e\u003cp lang=\"en\" dir=\"ltr\"\u003eToday was my last day at Sysco/Bicon. I am starting in a new role, in my own company. Will do Oracle, development, tuning, data whatever. Feeling good.\u003c/p\u003e","title":"Alter Pluggable Database Enesi Open;"},{"content":"2020 has been a tough year for many, but I can\u0026rsquo;t complain considering what other people have gone through. While others lost their job due to the shutdown, I could work from home for extended periods.\nBut I haven\u0026rsquo;t been satisfied with the way things were going and needed a change. I wanted to focus on what I enjoy doing, and after a checkpoint, I realised what things I wanted to avoid.\nThere was also this uncertainty about the future, how long could I continue where I was.\nSo, in October, I decided to do something else and leave the company I have worked for the last seven and a half years. From February I will be on my own in my company, Enesi Consulting.\nOne of my good friends said I was crazy doing this, probably hinting at the uncertainty we are going through. That just confirmed the concerns that had passed in my head, but not rational enough for me to stop. It is now or never.\nI have received plenty of encouragement from friends in the Oracle Community. That too reminded me how much I like to be in this environment with excellent friends, and working with the database, infrastructure on Linux, development, and most importantly, data analytics.\nOne of the advantages I am looking forward to is saying no to duties that I don\u0026rsquo;t do well, like silly meetings, bureaucratic processes, plans without results, etc. I am a techie guy, and I like to help people.\nOptimisation and troubleshooting of systems connected to Oracle is one area of priority; traditional DBA-work may be out of demand, but developers have just found new resources to abuse. Words like observability and metrics, mostly cover what I have been doing for a while; use the instrumentation and create a resource usage profile to find the bottleneck. But there is undoubtedly more to be done out there. Few systems are so heavily instrumented as the Oracle database, but now it is more accepted to make other systems observable. Prometheus is fun to work with, and coupled with Grafana ; proactive monitoring becomes less of a burden.\nI have moved away from the typical DBA role for the last years and done more database development, especially with Oracle APEX. Python is also cool, though it will take some time before I reach fluency. If somebody asks me to do integration, I will choose Apache Kafka. It is rock solid and gets the job done. Nothing is worse than delayed data to a data warehouse. It is time to stop these overloaded batch jobs that fail way too often. Please keep it simple, get a clear overview of the flow, and spend money on storage, not human labour.\nMy current hobby is machine learning; I am rediscovering old interests in mathematics and algorithms. The list of topics I want to look at won\u0026rsquo;t be empty for a while, which is good. The combination of people, data, and code will occupy me further on.\n","permalink":"https://enesi.no/2020/12/reboot/","summary":"\u003cp\u003e2020 has been a tough year for many, but I can\u0026rsquo;t complain considering what other people have gone through. While others lost their job due to the shutdown, I could work from home for extended periods.\u003c/p\u003e\n\u003cp\u003eBut I haven\u0026rsquo;t been satisfied with the way things were going and needed a change. I wanted to focus on what I enjoy doing, and after a checkpoint, I realised what things I wanted to avoid.\u003c/p\u003e","title":"Reboot"},{"content":" I don\u0026rsquo;t know why I stopped blogging, probably because I am lazy or easily distracted. But this annual opportunity to say thanks to some people who has helped me a lot shouldn\u0026rsquo;t be missed (I did last year). Tim Hall aka ORACLE-BASE organizes an annual virtual Thanksgiving for us Oracle nerds .\nExpressing gratitude is a strange thing. Why do we say thanks? Frequently we do it just to be polite, but for me, the most honest feeling of gratitude comes after when someone has solved a big problem for me. If I suspect that the person helping me is unaware of how much difference it made, I find it even more important to say \u0026ldquo;Thank you!\u0026rdquo; Giving thanks isn\u0026rsquo;t always easy, the more sincere it is the more awkard it gets. But given this opportunity when a good part of the Oracle community does the same, I have the excuse. So thank you, Tim, for giving us this opportunity (Is this a meta-thank you?)\nThere are people who do their day job, and others who go out of their way to help. Some give a presentation, while others give a presentation, allow for Q\u0026amp;A, and follows you to the bar after.\nWho am I writing about? It is a special group of Oracle employees who spend a substantial amount of time away from their families going to conferences in order to share new knowledge and insight. I don\u0026rsquo;t go to much regular training these days because I can learn most tech on my own, but after a while I have lots of questions that can\u0026rsquo;t be answered from ordinary material. On conferences I have had many a great conversation with product managers who are willing to listen and to give advice. Rarely am I suffering a sales pitch, quite often I get an off the record answer that sends me in the right direction. I have had many interesting conversations about the small and big issues in life, not necessarily connected to tech, with these people. It has helped to balance the impression one might get of the big Oracle Corp. Also knowing that the database is made and maintained by people who really know their computer science, by people I can meet at conferences makes me trust this old software even more.\nI could have listed some names, or at least a bunch of Twitter tags, but then I would surely forget some. So to all product managers, community advocates and general good people working for Oracle that I have met at conferences for the last 17 years or so: Thank you!\n","permalink":"https://enesi.no/2019/10/ogb-appreciation-day-2019-a-special-part-of-the-oracle-community/","summary":"\u003cfigure\u003e\u003ca href=\"https://community.oracle.com/community/groundbreakers/\"\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/groundbreaker.jpeg\"/\u003e \u003c/a\u003e\n\u003c/figure\u003e\n\n\u003cp\u003eI don\u0026rsquo;t know why I stopped blogging, probably because I am lazy or easily distracted. But this annual opportunity to say thanks to some people who has helped me a lot shouldn\u0026rsquo;t be missed (I did last year). Tim Hall aka ORACLE-BASE organizes \u003ca href=\"https://oracle-base.com/blog/2019/09/30/ogb-appreciation-day-2019-thanksogb/\" target=\"_blank\" rel=\"noopener\"\u003ean annual virtual Thanksgiving for us Oracle nerds\u003c/a\u003e\n.\u003c/p\u003e\n\u003cp\u003eExpressing gratitude is a strange thing. Why do we say thanks? Frequently we do it just to be polite, but for me, the most honest feeling of gratitude comes after when someone has\nsolved a big problem for me. If I suspect that the person helping me is unaware of how much difference it made, I find it even more important to say \u0026ldquo;Thank you!\u0026rdquo; Giving\nthanks isn\u0026rsquo;t always easy, the more sincere it is the more awkard it gets. But given this opportunity when \u003ca href=\"https://twitter.com/search?q=%23ThanksOGB\u0026amp;src=typed_query\" target=\"_blank\" rel=\"noopener\"\u003ea good part of the Oracle community\u003c/a\u003e\n does the same, I have the excuse. So thank you, Tim, for giving us this opportunity (Is this a meta-thank you?)\u003c/p\u003e","title":"OGB Appreciation Day 2019: A Special part of the Oracle Community"},{"content":"I just upgraded a small Confluent Kafka cluster at a customer. Almost everything went fine, just following these instructions: https://docs.confluent.io/current/installation/upgrade.html (our servers are running on CentOS). But at the end KSQL server wouldn\u0026rsquo;t start and failed with this error message:\n[2019-07-11 16:23:31,233] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:53) java.lang.IllegalStateException: Invalid replcation factor on topic _confluent-ksql-default__command_topic: 1 at io.confluent.ksql.rest.util.KsqlInternalTopicUtils.ensureTopic(KsqlInternalTopicUtils.java:91) at io.confluent.ksql.rest.server.KsqlRestApplication.initialize(KsqlRestApplication.java:237) at io.confluent.ksql.rest.server.KsqlRestApplication.startKsql(KsqlRestApplication.java:200) at io.confluent.ksql.rest.server.KsqlRestApplication.start(KsqlRestApplication.java:187) at io.confluent.ksql.rest.server.KsqlServerMain.tryStartApp(KsqlServerMain.java:65) at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:51) I checked the replication factor with:\nkafka-topics --zookeeper :2181 --topic _confluent-ksql-default__command_topic --describe and the replication factor from the stack trace was confirmed. Since this small cluster has three nodes and we use replication factor 3 I figured that was the best guess for correct replication factor.\nYou can change the replication factor using the instructions here: https://kafka.apache.org/documentation/#basic_ops_increase_replication_factor So, what I did was to create a JSON-file with:\n{\u0026#34;version\u0026#34;:1, \u0026#34;partitions\u0026#34;:[{\u0026#34;topic\u0026#34;:\u0026#34;_confluent-ksql-default__command_topic\u0026#34;,\u0026#34;partition\u0026#34;:0,\u0026#34;replicas\u0026#34;:[0,2,3]}]} The numbers at the end (0, 2, 3) are the broker-ids. (For a reason I can\u0026rsquo;t remember we don\u0026rsquo;t have a broker #1). Now, execute this command:\nkafka-reassign-partitions --zookeeper localhost:2181 --execute --reassignment-json-file increase-replication-factor.json You can verify success with:\nkafka-reassign-partitions --zookeeper localhost:2181 --verify --reassignment-json-file increase-replication-factor.json After this the KSQL server started without problems.\n","permalink":"https://enesi.no/2019/07/ksql-server-failed-to-start-after-upgrade/","summary":"\u003cp\u003eI  just upgraded a small Confluent Kafka cluster at a customer.  Almost everything went fine, just following these instructions: \u003ca href=\"https://docs.confluent.io/current/installation/upgrade.html\" target=\"_blank\" rel=\"noopener\"\u003ehttps://docs.confluent.io/current/installation/upgrade.html\u003c/a\u003e\n  (our servers are running on CentOS). But at the end KSQL server wouldn\u0026rsquo;t start and failed with this error message:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e[2019-07-11 16:23:31,233] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:53)\njava.lang.IllegalStateException: Invalid replcation factor on topic _confluent-ksql-default__command_topic: 1\n        at io.confluent.ksql.rest.util.KsqlInternalTopicUtils.ensureTopic(KsqlInternalTopicUtils.java:91)\n        at io.confluent.ksql.rest.server.KsqlRestApplication.initialize(KsqlRestApplication.java:237)\n        at io.confluent.ksql.rest.server.KsqlRestApplication.startKsql(KsqlRestApplication.java:200)\n        at io.confluent.ksql.rest.server.KsqlRestApplication.start(KsqlRestApplication.java:187)\n        at io.confluent.ksql.rest.server.KsqlServerMain.tryStartApp(KsqlServerMain.java:65)\n        at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:51)\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eI checked the replication factor with:\u003c/p\u003e","title":"KSQL server failed to start after upgrade"},{"content":"The slides from my talk at Riga Dev Days 2019 will be available here after the session.\n","permalink":"https://enesi.no/2019/05/presentation-at-riga-dev-days-2019/","summary":"\u003cp\u003eThe slides from my talk at Riga Dev Days 2019 will be available here after the session.\u003c/p\u003e","title":"Presentation at Riga Dev Days 2019"},{"content":"Here is the link to the presentation.\n","permalink":"https://enesi.no/2019/02/presentation-at-obug-2019/","summary":"\u003cp\u003e\u003ca href=\"/5_Things_Developers_do.pdf\"\u003eHere is the link\u003c/a\u003e\n to the presentation.\u003c/p\u003e","title":"Presentation at OBUG 2019"},{"content":"If you try to import large Excel files to SQL Developer, it may hang for a long time. But let say you like SQL Developer very much, or you are too lazy to find another tool, here is a way to load large Excel files. By converting the Excel files to CSV SQL Developer can load it much faster.\nThe tool requires Python, and this week I\u0026rsquo;m into creating virtual environments, so I will do here. This works where Python works:\npython3 -m venv venv source venv/bin/activate pip3 install xlsx2csv xlsx2csv -h The last command prints out the options, but usually this is enough:\nxlsx2csv whatever.xlsx whatever.csv Someone judged Excel to be the optimal format for a dataset and sent me a file with 175354 lines and 64 columns. The command above took 2 minutes and 34 seconds on my old Mac. The import of the CSV file with SQL Developer was not super fast, but it did complete after about a coffee break. With the Excel file, SQL Developer errored out during the preview on Java heap space.\nThe source code is available on Github .\n","permalink":"https://enesi.no/2019/01/import-large-excel-files-with-sql-developer/","summary":"\u003cp\u003eIf you try to import large Excel files to SQL Developer, it may hang for a long time. But let say you like SQL Developer very much, or you are too lazy to find another tool, here is a way to load large Excel files.  By converting the Excel files to CSV SQL Developer can load it much faster.\u003c/p\u003e\n\u003cp\u003eThe tool requires Python, and this week I\u0026rsquo;m into creating virtual environments, so I will do here.  This works where Python works:\u003c/p\u003e","title":"Import Large Excel Files with SQL Developer"},{"content":"Here is the link to my presentation at POUG 2018 in Sopot, Polen\n","permalink":"https://enesi.no/2018/09/presentation-at-poug-2018/","summary":"\u003cp\u003e\u003ca href=\"/POUG_2018_5_things.pdf\"\u003eHere is the link\u003c/a\u003e\n to my presentation at \u003ca href=\"http://poug.org/en/edycja/poug-2018/\" target=\"_blank\" rel=\"noopener\"\u003ePOUG 2018\u003c/a\u003e\n in Sopot, Polen\u003c/p\u003e","title":"Presentation at POUG 2018"},{"content":"Epoch time is common in some situations because it is easier to do date calculations than with normal dates. The most popular is UNIX Epoch time which is the number of seconds since the start of January 1, 1970.\nIf you have your data in Oracle you don\u0026rsquo;t have to worry about date arithmetic, since Oracle handles that for you. So in case you have such a table with epoch time and want to query it with functions that expect the DATE datatype, you can add a virtual column that returns the epoch time converted to DATE like this:\nalter table reviews add review_date date generated always as (date \u0026#39;1970-01-01\u0026#39; + time_epoch /60/60/24); If you wonder why epoch time is easier for calculations just think about how you will find the number of days between two dates. With epoch time you simply subtract the earliest from the latest and perform the proper division to convert back to days.\nBy the way, Oracle supports another epoch time, Julian Day, which is the number of days since January 1, 4712 BC. You can get todays date in Julian date like this:\nselect to_char(sysdate,\u0026#39;J\u0026#39;) from dual; Which returned 2458224 on the day this post was published.\nFun fact: the Oracle documentation says number of days since January 1, 4712 BC, but Wikipedia says 4713 BC.\nSome dates are better than others. ","permalink":"https://enesi.no/2018/04/converting-epoch-time-to-date-with-a-virtual-column/","summary":"\u003cp\u003eEpoch time is common in some situations because it is easier to do date calculations than with normal dates. The most popular is \u003ca href=\"https://en.wikipedia.org/wiki/Unix_time\" target=\"_blank\" rel=\"noopener\"\u003eUNIX Epoch time\u003c/a\u003e\n which is\nthe number of seconds since the start of January 1, 1970.\u003c/p\u003e\n\u003cp\u003eIf you have your data in Oracle you don\u0026rsquo;t have to worry about date arithmetic, since Oracle handles that for you. So in case you have such a table with epoch time and want to query it\nwith functions that expect\nthe DATE datatype, you can add a virtual column that returns the epoch time converted to DATE like this:\u003c/p\u003e","title":"Converting Epoch Time to Date with a Virtual Column"},{"content":"I was supposed to do something else that involved using the package UTL_HTTP from the database accessing an URL over HTTPS. Instead I lost several hours troubleshooting ORA-29024: Certificate validation failure and ORA-28750: unknown error (Nice error the last one).\nThis is not the first time I\u0026rsquo;ve been through this. It is getting quite common that sites insist on using HTTPS, even if you access them using HTTP, the client will typically be redirected to HTTPS (port 443).\nBut first, a simple test case. If you want to access a URL from the package UTL_HTTP you first need to create an ACL. For this demo I\u0026rsquo;m using the database user OUGN. Run this as SYSTEM:\nBEGIN DBMS_NETWORK_ACL_ADMIN.CREATE_ACL(acl =\u0026gt; \u0026#39;www.xml\u0026#39;, description =\u0026gt; \u0026#39;WWW ACL\u0026#39;, principal =\u0026gt; \u0026#39;OUGN\u0026#39;, is_grant =\u0026gt; true, privilege =\u0026gt; \u0026#39;connect\u0026#39;); DBMS_NETWORK_ACL_ADMIN.ADD_PRIVILEGE(acl =\u0026gt; \u0026#39;www.xml\u0026#39;, principal =\u0026gt; \u0026#39;OUGN\u0026#39;, is_grant =\u0026gt; true, privilege =\u0026gt; \u0026#39;resolve\u0026#39;); DBMS_NETWORK_ACL_ADMIN.ASSIGN_ACL(acl =\u0026gt; \u0026#39;www.xml\u0026#39;, host =\u0026gt; \u0026#39;*\u0026#39;); commit; END; / It may not be a good habit to open for the whole world, but this is a demo.\nThen you can test it with:\nselect utl_http.request(\u0026#39;ifconfig.co/ip\u0026#39;) from dual; That works nice on 11g and 12c and returns the IP address.\nBut if you try to access an URL starting with HTTPS you will get ORA-29024: Certificate validation failure, which is expected. The Oracle database needs to have the SSL certificate in its wallet to verify it.\nBy following this recipe from Oracle Base aka Tim Hall, you should be up and going in a few minutes.\nI downloaded the certificate using the browser and transferred it to the database server, the file is called COMODO_RSA.cer in the code below.\nThis procedure works great on 11g and 12c, from the commandline where Oracle Home lies (put the cursor in the box below and scroll with arrow keys):\norapki wallet create -wallet $ORACLE_BASE/admin/$ORACLE_SID/wallet -pwd Secret42 -auto_login orapki wallet add -wallet $ORACLE_BASE/admin/$ORACLE_SID/wallet -trusted_cert -cert /tmp/COMODO_RSA.cer -pwd Secret42 orapki wallet display -wallet $ORACLE_BASE/admin/$ORACLE_SID/wallet The last command will obviously show the certificates; you know the feeling so far so good:\nRequested Certificates: User Certificates: Trusted Certificates: Subject: OU=Class 2 Public Primary Certification Authority,O=VeriSign\\, Inc.,C=US Subject: OU=Secure Server Certification Authority,O=RSA Data Security\\, Inc.,C=US Subject: CN=COMODO RSA Certification Authority,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB Subject: OU=Class 3 Public Primary Certification Authority,O=VeriSign\\, Inc.,C=US Subject: OU=Class 1 Public Primary Certification Authority,O=VeriSign\\, Inc.,C=US Subject: CN=GTE CyberTrust Global Root,OU=GTE CyberTrust Solutions\\, Inc.,O=GTE Corporation,C=US On 11g this actually works:\nselect utl_http.request(\u0026#39;https://www.vegvesen.no/\u0026#39;,NULL,\u0026#39;file:/u01/app/oracle/admin/RIO/wallet\u0026#39;,NULL) from dual; But on 12c the same procedure results in ORA-28750: unknown error. And yes, there are changes in 12c. I\u0026rsquo;ve been through lots of documents on MOS without success.\nI got so frustrated that I started a trial in the Oracle Cloud to test it, assuming that if this was caused by a missing patch the database in the sky would be duly patched. Same error.\nI was thinking I should list some documents from MOS I had waded through, and entered again the terms utl_http ORA-28750 12c. Not sure why I had not seen it before, but I found Doc ID 2238096.1 How To Investigate And Troubleshoot SSL/TLS Issues on the Database And Client SQL Net Layer and stumbled upon Patch 26040483.\nThis patch actually solves the problem!\nHeck, I even patched my cloud database, and it works too!\nThe moral of this post: If you are really upset with something, blog about it, you may find the solution before you get to the end.\nAnd if you love patching the cloud won\u0026rsquo;t necessarily put an end to that.\n","permalink":"https://enesi.no/2018/02/oracle-12c-and-ssl/","summary":"\u003cp\u003eI was supposed to do something else that involved using the package \u003ccode\u003eUTL_HTTP\u003c/code\u003e from the database accessing an URL over HTTPS. Instead I lost several hours troubleshooting \u003cem\u003eORA-29024: Certificate validation failure\u003c/em\u003e and \u003cem\u003eORA-28750: unknown error\u003c/em\u003e (Nice error the last one).\u003c/p\u003e\n\u003cp\u003eThis is not the first time I\u0026rsquo;ve been through this. It is getting quite common that sites insist on using HTTPS, even if you access them using HTTP, the client will typically be redirected to HTTPS (port 443).\u003c/p\u003e","title":"Oracle 12c and SSL"},{"content":" Oracle Developer Community Most of my work in the Oracle world has been DBA-oriented. But for the last years, I have rediscovered the joy of development and data analysis. When I haven\u0026rsquo;t been writing the code myself, I have often helped other developers when they connect to the Oracle database, and their program does not perform as expected.\nDevelopers coding in Java, Python, or most other programming languages spend time on figuring out how to get the work done. They learn algorithms to discover the fastest way to do it, or the lazy coders just punch out code without wondering too much about efficiency. If their test data is small, performance problems usually get detected after release to production with ensuing frustrations.\nIf you have an idea but don\u0026rsquo;t want to worry too much about how to do it, then use SQL. You see, SQL is a declarative language. If you are capable of expressing what you want, then write it in SQL, and the algorithms behind will take care of the hard work.\nSQL runs in the database with code much closer to your data then all the alternatives (which are different solutions running somewhere else). It is challenging to beat the performance of well-written SQL running in the Oracle database.\nWhen you tell Oracle what you want, Oracle executes code that implements complex algorithms. What Oracle runs for you also runs a million of times around the world every day. Bugs do happen in software made by humans, but the probability that someone discovers them soon increases tremendously when millions of people use the same software instead of a comparatively small group.\nBesides not losing data , getting the answer correct is the #1 priority for the Oracle database. (Getting wrong answers from a databank is just as useful as getting counterfeit bills from an ATM.) Oracle has put down a lot of effort to make sure this works as expected.\nWriting good SQL can be an art. When I struggle to express a query in SQL, I ask myself if I have understood the problem. Once I have a grasp on the logic, the job is at least half done. Once the statement is written and sent for execution, the result comes back amazingly fast. Typically one answer brings another question, but the ad-hoc nature of this makes it easy to continue with many new enquires. No need to compile or rewrite half the code, just change a bit and learn from new results.\nI use the same language when I help people with optimisation (often called tuning by them because they think there is a knob we forgot to move.) You see, the Oracle database has a lot of information about the data and what is going on in the database. The same SQL I use for reporting I use to investigate performance. I spend most of the time analysing what is going on, and little on how to implement code to get the answer I want.\nSo for this year\u0026rsquo;s appreciation day, I thank Oracle for SQL, and for keeping life interesting for data professionals around the world.\n","permalink":"https://enesi.no/2017/10/odc-appreciation-day-sql/","summary":"\u003cp\u003e\u003cfigure\u003e\u003ca href=\"https://developer.oracle.com/\"\u003e\n    \u003cimg loading=\"lazy\" src=\"/images/oracle-developer-community.png\"/\u003e \u003c/a\u003e\u003cfigcaption\u003e\n            Oracle Developer Community\n        \u003c/figcaption\u003e\n\u003c/figure\u003e\n\nMost of my work in the Oracle world has been DBA-oriented. But for the last years, I have rediscovered the joy of development and data analysis. When I haven\u0026rsquo;t been writing the code myself, I have often helped other developers when they connect to the Oracle database, and their program does not perform as expected.\u003c/p\u003e\n\u003cp\u003eDevelopers coding in Java, Python, or most other programming languages spend time on figuring out \u003cem\u003ehow\u003c/em\u003e to get the work done. They learn algorithms to discover the fastest way to do it, or the lazy coders just punch out code without wondering too much about efficiency. If their test data is small, performance problems usually get detected after release to production with ensuing frustrations.\u003c/p\u003e","title":"ODC Appreciation Day: SQL"},{"content":"I\u0026rsquo;m writing this post so it can be found by googlers struggling with ORA-01861. Usually, ORA-01861 means that you missed something when using a date conversion function. But, here the other day I had created a table using some code generation tool, and I had accidentally declared a column with datatype DATE instead of NUMBER.\nWhen you are using the wrong datatype in SQL, Oracle tries to convert it for you. If you try to insert letters into a DATE column, Oracle will return error message ORA-01858 - a non-numeric character was found where a numeric was expected, if you try to insert a number into it, you\u0026rsquo;ll get ORA-00932 - inconsistent datatypes: expected DATE got NUMBER. But if you insert a number in quotes, and the string is in the right format, Oracle may succeed. ORA-01861 comes when you submit a number in quotes, but is not a complete date.\nSo, if you see ORA-01861 and you haven\u0026rsquo;t used a conversion function, check your table definition.\n","permalink":"https://enesi.no/2017/09/ora-01861/","summary":"\u003cp\u003eI\u0026rsquo;m writing this post so it can be found by googlers struggling with ORA-01861. Usually, ORA-01861 means that you missed something when using a date conversion function. But, here the other day I had created a table using some code generation tool, and I had accidentally declared a column with datatype DATE instead of NUMBER.\u003c/p\u003e\n\u003cp\u003eWhen you are using the wrong datatype in SQL, Oracle tries to convert it for you. If you try to insert letters into a DATE column, Oracle will return error message ORA-01858 - \u003cem\u003ea non-numeric character was found where a numeric was expected\u003c/em\u003e, if you try to insert a number into it, you\u0026rsquo;ll get ORA-00932 - \u003cem\u003einconsistent datatypes: expected DATE got NUMBER\u003c/em\u003e. But if you insert a number in quotes, and the string is in the right format, Oracle may succeed. ORA-01861 comes when you submit a number in quotes, but is not a complete date.\u003c/p\u003e","title":"ORA-01861"},{"content":"In an attempt to compare how long it will take to build a Docker image of Oracle XE (11.2) vs EE (12.2) on my Mac I ran into a problem. The building of XE failed, it complained that only 1023 MB swap space was available. I thought that adding swap during build with this workaround would do, but it turns out that with Docker things are different.\nI found suggestions here , but since I was not reading carefully enough, I missed the point that you add swap to the hypervisor Docker for Mac is running on. This is what I did to solve it. You connect to the hypervisor with:\nscreen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty In my case it did not ask for a password, you get connected as root. Just apply the normal procedure to add swap temporarily:\ndd if=/dev/zero of=/var/swap.file count=2000000 bs=1024 chmod 0600 /var/swap.file mkswap /var/swap.file echo \u0026#34;/var/swap.file swap swap defaults 0 0\u0026#34; \u0026gt;\u0026gt; /etc/fstab swapon -a Disconnect with Ctrl-A d. Start the building again with:\n./buildDockerImage.sh -v 11.2.0.2 -x The change above is not permanent. After a restart of Docker, you can check with free -m (when connected to the hypervisor) that available swap space is down to 1023MB.\nBy the way, I am using Docker for Mac, check out a comparison of Docker for Mac vs Docker Toolbox It is worth pointing out that while Docker Toolbox used a VirtualBox VM, Docker for Mac uses Hyperkit , a lightweight macOS virtualization solution built on top of Hypervisor.framework in macOS 10.10 Yosemite and higher.\nAnother strange issue was that when I connected with screen a second time, the terminal was a complete mess. After a restart of Docker, the terminal was OK.\nTo start the XE container I used the following command:\ndocker run -d -p 15210:1521 --name oracle-xe --shm-size=2g oracle/database:11.2.0.2-xe Without the shm-size option it will fail:\nError: The container doesn\u0026#39;t have enough memory allocated. A database XE container needs at least 1 GB of shared memory (/dev/shm). You currently only have 64 MB allocated to the container. Here I used the option -d, in order to run it in the background. Use this command to see the progress of the database creation:\ndocker logs -ft oracle-xe I also forward the standard 1521 port to 15210 since 1521 is likely occupied with something else.\nLast tip: SQL Developer Command Line version is very lightweight and useful when testing this, you probably can\u0026rsquo;t get connected faster than this:\n./sql system/ede49171a7c0349c@localhost:15210/XE (For once I am not worried about sharing a password, they are generated for each new container.)\n","permalink":"https://enesi.no/2017/09/docker-for-mac-increase-swap/","summary":"\u003cp\u003eIn an attempt to compare how long it will take to build a Docker image of Oracle XE (11.2) vs EE (12.2) on my Mac I ran into a problem. The building of XE failed, it complained that only 1023 MB swap space was available. I thought that adding swap during build with \u003ca href=\"https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Deployment_Guide/s2-swap-creating-file.html\" target=\"_blank\" rel=\"noopener\"\u003ethis workaround\u003c/a\u003e\n would do, but it turns out that with Docker things are different.\u003c/p\u003e\n\u003cp\u003eI found suggestions \u003ca href=\"https://github.com/oracle/docker-images/issues/294\" target=\"_blank\" rel=\"noopener\"\u003ehere\u003c/a\u003e\n, but since I was not reading carefully enough, I missed the point that you add swap to the hypervisor \u003cem\u003eDocker for Mac\u003c/em\u003e is running on. This is what I did to solve it. You connect to the hypervisor with:\u003c/p\u003e","title":"Docker for Mac - Increase Swap"},{"content":"Queries from monitoring software like BMC are not supposed to be among Top SQL, but that was happening in many databases at one site. This may happen if the dictionary statistics are not updated, leading Oracle to choose the wrong plan. Document with ID 1328789.1 on MOS shows an example of this.\nSo the SQL that came from BMC sometimes took up to a minute and was executed every 5 minutes, easily making it to the Top SQL list in Enterprise Manager / Cloud Control. This is the SQL:\nselect count(1) from v$lock where ctime \u0026gt; 120 and type in (\u0026#39;TM\u0026#39;, \u0026#39;TX\u0026#39;, \u0026#39;UL\u0026#39;) and lmode = 0; The sql_id was 9m8xksjywfy1x (I am curious if people actually google for sql_ids.) Anyway, checking the execution of this in SQL Monitor showed that the rows returned from some operations were much higher than expected by the Cost Based Optimizer (CBO). This reminded me of stale dictionary statistics and the search on MOS led me to the document above. The fix is easy, just execute as SYS:\nexec dbms_stats.GATHER_FIXED_OBJECTS_STATS It can take more than a minute for this to complete, even for a small database.\nIn one database, the cursor with the wrong plan wouldn\u0026rsquo;t go away, without figuring out why I followed the instructions in this blog post to remove it.\nFind the ADDRESS and HASH_VALUE of the cursor:\nselect ADDRESS, HASH_VALUE from V$SQLAREA where SQL_ID like \u0026#39;9m8xksjywfy1x\u0026#39;; ADDRESS HASH_VALUE ---------------- ---------- 00000002D314E160 2110224445 Then execute:\nexec DBMS_SHARED_POOL.PURGE (\u0026#39;00000002D314E160,2110224445\u0026#39;,\u0026#39;C\u0026#39;) The first argument is \u0026lsquo;ADDRESS,HASH_VALUE\u0026rsquo;, the second is C for cursor object.\nI had to run this on both instances in the RAC. You can add PLAN_HASH_VALUE to the query above in order to see if the plan differs across instances; next time Oracle executes this query a new plan will hopefully be generated, resulting in a new PLAN_HASH_VALUE.\nI personally prefer to use Enterprise Manager / Cloud Control from Oracle to monitor databases, since I reckon they know better how to monitor their own stuff. Critical alarms can be forwarded to a site-wide monitoring system like BMC, or whatever you are struggling with.\n","permalink":"https://enesi.no/2017/07/slow-sql-from-monitoring-software/","summary":"\u003cp\u003eQueries from monitoring software like BMC are not supposed to be among Top SQL, but that was happening in many databases at one site. This may happen if the dictionary statistics are not updated, leading Oracle to choose the wrong plan. Document with ID 1328789.1 on MOS shows an example of this.\u003c/p\u003e\n\u003cp\u003eSo the SQL that came from BMC sometimes took up to a minute and was executed every 5 minutes, easily making it to the Top SQL list in Enterprise Manager / Cloud Control. This is the SQL:\u003c/p\u003e","title":"Slow SQL From Monitoring Software"},{"content":"It is probably the developers\u0026rsquo; responsibility to run tests, but they may not be aware of features in the database that makes the testing easier. Here the other day an ETL developer asked for help with resetting tables after ETL-testing. But there was no need to create complicated scripts to do so. I showed him flashback tables so he could easily fix it himself and run the tests over and over again without any pain.\nIf you haven\u0026rsquo;t tried this, it is a good case for Swingbench. The example below is from a round with the SOE benchmark.\nFlashback tables is documented in Administrators Guide 20.10 , with a reference further to Backup and Recovery User\u0026rsquo;s Guide, 18.2 There are some requirements though. Materialized views, tables that are part of a cluster (table cluster, not RAC, you know), and other advanced stuff won\u0026rsquo;t work.\nYou need to set up a few things up before you start. Make sure you have room in your UNDO tablespace, and check the value for undo_retention_target. Documentation says that this limit is the minimal time Oracle attempts to preserve undo information before the space is recycled in the UNDO tablespace. Obviously you need to have enough space there for this to work, but you can monitor that as you go along.\nLastly, something called row movement must be enabled. That in a simple description means that Oracle can move a row and therefore give it another address (rowid). If you have some references to rowid\u0026rsquo;s in your code (which you shouldn\u0026rsquo;t have) you will not want to enable this. You enable row movement for a table called customers with this:\nalter table customers enable row movement; Then you flashback a set of tables with this command.\nflashback table ADDRESSES,CUSTOMERS,CARD_DETAILS,WAREHOUSES , ORDER_ITEMS,ORDERS,INVENTORIES,PRODUCT_INFORMATION ,LOGON,PRODUCT_DESCRIPTIONS,ORDERENTRY_METADATA to timestamp to_timestamp(\u0026#39;2017-06-26 16:10:00\u0026#39;,\u0026#39;yyyy-mm-dd hh24:mi:ss\u0026#39;); The list above contains the tables involved in the SOE benchmark from Swingbench. After this you will receive:\nFlashback succeeded. Of course one can flashback the whole database, but this method doesn\u0026rsquo;t require the intervention of a DBA to perform it, nor the restart of the database with open resetlogs (scary stuff). Meaning the developer can do it as much as he wants without bothering you.\nYou may also try out what happens if you flashback only one or a few of the tables involved in the transactions being flashed back. It shouldn\u0026rsquo;t work. Why?\n","permalink":"https://enesi.no/2017/06/flashback-tables/","summary":"\u003cp\u003eIt is probably the developers\u0026rsquo; responsibility to run tests, but they may not be aware of features in the database that makes the testing easier. Here the other day an ETL developer asked for help with resetting tables after ETL-testing. But there was no need to create complicated scripts to do so. I showed him flashback tables so he could easily fix it himself and run the tests over and over again without any pain.\u003c/p\u003e","title":"Flashback Tables"},{"content":"A colleague said that he had some tables that showed up in one tool, but not in another, I think it was Enterprise Manager/Cloud Control. This is easy to forget if you don\u0026rsquo;t work with non-relational tables regularly. What happens is that only relational tables are listed in DBA_TABLES or USER_TABLES, but all tables can be found if you search DBA_OBJECTS / USER_OBJECTS for objects with OBJECT_TYPE=TABLE.\nThe documentation actually says: USER_TABLES describes the relational tables owned by the current user. You can try this code to demonstrate it:\nselect table_name from user_tables; create or replace type my_object_type as object ( brewery varchar2(100), beer varchar2(50)); / create table tastings of my_object_type; select table_name from user_tables; select object_name from user_objects where object_type=\u0026#39;TABLE\u0026#39;; Note your user need the CREATE TYPE privilege to do this. This is the output I got from a newly created user:\nno rows selected Type MY_OBJECT_TYPE compiled Table TASTINGS created. no rows selected OBJECT_NAME -------------------------------------------------------------------------------------------------------------------------------- TASTINGS Anyway, this won\u0026rsquo;t be a problem if you use SQL Developer:\n","permalink":"https://enesi.no/2017/06/invisible-tables/","summary":"\u003cp\u003eA colleague said that he had some tables that showed up in one tool, but not in another, I think it was Enterprise Manager/Cloud Control. This is easy to forget if you don\u0026rsquo;t work with non-relational tables regularly. What happens is that only relational tables are listed in DBA_TABLES or USER_TABLES, but all tables can be found if you search  DBA_OBJECTS / USER_OBJECTS for objects with OBJECT_TYPE=TABLE.\u003c/p\u003e\n\u003cp\u003eThe documentation actually says: \u003cem\u003eUSER_TABLES describes the relational tables owned by the current user.\u003c/em\u003e You can try this code to demonstrate it:\u003c/p\u003e","title":"Invisible Tables"},{"content":"\nHave you been in a presentation with the feeling that the speaker is talking over your head? As if he is speaking to a select group, maybe trying to impress them? The selection of words and expressions, subject for the talk may sometimes give a feeling of cleverness, yet you can\u0026rsquo;t relate to it. Instead you walk away thinking you are not remotely smart enough to give a presentation.\nThat is too bad. Though we are supposed to be originals, our projects and environments mean that statistically speaking there are quite a few in your situation, and perhaps with your thinking. If all of your group have the same feeling; that your are too inexperienced, or concerned with trivial issues, noone is going to give talks you can relate to. Maybe you should give that presentation?\nI have a DBA background, but for the last years I have mostly been working in projects where I support developers. Looking for stuff that won\u0026rsquo;t work well with the database, and helping people to use the database better are my top priorities. (I don\u0026rsquo;t succeed all the time, far from it, but it seem to work.) I keep getting back to basic stuff; focusing on response time on several levels, let the database to do what it does better than a new application, and so on. There is nothing wrong with getting the basics right, and it usually saves you from a lot of frustration later.\nAnother example is data modelling. It is strange how many projects where nobody sat down to create a simple model before they started to create tables. What it tells me is that they didn\u0026rsquo;t spend time to think it through before they started the implementation. These issues happens in the zone where developers and administrators (like DBAs) interact. It sometimes looks like an abyss, or perhaps a DMZ.\nIf you have any experience from projects, where you discovered a method, or a tool, or a solution that moved the project forward, you should start writing it down and eventually you will have material to present to your group.\nSpeaking for the first time at a conference may sound like a nightmare or a suicidal act. But it doesn\u0026rsquo;t have to be. Fact is that speaking at the UKOUG Tech conference has been for me a great experience. The audience has always been nice and I always meet someone who has been in a similar situation.\nGiving a presentation together with someone is one option, if you reach out to the Oracle community it is likely that you will get a positive answer. If you ask me, I don\u0026rsquo;t mind being the clown in the outfit.\nOne rule matters: talk about something you care about, something that you believe in. The rest is just work. The call for paper closes soon, submit your proposal here: http://tech17.ukoug.org/default.asp?p=16612 .\n","permalink":"https://enesi.no/2017/06/you-should-speak-at-ukoug-tech17/","summary":"\u003cp\u003e\u003cimg alt=\"UKOUG Tech17 logo\" loading=\"lazy\" src=\"http://www.ukoug.org/public/images/tech17-c-e-pos-logo-rgb-smaller.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eHave you been in a presentation with the feeling that the speaker is talking over your head? As if he is speaking to a select group, maybe trying to impress them? The selection of words and expressions, subject for the talk may sometimes give a feeling of cleverness, yet you can\u0026rsquo;t relate to it. Instead you walk away thinking you are not remotely smart enough to give a presentation.\u003c/p\u003e","title":"You should speak at UKOUG Tech17"},{"content":"Here the other day I was helping my son with his homework. His task was to create a simple data model, create it in MySQL, and make a simple web page to access the data. I thought that after creating a few INSERT statements on his own I could help him with demo data. Since he had decided to create a movie database I googled for some movie data sets and decided to use the IMDB 5000. You can download it from Kaggle.com (free registration probably required).\nThese datasets are often delivered in one file, which means that the data are denormalized. For his simple data model I only needed a few tricks to get the datas into the tables.\nThe first step is to import the data using SQL Developer. I have one advice on this; do it as simple as possible for you; you are going to copy the data further so there is no reason to make it perfect from start. Just use VARCHAR2 with plenty of space unless you know that all rows have the exact same format. In other situations where you will import the same kind of data repeatedly (like your bank statement), it may be worth the effort to set the correct data type from start. If importing data to SQL Developer is difficult for you, leave me a note in the comment session and I will blog about it.\nIn my case I called the imported table for MOVIES_IMPORT. Here is a simple example to start with; create a table with all the directors:\ncreate table directors as select rownum id, name from ( select distinct director_name name from movies_import where director_name is not null ); Point here is that in the data set there is only one director for each film, so a simple column in a MOVIE table with a foreign key to this table will work. Adding a couple of key constraints:\nalter table directors add constraint director_pk primary key(id); alter table directors add constraint director_uk unique(name); The actors are a bit different. The data set contains three columns for the actors, with headings ACTOR_1_NAME, ACTOR_2_NAME, and ACTOR_3_NAME. No problem, this SQL will do:\ncreate table actors as select rn id,name from ( select rownum rn, name from ( select actor_1_name name from movies_import union select actor_2_name name from movies_import union select actor_3_name name from movies_import ) ); Adding a couple of constratins again:\nalter table actors add constraint actor_pk primary key(id); alter table actors add constraint actor_uk unique(name); In the data set several genres are stored in one column, with the genre names separated with |. This means that there is a many-to-many relationship between the movies and the genres. First I created an empty table with constraints:\ncreate table genres (id number,name varchar2(50), constraint genre_pk primary key(id), constraint genre_uk unique (name)); The following anonymous block uses a tool from APEX to split a column value into rows and store them in the table. If the genre is already there the exception catches the ORA-00001 error and continues:\ndeclare l_elements_arr apex_application_global.vc_arr2; l_str varchar2(2000) ; l_sep varchar2(10) := \u0026#39;|\u0026#39;; e_dupl exception; pragma exception_init(e_dupl,-1); j number; begin j := 0; for g in (select genres from movies_import) loop l_elements_arr := apex_util.string_to_table(g.genres, l_sep); for i in 1..l_elements_arr.count loop begin j := j +1; insert into genres values(j,l_elements_arr(i)); exception when e_dupl then null; end; end loop; end loop; end; / A note about constraints here; some will suggest to add the constraints later, but I have so many bugs in my code that I prefer the database to catch them early on. The logic above would not have worked without the unique constraint from the start.\nI like to keep it simple so I made a couple of simple helper functions to find the primary key in the DIRECTOR and GENRE table:\ncreate or replace function director_id(p_name in varchar2) return number is l_id number; begin select id into l_id from directors where NAME=p_name; return l_id; exception when no_data_found then return null; end; / create or replace function genre_id(p_name in varchar2) return number is l_id number; begin select id into l_id from genres where name=p_name; return l_id; exception when no_data_found then return null; end; / Here is the SQL for the MOVIE table:\ncreate table movies as select rownum id , movie_title , director_id(director_name) director_id , movie_imdb_link , language ,country , content_rating , budget , gross ,title_year ,imdb_score , movie_facebook_likes ,num_user_for_reviews ,duration , num_critic_for_reviews from movies_import; Almost done, just a couple of tables left. First one is to map the movies to the genres:\ncreate table movie_genres ( movie_id number, genre_id number, constraint movie_genres_pk primary key(movie_id,genre_id), constraint movie_genres_fk1 foreign key(movie_id) references movies(id), constraint movie_genres_fk2 foreign key(genre_id) references genres(id)) ; We could probably have used just genre as the primary key instead of the surrogate key. Here is the mapping table for movies and actors:\ncreate table movie_actors ( movie_id number, actor_id number, constraint movie_actors_pk primary key(movie_id,actor_id), constraint movie_actors_fk1 foreign key(movie_id) references movies(id), constraint movie_actors_fk2 foreign key(actor_id) references actors(id)); At this point while working on next step I discovered there were some duplicates in the data set. Just to teach myself a lesson; I hadn\u0026rsquo;t created a primary key on the MOVIE table so Oracle happily inserted all the rows. If you check there are two rows for The Avengers. They occur at line 19 and 796 in the downloaded file and look very similar to me.\nWell, sure there are several movies in the world with the same name, but at this time I started to get tired of this data set and just wanted to produce some insert statements for my son. So I decided to ignore it and remove the duplicates. Some day I may blog about how to find true duplicates in such a data set with SQL.\nHere is some SQL to list movies with more than one row for each movie name:\nselect * from ( select id,movie_title, row_number() over (partition by movie_title order by id) rn, count(*) over (partition by movie_title) cnt from movies order by movie_title ) where cnt\u0026gt;1; This SQL removes the duplicates:\ndelete from movies where id in (select id from ( select id , row_number() over (partition by movie_title order by id) rn, count(*) over (partition by movie_title) cnt from movies) where cnt\u0026gt;1 and rn\u0026gt; 1); Should probably add the keys then:\nalter table movies add constraint movies_pk primary key(id); alter table movies add constraint movies_uk unique (movie_title); After some trial and errors I ended up with this code to update the mapping tables:\ndeclare l_movie_id number; l_elements_arr apex_application_global.vc_arr2; l_str varchar2(2000) ; l_sep varchar2(10) := \u0026#39;|\u0026#39;; begin for m in (select * from (select mov.id movie_id, i.movie_title, i.genres, i.actor_1_name,i.actor_2_name,i.actor_3_name, row_number() over (partition by i.movie_title order by i.movie_title) rn from movies_import i, movies mov where i.movie_title=mov.movie_title and mov.director_id = director_id(i.director_name)) where rn=1) loop l_elements_arr := apex_util.string_to_table(m.genres, l_sep); for i in 1..l_elements_arr.count loop exit when l_elements_arr(i) is null; dbms_output.put_line(m.movie_id || \u0026#39;:\u0026#39; || l_elements_arr(i)); insert into movie_genres values (m.movie_id,genre_id(l_elements_arr(i))); end loop; continue when m.actor_1_name is null; insert into movie_actors values(m.movie_id,actor_id(m.actor_1_name)); continue when m.actor_2_name is null; insert into movie_actors values(m.movie_id,actor_id(m.actor_2_name)); continue when m.actor_3_name is null; insert into movie_actors values(m.movie_id,actor_id(m.actor_3_name)); end loop; end; / I have to be honest and say the process was not all that simple, duplicates and other errors ment I had to rewrite code and go back a few steps. But it was usefull to see that adding constraints from the start stops you from inserting wrong data, and detect problems in the data set.\nIn other words, trying to normalize a data set used for mining lets you find issues and learn about the data set before you write a lot of code. This was an iterative process that I may try again. What if the data set is huge? Maybe a sample will work, but your Oracle database will handle data sets far above the size when downloading them becomes impractical.\n","permalink":"https://enesi.no/2017/05/normalizing-data-sets/","summary":"\u003cp\u003eHere the other day I was helping my son with his homework. His task was to create a simple data model, create it in MySQL, and make a simple web page to access the data. I thought that after creating a few INSERT statements on his own I could help him with demo data. Since he had decided to create a movie database I googled for some movie data sets and decided to use the IMDB 5000. You can download it \u003ca href=\"https://www.kaggle.com/deepmatrix/imdb-5000-movie-dataset\" target=\"_blank\" rel=\"noopener\"\u003efrom Kaggle.com\u003c/a\u003e\n (free registration probably required).\u003c/p\u003e","title":"Normalizing Data Sets"},{"content":"I just got started.\nWith a new blogging platform. I was frustrated with the way things went and wanted to start over with something else, something simple and not pretentious at all. Something that let me spend time on writing the post, and not on formatting the code when I had some SQL to show. Just plain and simple.\nMy previous blog used to be mostly about Oracle, but I spend more and more time looking into Hadoop, Spark and related software. I wanted a new place where I can blog exactly about what I am working on without worrying if it fits in. If I solve a problem with Spark on my mini-cluster I now have a place to share it.\nI\u0026rsquo;m still an Oracle DBA, but my interest has shifted towards the data, and not so much about backup and recovery. Maybe it was the anxiety when doing alter database open resetlogs that made me move on.\nI may add some pictures, right now I just want to get started with the blogging. We\u0026rsquo;ll see.\n","permalink":"https://enesi.no/2017/05/hi-there/","summary":"\u003cp\u003eI just got started.\u003c/p\u003e\n\u003cp\u003eWith a new blogging platform. I was frustrated with the way things went and wanted to start over with something else, something simple and not pretentious at all. Something that let me spend time on writing the post, and not on formatting the code when I had some SQL to show. Just plain and simple.\u003c/p\u003e\n\u003cp\u003eMy previous blog used to be mostly about Oracle, but I spend more and more time looking into Hadoop, Spark and related software. I wanted a new place where I can blog exactly about what I am working on without worrying if it fits in. If I solve a problem with Spark on my mini-cluster I now have a place to share it.\u003c/p\u003e","title":"Hi there!"},{"content":"If you like the simplicity of Docker, but would like to continue with Virtualbox, then Vagrant is for you. After the conference in Dublin, OUG Ireland, I got this challenge:\n@OyvindIsene agreed to be my guinea pig for my @vagrantup files. That\u0026rsquo;s what you get when going to a conference! ;) https://t.co/pJDisr4Mhp — Gerald Venzl (@GeraldVenzl) March 24, 2017 Not much of a challenge, of course, since he has done all the work, it is just \u0026ldquo;Download and Go\u0026rdquo;. Just follow the instructions in the README part on the page linked to in his Tweet. I did this on my Mac at the airport (I found a downloaded copy of 12.2 after all). The only comment I could make is that the git command creates a directory (oracle-12.2-vagrant) and you copy the downloaded zip file (linuxx64_12201_database.zip) into that directory.\nIn the oracle-12.2-vagrant directory there is of course the Vagrantfile that you can edit if you want. In case you have a listener already running on port 1521, you can easily change the Vagrantfile so the port 1521 from the VM will be forwarded to another port. I actually just did vagrant up on my server at home when preparing for this post, and it failed because port 1521 was already in use. No problem, just edit Vagrantfile (line number 19), and repeat the command, nothing to clean up first.\nDo check out this blog post that came out today: Oracle Linux Vagrant boxes by Mikael Sandström. He has been using Vagrant a lot and actually created a box for many versions of Oracle Linux. Which means that if you want to try an upgrade on one specific version of Oracle Linux you can use one of his boxes easily. There are some other useful tips on Vagrant in the post too.\nI will continue to play around with both Docker and VirtualBox/Vagrant. Creating new lab environments has become very easy so we can focus on more important stuff than just installing software.\n","permalink":"https://enesi.no/2017/03/too-easy-for-a-post-install-oracle-12.2-with-vagrant/","summary":"\u003cp\u003eIf you like the simplicity of Docker, but would like to continue with Virtualbox, then Vagrant is for you. After the conference in Dublin,  OUG Ireland, I got this challenge:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003ca href=\"https://twitter.com/OyvindIsene\" target=\"_blank\" rel=\"noopener\"\u003e@OyvindIsene\u003c/a\u003e\n agreed to be my guinea pig for my \u003ca href=\"https://twitter.com/vagrantup\" target=\"_blank\" rel=\"noopener\"\u003e@vagrantup\u003c/a\u003e\n files. That\u0026rsquo;s what you get when going to a conference! ;) \u003ca href=\"https://t.co/pJDisr4Mhp\" target=\"_blank\" rel=\"noopener\"\u003ehttps://t.co/pJDisr4Mhp\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e— Gerald Venzl (@GeraldVenzl) \u003ca href=\"https://twitter.com/GeraldVenzl/status/845353278598631424\" target=\"_blank\" rel=\"noopener\"\u003eMarch 24, 2017\u003c/a\u003e\n\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eNot much of a challenge, of course, since he has done all the work, it is just \u0026ldquo;Download and Go\u0026rdquo;. Just follow the instructions in the README part \u003ca href=\"https://github.com/gvenzl/oracle-12.2-vagrant\" target=\"_blank\" rel=\"noopener\"\u003eon the page\u003c/a\u003e\n linked to in his Tweet. I did this on my Mac at the airport (I found a downloaded copy of 12.2 after all). The only comment I could make is that the git command creates a directory (oracle-12.2-vagrant) and you copy the downloaded zip file (linuxx64_12201_database.zip) into that directory.\u003c/p\u003e","title":"Too Easy for a Post: Install Oracle 12.2 with Vagrant "},{"content":"Oracle Database 12.2 became available for download last week. This weekend I\u0026rsquo;ve been playing with Docker and created a container with it. The whole process is so easy, thanks to the work by Gerald Venzl at Oracle. You\u0026rsquo;ll find all the information you need in his blog post . Though the post is for version 12.1.0.2, the dockerfiles have already been updated for 12.2.0.1, and you can download it from Github using the link he provides in the post.\nIt takes sometime to create a container (depending on your hardware, of course), so when I discovered that the examples / demos that are distributed in another file were not included, I decided to try to add it to the running container. It is easy, just download the linuxx6412201examples.zip file and copy it into the running container, which in my case is named ora12.2:\ndocker cp linuxx6412201examples.zip ora12.2:/tmp Also unzip a copy of the included response file locally:\nunzip -j -d /tmp linuxx6412201examples.zip \\ examples/response/demosinstall.rsp Edit it and change the variables so they look like this:\nUNIXGROUPNAME=oinstall ORACLEHOME=/opt/oracle/product/12.2.0.1/dbhome1 ORACLEBASE=/opt/oracle Copy it to the container:\ndocker cp /tmp/demosinstall.rsp ora12.2:/tmp Connect to your container with a command similar to this (again, ora12.2 is the name of my container):\ndocker exec -ti ora12.2 /bin/bash Inside the container, unzip the transferred zip file and start the installation with:\ncd /tmp unzip linuxx6412201examples.zip cd examples ./runInstaller -silent -force -waitforcompletion \\ responsefile /tmp/demosinstall.rsp -ignoresysprereqs -ignoreprereq You can now verify that you have more demos installed, for instance under $ORACLEHOME/md/demo.\nOf course you can achieve the same effect by adding a few lines to the Dockerfile, build, and run again. Something like this should do for the Dockerfile:\nAdded install of examples ENV EXAMPLERSP=\u0026#34;demosinstall.rsp\u0026#34; \\ EXAMPLEFILE=\u0026#34;linuxx6412201examples.zip\u0026#34; COPY $EXAMPLERSP $EXAMPLEFILE /tmp/ RUN cd /tmp \u0026amp;\u0026amp; \\ unzip linuxx6412201examples.zip \u0026amp;\u0026amp; \\ cd examples \u0026amp;\u0026amp; \\ ./runInstaller -silent -force -waitforcompletion \\ -responsefile /tmp/demosinstall.rsp -ignoresysprereqs -ignoreprereq It turned out to run just as fast since the new build builds on the previous images, oh well.\n","permalink":"https://enesi.no/2017/03/adding-examples-to-a-docker-container-with-oracle-database-12.2/","summary":"\u003cp\u003eOracle Database 12.2 became available for download last week. This weekend I\u0026rsquo;ve been playing with Docker and created a container with it. The whole process is so easy, thanks to the work by Gerald Venzl at Oracle. You\u0026rsquo;ll find all the information you need in \u003ca href=\"https://blogs.oracle.com/developer/entry/creating_and_oracle_database_docker\" target=\"_blank\" rel=\"noopener\"\u003ehis blog post\u003c/a\u003e\n. Though the post is for version 12.1.0.2, the dockerfiles have already been updated for 12.2.0.1, and you can download it from Github using the link he provides in the post.\u003c/p\u003e","title":"Adding Examples to a Docker Container with Oracle database 12.2"},{"content":"Every 4 months or so I need a simple way to split a string (VARCHAR2) into elements, where the elements are separated with some fixed value (a comma, a colon, or perhaps a longer string). Since my short-term memory is too short0, I figured I should make a reminder here. Of course, you\u0026rsquo;ll find this on Stackoverflow as well.\nThere is this function in APEX, which is usually1 available for you in the database, even if you are not using APEX. Here is a short demo:\ndeclare l_elements_arr apex_application_global.vcarr2; l_str varchar2(2000) := \u0026#39;IPAStoutPorterPale Ale\u0026#39;; l_sep varchar2(10) := \u0026#39;\u0026#39;; begin l_elements_arr := apex_util.string_to_table(l_str, l_sep); for i in 1..l_elements_arr.count loop dbms_output.put_line(l_elements_arr(i)); end loop; l_str := apex_util.table_to_string(l_elements_arr,\u0026#39;|\u0026#39;); dbms_output.put_line(l_str); end; / The variables for the string and the separator can of course be part of a procedure / function. Also not that the separator can have more than one character. I used three stars because beer is probably good for you.\nAnother demo inspired by this Oracle blog post :\ndeclare l_str varchar2(100) := \u0026#39;I$$bought$$a$$3D printer$$this weekend\u0026#39;; l_sep varchar2(10) := \u0026#39;$$\u0026#39;; l_exp varchar2(200) := \u0026#39;^\u0026#39; || lsep || \u0026#39;+\u0026#39; ; begin for e in (select regexp_substr(l_str,l_exp, 1, level) str from dual connect by regexp_substr(l_str,l_exp, 1, level) is not null) loop dbms_output.put_line(e.str); end loop; end; / Since the work is done by the SQL statement in the FOR-LOOP, you can easily use this in pure SQL as well.\nLet\u0026rsquo;s just say that my buffer cache has surrendered space to the library cache, because there is so much interesting stuff going on, and all that parsing takes space. \u0026ldquo;Usually\u0026rdquo;, because APEX is installed when creating the database, unless you deselect one of the standard components, which would, in almost all cases (that is, more often than what \u0026ldquo;usually\u0026rdquo; means), would be a mistake. If you regret your mistake, just download the latest version and install it. That will take less time than it takes to teach me write proper sentences with an adequate use of strange words, and commas. ","permalink":"https://enesi.no/2017/02/splitting-a-string-into-elements/","summary":"\u003cp\u003eEvery 4 months or so I need a simple way to split a string (VARCHAR2) into elements, where the elements are separated with some fixed value (a comma, a colon, or perhaps a longer string). Since my short-term memory is too short0, I figured I should make a reminder here. Of course, you\u0026rsquo;ll find this on Stackoverflow as well.\u003c/p\u003e\n\u003cp\u003eThere is this function in APEX, which is usually1 available for you in the database, even if you are not using APEX. Here is a short demo:\u003c/p\u003e","title":"Splitting a String into Elements"},{"content":"Yesterday I was importing some data I downloaded. When creating a new table I tried to convert two columns with latitude and longitude, stored as VARCHAR2, to numbers (in order to use them in the SDO_GEOMETRY constructor). The create table as select (CTAS) statement failed with ORA-01722. Problem is it does not tell you which line or what value is the offender. You may turn on some tracing with event 1722, but using regular expression is much easier:\nselect longitude from data_import where regexp_instr(longitude,\u0026#39;\\\\d+.\\\\d{7}\u0026#39;) = 0; This will show you rows that does match the format used in the to_number function. Which in this case was \u0026lsquo;99.9999999\u0026rsquo;. Turned out that some of the rows had \u0026ldquo;(blank)\u0026rdquo; in them\u0026hellip; they were updated to NULL.\nOracle database has support for regular expressions in theses functions:\nREGEXP_COUNT REGEXP_INSTR REGEXP_REPLACE REGEXP_SUBSTR ","permalink":"https://enesi.no/2017/02/ora-1722-and-regular-expressions/","summary":"\u003cp\u003eYesterday I was importing some data I downloaded. When creating a new table I tried to convert two columns with latitude and longitude, stored as VARCHAR2, to numbers (in order to use them in the SDO_GEOMETRY constructor). The create table as select (CTAS) statement failed with ORA-01722. Problem is it does not tell you which line or what value is the offender. You may turn on some tracing with event 1722, but using regular expression is much easier:\u003c/p\u003e","title":"ORA-1722 and Regular Expressions"},{"content":"If you need to delete all rows in a table that has parent keys for other tables\u0026rsquo; foreign keys, and the foreign keys constraints have not been defined with \u0026ldquo;on delete cascade\u0026rdquo;, you can do a recursive delete with the following simple procedure.\nThis is typically something you will do only in a test or development database, and not in production. As always, it is a good thing to understand this procedure before you execute it:\ncreate or replace procedure delete_cascade( p_table_name in user_tables.table_name%type) is l_pk user_constraints.constraint_name%type; begin select constraint_name into l_pk from user_constraints where Constraint_Type=\u0026#39;P\u0026#39; and table_name=upper(p_table_name); for c in ( select table_name from user_constraints where R_CONSTRAINT_NAME=l_pk) loop delete_cascade(c.table_name); end loop; execute immediate \u0026#39;delete from \u0026#39; || upper(p_table_name); exception when no_data_found then -- Tables without constraint execute immediate \u0026#39;delete from \u0026#39; || upper(p_table_name);; end; / ","permalink":"https://enesi.no/2016/11/delete-cascade-with-recursive-pl-sql/","summary":"\u003cp\u003eIf you need to delete all rows in a table that has parent keys for other tables\u0026rsquo; foreign keys, and the foreign keys constraints have not been defined with \u0026ldquo;on delete cascade\u0026rdquo;, you can do a recursive delete with the following simple procedure.\u003c/p\u003e\n\u003cp\u003eThis is typically something you will do only in a test or development database, and not in production. As always, it is a good thing to understand this procedure before you execute it:\u003c/p\u003e","title":"Delete Cascade with Recursive PL/SQL"},{"content":"There are many ways to display spatial data, but when exploring a new data set in SQL Developer I think the built in Map View is practical. For this demo I used a list of nuclear power stations in the world, found via http://freegisdata.rtwilson.com/ . The Google Fusion Table can be exported to CSV format, and it includes the longitude and latitude which makes it easy to convert to SDO_GEOMETRY. After importing the file with SQL Developer into a table called NUCLEAR, I did this to add native Spatial data:\nalter table nuclear add geom sdo_geometry; update nuclear set geom=sdo_geometry(2001, 8307, sdo_point_type(to_number(substr(location,instr(location,\u0026#39;,\u0026#39;)+1)), to_number(substr(location,1,instr(location,\u0026#39;,\u0026#39;)-1)), null), null, null); Since I want to try out a spatial operator, I need to add metadata for the column (or layer) and add a spatial index:\ninsert into user_sdo_geom_metadata(TABLE_NAME,COLUMN_NAME,diminfo,srid) values (\u0026#39;NUCLEAR\u0026#39;,\u0026#39;GEOM\u0026#39;, sdo_dim_array(sdo_dim_element(\u0026#39;Longitude\u0026#39;,-180,180,3), sdo_dim_element(\u0026#39;Latitude\u0026#39;,-90,90,3)),8307); create index nuclear_si on nuclear(geom) indextype is mdsys.spatial_index parameters (\u0026#39;layer_gtype=POINT\u0026#39;); The parameter is not necessary, but when you use this, Oracle can check that all new data has the same GTYPE and will speed up index creation. Not a big deal here, but useful to know for later.\nWith this in place I can search for active reactors within a distance of 1000km from my home:\nselect name,country ,round(sdo_geom.sdo_distance(geom ,sdo_geometry(2001,8307,sdo_point_type(10.8000994,59.8301125,null) ,null,null) ,10, \u0026#39;unit=km\u0026#39;)) \u0026#34;Distance in km\u0026#34; from nuclear where sdo_within_distance(geom, sdo_geometry(2001,8307 ,sdo_point_type(10.8000994,59.8301125,null),null,null) ,\u0026#39;distance=1000 unit=km\u0026#39;) = \u0026#39;TRUE\u0026#39; and active_reactors\u0026gt;0 order by 3; NAME\nCOUNTRY\nDistance in km\nRINGHALS\nSWEDEN\n297\nFORSMARK\nSWEDEN\n415\nOSKARSHAMN\nSWEDEN\n434\nBARSEBECK\nSWEDEN\n471\nOLKILUOTO\nFINLAND\n604\nBRUNSBUETTEL\nGERMANY\n666\nBROKDORF\nGERMANY\n672\nSTADE\nGERMANY\n696\nKRUEMMEL\nGERMANY\n716\nUNTERWESER\nGERMANY\n726\nEMSLAND\nGERMANY\n848\nLOVIISA\nFINLAND\n865\nGROHNDE\nGERMANY\n872\nTORNESS\nUNITED KINGDOM\n891\nHARTLEPOOL\nUNITED KINGDOM\n924\nCHAPELCROSS\nUNITED KINGDOM\n995\nA table looks nice, but it sure would be more informative on a map. I added the GEOM column to the query (and removed the others since I don\u0026rsquo;t need them, and also the ORDER BY). The result is this:\nBring up the menu on the result set (right-click or similar on one of the rows), and select Invoke Map View on result set:\nYou get something like this:\nThat is not phenomenal. I had this idea that there should be a map in here. Well, a map with countries and their borders are spatial objects in themselves, and since I had downloaded the data files for the book Pro Oracle Spatial for Oracle Database 11g , I could just display the countries within the same distance:\nselect geom from spatial.world_countries where sdo_within_distance(geom, sdo_geometry(2001,8307 ,sdo_point_type(10.8000994,59.8301125,null),null,null) ,\u0026#39;distance=1000 unit=km\u0026#39;) = \u0026#39;TRUE\u0026#39;; and repeat the procedure. I also made this simple query to get an SDO_GEOMETRY object for my own location:\nselect sdo_geometry(2001,8307,sdo_point_type(10.8000994,59.8301125,null),null,null) from dual; After changing the order, titles, colors, and zoom, the result look like this:\nAlso note that I didn\u0026rsquo;t make any comments about our neighbours ;-)\n","permalink":"https://enesi.no/2016/10/displaying-spatial-data-in-sql-developer/","summary":"\u003cp\u003eThere are many ways to display spatial data, but when exploring a new data set in SQL Developer I think the built in Map View is practical. For this demo I used a \u003ca href=\"https://www.google.com/fusiontables/DataSource?dsrcid=579353\" target=\"_blank\" rel=\"noopener\"\u003elist of nuclear power stations\u003c/a\u003e\n in the world, found via \u003ca href=\"http://freegisdata.rtwilson.com/\" target=\"_blank\" rel=\"noopener\"\u003ehttp://freegisdata.rtwilson.com/\u003c/a\u003e\n. The Google Fusion Table can be exported to CSV format, and it includes the longitude and latitude which makes it easy to convert to SDO_GEOMETRY. After importing the file with SQL Developer into a table called NUCLEAR, I did this to add native Spatial data:\u003c/p\u003e","title":"Displaying Spatial Data in SQL Developer"},{"content":"The previous post showed how to import a file in GPX-format into a table that uses the XMLType. This post shows how to import the waypoints from the GPS logger into a table with the native SDO_GEOMETRY type.\nFirst create a table:\ncreate table gps_log ( log_time timestamp with time zone, waypoint sdo_geometry, src varchar2(20)); The datatype for log_time is chosen like shown because the type used in the XML-file (xsd:dateTime) maps to this, which makes importing the time data much easier. (Have a look at the previous post to see this mapping.)\nNot strictly necessary at this point, but I want to add metadata about the spatial column (aka layer). This is done by doing an insert on a view that resides in the MDSYS-schema (with a public synonym added). Insert on the view works by instead-of triggers defined on the view, in case you wondered how this works:\ninsert into user_sdo_geom_metadata(TABLE_NAME,COLUMN_NAME,diminfo,srid) values (\u0026#39;GPS_LOG\u0026#39;,\u0026#39;WAYPOINT\u0026#39;, sdo_dim_array(sdo_dim_element(\u0026#39;Latitude\u0026#39;,-90,90,3), sdo_dim_element(\u0026#39;Longitude\u0026#39;,-180,180,3)), 8307); commit; If you don\u0026rsquo;t want to write this SQL yourself, you can use a nice feature in SQL Developer. Bring up the menu for the table in the Connections window, select Spatial and then Update Spatial Metadata\u0026hellip;:\nThen you can add the information in the pop-up window:\nYou can verify that this actually worked with:\nselect * from user_sdo_geom_metadata; The app I was using used the position from the mobile network sometimes. I guess there was problem with the GPS reception at times. Anyway, the position reported from the mobile network does not include elevation, so I decided to leave it out. This has an implication when choosing the _spatial reference ID; _8307 is commonly used for waypoints given by latitude and longitude (and without elevation), see this chapter for details . Another detail is the chosen tolerance; the app reported that the accuracy was around 4 meters from the GPS, usually worse from the mobile network, so I figured 3 meters is OK. (For geodetic coordinate systems the unit for tolerance is meter.)\nInserting the data into the table is pretty easy by adapting the SQL code at the end of previous post:\ninsert into gps_log(log_time,waypoint,src) SELECT EXTRACTVALUE(VALUE(t), \u0026#39;trkpt/time\u0026#39;), sdo_geometry(2001, 8307, sdo_point_type(EXTRACTVALUE(VALUE(t), \u0026#39;trkpt/@lat\u0026#39;), EXTRACTVALUE(VALUE(t), \u0026#39;trkpt/@lon\u0026#39;),null), null, null), EXTRACTVALUE(value(t),\u0026#39;trkpt/src\u0026#39;) FROM GPX g, TABLE(XMLSequence(extract(g.OBJECT_VALUE, \u0026#39;/gpx/trk/trkseg/trkpt\u0026#39;, \u0026#39;xmlns=\u0026#34;http://www.topografix.com/GPX/1/0\u0026#34;\u0026#39;))) t; commit; The following creates a spatial index on the WAYPOINT-column, but before you execute this, make sure the user has the privilege CREATE SEQUENCE in addition to CREATE TABLE:\ncreate index gps_log_si on gps_log(waypoint) indextype is mdsys.spatial_index The statement above will return an error stack including ORA-29855 and ORA-13203 if metadata on the column is missing in USER_SDO_GEOM_METADATA. Probably it is a good habit to add it right after the creation of the table as shown previously. Note, if the statement failed, it may have created the index object (although not a useful one), and you may need to drop the index before you try again.\n","permalink":"https://enesi.no/2016/10/importing-gpx-files-to-oracle-database-part-2/","summary":"\u003cp\u003eThe \u003ca href=\"https://enesi.no/2016/10/importing-gpx-files-to-oracle-database-part-1/\"\u003eprevious post\u003c/a\u003e\n showed how to import a file in GPX-format into a table that uses the XMLType. This post shows how to import the waypoints from the GPS logger into a table with the native SDO_GEOMETRY type.\u003c/p\u003e\n\u003cp\u003eFirst create  a table:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003ecreate table gps_log (  \n  log_time timestamp with time zone,  \n  waypoint sdo_geometry,  \n  src varchar2(20));  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eThe datatype for log_time is chosen like shown because the type used in the XML-file (xsd:dateTime) maps to this, which makes importing the time data much easier. (Have a look at the previous post to see this mapping.)\u003c/p\u003e","title":"Importing GPX files to Oracle database, part 2"},{"content":" As suggested by Tim Hall , aka Oracle-Base.com, this post is about my favourite feature in the Oracle world. My favourite product is the good old database, and my favourite feature is simply the fact that I don\u0026rsquo;t lose data!\nNever once have I lost data due to software errors, and after many hard landings the database always manages to come back online after wading through the redo logs and make sure the data remains consistent.\nI have been working as a DBA at sites where some decided to do maintenance on their UPS, the SAN, the network, or whatever in peak hours, resulting in hard landings that many databases won\u0026rsquo;t recover from. The worst nightmare for a DBA is to lose data. Yes, we have backups, and we spend time learning disaster recovery. But nothing beats the relief when you initiate \u0026ldquo;startup\u0026rdquo; after a crash and Oracle after a little while responds with \u0026ldquo;Database opened\u0026rdquo;. You may say I\u0026rsquo;ve been lucky; software errors that corrupt data do exists, but after thousands of databases and many years with Oracle Database administration, not losing data remains my favourite feature.\nMy second favourite is all the instrumentation in there, the ability to figure out what is going on when someone complains. The database instrumentation is frequently a good starting place, even when the fault is outside somewhere, like in a badly designed application in another layer.\nThank you, OTN, for the support, information, articles, easy downloads, the ACE program, events, and much more. Have a great Appreciation Day!\n","permalink":"https://enesi.no/2016/10/otn-appreciation-day-a-database-that-is-reliable/","summary":"\u003cp\u003e\u003ca href=\"/images/blogger/otn-appreciation-day-database-that-is-1.jpg\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/otn-appreciation-day-database-that-is-1.jpg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eAs \u003ca href=\"https://oracle-base.com/blog/2016/09/28/otn-appreciation-day/\" target=\"_blank\" rel=\"noopener\"\u003esuggested by Tim Hall\u003c/a\u003e\n, aka Oracle-Base.com,  this post is about my favourite feature in the Oracle world. My favourite product is the good old database, and my favourite feature is simply the fact that I don\u0026rsquo;t lose data!\u003c/p\u003e\n\u003cp\u003eNever once have I lost data due to software errors, and after many hard landings the database always manages to come back online after wading through the redo logs and make sure the data remains consistent.\u003c/p\u003e","title":"OTN Appreciation Day: A database that is reliable "},{"content":"Update 2016-10-16: Found and error in procedure, see below.\nIn preparation for a presentation on Oracle Spatial for DBAs I decided to import some GPS-data. I have an app called GPSLogger on my Android mobile that logs GPS data to a file. The format for this is GPS Exchange Format (GPX). Since I spent some time understanding the XML side of this I decided to blog about it.\nNote, there are two versions of the GPX schema, 1.0 and 1.1. My app uses the version 1.0, if you try to import a GPX file, have a look in it to verify which schema definition it uses (Look for xsi:schemaLocation in the head of the GPX file).\nI followed the information from this blog post (that actually uses version 1.1) together with info from Oracle Support, and lots of googling.\nThe app can upload to Dropbox which makes it easy to transfer the file to the database server. In addition I needed the schema definition, which you can get from Topographix.com, you\u0026rsquo;ll find the link to the XSD file in the GPX file as mentioned, in my case it was http://www.topografix.com/GPX/1/0/gpx.xsd However, the file needs some adjustment in order to have Oracle parse values of type xsd:dateTime correctly. Open it in an editor and look for the following block:\n\u0026lt;xsd:schema xmlns:xsd=\u0026#34;http://www.w3.org/2001/XMLSchema\u0026#34; xmlns:gpx=\u0026#34;http://www.topografix.com/GPX/1/0\u0026#34; targetNamespace=\u0026#34;http://www.topografix.com/GPX/1/0\u0026#34; elementFormDefault=\u0026#34;qualified\u0026#34;\u0026gt; At the end of this, add one line so it looks like this:\n\u0026lt;xsd:schema xmlns:xsd=\u0026#34;http://www.w3.org/2001/XMLSchema\u0026#34; xmlns:gpx=\u0026#34;http://www.topografix.com/GPX/1/0\u0026#34; targetNamespace=\u0026#34;http://www.topografix.com/GPX/1/0\u0026#34; elementFormDefault=\u0026#34;qualified\u0026#34; xmlns:xdb=\u0026#34;http://xmlns.oracle.com/xdb\u0026gt;\u0026#34; Then you have to search through the file and search for every definition that uses type \u0026ldquo;xsd:dateTime\u0026rdquo; and add:\nxdb:sqltype=\u0026ldquo;TIMESTAMP WITH TIME ZONE\u0026rdquo;\nso it looks like this:\nxsd:element minoccurs=\u0026#34;0\u0026#34; name=\u0026#34;time\u0026#34; type=\u0026#34;xsd:dateTime\u0026#34; xdb:sqltype=\u0026#34;TIMESTAMP WITH TIME ZONE\u0026#34; (Take away the line break, I added it to make it readable). There are four entries, so this manual job does not take a long time. I transferred this file together with my GPX file to a directory on my server (/u01/app/oracle/load_dir). Here is the first part to get started, the first part is executed as SYSTEM:\ncreate user oyvind identified by oracle default tablespace users temporary tablespace temp quota unlimited on users; grant create session, create table, create view, create procedure to oyvind; grant alter session to oyvind; grant create type to oyvind; create directory LOAD_DIR as \u0026#39;/u01/app/oracle/load_dir\u0026#39;; grant read, write on directory load_dir to oyvind; As you can see, I did not granted XDBADMIN to this user. Update 2016-10-16: The use of DBMS_XDB does indeed require being SYS, the owner of the package (XDB), or having one of the roles XDBADMIN / DBA. Since I did not want an ordinary user to have an admin role, I executed the first part that uses the DBMS_XDB package as SYSTEM. The function below was created in the schema of the ordinary user as well. I must have been experimenting with granting and revoking the XDBADMIN without noticing that the first part did nothing since the resource was already there.\nThe first step is to load the XML schema definition to the database. I copied the function getClobDocument from the mentioned blog, it is used to load files as CLOB:\ncreate or replace function getClobDocument( p_directoryname In varchar2, p_filename In varchar2, p_charset In varchar2 default NULL) return CLOB deterministic is v_file bfile := bfilename(p_directoryname,p_filename); v_charContent CLOB := \u0026#39; \u0026#39;; v_targetFile bfile; v_lang_ctx number := DBMS_LOB.default_lang_ctx; v_charset_id number := 0; v_src_offset number := 1 ; v_dst_offset number := 1 ; v_warning number; begin if p_charset is not null then v_charset_id := NLS_CHARSET_ID(p_charset); end if; v_targetFile := v_file; DBMS_LOB.fileopen(v_targetFile, DBMS_LOB.file_readonly); DBMS_LOB.LOADCLOBFROMFILE(v_charContent, v_targetFile, DBMS_LOB.getLength(v_targetFile), v_src_offset, v_dst_offset, v_charset_id, v_lang_ctx, v_warning); DBMS_LOB.fileclose(v_targetFile); return v_charContent; end; / The following code worked for me to create resource:\ndeclare schemaURL varchar2(256) := \u0026#39;http://www.topografix.com/GPX/1/0/gpx.xsd\u0026#39;; schemaDoc varchar2(30) := \u0026#39;gpx.xsd\u0026#39;; xmlSchema xmlType; res boolean; begin xmlSchema := XMLTYPE(getCLOBDocument(\u0026#39;LOAD_DIR\u0026#39;,schemaDoc,\u0026#39;UTF8\u0026#39;)); if (dbms_xdb.existsResource(\u0026#39;/home/\u0026#39; ||schemaDoc)) then dbms_xdb.deleteResource(\u0026#39;/home/\u0026#39; ||schemaDoc); end if; res := dbms_xdb.createResource(\u0026#39;/home/\u0026#39; || schemaDoc,xmlSchema); if res then dbms_output.put_Line(\u0026#39;OK\u0026#39;); else dbms_output.put_Line(\u0026#39;NOK\u0026#39;); end if; end; / Make sure you turn on server output and get a proper message to indicate if it succeeded or not; no error message was returned when it failed.\nThe rest of this is executed as user oyvind (Update: also create the getClobDocument from above to this user). Next step is to register the schema, I used the following code:\ndeclare schemaURL varchar2(256) := \u0026#39;http://www.topografix.com/GPX/1/0/gpx.xsd\u0026#39;; schemaDoc varchar2(30) := \u0026#39;gpx.xsd\u0026#39;; begin dbms_xmlschema.registerSchema ( schemaURL, XMLTYPE(getCLOBDocument(\u0026#39;LOAD_DIR\u0026#39;,schemaDoc,\u0026#39;AL32UTF8\u0026#39;)), local =\u0026gt; true, genTypes =\u0026gt; true, genTables =\u0026gt; false, enableHierarchy =\u0026gt; dbms_xmlschema.ENABLE_HIERARCHY_NONE ); end; / Use the following code to create a table to store the document as XMLType:\nCREATE TABLE gpx OF XMLType (CHECK (XMLIsValid(object_value) = 1)) XMLSCHEMA \u0026#34;http://www.topografix.com/GPX/1/0/gpx.xsd\u0026#34; ELEMENT \u0026#34;gpx\u0026#34;; Then, finally, you can load the GPX file with this:\nINSERT INTO GPX VALUES (XMLTYPE(getCLOBDocument(\u0026#39;LOAD_DIR\u0026#39;,\u0026#39;20161001.gpx\u0026#39;))); commit; To see some of the data, you can extract what you find interesting with:\nSELECT to_number(EXTRACTVALUE(VALUE(t), \u0026#39;trkpt/@lon\u0026#39;)) longitude, to_number(EXTRACTVALUE(VALUE(t), \u0026#39;trkpt/@lat\u0026#39;)) latitude, TO_NUMBER(EXTRACTVALUE(VALUE(t), \u0026#39;trkpt/ele\u0026#39;)) Elevation, EXTRACTVALUE(value(t),\u0026#39;trkpt/src\u0026#39;) src FROM GPX g, TABLE(XMLSequence(extract(g.OBJECT_VALUE,\u0026#39;/gpx/trk/trkseg/trkpt\u0026#39;,\u0026#39;xmlns=\u0026#34;http://www.topografix.com/GPX/1/0\u0026#34;\u0026#39;))) t; This is an example of what comes out (it was a short walk):\nLONGITUDE\nLATITUDE\nELEVATION\nSRC\n10.79831903\n59.83451229\n85\ngps\n10.80009538\n59.83263633\n170\ngps\n10.80033655\n59.83156553\n146\ngps\n10.8003321\n59.8303762\nnetwork\nIn next post I plan to use the Spatial datatype to store these points. I may do another test with GPX version 1.1 in the mean time since that is the version used by my Garmin GPS.\nSome tips if you need to clean up in case of errors; if you have the recyclebin enabled, do a \u0026ldquo;purge recyclebin\u0026rdquo; after dropping the table with the XMLType. I used to following command to delete the schema:\npurge recyclebin; exec dbms_xmlschema.deleteschema(schemaurl =\u0026gt; \u0026#39;http://www.topografix.com/GPX/1/0/gpx.xsd\u0026#39;); Get in touch if you need a copy of the final gpx.xsd file.\n","permalink":"https://enesi.no/2016/10/importing-gpx-files-to-oracle-database-part-1/","summary":"\u003cp\u003eUpdate 2016-10-16: Found and error in procedure, see below.\u003c/p\u003e\n\u003cp\u003eIn preparation for a presentation on Oracle Spatial for DBAs I decided to import some GPS-data. I have an app called \u003ca href=\"https://play.google.com/store/apps/details?id=com.mendhak.gpslogger\" target=\"_blank\" rel=\"noopener\"\u003eGPSLogger\u003c/a\u003e\n on my Android mobile that logs GPS data to a file. The format for this is \u003ca href=\"https://en.wikipedia.org/wiki/GPS_Exchange_Format\" target=\"_blank\" rel=\"noopener\"\u003eGPS Exchange Format\u003c/a\u003e\n (GPX).  Since I spent some time understanding the XML side of this I decided to blog about it.\u003c/p\u003e\n\u003cp\u003eNote, there are two versions of the GPX schema, 1.0 and 1.1. My app uses the version 1.0, if you try to import a GPX file, have a look in it to verify which schema definition it uses (Look for xsi:schemaLocation in the head of the GPX file).\u003c/p\u003e","title":"Importing GPX files to Oracle database, part 1"},{"content":"So I went to Oracle Open World again. For me this year it was partly vacation, networking, and inspirations for work. I went on my own, to a few sessions, spent a lot of time on Oak Table World, hanging out in the OTN Lounge, socialising, and at meet-ups.\nOn my way home I put down some of the takeaways:\nThere were a lot of clouds, but it didn’t rain. There are a some speakers who manage to go back to stuff I’ve seen before, yet inspire me with new ideas. Some presenters are so inspiring that I went to sessions that I thought were almost irrelevant for me. It happened more than once that I had a “Huh, this is cool!”-moment. I am thankful for being part of the ACE program, and it is a great privilege to meet more or less likeminded people from all around the world. To stay inside it, I have to keep learning and share with the community, another good motivation for a better career. OOW, despite being one of the most commercial conferences I go to, is the best opportunity to meet all these people, I’ll do the same next year. Again I met some very helpful people from Oracle, people I like to talk to about everything, from Apocalypse to Zookeeper. These people reminds me that Oracle is more than a machine, and the software I work with every day is made by both smart and nice experts. Next year, join the biking on Saturday, or the running over the bridge on Sunday to see what I mean. OTN which is responsible for the ACE program does a lot of cool stuff like the IoT workshop, and much more. I hope the team do understand they have many friends around the world. Some of us plan to show this in near future, check this blog post by Tim Hall aka Oracle-Base: https://oracle-base.com/blog/2016/09/28/otn-appreciation-day/ Next year, October 1.\n","permalink":"https://enesi.no/2016/09/oracle-open-world-2016-afterthoughts/","summary":"\u003cp\u003eSo I went to Oracle Open World again. For me this year it was partly vacation, networking, and inspirations for work. I went on my own, to a few sessions, spent a lot of time on Oak Table World, hanging out in the OTN Lounge, socialising, and at meet-ups.\u003c/p\u003e\n\u003cp\u003eOn my way home I put down some of the takeaways:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThere were a lot of clouds, but it didn’t rain.\u003c/li\u003e\n\u003cli\u003eThere are a some speakers who manage to go back to stuff I’ve seen before, yet inspire me with new ideas.\u003c/li\u003e\n\u003cli\u003eSome presenters are so inspiring that I went to sessions that I thought were almost irrelevant for me.  It happened more than once that I had a “Huh, this is cool!”-moment.\u003c/li\u003e\n\u003cli\u003eI am thankful for being part of the ACE program, and it is a great privilege to meet more or less likeminded people from all around the world. To stay inside it, I have to keep learning and share with the community, another good motivation for a better career.\u003c/li\u003e\n\u003cli\u003eOOW, despite being one of the most commercial conferences I go to, is the best opportunity to meet all these people, I’ll do the same next year.\u003c/li\u003e\n\u003cli\u003eAgain I met some very helpful people from Oracle, people I like to talk to about everything, from Apocalypse to Zookeeper.\u003c/li\u003e\n\u003cli\u003eThese people reminds me that Oracle is more than a machine, and the software I work with every day is made by both smart and nice experts. Next year, join the biking on Saturday, or the running over the bridge on Sunday to see what I mean.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eOTN which is responsible for the ACE program does a lot of cool stuff like the IoT workshop, and much more. I hope the team do understand they have many friends around the world. Some of us plan to show this in near future, check this blog post by Tim Hall aka Oracle-Base: \u003ca href=\"https://oracle-base.com/blog/2016/09/28/otn-appreciation-day/\" target=\"_blank\" rel=\"noopener\"\u003ehttps://oracle-base.com/blog/2016/09/28/otn-appreciation-day/\u003c/a\u003e\n\u003c/p\u003e","title":"Oracle Open World 2016 Afterthoughts"},{"content":"I have been using SQL Developer since it required some goodwill to work. Some of the motivation came from an annoyance with Toad sessions in the database. Iremember that I back then googled \u0026ldquo;how to block Toad users from the database\u0026rdquo;. Anyway, these days most developers around me are using SQL Developer. But quite a few are learning only the least to get their job done. The following list is what I use often, and some of itresults in \u0026ldquo;How did you do that?\u0026rdquo; when they are looking over my shoulder. The shortcuts here refer to the Windows version since that is what I\u0026rsquo;m currently using at work. You can easily remap them from Preferences.\nUse templates from preferences to create shortcuts to frequently used SQL statements, or fraction of it. You\u0026rsquo;ll find it under Database - SQL Editor Code Templates\nSpeaking of preferences, there are lots of stuff to tweak, if you are looking for something special use the search field in the upper left corner\nTemplates are useful, so are snippets which can be anything from functions and hints to small scripts. Check out View -\u0026gt; Snippets, and when you see something cool, just drag it into the editor window.\nDrag the table from the list of tables in the Connections window to the worksheet, and it will offer to generate SQL for you (INSERT, SELECT, etc).\nCopy rows from Query Result with Shift-Ctrl-C to get the column headers. Paste it directly into Excel or in Outlook Email. In the latter you can use the \u0026ldquo;Convert to text\u0026rdquo; function to get a nice looking table in a few steps.\nAttach shortcuts to worksheets with Alt-Shift-number (1-9). Switch between them with Alt-number\nIf you have used one of those tools that requires you to mark the statement you want to execute, you don\u0026rsquo;t have to do this with SQL Dev; with the cursor somewhere on the statement just hit Ctrl-Enter.\nTo convert text to and from upper, lower, or initcap, mark the text and hit Ctrl-\u0026rsquo; (this one was not set on my Mac version, if that is your case, go to Preferences, search for shortcut, and in that window search for \u0026lsquo;other\u0026rsquo;). Repeating it will cycle through upper, lower, and initcap.\nHow to insert a list you receive into a table: Say someone sends you a list of 10 rows in an email, go to the Data tab for the table and add 10 rows. Copy the rows from the email and put the cursor on the first row in the data tab, and paste them in with Ctrl-V. SQL Dev will paste them in as you expect in Excel. This even works with two or more columns if the columns are separated with tabs. Using Subversion (or Git) from SQL Dev saves you a mental context switch compared to using an external client. By opening SQL scripts in SQL Developer, edit, save, and commit them (to the source control system) your work flows better.\nReformat code with Ctrl-F7. Makes it easier to read when you have copied and pasted generated code from somewhere.\nGet instant info about a table. Put the cursor on the table name in the worksheet and hit Shift-F4. A window pops up with lots of vital information, and it is quicker than to search for it in the hierarchy. Hit Ctrl-G to go to a specific line in your editor window. After you have become more productive and live your life in SQL Dev, it may need more memory. See this post for details: http://www.thatjeffsmith.com/archive/2014/06/configuring-memory-usage-in-oracle-sql-developer/ In short, uncomment the line in product.conf that starts with AddVMOption, and change the value from 800m (default) to say 1024m. Save query output for future reference by using the red pin in the Query result window. You can go back and see what SQL produced it by clicking the blue \u0026ldquo;SQL\u0026rdquo;.\nWhen you look at the indexes for a table in the Pop-up info box (Shift-F4), you can open the declaration for the index by right-clicking on a row for an index, and choose Go To\u0026hellip;\nRight-click on an editor tab and choose Float. It will move the worksheet tab to a separate window. Then you can click on other tabs and drag them to that window and group connections that belong together.\nChange the DATE-format in the Preferences, the default format yyyy.mm.dd is usually not enough for me, so I add HH24:MI:SS DBAs and others often work on many tasks at the same time, some scripts or SQL commands take long time to execute. Instead of checking back once in a while for its completion open the Task Progress from the View menu. It will show tasks that are running, completed or failed. Click the icon on the right side in order to remove finished tasks.\nSQL History (F8) is useful, even as backup of previous complex SQL that you have been working on, but didn\u0026rsquo;t save before the crash\u0026hellip;\nUse Connection Color when setting up a connection towards a production database. I use red and it will add a discrete line around the edge to remind me not to do something stupid. This is much better than having the entire background bleeding when doing serious business.\nHit F10 to get the execution plan for the statement where your cursor is.\nIf you need to get to your database through a jump server, see my blogpost about it.\nCode completion is very useful, but sometimes it doesn\u0026rsquo;t work. There is a limit to how much processing and parsing SQL Developer can do before your patience runs out. It helps to at least finish previous statement with a semi-colon or something that clearly marks the start of next statement, and write correct SQL.\nThere is a lot smartness built into the tool, sometimes a bit hidden; to not get in your way. If you think SQL Developer can\u0026rsquo;t do something, just try it out. So far I have never received any obscene error messages.\n","permalink":"https://enesi.no/2016/07/24-basic-tips-on-using-sql-developer/","summary":"\u003cp\u003eI have been using \u003ca href=\"http://www.oracle.com/technetwork/developer-tools/sql-developer/overview/index.html\" target=\"_blank\" rel=\"noopener\"\u003eSQL Developer\u003c/a\u003e\n since it required some goodwill to work. Some of the motivation came from an annoyance with Toad sessions in the database. Iremember that I back then googled \u0026ldquo;how to block Toad users from the database\u0026rdquo;.  \u003c/p\u003e\n\u003cp\u003eAnyway, these days most developers around me are using SQL Developer. But quite a few are learning only the least to get their job done. The following list is what I use often, and some of itresults in \u0026ldquo;How did you do that?\u0026rdquo; when they are looking over my shoulder. The shortcuts here refer to the Windows version since that is what I\u0026rsquo;m currently using at work. You can easily remap them from Preferences.\u003c/p\u003e","title":"24 Basic Tips on Using SQL Developer"},{"content":"This blog post from Oracle explains well how to connect to Hive from SQL Developer. In short, you need to download the JDBC driver from Cloudera and make sure you use the correct version, version 4 and not 4.1 Three zip files are included, use the one that starts with Cloudera_HiveJDBC4_2.*.\nAnyway, after adding the jar files to SQL Developer, and restarting it, the connection failed with:\nError setting/closing session: Open Session Error\nNot really informative for anyone used to ORA error codes.\nI tested the connection locally with the beeline command that is included with Hive. It worked OK when connecting with the URL jdbc:hive2://, but when I connected with URL jdbc:hive2://rio:10000 (rio is the hostname), it returned the error:\nhadoop is not allowed to impersonate anonymous I tried options -n and -p to connect as user hadoop, but the error message was similar:\nhadoop is not allowed to impersonate hadoop\nTurned out that I needed to change the configuration of Hadoop itself, and added the following to core-site.xml:\n\u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hadoop.proxyuser.hadoop.hosts\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;*\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hadoop.proxyuser.hadoop.groups\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;*\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; Yes, this is lax security, but OK in my lab. After a restart of hdfs etc, SQL Developer connected successfully to Hive. Proxy user in Hadoop is explained here .\nRunning simple queries from Hive, either from beeline or SQL Developer is slow, which makes you think twice before you submit a query, unless you are going for a cup of coffee. That is a good thing, I guess.\n","permalink":"https://enesi.no/2016/05/open-session-error-when-connecting-to-hive-from-sql-developer/","summary":"\u003cp\u003e\u003ca href=\"https://blogs.oracle.com/datawarehousing/entry/oracle_sql_developer_data_modeler\" target=\"_blank\" rel=\"noopener\"\u003eThis blog post\u003c/a\u003e\n from Oracle explains well how to connect to Hive from SQL Developer. In short, you need to download the JDBC driver from Cloudera and make sure you use the correct version, version 4 and not 4.1 Three zip files are included, use the one that starts with Cloudera_HiveJDBC4_2.*.\u003c/p\u003e\n\u003cp\u003eAnyway, after adding the jar files to SQL Developer, and restarting it,  the connection failed with:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eError setting/closing session: Open Session Error\u003c/p\u003e","title":"Open Session Error when connecting to Hive from SQL Developer"},{"content":"There are times when you as a DBA wished you were involved before someone designed a data model. After release to production correcting mistakes is usually complicated and with risks.\nUpdate 2016-04-20: Fixed errors in code.\nThere is an exception to this. Online table redefinition lets you change the structure of a table while everything is up and running. I have used it lately to partition large tables. During the process I also added compression, and moved some of the partitions to another tablespace that I have set read-only, in order to reduce backup time as well. LOBs in one table were moved to SecureFiles with deduplication.\nRedefining tables online is explained in the Administrator\u0026rsquo;s Guide, chapter 20 (11g and 12c). The PL/SQL package DBMS_REDEFINITION is used along SQL to create a new table with the desired structure.\nThe whole process can be quite simple, depending on how much you want to change, though it may require some time. Of course, in development, you can repeat the whole process in order to improve on the design in case you are not satisfied. In theory this can also be achieved in production, but I do recommend that you get tired of fixing this in development before you move on to production.\nThe example is made simple to get an overview, check the documentation for more details.\nI did the whole process as user SYSTEM, but it may be executed as the owner of the table provided execute privilege on the package has been granted along with CREATE TABLE and CREATE MVIEW.\nVerify that your table can be redefined this way: begin DBMS_REDEFINITION.CAN_REDEF_TABLE(\u0026#39;SCOTT\u0026#39;,\u0026#39;BIG_TABLE\u0026#39; ,dbms_redefinition.cons_use_rowid); end; / The last parameter is for tables without a primary key (PK) defined, the default value here is dbms_redefinition.cons_use_pk that applies to tables with a PK. In other words, if your table has a PK (as it should have), skip this last parameter here and in step 3. If no error is returned you are good to go.\nCreate an empty interim table with the structure you want. Spend some time here in order to avoid new issues. I used SQL Developer to generate the SQL for the original table and edited it to create the new table. This is rather simple if all you want is to introduce partitioning or similar that doesn\u0026rsquo;t change the columns and datatypes. Also note, remove all constraints on this interim table, they will be copied later. Start the process with:\nbegin DBMS_REDEFINITION.START_REDEF_TABLE (\u0026#39;SCOTT\u0026#39;,\u0026#39;BIG_TABLE\u0026#39; ,\u0026#39;BIG_TABLE_TMP\u0026#39;,null ,dbms_redefinition.cons_use_rowid); end; / The 4th parameter is null in this example, but is used when you want to reorder columns and need to use column mapping between the two tables. This statement starts the copying of rows from the original to the interim table, and will take some time depending on the amount of data. You may speed things up by enable parallel DML and query with:\nALTER SESSION FORCE PARALLEL DML PARALLEL 8; ALTER SESSION FORCE PARALLEL QUERY PARALLEL 8; Next step is to have Oracle copy all dependent objects of the table. That is, indexes, constraints, triggers, privileges, stats, and more.\ndeclare l_err pls_integer; begin DBMS_REDEFINITION.COPY_TABLE_DEPENDENTS(uname=\u0026gt;\u0026#39;SCOTT\u0026#39; ,orig_table=\u0026gt;\u0026#39;BIG_TABLE\u0026#39; , int_table=\u0026gt;\u0026#39;BIG_TABLE_TMP\u0026#39;,num_errors=\u0026gt;l_err); dbms_output.put_line(l_err); end; / This procedure actually has options to skip certain object types like indexes, or triggers, etc. Check the [documentation](http://docs.oracle.com/cd/E11882_01/appdev.112/e40758/d_redefi.htm#i999656) for details. Again, the amount of time this takes depends on the size of it all. But it is kind of cool that the Oracle database is working hard for you while you read Oracle blogs. If this procedure fails, it may be that you created the interim table (BIG_TABLE_TMP) with a NOT NULL constraint; easy to leave in there. Just remove the constraint and repeat this step; no need to start over from scratch. Next step is to finish it all with: begin DBMS_REDEFINITION.FINISH_REDEF_TABLE (\u0026#39;SCOTT\u0026#39; ,\u0026#39;BIG_TABLE\u0026#39; ,\u0026#39;BIG_TABLE_TMP\u0026#39;); end; / When this completes the dictionary has been updated and the table with name BIG_TABLE will have the structure you created in BIG_TABLE_TMP, and vice versa. You can now drop the BIG_TABLE_TMP table.\nAs usual with online operations there are some locks involved when the data dictionary is updated, so you may chose to finish the operation off peak time. Also in order to shorten the time spent in step 5 you may use DBMS_REDEFINITION.SYNC_INTERIM_TABLE between step 4 and 5 to refresh the interim table first. Lastly, if you want to abort the procedure between step 3 and 5, just execute the ABORT_REDEF_TABLE procedure.\nThis feature has been around for some time, and is very stable. If you have SQL not performing well because a table grew too fast, or you have some other structural problems, this feature is really a low-hanging fruit. The idea behind here is that you don\u0026rsquo;t touch the original table, but let Oracle do it for you when everything is ready (in step 5).\nAs always, keep it simple, life as a DBA has more than enough of excitements.\n","permalink":"https://enesi.no/2016/04/undo-old-mistakes-with-online-table-redefinition/","summary":"\u003cp\u003eThere are times when you as a DBA wished you were involved before someone designed a data model. After release to production correcting mistakes is usually complicated and with risks.\u003c/p\u003e\n\u003cp\u003eUpdate 2016-04-20: Fixed errors in code.\u003c/p\u003e\n\u003cp\u003eThere is an exception to this. Online table redefinition lets you change the structure of a table while everything is up and running. I have used it lately to partition large tables. During the process I also added compression, and moved some of the partitions to another tablespace that I have set read-only, in order to reduce backup time as well. LOBs in one table were moved to SecureFiles with deduplication.\u003c/p\u003e","title":"Undo old mistakes with online table redefinition"},{"content":"In version 4.0.3 SSH was introduced in SQL Developer; here\u0026rsquo;s a post from Jeff Smith about it. Now this feature has become even easier to use with SSH configuration made independent of the database connections. DBAs typically have a jump server they connect to in order to login further to the database servers. With the new implementation (I\u0026rsquo;m using version 4.1.3 at the moment) you setup one SSH host, and for every listener you want to connect to you configure a tunnel through the jump server.\nGo to View -\u0026gt; SSH to show the window for SSH configuration. Add a host with a right-click on \u0026ldquo;SSH Hosts\u0026rdquo;. To begin with it is sufficient to add just a descriptive name, a hostname or IP address, and your username. Then with a right-click on the host select \u0026ldquo;New Local Port Forward\u0026rdquo; as shown here:\nThe name should be the database name if you use dedicated listeners for every database, or a descriptive name of the listener if it is used by many. If you are using RAC, don\u0026rsquo;t use the hostname for the SCAN listener, but one of the vip-addresses. Also use the default \u0026ldquo;Automatically assign local port\u0026rdquo;, because there is some nice magic behind the scene here.\nWhere I work I typically have many connections to the same database, each with a different Oracle user. With the new SSH feature in SQL Developer it gets very easy to setup. For every new connection select Connection Type SSH and you pick from the Port Forward list as shown here:\n(By the way, did you notice I changed the Connection Color? One of my favourite features when dealing with production databases).\nThat\u0026rsquo;s it! The point here is that you reuse each port forward aka tunnel as often as you need, you no longer need to configure the same SSH tunnel for each database connection. And if the listener is moved you change it only once. Exactly what port is used locally is taken care of, which makes this much easier to setup and use than creating the tunnels in Putty or SSH command line since you don\u0026rsquo;t have to remember the local port.\n","permalink":"https://enesi.no/2016/02/sql-developer-and-ssh-for-dbas/","summary":"\u003cp\u003eIn version 4.0.3 SSH was introduced in SQL Developer; \u003ca href=\"http://www.thatjeffsmith.com/archive/2014/09/30-sql-developer-tips-in-30-days-day-17-using-ssh-tunnels/\" target=\"_blank\" rel=\"noopener\"\u003ehere\u0026rsquo;s a post\u003c/a\u003e\n from Jeff Smith about it. Now this feature has become even easier to use with SSH configuration made independent of the database connections. \u003c/p\u003e\n\u003cp\u003eDBAs typically have a jump server they connect to in order to login further to the database servers. With the new implementation (I\u0026rsquo;m using version 4.1.3 at the moment) you setup one SSH host, and for every listener you want to connect to you configure a tunnel through the jump server.\u003c/p\u003e","title":"SQL Developer and SSH for DBAs"},{"content":"In SQL Developer Data Modeler you have Libraries under the meny Tools -\u0026gt; Design Rules and Transformations. But when you select that one you may have seen this error message:\nNotice the second line with Jruby lib is red. This is because the JRuby is not installed (or cannot be found) on your machine.\nI found the solution to this in Heli\u0026rsquo;s book on SQL Developer Data Modeler, but thought I could add the easy way to solve this on your Mac. You need to download an extension to your JDK installation. A jar-file can be downloaded from http://jruby.org/download (There are two main versions, I decided to go for the the 1.X one). Look for the jar-file, currently I downloaded jruby-complete-1.7.23.jar.\nYou need to figure out where to put this jar file. In SQL Developer Data Modeler, check your java.library.path setting by going to Help -\u0026gt; About. Then click on the Properties tab. There are a lot of them, so in stead of scrolling down, start writing in the search field \u0026ldquo;java.l\u0026rdquo; as shown:\nIn my case, one of the directories in the search path is /Users/oisene/Library/Java/Extensions. It did not exist on my system, so I created it and copied the downloaded Jruby jar file into this directory. After a restart of SQL Developer Data Modeler you can go to Tools -\u0026gt; Design Rules and Transformations -\u0026gt; Libraries to verify that you no have the error message and the line with Jruby lib is black.\n","permalink":"https://enesi.no/2015/12/jruby-in-sql-developer-data-modeler-on-mac/","summary":"\u003cp\u003eIn SQL Developer Data Modeler you have Libraries under the meny Tools -\u0026gt; Design Rules and Transformations. But when you select that one you may have seen this error message:\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"/images/blogger/jruby-in-sql-developer-data-modeler-on-1.png\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/jruby-in-sql-developer-data-modeler-on-1.png\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eNotice the second line with Jruby lib is red. This is because the JRuby is not installed (or cannot be found) on your machine.\u003c/p\u003e\n\u003cp\u003eI found the solution to this in \u003ca href=\"http://www.amazon.com/Oracle-Developer-Modeler-Database-Mastery/dp/0071850090/ref=sr_1_1?ie=UTF8\u0026amp;qid=1451409899\u0026amp;sr=8-1\u0026amp;keywords=sql\u0026#43;developer\u0026#43;data\u0026#43;modeler\" target=\"_blank\" rel=\"noopener\"\u003eHeli\u0026rsquo;s book\u003c/a\u003e\n on SQL Developer Data Modeler, but thought I could add the easy way to solve this on your Mac. You need to download an extension to your JDK installation. A jar-file can be downloaded from \u003ca href=\"http://jruby.org/download\" target=\"_blank\" rel=\"noopener\"\u003ehttp://jruby.org/download\u003c/a\u003e\n (There are two main versions, I decided to go for the the 1.X one). Look for the jar-file, currently I downloaded jruby-complete-1.7.23.jar.\u003c/p\u003e","title":"JRuby in SQL Developer Data Modeler on Mac"},{"content":"This post is meant to be found by people googling the error message ORA-01105 and ORA-19808 on RAC. The following error messages may be seen after starting an instance:\nORA-01105: mount is incompatible with mounts by other instances ORA-19808: recovery destination parameter mismatch When this happened to me I ran this command on both instances:\nshow parameter recovery it showed there was a mismatch for the parameter db_recovery_file_dest_size. Probably I forgot to add \u0026ldquo;sid=*\u0026rdquo; to the alter system command one time when I increased the limit. Since one instance was up I could update the parameter for both instances with:\nalter system set db_recovery_file_dest_size=700G sid=\u0026#39;*\u0026#39;; The instance that was started must be shut down and started again.\nThis means that this parameter has to be the same across the instances during startup, but Oracle allows you to set them different with alter system.\nAs shown here you can correct parameters in spfile for an instance that won\u0026rsquo;t start from a running instance. This is a favourite feature, and whenever I have to change memory parameters that requires restart of the instance, I keep one instance running so I can correct the settings from there if needed.\n","permalink":"https://enesi.no/2015/11/ora-01105/","summary":"\u003cp\u003eThis post is meant to be found by people googling the error message ORA-01105 and ORA-19808 on RAC. The following error messages may be seen after starting an instance:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eORA-01105: mount is incompatible with mounts by other instances  \nORA-19808: recovery destination parameter mismatch  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eWhen this happened to me I ran this command on both instances:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eshow parameter recovery  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eit showed there was a mismatch for the parameter db_recovery_file_dest_size. Probably I forgot to add \u0026ldquo;sid=*\u0026rdquo; to the alter system command one time when I increased the limit. Since one instance was up I could update the parameter for both instances with:\u003c/p\u003e","title":"ORA-01105"},{"content":"A big conference like Oracle Open World is an excellent opportunity to learn more. But with all the books, blog posts, and on-demand learning on Internet, I really did not have to go to OOW this year to keep myself busy learning. Of course, I learned a lot, but for me, the biggest reward is the inspiration and ideas for future projects, as well as meeting friends in the community including Oracle employees who take part in it.\nRegarding inspiration I think that the attention given to SQL and PL/SQL is awesome. This is much more than the code itself. It is about the ability to do more and more complex analysis for each new version of the database. You can do it with minimal amount of code, code that executes very close to the data.\nIt is also about Application Express that is a quick and responsive web interface for your data; web pages generated in the database with the ability to manipulate and report on the data. I am a DBA that keeps telling other DBAs that if they can choose APEX for a project they should do so. Anything else will be more complex with more layers (even with Oracle REST Data Service as a proxy). It is built on technology the DBA understands and can control.\nIn the era of Big Data and Cloud computing I think the most fascinating stuff is data mining. It has been around for years, but thanks to an renewed focus, data mining algorithms are cool again. Oracle Advanced Analytics is more than complex algorithms. You need to explore the data, and the more you know SQL in general, and analytical functions in particular, the better you are prepared for such a project.\nI\u0026rsquo;m getting the impression that beside the Oracle database, preferably running on Exadata, I need nothing more than perhaps a couple of tiny applications servers in front, and network, of course.\n","permalink":"https://enesi.no/2015/11/my-take-away-from-oow15/","summary":"\u003cp\u003eA big conference like Oracle Open World is an excellent opportunity to learn more. But with all the books, blog posts, and on-demand learning on Internet, I really did not have to go to OOW this year to keep myself busy learning. Of course, I learned a lot, but for me, the biggest reward is the inspiration and ideas for future projects, as well as meeting friends in the community including Oracle employees who take part in it.\u003c/p\u003e","title":"My take away from OOW15"},{"content":"The biggest conference for Oracle users in the Nordics is not far away. Be there as a speaker and attendee. Sure you have a war story or another experience worth sharing. Last year OUGN received 250 abstracts, please, do it again!\nFollow this link to submit your abstracts:\nhttp://c4p.ougn.no/ Deadline is November 15.\nSee you on the ship (which is a boat with other boats on it, according to Millsap)!\n","permalink":"https://enesi.no/2015/10/ougn16-call-for-paper/","summary":"\u003cp\u003eThe biggest conference for Oracle users in the Nordics is not far away. Be there as a speaker and attendee. Sure you have a war story or another experience worth sharing. Last year OUGN received 250 abstracts, please, do it again!\u003c/p\u003e\n\u003cp\u003eFollow this link to submit your abstracts:\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"http://c4p.ougn.no/\" target=\"_blank\" rel=\"noopener\"\u003ehttp://c4p.ougn.no/\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDeadline is November 15\u003c/strong\u003e.\u003c/p\u003e\n\u003cp\u003eSee you on the ship (which is a boat with other boats on it, according to Millsap)!\u003c/p\u003e","title":"OUGN16 - Call for paper"},{"content":"I have my mental list of people in the Oracle community I look up to. They are people who help through social networks, emails, presentations, books, blog posts, or in a friendly conversation. Now, Oracle has recognised that there are many such individuals in the community, people that know a lot and spend much time sharing it with others.\nIn fact they have asked us to vote on our developer heroes, and they will award them during a big celebration at Oracle Open World. Sure you have met one SQL guru, or a PL/SQL programmer, or someone really concerned about database design? Since I am a DBA I think that we should support our database developers since they are doing their part to make sure that the database, and database application that we will support later is as good as possible.\nIt starts with a good design, then hopefully as much as possible of the hard work will be done in SQL or PL/SQL close to the data, and finally presented beautifully in a web application done in APEX, possibly delivered through Oracle REST Data Services to make it extra secure.\nWhat are the alternatives? A bunch of coders that starts out without any planning or previous knowledge of the database (because it is just a persistent storage they don\u0026rsquo;t want to relate to). They also want to do as much as possible in their own app far away, so they happily offload half the database to do so, and blame it on the network if it does not perform. The end result is a a mess that does not perform or scale, but you get to manage it from release until eternity.\nHere is what you need to give something back before it is too late:\nhttps://community.oracle.com/community/database/awards ","permalink":"https://enesi.no/2015/09/who-is-your-hero-in-the-oracle-community/","summary":"\u003cp\u003eI have my mental list of people in the Oracle community I look up to. They are people who help through social networks, emails, presentations, books, blog posts, or in a friendly conversation. Now, Oracle has recognised that there are many such individuals in the community, people that know a lot and spend much time sharing it with others.\u003c/p\u003e\n\u003cp\u003eIn fact they have asked us to vote on our developer heroes, and they will award them during a big celebration at Oracle Open World. Sure you have met one SQL guru, or a PL/SQL programmer, or someone really concerned about database design? Since I am a DBA I think that we should support our database developers since they are doing their part to make sure that the database, and database application that we will support later is as good as possible.\u003c/p\u003e","title":"Who is your hero in the Oracle community?"},{"content":"In case you do not want to grant a user access to data dictionary tables like DBA_TABLES, but will let user B see the list of all tables belonging to user A, you can work around it with a pipelined function in schema A:\ncreate type str_set as table of varchar2(30); / create or replace function a_tables return str_set pipelined is l_str varchar2(30); begin for l_str in (select table_name from user_tables) loop pipe row(l_str.table_name); end loop; return; end; / grant execute on a_tables to B; Then user B can see the list of A\u0026rsquo;s table with:\nselect * from table(a.a_tables); Trying to solve this with a view in schema A that selects on USER_TABLES does not work, but prove me wrong, please.\n","permalink":"https://enesi.no/2015/09/function-to-let-user-b-see-all-tables-of-user-a/","summary":"\u003cp\u003eIn case you do not want to grant a user access to data dictionary tables like DBA_TABLES, but will let user B see the list of all tables belonging to user A, you can work around it with a pipelined function in schema A:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003ecreate type str_set as table of varchar2(30);  \n/  \n  \ncreate or replace function a_tables return str_set pipelined is  \nl_str varchar2(30);  \nbegin  \nfor l_str in (select table_name from user_tables)  \nloop  \npipe row(l_str.table_name);  \nend loop;  \nreturn;  \nend;  \n/  \ngrant execute on a_tables to B;  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eThen user B can see the list of A\u0026rsquo;s table with:\u003c/p\u003e","title":"Function to let user B see all tables of user A"},{"content":"When you run dbms_stats.gather_table_stats on tables with spatial indexes or dbms_stats.gather_index_stats directly on the spatial index the routine may return without an error even if no statistics gathering took place. You can verify this by looking at the LAST_ANALYZED column in DBA_INDEXES:\nselect owner,index_name,last_analyzed from dba_indexes where index_type=\u0026#39;DOMAIN\u0026#39;; Another way to verify if statistics gathering for these indexes took place is to look for tables with names that start with MDXT:\nselect owner,table_name from dba_tables where table_name like \u0026#39;MDXT_%\u0026#39;; The user that owns these indexes must be granted CREATE TABLE directly and not through a role. You can find some details about these tables in Doc ID 1610877.1 on My Oracle Support. In case you have many tables that starts with MDXT you can see what are actually needed by running this:\nselect sdo_index_owner, index_name, sdo_index_table from mdsys.all_sdo_index_info; MDXT-tables whose number does not appear among the tables listed in sdo_index_table (name starts with MDRT) can be dropped according to the note at MOS.\n","permalink":"https://enesi.no/2015/07/gather-statistics-on-spatial-index/","summary":"\u003cp\u003eWhen you run dbms_stats.gather_table_stats on tables with spatial indexes or dbms_stats.gather_index_stats directly on the spatial index the routine may return without an error even if no statistics gathering took place. You can verify this by looking at the LAST_ANALYZED column in DBA_INDEXES:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eselect owner,index_name,last_analyzed  \nfrom dba_indexes   \nwhere index_type=\u0026#39;DOMAIN\u0026#39;;  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eAnother way to verify if statistics gathering for these indexes took place is to look for tables with names that start with MDXT:\u003c/p\u003e","title":"Gather statistics on spatial index"},{"content":"When you need to find the OS-process on the database server for an oracle session (dedicated server) you can join v$session with v$process:\nselect p.spid from v$session s join v$process p on(s.paddr=p.addr) where s.sid=42; But if you kill a session with \u0026lsquo;alter system kill session \u0026hellip;\u0026rsquo; the link between these views are broken because the value in v$session.addr changes. In order to look for these orphan processes run this query:\nselect spid, program from v$process where program!= \u0026#39;PSEUDO\u0026#39; and addr not in (select paddr from v$session) and addr not in (select paddr from v$bgprocess) and addr not in (select paddr from v$shared_server); You may check with OS tools like ps on Linux to see that these are indeed dead processes or with strace to see what they are doing and eventually kill them.\n","permalink":"https://enesi.no/2015/07/orphan-processes-in-the-database/","summary":"\u003cp\u003eWhen you need to find the OS-process on the database server for an oracle session (dedicated server) you can join v$session with v$process:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eselect p.spid  \nfrom v$session s join v$process p on(s.paddr=p.addr)  \nwhere s.sid=42;  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eBut if you kill a session with \u0026lsquo;alter system kill session \u0026hellip;\u0026rsquo;  the link between these views are broken because the value in v$session.addr changes. In order to look for these orphan processes run this query:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eselect spid, program from v$process   \n    where program!= \u0026#39;PSEUDO\u0026#39;   \n    and addr not in (select paddr from v$session)  \n    and addr not in (select paddr from v$bgprocess)  \n    and addr not in (select paddr from v$shared_server);  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eYou may check with OS tools like ps on Linux to see that these are indeed dead processes or with strace to see what they are doing and eventually kill them.\u003c/p\u003e","title":"Orphan Processes in the database"},{"content":"When I had my first Oracle DBA course some years before the start of this millennium I made a special mental note about a very common error message, ORA-00942, \u0026ldquo;table or view does not exist\u0026rdquo;.\nIn most programming languages there are a similar message for a common situation; when you refer to an object that does not exist, quite often because you wrote the name wrong. What I found a bit remarkable when I heard about it first time was that ORA-942 may mean two things, either that the table or view does indeed not exist, or you do not have access to it. An example of the latter is when some user has created a table in her schema, and you try to query it, but you have not been granted a privilege to do so. Instead of you receiving some error message like \u0026ldquo;access denied\u0026rdquo;, Oracle responds with the same error message; \u0026ldquo;table or view does not exist\u0026rdquo;, even if it does exist. It may have added \u0026ldquo;for you\u0026rdquo; to make it more correct. I think there is a good reason why this is so, and I think it has to do with optimisation.\nIf you enable sql trace, you can see Oracle runs queries against obj$ and objauth$.\nselect obj#,type#,ctime,mtime,stime, status, dataobj#, flags, oid$, spare1, spare2 from obj$ where owner#=:1 and name=:2 and namespace=:3 and remoteowner is null and linkname is null and subname is null select grantee#,privilege#,nvl(col#,0),max(mod(nvl(option$,0),2)) from objauth$ where obj#=:1 group by grantee#,privilege#,nvl(col#,0) order by grantee# The first checks for the object\u0026rsquo;s existence, the second looks for privileges granted on the object. By the way, the results from these queries may be cached so you won\u0026rsquo;t always see these in the trace file. Exactly what is going on behind the scene here is unknown to me, and I also think that the implementation of this has changed during the years with different versions of the database. But if you look at the second query, that fact that no row is returned may come from either the fact that no privileges have been given, or that the table does not exist. So, instead of going an extra round in the dictionary to see if the table exists since no privileges exist on it, Oracle simply returns the same error message \u0026ldquo;table or view does not exist\u0026rdquo;.\nIt is probably a good security practise to not inform unprivileged users about the existence of objects they have no access to, but I thought it was a smart optimisation by Oracle back then, because I was used to see error messages like \u0026ldquo;access denied\u0026rdquo;. The way they have done this saves extra work. By know I think I have lost the attention of everyone except my best oracle nerd friends, so I thought I could sneak in a confession here at the end. Some years ago I started to practise ORA-942 in my own life. You know in those situations where people suspect you know some secret and start asking away about stuff you are not allowed to pass on? Instead of me having the burden of creating excuses or explaining why I can\u0026rsquo;t tell, I simply answer \u0026ldquo;I don\u0026rsquo;t know\u0026rdquo;. That saves me a lot of extra work and I don\u0026rsquo;t consider it a worse practice than mining other people\u0026rsquo;s brain.\nHave a nice weekend!\n","permalink":"https://enesi.no/2015/05/ora-00942/","summary":"\u003cp\u003eWhen I had my first Oracle DBA course some years before the start of this millennium I made a special mental note about a very common error message, ORA-00942, \u0026ldquo;table or view does not exist\u0026rdquo;.\u003c/p\u003e\n\u003cp\u003eIn most programming languages there are a similar message for a common situation; when you refer to an object that does not exist, quite often because you wrote the name wrong. What I found a bit remarkable when I heard about it first time was that ORA-942 may mean two things, either that the table or view does indeed not exist, or you do not have access to it. An example of the latter is when some user has created a table in her schema, and you try to query it, but you have not been granted a privilege to do so. Instead of you receiving some error message like \u0026ldquo;access denied\u0026rdquo;, Oracle responds with the same error message; \u0026ldquo;table or view does not exist\u0026rdquo;, even if it does exist. It may have added \u0026ldquo;for you\u0026rdquo; to make it more correct. I think there is a good reason why this is so, and I think it has to do with optimisation.\u003c/p\u003e","title":"ORA-00942"},{"content":"Should be obvious, but I just discovered that it is much easier to change preferences if you search for it in the search field instead of wandering around in the Preferences GUI looking for the place where you can set it:\nAnd it is pretty fast, try to look for the place where you choose the OCI client, just write OCI in the field. And similarly, if you think you wrote some smart code a few days ago, just search for it in the SQL history:\nIf you can\u0026rsquo;t find it, it is because you have forgotten to increase Days to keep history from 7 which seems to be default. Go back to preferences and crank it up. 30 days, why not?\n","permalink":"https://enesi.no/2015/03/search-and-history-in-sql-developer/","summary":"\u003cp\u003eShould be obvious, but I just discovered that it is much easier to change preferences if you search for it in the search field instead of wandering around in the Preferences GUI looking for the place where you can set it:\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"/images/blogger/search-and-history-in-sql-developer-1.png\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/search-and-history-in-sql-developer-1.png\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eAnd it is pretty fast, try to look for the place where you choose the OCI client, just write OCI in the field. And similarly, if you think you wrote some smart code a few days ago, just search for it in the SQL history:\u003c/p\u003e","title":"Search and history in SQL Developer"},{"content":"Here the other day I came across this package in a PL/SQL procedure written by someone else. From the name I reckoned it was a standard package from Oracle, but I had never seen it before. It is not mentioned in the manual Database PL/SQL Packages and Types Reference, and I could not find much about it at My Oracle Support either. Anyway, with SQL Developer you\u0026rsquo;ll get what you need by hitting Shift-F4 (with the cursor at the name of the package). The API is pretty good documented in the comments. The package is used to rebuild indexes, either for a named table, a named schema, or a list of indexes plus some stuff I didn\u0026rsquo;t bother to look into.\nNow, there may be a reason why this is not well documented and better known.\n","permalink":"https://enesi.no/2015/03/dbms_index_utl/","summary":"\u003cp\u003eHere the other day I came across this package in a PL/SQL procedure written by someone else. From the name I reckoned it was a standard package from Oracle, but I had never seen it before. It is not mentioned in the manual \u003cem\u003eDatabase PL/SQL Packages and Types Reference\u003c/em\u003e, and I could not find much about it at \u003cem\u003eMy Oracle Support\u003c/em\u003e either. Anyway, with SQL Developer you\u0026rsquo;ll get what you need by hitting Shift-F4 (with the cursor at the name of the package). The API is pretty good documented in the comments. The package is used to rebuild indexes, either for a named table, a named schema, or a list of indexes plus some stuff I didn\u0026rsquo;t bother to look into.\u003c/p\u003e","title":"DBMS_INDEX_UTL"},{"content":"At UKOUG Tech conference in Liverpool last year I had an interesting talk with a DBA that had two years of experience. Being 23 years the DBA naturally stood out from the crowd. I talked about my thoughts on what is important for a DBA, and since I spend way too much on user group activities and conferences I argued that going to such events are useful for a DBA, and continued with other ramblings about what I think is important for a DBA to do. I hope I stopped soon enough, but stop talking and start listening is an art. Anyway, I decided to write down a list of recommendations for a new DBA. This is what I think is important.\nBackup and recovery stands out. The database is your responsibility, that is why you are hired as a DBA. In the database, which in earlier times also was called a databank, the data is the money. If you can\u0026rsquo;t get the database back after a disaster they will blame you and no other. In fact if you can imagine one reason why you in some situation will not be able to recover the database you should mail it to your manager now. The concept of backup and recovery should be easy to grasp, but realities are that many different scenarios exist and how to deal with them depends on the situation. There are plenty of stuff written on this from masters of Recovery Manager (RMAN) to people more dedicated to risk management. This subject is more than just getting the database back running, it is also about keeping the system available, guard against data loss and testing systems for faults.\nHave a fire drill once in a while to test your routines and skills. Get some clear policies in place like, how much data are you allowed to lose, and how much time (downtime) do you have to recover your database. The costs and implementation will depend on this. If you cannot lose any data and management starts talking about 99.999% uptime a simple backup is not enough. \u0026ldquo;How much data can you lose?\u0026rdquo; Is a phrase that can freak people out, but if you can\u0026rsquo;t lose data, is your system and routines really prepared for that?\nTuning and Optimisation is by many thought of as your problem and your problem only. Reality is that when a system is not performing it is usually not the DBA\u0026rsquo;s fault. Obviously, a badly configured database will not work well, but the Oracle database performs remarkably well with default configurations. Your database depends on the server it is running on, the network between the database and connected clients, and so on. But most frequently you will discover this: the reason for bad performance is bad design.\nDesign as in \u0026ldquo;invented by a developer who is a single user on his own PC with lots of CPU and memory, but little data and load to reveal bad algorithms\u0026rdquo;. Tuning is a word looked down on by some in the community since it came from an era where quick fixes and esoteric queries would reveal problems in the database; with the right secret revealed to you, you could do some magic and have everything run perfectly. Still some people are tempted by black magic, so be critical to what Google brings you.\nOptimisation on the other hand is taking what you have, see where the time is wasted, and take away as much as you can (or want) by replacing one or a few parts of the job by something that runs better. Often a redesign is necessary, or if you are lucky, fiddling with an index may do the job.\nMy best advice is to find a way to identify where time is wasted and zooming in on that. This means profiling in some way and work from there. By having some methods ready to detect waste or a time sink, you can stop spending time improving on what is already working reasonably well. In other words, stop the parameter tweaking (nobody changes the db_block_size these days), take one step back and ask, what is going on here. What you learned in computer science at the university may come very useful here. To put your mind on the right track I cannot think of a better place to start than to read through the papers at Method R, especially Thinking Clearly About Performance .\nGet involved in a user group and go to conferences, meetups or whatever is easy for you. Many companies have many developers or IT-workers, but only one or two DBAs. In the community there are people who love to talk about their work, they will share experience with you, and tell you what is not in the manuals. If your manager won\u0026rsquo;t send you to a conference, here are some arguments you may try:\nOne course covers one subject, but a conference covers many, during one intensive conference you can get inputs and learning in many fields. Problems you may have at work can be discussed by others at the conference. Chances are that what you are going through is interesting and valuable input to others and you are likely not the first one to go through this. Make friends and discover that you are not alone. Getting friends and contacts outside your pool is possibly the best part of it. Meeting people IRL and being in touch on many channels is priceless to me. Many conferences are run by the community, meaning that a demand for a profit margin is lower at conferences than at a course delivered by a company. It translates to more learning for the money than at most other places. Find a social network that works for you and become visible. Twitter works well for me, I meet likeminded people there and discover other things we have in common besides Oracle tech. Most of my friends I have now come from the Twitter community.\nTake notes and start a blog Starting as a DBA today must be different from how it was 10-20 years ago. As you go, take notes and post it on your own blog. Even mistakes; DBAs are busy, if other can learn from your mistakes it is a valuable contribution. The problems you have now may be more relevant for another junior DBA than a beginner\u0026rsquo;s post written many years ago. And, you will be surprised how useful your procedures will be later when you have to go back and perform a task you haven\u0026rsquo;t done for a while.\nIf you are a senior DBA reading this, maybe you should blog about what you learned and want to pass on? I think many have had an experience that stood out, something that put their mind on the right track. Blog about it!\n","permalink":"https://enesi.no/2015/01/to-a-junior-dba/","summary":"\u003cp\u003eAt UKOUG Tech conference in Liverpool last year I had an interesting talk with a DBA that had two years of experience. Being 23 years the DBA naturally stood out from the crowd. I talked about my thoughts on what is important for a DBA, and since I spend way too much on user group activities and conferences I argued that going to such events are useful for a DBA, and continued with other ramblings about what I think is important for a DBA to do. I hope I stopped soon enough, but stop talking and start listening is an art. Anyway, I decided to write down a list of recommendations for a new DBA. This is what I think is important.\u003c/p\u003e","title":"To a junior DBA"},{"content":"You know, this is my blog, but I am aware of my role in our user group. The opinions here are mine; I haven\u0026rsquo;t stolen them. Selecting abstracts for our conference is done by many in our board, but I wanted to clarify how I look at the task, what speakers I look for to our conference.\nThursday this week we sent out emails to 71 people who will present on our next conference that starts March 12. Another group received a more disappointing email. Some have questioned why they did not make it (again).\nAs a user group I think we have some community duties. These came to my mind now:\nBe there for our members, independent from any vendor, but still cooperating with Oracle and others. Enlighten and show new stuff to our members. Help them finding solutions to problems they may have at work - share experiences. Networking; remind DBAs and other professionals that may be alone in their day job that they are not alone in the community. Also network related: There are a lot of experts inside and outside Oracle worldwide that believe in what they are doing, and are willing to help. The community needs to know about them. Assist in recruiting new people into our fields of technology; more members and new colleagues. Help new speakers into giving presentations, develop the community with even more good speakers. This post is mostly about that last bullet point.\nI have my heroes among the speakers. First I discovered Tom Kyte, and when he started blogging I discovered the Oaktable Network . From that group I can mention Cary Millsap, Jonathan Lewis, Christian Antognini, Kellyn Pot\u0026rsquo;Vin-Gorman and Douglas Burns, to name a few. They always attract a lot of people. And I really enjoy the company of these, really smart, yet levels with you every time. The community is even more than the Oaktable, there are plenty of people who just love what they are doing without carrying any reward for it.\nMany of these speakers have been rewarded for their community contributions through the Oracle Ace Program . OUGN could never have staged such a good conference without the support from Oracle through the ACE program, we have several ACEs every year and the ACE program covers a substantial amount of expenses. So if someone from OTN is reading this: Thank you!\nIn this program the best (according to some metric) are rewarded with status ACE Director (ACED). Before they got that far some started as ACE, this year the new entry level is ACE Associate. Now, according to someone\u0026rsquo;s logic we should go through the list of submitted abstracts and pick all the ACEDs first, right?\nNot according to mine. I think we have a duty in helping new speakers to the stage. One of the most uplifting experience I have in the community is to discover (new) people that spend time on becoming good at what they are doing through hard work, lots of lab testing, and blogging. The presentations that gave me most came from people who really care about what they talk about (call it passion, if you insist), not from a very slick and polished presentation.\nThis is one way we in OUGN can give back to the ACE program and the rest of the community: help the not so famous or people who aspire to become an ACE to get better at presenting their stuff. I like to think that some of those new ACEs became an ACE because they presented at our conference. One of our Norwegian members came to me after a conference was finished and said one certain presentation made the whole conference for him. It was delivered by one I invited to speak, he was not an ACE then, but is now. It is a lot of work to become an ACE, more then one conference experience, of course, but still I think we can contribute to it.\nIn addition to this group of speakers who I personally care much about, here is a list of others I look for:\nProduct managers and others in Oracle who shares and cares about the community. PMs for SQL Developer, APEX, among others, are the best ambassadors for Oracle. People who are in the front with new trends and technologies. Famous people who attract attendees, because we need them. Really hard-core nerds that may not appeal to many, but makes the conference for the ones that go to the deep-dive session. So, if your abstract did not make it, it means among other things:\nWe were blessed with lots of submitted abstracts (over 250 this year, not counting people who said they was going to submit, but forgot about it.) We have to choose from different groups of speakers. Something else happened. (This is the miscellaneous bucket including faults on our side.) If you made it, it means that we literally want you on board and that you will succeed. You may have the best experience after your presentation from introverts who want to know more about what you are doing. And the last night while sailing back to Oslo and you will have no presentation to worry about, you will understand why quite a few want to come back to the OUGN Spring Seminar, aka The Boat Conference :-)\nBy the way, the worst thing about abstract selections is not to say no to the pros (ACEDs), they can handle it, I assume, but there are some less experienced people that deserves to be speaking, but we couldn\u0026rsquo;t find slots for them.\nCheers to everybody.\n","permalink":"https://enesi.no/2014/12/why-was-my-abstract-not-accepted/","summary":"\u003cp\u003eYou know, this is my blog, but I am aware of my role in our user group. The opinions here are mine; I haven\u0026rsquo;t stolen them. Selecting abstracts for our conference is done by many in our board, but I wanted to clarify how I look at the task, what speakers I look for to our conference.\u003c/p\u003e\n\u003cp\u003eThursday this week we sent out emails to 71 people who will present on our \u003ca href=\"http://ougn.no/?page_id=273\" target=\"_blank\" rel=\"noopener\"\u003enext conference\u003c/a\u003e\n that starts March 12. Another group received a more disappointing email. Some have questioned why they did not make it (again).\u003c/p\u003e","title":"Why was my abstract (not) accepted?"},{"content":"How long is an email address? 30 characters should be enough, I thought. And I was right for about three weeks or so until one of the smartest Oracle experts in Europe tried to submit an abstract at our call for paper site . His email address had 32 chars and therefore failed already at registration. A bit embarrassing.\nThe data model was created in SQL Developer Data Modeler (SDDM) and I had done something right. In the data model EMAIL was defined as a domain and not simply as VARCHAR2(30) in different places in the data model.\nWhat I needed to do was to update the properties of this domain. Before you change a domain you may want to know where the domain has been used so you get an idea of the consequences of the change. Just go to the Browser and expand the stuff under Domains, and by right clicking on the domain you select Properties and in the window that pops up you can see where the domain has been used by clicking on Used In:\nYou can change the definition of the domain in one place, of course, under Tools -\u0026gt; Domain Administration:\nSimply select the domain, you have to click on the Modify button before you can update the Domain Properties on the right. The fix in the schema was to perform a few ALTER TABLE statements on the involved tables followed by some statements to compile objects that became INVALID afterwards.\n","permalink":"https://enesi.no/2014/11/use-domains-in-sql-developer-data-modeler/","summary":"\u003cp\u003eHow long is an email address? 30 characters should be enough, I thought. And I was right for about three weeks or so until one of the smartest Oracle experts in Europe tried to submit an abstract at our \u003ca href=\"http://c4p.ougn.no/apex/f?p=102\" target=\"_blank\" rel=\"noopener\"\u003ecall for paper site\u003c/a\u003e\n. His email address had 32 chars and therefore failed already at registration. A bit embarrassing.\u003c/p\u003e\n\u003cp\u003eThe data model was created in SQL Developer Data Modeler (SDDM) and I had done something right. In the data model EMAIL was defined as a domain and not simply as VARCHAR2(30) in different places in the data model.\u003c/p\u003e","title":"Use Domains in SQL Developer Data Modeler"},{"content":"Of course there is nothing special about restoring a database from the cloud on your own down-on-earth server, but it helps making this post cool and cloudy.\nWhen I deployed XE to an AWS instance I made a small script to backup the database once pr day (scheduled in crontab), and another to fetch the backupset to my server here on earth. Backup script looks like this:\n#!/bin/sh ORACLE_SID=XE ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe PATH=$ORACLE_HOME/bin:$PATH export ORACLE_SID ORACLE_HOME PATH rman target / \u0026lt;\u0026lt; EOF \u0026gt; $HOME/script/rman.$$ backup database plus archivelog delete input; delete noprompt obsolete; list backup of database summary; EOF By default the XE is installed without archiving so you need to change that when you install your production XE database. I also turned on auto backup of the controlfile. The RMAN configuration for the XE database looks like this:\nCONFIGURE RETENTION POLICY TO REDUNDANCY 2; CONFIGURE BACKUP OPTIMIZATION ON; CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default CONFIGURE CONTROLFILE AUTOBACKUP ON; CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO \u0026#39;%F\u0026#39;; # default CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default CONFIGURE MAXSETSIZE TO UNLIMITED; # default CONFIGURE ENCRYPTION FOR DATABASE OFF; # default CONFIGURE ENCRYPTION ALGORITHM \u0026#39;AES128\u0026#39;; # default CONFIGURE COMPRESSION ALGORITHM \u0026#39;BASIC\u0026#39; AS OF RELEASE \u0026#39;DEFAULT\u0026#39; OPTIMIZE FOR LOAD TRUE ; # default CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default CONFIGURE SNAPSHOT CONTROLFILE NAME TO \u0026#39;/u01/app/oracle/product/11.2.0/xe/dbs/snapcf_XE.f\u0026#39;; # default It is mostly default values.\nA script that runs on my own box simply does a recursive secure copy (scp):\nscp -r -i ssh_key.pem ec2-user@42.42.42.42:/u01/app/oracle/fast_recovery_area/XE/backupset/\\`date +%Y_%m_%d\\` /home/alone/RMAN/backupset scp -r -i ssh_key.pem ec2-user@42.42.42.42:/u01/app/oracle/fast_recovery_area/XE/backupset/\\`date +%Y_%m_%d\\` /home/alone/RMAN/autobackup This fetches today\u0026rsquo;s backup and stores it locally.\nI installed XE on my own box and executed the following in RMAN to restore the database. (If the instance is up, take it down with shutdown abort first):\nrman target / startup nomount restore controlfile from \u0026#39;/home/alone/RMAN/autobackup/2014_10_21/o1_mf_s_861554053_b4dvq5pl_.bkp\u0026#39;; alter database mount; catalog start with \u0026#39;/home/alone/RMAN/autbackup/2014_10_21\u0026#39;; restore database; recover database; alter database open resetlogs; That\u0026rsquo;s it.\nBut don\u0026rsquo;t trust me on this procedure without trying it yourself. Remember a DBA can make mistakes, except when it comes to backup AND recovery.\n","permalink":"https://enesi.no/2014/10/restore-xe-database-from-an-aws-instance/","summary":"\u003cp\u003eOf course there is nothing special about restoring a database from the cloud on your own down-on-earth server, but it helps making this post cool and cloudy.\u003c/p\u003e\n\u003cp\u003eWhen I \u003ca href=\"https://enesi.no/2014/10/oracle-database-xe-and-apex-on-amazon-web-services/\"\u003edeployed XE to an AWS instance\u003c/a\u003e\n I made a small script to backup the database once pr day (scheduled in crontab), and another to fetch the backupset to my server here on earth. Backup script looks like this:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e#!/bin/sh  \n  \nORACLE_SID=XE  \nORACLE_HOME=/u01/app/oracle/product/11.2.0/xe  \nPATH=$ORACLE_HOME/bin:$PATH  \nexport ORACLE_SID ORACLE_HOME PATH  \n  \nrman target / \u0026lt;\u0026lt; EOF \u0026gt; $HOME/script/rman.$$  \nbackup database plus archivelog delete input;  \ndelete noprompt obsolete;  \nlist backup of database summary;  \nEOF  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eBy default the XE is installed without archiving so you need to change that when you install your production XE database. I also turned on auto backup of the controlfile. The RMAN configuration for the XE database looks like this:\u003c/p\u003e","title":"Restore XE database from an AWS instance"},{"content":"One of the cool things with Oracle VM is that Oracle offers many templates you can download from edelivery.oracle.com and import into your repository with Oracle VM Manager (OVMM). Well, since I did the same mistake that you did, that is, not reading the entire README file, I confused templates with assemblies.\nSay you want to download a template for Oracle Linux 6, you have to select product pack Oracle VM Templates. The downloaded zip contains one file with extension .ova (does not mean Oracle Virtual Assembly, but Open Virtualization Format Archive). Since it is written \u0026ldquo;VM Templates\u0026rdquo; several places on the web site, it is easy to think it should be imported under VM Templates, but you need to select Assemblies as shown here and do the import from there:\nTrying to import the file with extension .ova under VM Templates will fail and throw an error message at you similar to:\nNo VM configure file found\nThis is obvious if you pay attention to section 2 in the README file, that states:\n2. Creating an Oracle VM Virtual Machine from the Template\n======================================================\nThe OVF based Oracle VM Templates should be imported as Assemblies.\nPlease review the Assemblies section of Oracle VM User\u0026rsquo;s Guide for details\nabout importing an assembly, and create a virtual machine template from the assembly. Assemblies are explained in 7.5.4 in the User Guide (for version 3.2 at least). You can create one or many VM Templates from an assembly and from there your new and shiny VMs.\nBy the way, one easy way to import them is to use a local HTTP server that comes with Python.\n","permalink":"https://enesi.no/2014/10/oracle-vm-templates-and-assemblies/","summary":"\u003cp\u003eOne of the cool things with Oracle VM is that Oracle offers many templates you can download from edelivery.oracle.com and import into your repository with Oracle VM Manager (OVMM). Well, since I did the same mistake that you did, that is, not reading the entire README file, I confused \u003cem\u003etemplates\u003c/em\u003e with \u003cem\u003eassemblies\u003c/em\u003e.\u003c/p\u003e\n\u003cp\u003eSay you want to download a template for Oracle Linux 6, you have to select product pack Oracle VM Templates. The downloaded zip contains one file with extension .ova (does not mean Oracle Virtual Assembly, but Open Virtualization Format Archive). Since it is written \u0026ldquo;VM Templates\u0026rdquo; several places on the web site, it is easy to think it should be imported under VM Templates, but you need to select Assemblies as shown here and do the import from there:\u003c/p\u003e","title":"Oracle VM: Templates and Assemblies"},{"content":"In our user group OUGN we used to collect abstracts for our annual conference with a simple Google Form.\nAdvantage:\nEasy to set up Disadvantages\nNo possibility for users to reconnect and correct what they had already submitted Lots of manual work Difficult to work with in the process later (selection, agenda, late submissions, special guests) Not really cool or cloudy Google Forms is really just a shared spreadsheet with a form interface wrapped around it. And like any other spread sheet it is easy to work with in the beginning, but as the number of people involved increases and data grows it soon gets cumbersome to work with.\nThis year we moved it to Oracle Application Express (APEX). We should have done that long time ago, especially since we are an Oracle User Group\u0026hellip;\nI figured out that the Express Edition (XE) would be sufficient, even 300 abstracts with speaker info won\u0026rsquo;t take all of the 11GB of space available. Next issue was to figure out where to host it. I looked around and decided that a micro-instance at Amazon Web Services (AWS) should work. It is actually located in Ireland; Hi, Brendan !\nThe rest of this post is about experiences I made with running APEX in XE on AWS.\nOS One of the default Red Hat Linux 6.5 images from AWS works well. You can then download the rpm-file for XE from OTN . The rpm-file is quite complete, no need to install a lot of extra packages or install the oracle-rdbms-server-11gR2-preinstall package.\nSwap space During installation of XE the installer may complain about not enough or missing swap space. The default installation that I used did not come with much swap space, and since you don\u0026rsquo;t want your database server to start swapping you shouldn\u0026rsquo;t need any either. But, I like to get rid of warning messages so I created temporary swap space like this to stop the complaints from runInstaller:\ndd if=/dev/zero of=/tmp/swap bs=1024 count=1300000 mkswap /tmp/swap swapon /tmp/swap Upgrade APEX in db The version of APEX that comes with XE 11g is old, but it is easy to upgrade. Download a new version from OTN . Start with 3.5.2 in the Installation Guide Installing full version of APEX Though strictly not necessary I find it easier to have the full version of APEX installed, not only the run-time version. Running the script apexins.sql took 3.5 hours on this micro-instance, so plan ahead for some other work to do during installation.\nURL not found If you start getting messages like The requested URL /apex/f was not found on this server then check the alert log, you may find ORA-04031 errors there, see next bullet.\nShared pool size too small The XE instance is configured during installation with a small shared pool. Try to increase it in small steps (to avoid swapping), our instance is now running with 250MB shared pool. We\u0026rsquo;ll see how it works out as people connects to it. You may well reduce the java pool to a minimum too; set it to 0 and Oracle will allocate 1 granule for it, which is 4MB on this instance. As always, make a pfile copy of the spfile before you do any changes to the spfile that requires a restart of the instance. Certain combinations of memory parameters are not supported and I had to revert the changes a couple of times to get the instance up.\nDeploying new versions in APEX I developed the application on a local database and for each round of fixes I exported the application to a file and then uploaded it to the AWS instance with build status Run Application Only. With a good separation of data and application it was easy to deploy new versions as feedback from the user group board members came in. I also learned that in order to deploy a new version you have to change the build status to Run and build application before you can delete or replace it. That is obvious when you\u0026rsquo;ve done it once\u0026hellip;\nDisable access to workspace login With the full installation installed the database will by default allow you to login to the APEX workspace, e.g. c4p.ougn.no/apex. This is typically not something you want in this case, but can be turned off easily with the following executed as SYS or other privileged user:\nalter session set current_schema=APEX_040200; BEGIN APEX_INSTANCE_ADMIN.SET_PARAMETER(\u0026#39;DISABLE_WORKSPACE_LOGIN\u0026#39;,\u0026#39;Y\u0026#39;); commit; END; / When you need to redeploy you turn it on again replacing Y with N.\nUsing standard http port 80 You need to change the ownership of the listener to root and start it as root:\ntnslsnr LISTENER -user 0 -group 0 You will to need to run something similar to this as SYSTEM:\nexec DBMS_XDB.SETLISTENERENDPOINT(DBMS_XDB.XDB_ENDPOINT_HTTP, \u0026#39;10.212.42.42\u0026#39;, 80, DBMS_XDB.XDB_PROTOCOL_TCP); The IP address is the internal IP address for the network interface. Check with ifconfig eth0 to see what address is being used.\nNot using SSL / HTTPS Of course we could have bought an SSL certificate to protect the usernames and passwords from being sent in clear text. But there is really not much sensitive material here, if you disagree strongly I would like to hear your opinion. Most of the work was actually spent on login and account administration. I can say that the passwords are MD5 hashed before being stored in the database.\nStart early with a good data model This has nothing to do with APEX, XE or AWS. It is just that you need to be reminded that everything goes smoother when you start modelling the whole thing in SQL Developer Data Modeller (SDDM) before you start developing code and APEX-application. Keep future features in mind a little without actually including them in the current data model, meaning imagine how you can expand it later for other usage like a membership administration tool. Me being not too much experienced with modelling think that a good understanding of entities and the relationship between them brings you a long way in the right direction. When Heli has finished her book we all will become experts.\n","permalink":"https://enesi.no/2014/10/oracle-database-xe-and-apex-on-amazon-web-services/","summary":"\u003cp\u003eIn our user group OUGN we used to collect abstracts for our annual conference with a simple Google Form.\u003c/p\u003e\n\u003cp\u003eAdvantage:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eEasy to set up \u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eDisadvantages\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eNo possibility for users to reconnect and correct what they had already submitted\u003c/li\u003e\n\u003cli\u003eLots of manual work\u003c/li\u003e\n\u003cli\u003eDifficult to work with in the process later (selection, agenda, late submissions, special guests)\u003c/li\u003e\n\u003cli\u003eNot really cool or cloudy\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eGoogle Forms is really just a shared spreadsheet with a form interface wrapped around it. And like any other spread sheet it is easy to work with in the beginning, but as the number of people involved increases and data grows it soon gets cumbersome to work with.\u003c/p\u003e","title":"Oracle database XE and APEX on Amazon Web Services"},{"content":"For schema level exports it may be useful to include roles and public synonyms relevant to the schemas exported. Instead of generating them manually they can be included in the DataPump export. The following example of a parameter file shows how this can be done:\nFULL=YES INCLUDE=SCHEMA:\u0026#34;IN (SELECT \u0026#39;PUBLIC\u0026#39; FROM DUAL UNION SELECT username FROM dba_users WHERE username in (\u0026#39;APEX\u0026#39;,\u0026#39;SCOTT\u0026#39;))\u0026#34; INCLUDE=PUBLIC_SYNONYM/SYNONYM:\u0026#34;IN (SELECT synonym_name FROM dba_synonyms WHERE owner = \u0026#39;PUBLIC\u0026#39; AND table_owner in (\u0026#39;APEX\u0026#39;,\u0026#39;SCOTT\u0026#39;))\u0026#34; DIRECTORY=DB_EXPORT DUMPFILE=apex_expdp.dmp LOGFILE=apex_expdp.log INCLUDE=ROLE:\u0026#34;IN (select role from dba_roles where role in (select grantee from dba_tab_privs where grantor in (\u0026#39;APEX\u0026#39;,\u0026#39;SCOTT\u0026#39;)))\u0026#34; When deciding what to include in an export you may want to check the table DATABASE_EXPORT_OBJECTS to find the correct path as in this example:\nselect * from DATABASE_EXPORT_OBJECTS where object_path like \u0026#39;%ROLE%\u0026#39;; ","permalink":"https://enesi.no/2014/10/include-roles-and-synonyms-in-datapump-export/","summary":"\u003cp\u003eFor schema level exports it may be useful to include roles and public synonyms relevant to the schemas exported. Instead of generating them manually they can be included in the DataPump export. The following example of a parameter file shows how this can be done:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eFULL=YES  \nINCLUDE=SCHEMA:\u0026#34;IN (SELECT \u0026#39;PUBLIC\u0026#39; FROM DUAL UNION SELECT username FROM dba_users WHERE username in (\u0026#39;APEX\u0026#39;,\u0026#39;SCOTT\u0026#39;))\u0026#34;  \nINCLUDE=PUBLIC_SYNONYM/SYNONYM:\u0026#34;IN (SELECT synonym_name FROM dba_synonyms WHERE owner = \u0026#39;PUBLIC\u0026#39; AND table_owner in (\u0026#39;APEX\u0026#39;,\u0026#39;SCOTT\u0026#39;))\u0026#34;  \nDIRECTORY=DB_EXPORT  \nDUMPFILE=apex_expdp.dmp  \nLOGFILE=apex_expdp.log  \nINCLUDE=ROLE:\u0026#34;IN (select role from dba_roles where role in (select grantee from dba_tab_privs where grantor in (\u0026#39;APEX\u0026#39;,\u0026#39;SCOTT\u0026#39;)))\u0026#34;    \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eWhen deciding what to include in an export you may want to check the table DATABASE_EXPORT_OBJECTS to find the correct path as in this example:\u003c/p\u003e","title":"Include roles and synonyms in DataPump export "},{"content":"There is one night every year I can stay up as long as I can manage without fearing the next day. Because, on that particular day I can sleep as much as I want on the plane, trying to fool my body into GMT-8, better known as PDT or Pacific Daylight Time. Yes, it is Oracle Open World again, in the second nicest city in the world, only beaten by Rio de Janeiro.\nIt is also an opportunity to blog at an unusual hour (at least for me). Though it is not like many have inquired about my travel plans, but it feels good to put it down in writing that I am happy to see Oracle friends again, some I only see in SFO.\nSince last OOW, I became an Oracle ACE Associate, and more than ever I am looking forward to meet friends in the OTN Lounge, and that party, you know.\nWhat else\u0026hellip; OUGN is paying for most of my expenses and my employer is letting me go, so I\u0026rsquo;ll try to give them both some satisfaction back. Mostly I will try to target and convince some speakers to come to our conference next year, increasingly known as the Boat Conference. If you are reading this and are curious, please interrupt me whatever I\u0026rsquo;m doing (mostly).\nI\u0026rsquo;ve learned a few lessons and so far not overloaded my agenda. This week I will go to the global user group leaders meeting (IOUC) on Thursday and Friday. Wednesday morning until 2 pm I\u0026rsquo;ll be at the EMEA user group stand. I will go to different sessions at the Oaktable World, hang out on different social events and of course a bunch of sessions.\nSomewhat related, Tim Hall, aka Oracle-Base blogged this today. Surely he will feel reinvigorated after the Bloggers\u0026rsquo; Meetup and the small talk with Oracle Nerds from all over the world looking for freebies in the OTN Lounge.\nThe easiest way to get in touch during OOW next week, is probably on Twitter.\nYours,\n@OyvindIsene\nPS. I did not know how to spell reinvigorated, the spell checker helped me.\nPS2. Come to think about it, there are probably a few more late nights without anxiety the following day. But I can\u0026rsquo;t find a better opening right now.\n","permalink":"https://enesi.no/2014/09/oow-2014/","summary":"\u003cp\u003eThere is one night every year I can stay up as long as I can manage without fearing the next day. Because, on that particular day I can sleep as much as I want on the plane, trying to fool my body into GMT-8, better known as PDT or  Pacific Daylight Time. Yes, it is Oracle Open World again, in the second nicest city in the world, only beaten by Rio de Janeiro.\u003c/p\u003e","title":"OOW 2014"},{"content":"Oracle Enterprise Manager (EM) reports on many security policies when they are violated. One of them are about execute privileges on standard PL/SQL packages granted to PUBLIC. One a day this summer with nothing better to do I went on to revoke these privileges. The wages for my efforts was several ORA-600 errors:\nORA-00600: internal error code, arguments: [qmxqtmChkXQAtomMapSQL:2], [], [], [], [], [], [], [], [], [], [], []\n00600. 00000 - \u0026ldquo;internal error code, arguments: [%s], [%s], [%s], [%s], [%s], [%s], [%s], [%s]\u0026rdquo;\n*Cause: This is the generic internal error number for Oracle program\nexceptions. This indicates that a process has encountered an\nexceptional condition.\n*Action: Report as a bug - the first argument is the internal error number\nA quick research on My Oracle Support indicated that some standard packages depends on these privileges granted to PUBLIC. The solution is to grant back to PUBLIC if you are in a hurry, or grant the privilege to the owners of the individual packages (like XDB). The errors continued after the granting was done, but stopped after I bounced the database instance.\nJust because EM reports something and the solution seems obvious, it is not alway wise to apply remedy immediately, especially in production. It surprised me a little that EM suggests a security policy that Oracle\u0026rsquo;s own packages do not adhere to.\n","permalink":"https://enesi.no/2014/08/revoke-privileges-from-public-may-cause-ora-600/","summary":"\u003cp\u003eOracle Enterprise Manager (EM) reports on many security policies when they are violated. One of them are about execute privileges on standard PL/SQL packages granted to PUBLIC. One a day this summer with nothing better to do I went on to revoke these privileges. The wages for my efforts was several  ORA-600 errors:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eORA-00600: internal error code, arguments: [qmxqtmChkXQAtomMapSQL:2], [], [], [], [], [], [], [], [], [], [], []\u003cbr\u003e\n00600. 00000 -  \u0026ldquo;internal error code, arguments: [%s], [%s], [%s], [%s], [%s], [%s], [%s], [%s]\u0026rdquo;\u003cbr\u003e\n*Cause:    This is the generic internal error number for Oracle program\u003cbr\u003e\n           exceptions.     This indicates that a process has encountered an\u003cbr\u003e\n           exceptional condition.\u003cbr\u003e\n*Action:   Report as a bug - the first argument is the internal error number\u003c/p\u003e","title":"Revoke privileges from PUBLIC may cause ORA-600"},{"content":"Recently during restore of a large database (over 10 TB) we monitored the alert log looking for messages like «Full restore complete of datafile 42 to datafile copy … Elapsed time: 0:39:32»\nWe saw that the time to restore a 32GB file varied a lot, from around 20 minutes to around 75 minutes. This made me wonder why. We noticed that some of the files belonging to the UNDO tablespace took usually little time, and then I found a couple of more data files that took little time to restore. Then I thought that maybe the UNDO tablespace at the time of the backup had many unused blocks (because there weren’t many active transactions at the time). So I checked the other two files that did not take long time to restore and found out they had lots of free space in them too. So I decided to take the file id and the restore time from all the messages in the alert log and load it into Oracle, in the source database (this all happened when we were making a clone for a RAT session). Then I converted the elapsed time to seconds, found file size from DBA_DATA_FILES and free space for each file from DBA_FREE_SPACE. Calculating the correlation between restore time and (file size - free space) resulted in 0.96, but for the correlation between restore time and file size, resulted in 0.38. (I used the CORR function.)\nBy the way the backup was stored on a file system created on a logical volume from a group whose LUNs where stored on an EMC VMAX 20, but the data files were restored to ASM on LUNs in an EMC XtremIO. As pointed out on Twitter:\n@OyvindIsene you mean it restores datafiles faster even they are bigger but empty? it\u0026rsquo;s because of RMAN unused block compression.\n— Gokhan Atil (@gokhanatil) August 6, 2014 it could be caused by the RMAN compression of unused blocks. Next step would be a test with creating a database with several files with different degree of free space, and doing two RMAN backups, one with and another without compression and compare restore time.\nThis may sounds like a lot of fuzz about nothing, but what we had in mind was the deduplication that the XtremIO does; wondering if the deduplication of empty blocks could take less time than blocks with real data in it.\n","permalink":"https://enesi.no/2014/08/correlation-between-restore-time-and-used-space-in-a-datafile/","summary":"\u003cp\u003eRecently during restore of a large database (over 10 TB) we monitored the alert log looking for messages like  «Full restore complete of datafile 42 to datafile copy … Elapsed time: 0:39:32»\u003c/p\u003e\n\u003cp\u003eWe saw that the time to restore a 32GB file varied a lot, from around 20 minutes to around 75 minutes. This made me wonder why. We noticed that some of the files belonging to the UNDO tablespace  took usually little time, and then I found a couple of more data files that took little time to restore. Then I thought that maybe the UNDO tablespace at the time of the backup had many unused blocks (because there weren’t many active transactions at the time). So I checked the other two files that did not take long time to restore and found out they had lots of free space in them too. \u003c/p\u003e","title":"Correlation between restore time and used space in a datafile"},{"content":"In case you are using the DBMS_LDAP package from a reasonable modern environment you may run into the error:\nORA-12703 this character set conversion is not supported\nThe simple solution to this is to run:\nbegin DBMS_LDAP.UTF8_CONVERSION := false; end; / or if you are using SQL*Plus or SQL Developer, just:\nexec DBMS_LDAP.UTF8_CONVERSION := false ","permalink":"https://enesi.no/2014/07/dbms_ldap-and-ora-12703/","summary":"\u003cp\u003eIn case you are using the DBMS_LDAP package from a reasonable modern environment you may run into the error:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eORA-12703 this character set conversion is not supported\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe simple solution to this is to run:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003ebegin  \n  DBMS_LDAP.UTF8_CONVERSION := false;  \nend;  \n/  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eor if you are using SQL*Plus or SQL Developer, just:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eexec DBMS_LDAP.UTF8_CONVERSION := false  \n\u003c/code\u003e\u003c/pre\u003e","title":"DBMS_LDAP and ORA-12703"},{"content":"I struggled to connect to the console of a newly created VM. The VM was created using Oracle VM Manager (OVMM) and despite efforts to install the required packages I could not use the console button in the Manager interface.\nI noticed that the VNC server built into the VM was listening on 127.0.0.1 only (on the OVM server), you can see this if you check the file vm.cfg for the guest. Without analysing how the OVMM would try to connect to the VNC server on the other host I decided to create an ssh tunnel from my Mac to the OVM server:\nssh -L 5901:localhost:5901 root@fu.bar.com (replace fu.bar.com with IP address or host name of your OVM server).\nNow, I think I have found a reason why Oracle specifically recommends TightVNC as a VNC viewer; I tried two other VNC viewers on my Mac, but I could not make them work with the console on the VM, sometimes it connected, but later it would hang, another just aborted. I downloaded the java version of the VNC viewer from http://www.tightvnc.com/download.php and connected without any problems to localhost and port 5901 (using the tunnel created above).\n","permalink":"https://enesi.no/2014/07/connecting-to-a-guest-console-in-oracle-vm/","summary":"\u003cp\u003eI struggled to connect to the console of a newly created VM. The VM was created using Oracle VM Manager (OVMM) and despite efforts to install the required packages I could not use the console button in the Manager interface.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"/images/blogger/connecting-to-guest-console-in-oracle-vm-1.png\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/connecting-to-guest-console-in-oracle-vm-1.png\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eI noticed that the VNC server built into the VM was listening on 127.0.0.1 only (on the OVM server), you can see this if you check the file vm.cfg for the guest. Without analysing how the OVMM would try to connect to the VNC server on the other host I decided to create an ssh tunnel from my Mac to the OVM server:\u003c/p\u003e","title":"Connecting to a guest console in Oracle VM"},{"content":"Just a simple trick to import ISO files to the repository using Oracle VM Manager (OVMM). The import function supports imports through protocols http, https and ftp. If you want to do this in the supported way you can use a simple web server to serve the ISO files for you.\nPython comes with a simple http server built in. If you have python installed (which is the case on both the OVM server and the OVMM host) you can start it with:\npython -m SimpleHTTPServer To make it easy for you, go to directory where the ISO file is located, start the http server and verify that you can see the file listed in a web browser. Then do the import in Oracle VM Manager.\nIn this case I copied the ISO file to OVM server (dom0) and started the http server there, and the URL for the import becomes as shown in the capture below (or use the IP address of the server where the ISO file is located and remember to use port 8000):\nThe name of the imported ISO is the same as the file name, rename to a say Oracle Linux 6 U5 and put in the origin of the file in description.\n","permalink":"https://enesi.no/2014/07/importing-isos-to-oracle-vm-using-ovm-manager/","summary":"\u003cp\u003eJust a simple trick to import ISO files to the repository using Oracle VM Manager (OVMM). The import function supports imports through protocols http, https and ftp. If you want to do this in the supported way you can use a simple web server to serve the ISO files for you.\u003c/p\u003e\n\u003cp\u003ePython comes with a simple http server built in. If you have python installed (which is the case on both the OVM server and the OVMM host) you can start it with:\u003c/p\u003e","title":"Importing ISOs to Oracle VM using OVM Manager"},{"content":"This post is a mixture of experiences from a recent project and an overview on how to use RAT in a project.\nReal Application Testing (RAT) is a licensed feature for the Enterprise Edition (EE) of the Oracle database. It was released on 11g, but can be used to test migration of databases from 9i and 10g to 11g (assuming that the source databases have a required version/patch). The term Real is, I reckon, used to hint at a more authentic testing process. The tests are executed based on a capture from a real production system.\nRAT can be used to:\nEvaluate new hardware, including migration from one platform to something completely different. Verify execution of SQL from today’s production on a newer version of the database Analyze the effect of changes in configuration Scaling up, see how current system works with higher load; this works best for queries / reporting and not so well for data manipulation (DML). A project involving RAT can be divided into these activities:\nCapture workload from a production system Create a clone of production system with flashback database enabled and a restore point created. Prepare for replay on test database and with agents on application servers or other client machines. Execute replay Generate reports (RAT and AWR) Flashback database If you haven’t reached a conclusion yet, repeat from 4. In two projects I was asked to do step 2 - 6 and figured out that I should write down what I learned for posterity. (The fact that I have a friend in Belgium that reminded me about blogging more is a mere coincidence.) I will not put many details on step 1 and 2 here.\nSome think it is a good idea with a long capture, but keep the replay part in mind and the time it takes to actually execute the replay. Though the flashback of the database is impressively fast, a longer replay (with much updates) will lead to more time spent on setting the database back with flashback. If you can limit the capture to say two hours that is a time saver later, but if you need to confirm say scalability for one night\u0026rsquo;s ETL jobs, you may need to capture several hours. Before you start with capture, make sure you have all the backups you need to duplicate the database with point in time recovery as close to the starting point as possible. This should not be an issue since all productions system (IMHO) should run in archiving mode and being backed up. This is an opportunity to verify your backups.\nFor large databases it may take quite some time to transport the backup (and capture files) to new platform and have it restored to a clone. Flashback database is a wonderful feature since it takes litle time to flashback a database compared to a full restore for large databases. Flashback is turned on with command:\nALTER DATABASE FLASHBACK ON; A restore point is created with:\nCREATE RESTORE POINT B4_REPLAY GUARANTEE FLASHBACK DATABASE; The name of the restore point is your decision (within limits). You will need some space for flashback logs, depending of the amount of changes to the database during replay.\nIn a bypass; yes, there are as usual two ways to do this. Enterprise Manager for the young and aspiring DBAs, or you can use the PL/SQL packages. When I have the routines ready for using the API I tend to favour them, so I did most of the work with the packages DBMS_WORKLOAD_CAPTURE and DBMS_WORKLOAD_REPLAY together with the packages for creating AWR reports DBMS_WORKLOAD_REPOSITORY. Sure, I had some looks at nice graphs and reports in EM too.\nGoing into more details on step 3; the result of the capture is a collection of many files that needs to be transported to the new platform. On the test database create a database DIRECTORY object that points to the file directory where they are stored. First step is to process the capture with:\nexec DBMS_WORKLOAD_REPLAY.PROCESS_CAPTURE(\u0026#39;REPLAY\u0026#39;) where REPLAY is the name for the directory. The result of this is a new directory with a name that starts with pp followed by a version number. Copy it all to the machine(s) that will host the replay agents (actually replay the workload and send it to database.) If you are doing this on a RAC-database, copy the same stuff to other nodes with same path for the directory object.\nIn each installation of the Oracle database and client you have the workload replay client (wrc) included under $ORACLE_HOME/bin. It’s task is to send SQL from the captured workload to the database. Before you start the actual replay a calibration should be run first. With ORACLE_HOME and PATH set, execute:\nwrc mode=calibrate replaydir=/tmp/replay This will report the recommended minimum number of clients. Too few means that the replay clients cannot replay the workload fast enough. The same will happen if the servers where the wrc binaries are overloaded, or the network between them and the database have any issues.\nSo if you have to execute 10 clients, put 10 copies of this in a script:\n#!/bin/bash wrc system/welcome1@clonedb mode=replay replaydir=/tmp/replay sleep 1 wrc system/welcome1@clonedb mode=replay replaydir=/tmp/replay sleep 1 wrc system/welcome1@clonedb mode=replay replaydir=/tmp/replay sleep 1 Don’t run it yet.\nNow we’re onto step 4. You repeat the following sequence if you decide to repeat a replay, but the preparation hitherto is only done once. Initialize the replay with the following command, using a distinct name for the replay each time:\nexec DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY(replay_name=\u0026gt;’run_1’, replay_dir=\u0026gt;’REPLAY’) In the middle of this we have to do some remapping\u0026hellip; The connections to the database used during production capture are not the same as used during replay. After the previous command has completed you can query the view DBA_WORKLOAD_CONNECTION_MAP and see what was used during capture. This view is empty before the initialize_replay procedure is executed, after it has values in the column CAPTURE_CONN, but REPLAY_CONN is empty. First time you have to look at the output and see if you have many different connections to the database and figure out how you will remap them. Often, only one connection string is used (connection string as the key in TNSNAMES.ORA) and the mapping is simply replacing it with the one you are now using to reach the new clone database. In that case you can use the following PL/SQL block to remap:\nbegin for i in (select conn_id, capture_conn from dba_workload_connection_map m, dba_workload_replays r where replay_id = id and name = ‘run_1’) loop dbms_workload_replay.remap_connection(connection_id=\u0026gt;i.conn_id, replay_connection=\u0026gt;\u0026#39;cloneserver:1521/clonedb\u0026#39;); end loop; commit; end; The name =\u0026gt; ‘run_1’ refers to the name you gave to the replay. You can either use an entry from tnsnames.ora or connection string as shown above when setting the new connection. If you have some load balancing or application partitioning where you use different connections to the database they have to be remapped accordingly. Though you have to execute this for every replay, you only spend time on this first time and keep the PL/SQL ready for next time you do a replay.\nNow after initialize, next step is prepare (!) In this step you set various parameters that how the replay is executed. In it simpliest way:\nexec DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY(synchronization =\u0026gt; TRUE, think_time_scale=\u0026gt;100, scale_up_multiplier=\u0026gt;1) synchronization set to ‘TRUE’ is equal to ‘SCN’, another more advanced (according to manual) is ‘OBJECT_ID’. This sometimes causes problems for the replay and testing several times with synchronization to ‘OFF’, ‘SCN’ or ‘OBJECT_ID’ may be necessary.\nLater on when a normal replay has been executed it is always interesting to see what happens when you reduce think time or scale up the load. Note, that DML, when scaled up won’t actually update the same way, so this needs some testing from you. For reporting usage scaling up should work, however, note if you double the number of users, they are in realilty not likely to send the same SQL as today twice as many times, that is, users usually will run slightly different queries. But if this is a problem you have to find out by analysing the reports - if an a few statements are popular in an OLTP system you can assume they will remain so after you scale up.\nNow you can add filter for what will be actually replayed, but I am going to leave it out. After you have exuted the prepare_replay procedure above, you can kick off the script on the clients, that is the script that includes the wrc commands. They will start and write out a message that they are waiting for replay to start.\nYou can start the replay with:\nexec DBMS_WORKLOAD_REPLAY.START_REPLAY() When this is executed the replay is running and the agents (wrc) writes out a message that replay has started.\nEM has a nice page for monitoring a replay. A few times we have seen that the replay lags behind compared to capture, but catches up later on, or sometimes takes longer time than the capture. Which sometimes means an error with RAT itself and not necessarily a problem with the new platform.\nIn step 5, after replay is finished, you generate a RAT report and also AWR reports. In DBA_WORKLOAD_REPLAYS you’ll find the replay_id of the replay and the RAT report can be generated with:\nselect dbms_workload_replay.report(42,’HTML’) from dual; Generate the AWR reports as usual. If you want to compare AWR report from capture with replay, you will have to import the AWR data (AWR data is automatically exported after the capture and included in the same directory). First verify that the capture is visible in DBA_WORKLOAD_CAPTURE. If not you can import all the data about captures found in the directory with:\nexec dbms_workload_capture.get_info(‘REPLAY’) where ‘REPLAY’ is the name of the DIRECTORY-object where capture is stored. The capture(s) should now be visible in DBA_WORKLOAD_CAPTURES. The AWR-data belonging to it can be imported with\nselect dbms_workload_capture.import_awr(42,’LUSER’) from dual; Take a note of the number returned. This is a generated DBID that you will use later to refer to the database of the capture (this means that this is a made up DBID that is different than the DBID of the database where you executed the capture). 42 above is the capture_id from DBA_WORKLOAD_CAPTURES and LUSER is the name of a schema used for staging. Just create on temporary if you don’t have one ready.\nThe import can take some time. When it is finished you can see in DBA_HIST_SNAPSHOT what snapshots are available, check for the DBID that was returned in last command. You can now create comparsion reports with:\nselect * from table(dbms_workoad_repository.awr_diff_report_html(123456789,1,120,130,734501234,1,10,20)); The first three parameters are DBID, start snap id, end snap id; all from the first database (source/capture) and the last three the same respectively for the second database where replay took place. Creating these reports work pretty well in SQL Developer, use Ctrl-A and Ctrl-C to copy the html code and paste it into an editor and save it. You can actually create more reports than I have shown here, see the documentation for the mentioned packages to get an idea. AWR data is automatically exported to the same directory after a replay. This means you can later be import them into another incarnation of the database to compare different replays.\nStep 6, flashback database is pretty easy. Shutdown the database, start it in mount, execute:\nflashback database to restore point b4_replay; Then open it with:\nalter database open resetlogs; If you are on RAC you have to shutdown the other nodes and start them after the previous command is completed.\nThere used to be some bugs that caused the database to crash spectacularly with an ORA-600 during flashback leaving the database useless and forcing you to do a new restore/duplicate from backup. Since I’m stressed during a project like this, and therefore I don’t have time to waste, I tend to avoid structural changes (like moving tablespaces around after creation of the restore point) that can challenge the flashback. I am not sure if the bugs are still there, though.\nStep 7, are you done yet? I usually have to run the replay over and over several times trying out with different parameters for procedure dbms_workload_replay.prepare_replay(\u0026hellip;). This takes a lot of time, don’t underestimate the project. Also remember that people will question your results.\nIn conclusion, try to figure out before you start what you want to prove. Are you comparing two systems, you need to agree on a common set of parameters; or are you trying to prove that a system is good enough (without comparing it to the old?) Try to lower you ambitions or scope, the time it takes with restore, replay, generating reports, looking over them, and possibly analyze further with extended trace, starting over,\u0026hellip; it may feel like a walk in the desert sometimes.\n","permalink":"https://enesi.no/2014/07/real-application-testing-some-notes/","summary":"\u003cp\u003eThis post is a mixture of experiences from a recent project and an overview on how to use RAT in a project.\u003c/p\u003e\n\u003cp\u003eReal Application Testing (RAT) is a licensed feature for the Enterprise Edition (EE) of the Oracle database. It was released on 11g, but can be used to test migration of databases from 9i and 10g to 11g (assuming that the source databases have a required version/patch). The term Real is, I reckon, used to hint at a more authentic testing process. The tests are executed based on a capture from a real production system.\u003c/p\u003e","title":"Real Application Testing - some notes"},{"content":"For the second time I have been involved with a proof of concept (PoC) where testing of an alternative hardware platform was the goal. Common for both PoCs were:\nThe other option was Exadata\nReal Application Testing (RAT) was used to replay production recordings on the new hardware.\nAnd for the second time I have concluded that sales people take this too easy. It is not easy and assuming you will outperform previous platform just because you have a bunch of less expensive flash disks, cache cards, and more is arrogant at best.\nA decent PoC requires lots of preparation and you need to have done some planning for capacity and load. If you have no idea of throughput, expected load, the nature of the applications and more, you are likely to fail.\nExadata is not the optimal solution for everybody, that is not why Oracle charges lots of money for it; it is not like they are thinking “Everybody will want this, so we can double the costs”. Not that I know from the inside, but some in Oracle probably thought they needed to create a solution for processing large amount of data fast. In order to do that a congestion of the CPU has to be avoided by filtering out data that is not needed before it reaches the database. As an added benefit of not shipping useless data the system will perform less I/O. If you don’t need that, Exadata may not be for you. On the other hand, if a customer needs this for a BI solution and you are selling what you like to call “standard commodity solution”, be prepared to have something that scales and can replace the Exadata features (if Exadata is what you are competing against.)\nIf you sell a solution to the 10% of the costs, it will not succeed simply because your bidding price is low; without performance it will fail. Without careful planning and analysis you will fail. Cheap food is nice, but if it makes people sick, they are not happy to spend the money on hospital bills. And they will go somewhere else.\nThere is no good substitute for proper understanding. DBAs have heard for many years that we will become redundant because modern systems are self-managing or something like that. Well, I gladly take up brewing if that becomes a reality, but what has changed is the amount of decibels from the CIO screaming when new and more expensive hardware does not perform after all.\nThe fun with Exadata does not all come from the fact that it is expensive and state of the art. Rather we have had these experiences where you combine your understanding of the platform with old wisdom like “filter early” and end up with incredible performance improvements. Like one day I asked a domain expert about the data we were querying, and after adding one redundant predicate to the SQL statement, the response time went from 90 seconds to less than 1 second. The whole improvement was due to smart scan taking place.\nBack to the PoC; we spent weeks waiting for proper integration between servers (running Linux) and the storage. Wrong interface was used for the interconnect (this was RAC), RAID 5 was used because after all, they hadn’t provided enough disks. The devices that would provide the caching features was not used, but the low-level devices was discovered by ASM instead due to wrong configuration (parameter ASM_DISKSTRING was unchanged leading to the sd* devices being discovered). After a long time the RAT replay showed that for a small number of users the performance was acceptable, but as load increased the throughput stalled and we concluded that the new solution was not dimensioned properly. The vendor suggested a new and even better storage solution with even more flash…\nProbably without seeing it themselves, the vendors in both cases demonstrated the benefit with an engineered solution. When the customer sees that you need weeks to have the storage play with your server, they start to think that maybe Oracle has a point with their preconfigured and tested solution, ready to plug in. (Yes, we know that reality is not that simple, search Twitter for #patchmadness for an illustration.)\nThere are a few takeaways from the RAT testing itself, which is material for the next post. What I can say now is that restore of database, performing replays and flashback and start over again takes a long time. Keep it simple and try to keep the capture as short as possible without losing value for what you are testing.\nThe goal with a PoC is to qualify or disqualify a new solution. If a new suggested solution is disqualified, the PoC should be deemed a success; this was actually stated by the project manager and not by me. PoCs should not be taken too easy. Think about all the problems you can avoid by stopping the wrong solution to enter your data center.\n","permalink":"https://enesi.no/2014/06/some-thoughts-from-last-poc/","summary":"\u003cp\u003eFor the second time I have been involved with a proof of concept (PoC) where testing of an alternative hardware platform was the goal. Common for both PoCs were:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003eThe other option was Exadata\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eReal Application Testing (RAT) was used to replay production recordings on the new hardware.\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eAnd for the second time I have concluded that sales people take this too easy. It is not easy and assuming you will outperform previous platform just because you have a bunch of less expensive flash disks, cache cards, and more is arrogant at best.\u003c/p\u003e","title":"Some thoughts from last PoC"},{"content":" Oracle User Group Norway (OUGN) organizes the biggest Oracle user conference in Scandinavia.\nAs of March 28, 341 DBAs, developers, managers, sales people, and more have figured out why. Read on for more reasons if you have not registered yet.\nThis year we are breaking many of our own records, for example we have 66 speakers from 15 different countries.\nWe have added another track with a total of 8 different tracks.\nOur conference, also known as the Boat Conference is getting famous worldwide. The call for paper this time resulted in more than 200 submitted abstracts, from more speakers than any time.\nWe start even earlier to cram in more sessions - it is hard to say no to all good speakers out there, we have never had so many sessions, more than 100 in total.\nThis year we have two roundtable sessions - meet the experts and join the conversations.\nSomething completely new: Women in Technology - WIT. See the abstract here http://bit.ly/OVrwgR If you have been to conferences before you know why this is important.\nMeet Oracle’s coolest product manager, for SQL Developer Jeff Smith, check his blog here http://www.thatjeffsmith.com/ and you get the idea.\nAre you a DBA and wonder why Java? Do you code in Java and wonder “What now, Oracle?” Meet Oracle’s PM on Java, Bruno Borges . Coming all the way from São Paulo, Brazil to talk about Java and more.\nThe Java track has never been better, with more Norwegian speakers, including Aslak Knutsen from Red Hat. Markus Eisele (who knows everybody in the Java community) will be there too.\nWe can’t of think any better place in Norway to mend the sometimes broken relationship between Java coders and DBAs.\nHow is your system performance? Learn optimization from the expert Cary Millsap .\nDBAs blame developers for bad design, developers blame the database for being old and not scalable. Who is right? Learn how these groups can work together from the same man.\nHave you been to an Oracle presentation and been so entertained that you lost sense of time? You will after going to one of Connor McDonnald’s presentations . Last year his presentations received record high scores on content and delivery.\nStill confused about fusion? The middleware track will help you and your future will look green.\nHow often have you googled for a solution and ended up with a nice write-up at oracle-base.com ? Now you can meet the man who wrote all of it, Tim Hall aka Oracle Base, look for a man in an Oraclenerd T-shirt.\nNot sure if Norway is big enough for Big Data? Surely Big Data Solutions Director Luis Campos from Oracle PT will tell us.\nSoftware projects are difficult, get off with a good start and a good data model in this presentation with Heli (the world has only one Heli like Heli, so first name is OK, we think).\nDo you like Scottish mood and complicated humour? You’ll be able to read 10053 trace files backwards before you understand all his jokes, which make his presentations never boring. Meet OUGN best supporter and a long-time friend, Doug Burns .\nDo you usually wait three years after release of R2 before you upgrade? You will have no reason to wait after you’ve got all the details in the Hitchhiker’s Guide to Oracle Database Upgrades With 12c PL/SQL is stronger than ever, according to Bryn in his presentation P133. Some worried that Oracle would kill MySQL once they bought it. They were wrong, our MySQL track is full of good content from Oracle’s own developers and users out there. Meet attendees and Oracle staff and ask about MySQL (Care for an meetup btw? We’ve got an open slot for you.)\nHas everybody moved to Linux? Check out this presentation on Sparc and LDOMS.\nThe apps track is back!\nYou too are late learning APEX? Don’t worry, check out this .\nSeveral managers have signed up. If you manage DBAs and other techies this is your opportunity to see them in their preferred habitat, and you can exchange experience with other managers.\nDo you have challenges in your job? Alone with a new design, or maybe you have learned something unusual? Share your experience and discuss it with other techies. Learn what is not in the documentation.\nIntroverts and control freaks have no better place to socialize then on a user conference. Not tired of discussing RMAN configuration? You will not be alone on this ship.\nHave a question for Oracle sale, pre-sale, technologists or someone from the C-level? Oracle Norway, US, DE and more are sending several to support us, and they don’t mind sitting down for a talk, be it in a bar or in the Boardroom. Who knows what can happen in international waters . Just don’t mention licenses on the dance floor, that is bad mojo.\nThis year we have a new native app for smartphones. It will work even when you are offline. Make your own agenda and share it on Twitter.\nOne of two frequent feedback from speakers and attendees is that the conference is well organized. Thanks to Color Line and the venue the conference is usually a smooth sailing. Actually we can’t think of any better venue, so we will probably use the same setup for many years.\nThe other feedback is that the food is quite good. Breakfast, lunch and dinner are served in a restaurant with excellent standard. More info about the ship here .\nBeing on a ship, people stick around and socialize with other speakers and attendees. Not too big and not to small. This is a fantastic networking opportunity; you’ll always find someone, when you’ve had enough you can always retreat to your cabin or a calm restaurant.\nThe weather forecast looks quite promising, with sun, no rain and 4º C the Oslofjord will be a nice scenery going and returning.\nTax free\u0026hellip;\nThough we have wifi in the conference area, you may actually enjoy being offline sometimes. It is a good excuse to not read email and check whatever addiction you have when we are going through Skagerrak.\nIf you come from abroad and want to know more about the Norwegian market you get to meet a variety of large and small customers.\nActually more attendees from outside Norway has registered than any time before.\nForgot to mention the cloud. Surely, someone will mention it on the conference. There will probably be more clouds in the sessions than in the sky, because we will have no-cloud weather during the conference.\nWe also have many presentations with tons of no-hype stuff, proven work from experienced people.\nSeeing first-time speakers freak out when the ship starts moving is priceless.\nIf you are still reading this, it may be to late. Ticket sale ends Monday March 31, 09 hs CET. Mail us if you are desperate.\n","permalink":"https://enesi.no/2014/03/42-reasons-to-join-me-at-ougn14/","summary":"\u003cp\u003e\u003ca href=\"/images/blogger/42-reasons-to-join-me-at-ougn14-1.jpg\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/42-reasons-to-join-me-at-ougn14-1.jpg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003eOracle User Group Norway (OUGN) organizes the biggest Oracle user conference in Scandinavia.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eAs of March 28, 341 DBAs, developers, managers, sales people, and more have figured out why. Read on for more reasons if you have not registered yet.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eThis year we are breaking many of our own records, for example we have 66 speakers from 15 different countries.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eWe have added another track with a total of 8 different tracks.\u003c/p\u003e","title":"42 Reasons to join me at #OUGN14"},{"content":"Vårens store høydepunkt er OUGN sitt vårseminar. Her er programmet og her kan du melde deg på . Drar du til Oslo for å delta på konferansen anbefaler jeg at du flyr inn dagen før for å bli med på et annet stort høydepunkt, nemlig et dagskurs med Cary Millsap i Oslo .\nDette er et kurs både for utviklere som integrerer mot Oracle database og for Oracle DBA-er. En bedre presentasjon av kurset finner du på påmeldingssiden .\nPersonlig så ser jeg på det som et skille i karrieren min, mellom før og etter at jeg lærte metodene til Cary. \u0026ldquo;Tuning\u0026rdquo; som det ofte kalles er ofte forbundet med myter og råd som baserer seg på prøving og feiling, sjelden effektivt i komplekse systemer. Ved å bruke SQL trace og de riktige verktøyene til å finne frem til hva som tar tid, kommer man raskere frem til målet. Målet er altså å optimalisere responsen til en transaksjon mot databasen, en tung rapport, interaktiv bruk fra en web-side, osv.\nKurset begynner kl 10 og holder til i Bjørvika, det blir dermed mulig å fly inn til Oslo på onsdag morgen før kurset.\nHar du spørsmål rundt kurset ta gjerne kontakt på Twitter, email eller telefon. Kurset arrangeres av Method R .\n","permalink":"https://enesi.no/2014/01/kurs-med-cary-millsap-i-oslo-dagen-for-ougn14/","summary":"\u003cp\u003eVårens store høydepunkt er OUGN sitt vårseminar. Her er \u003ca href=\"http://ougnvarseminar2014.sched.org/\" target=\"_blank\" rel=\"noopener\"\u003eprogrammet\u003c/a\u003e\n og \u003ca href=\"http://www.eventbrite.com/e/ougn-varseminar-2014-tickets-9426289285\" target=\"_blank\" rel=\"noopener\"\u003eher kan du melde deg på\u003c/a\u003e\n. Drar du til Oslo for å delta på konferansen anbefaler jeg at du flyr inn dagen før for å bli med på et annet stort høydepunkt, nemlig et \u003ca href=\"http://motd-oslo-2014-04-02.eventbrite.com/\" target=\"_blank\" rel=\"noopener\"\u003edagskurs med Cary Millsap i Oslo\u003c/a\u003e\n.\u003c/p\u003e\n\u003cp\u003eDette er et kurs både for utviklere som integrerer mot Oracle database og for Oracle DBA-er. En bedre presentasjon av kurset finner du på \u003ca href=\"http://motd-oslo-2014-04-02.eventbrite.com/\" target=\"_blank\" rel=\"noopener\"\u003epåmeldingssiden\u003c/a\u003e\n.\u003c/p\u003e","title":"Kurs med Cary Millsap i Oslo dagen før #OUGN14"},{"content":"OSL - CPH - SFO. Starting on my route in a few hours. Really look forward to see tweeps and a bunch of people from the worldwide Oracle community. Lots of good presentations of course, but also bloggers\u0026rsquo; meetup, Oaktable world, RAC Attack, Anchor Steam, and more.\nI\u0026rsquo;ll speak at the Bulgarian Oracle User Group conference 22nd to 24th of November. I have heard from many that this is a good conference. First time in Bulgaria means this will be a triple excitement; giving a presentation about something I have thought a lot about, seeing a new country and meeting a possibly different culture.\nThe UKOUG Tech conference starts the week after. That is another conference I\u0026rsquo;ve never been to (strangely enough since it is rather close compared to OOW).\nIf you happen to be at one of these conferences, please say Hi!\n","permalink":"https://enesi.no/2013/09/upcoming-conferences/","summary":"\u003cp\u003eOSL - CPH - SFO. Starting on my route in a few hours. Really look forward to see tweeps and a bunch of people from the worldwide Oracle community. Lots of good presentations of course, but also bloggers\u0026rsquo; meetup, Oaktable world, RAC Attack, Anchor Steam, and more.\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;ll speak at the Bulgarian Oracle User Group \u003ca href=\"http://www.bgoug.org/en/events/details/90.html\" target=\"_blank\" rel=\"noopener\"\u003econference\u003c/a\u003e\n  22nd to 24th of November. I have heard from many that this is a good conference. First time in Bulgaria means this will be a triple excitement; giving a presentation about something I have thought a lot about, seeing a new country and meeting a possibly different culture.\u003c/p\u003e","title":"Upcoming conferences"},{"content":"Here is an easy command to find corrupt XML-files on a Linux box.\nA couple of days ago I had a problem with OBIEE and weblogic. Googling and search on My Oracle Support indicated that the failure may be caused by a corrupt XML file. And you know that there are many of them in an OBIEE installation. In fact MOS in Doc 1475146.1 blindly suggests to move a bunch of them to a backup location, because in some cases they had become corrupt for some strange reason. I was not satisfied with just deleting a few random files that have caused problems for others earlier, I wanted to find the file.\nI then came up with the idea that I could find the corrupt file by parsing every XML file below a point in the file hierarchy. On Linux this is quite easy. The command xmlint is possibly not standard on every system, but should be easy to get with yum or similar. In my case the file I was looking for belonged to user weblogic, meaning the corrupt file would reside in a directory called weblogic, hence the command. But you can of course tweak the arguments to the find command to search the files you want:\nfind . -wholename \\\\*weblogic\\\\*.xml -ls -exec xmlint {} \u0026gt; /dev/null \\\\; Any normal output will be discarded and only failures sent to stderr (on your screen). A similar command to parse every file with extension xml below $ORACLE_BASE:\nfind $ORACLE_BASE -name \\\\*.xml -exec xmlint {} \u0026gt; /dev/null \\\\; If you (re)move the file you usually have to restart stuff with opmnctl.\n","permalink":"https://enesi.no/2013/09/finding-corrupt-xml-files-on-linux/","summary":"\u003cp\u003eHere is an easy command to find corrupt XML-files on a Linux box.\u003c/p\u003e\n\u003cp\u003eA couple of days ago I had a problem with OBIEE and weblogic. Googling and search on My Oracle Support indicated that the failure may be caused by a corrupt XML file. And you know that there are many of them in an OBIEE installation. In fact MOS in Doc 1475146.1 blindly suggests to move a bunch of them to a backup location, because in some cases they had become corrupt for some strange reason. I was not satisfied with just deleting a few random files that have caused problems for others earlier, I wanted to find the file.\u003c/p\u003e","title":"Finding corrupt XML files on Linux"},{"content":"If you don\u0026rsquo;t know what procrastination is you could start your own blog. Start a post with something like \u0026ldquo;I worked with \u0026hellip; and decided to write three posts about it\u0026rdquo;. Procrastination is what you experience before the last post is ready; seemingly simple things take forever to get out the door.\nThis post is important for my own record and shows how to salvage data from a large table with several corrupt blocks. I also had a learning experience I wanted to remember, explaining the redundancy of information here. Previous post was about how to map out tables and the corrupt blocks in a temporary table (that is, not a table created as a global temporary table, but a table you\u0026rsquo;ll drop when you\u0026rsquo;re finished.)\nNote 61685.1 on My Oracle Support (MOS) shows how to extract data from a corrupt table. This post builds on some of that. Before you begin; data in corrupt blocks are gone. Unless you have special tools and start reading the data files with a tool like DUDE you won\u0026rsquo;t recover those data. This is about how to recover the data that reside in blocks that have not been marked as corrupt.\nOne essential function gives you the ROWID from the block number and its file number: DBMS_ROWID.ROWID_CREATE:\nselect DBMS_ROWID.ROWID_CREATE(1,253071,89,660864,0) from DUAL; In the example above 1 means extended rowid; no need to change this, 253071 is the DATA_OBJECT_ID from DBA_OBJECTS, 89 is FILE# (file number) from V$DATABASE_BLOCK_CORRUPTION , and 660864 is BLOCK# (block number) from the same view. The last parameter is for the row number in that block, in this case we want the whole block and therefore starts with row 0.\nThis is an example of the content of V$DATABASE_BLOCK_CORRUPTION:\nFILE# BLOCK# BLOCKS CORRUPTION_CHANGE# CORRUPTION_TYPE 89 661112 8 0 CORRUPT 89 661048 16 0 CORRUPT 89 660984 16 0 CORRUPT 89 660920 16 0 CORRUPT 89 660864 8 0 CORRUPT In previous post I created a table SEGMENTS_WITH_CORRUPTION. The following query gives a report of corrupt segments with an interval by rowid using the function above:\nselect O.OWNER,O.OBJECT_NAME,O.OBJECT_TYPE, DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT,0) ROWID_OF_FIRST_CORR, DBMS_ROWID.ROWID_CREATE(1,o.DATA_OBJECT_ID,c.file_id,c.start_corrupt+c.corrupt_blocks,0) ROWID_OF_NEXT_CLEAN from segments_with_corruption c,DBA_OBJECTS O where C.OWNER=O.OWNER and c.segment_name=o.object_name; An example of output (slightly redacted so I don\u0026rsquo;t reveal the source)\nOWNER OBJECT_NAME OBJECT_TYPE ROWID_OF_FIRST_CORR ROWID_OF_NEXT_CLEAN LUSER FUBAR TABLE AAA9yPABZAAChZ4AAA AAA9yPABZAAChaAAAA LUSER FUBAR TABLE AAA9yPABZAAChY4AAA AAA9yPABZAAChZIAAA LUSER FUBAR TABLE AAA9yPABZAAChX4AAA AAA9yPABZAAChYIAAA LUSER FUBAR TABLE AAA9yPABZAAChW4AAA AAA9yPABZAAChXIAAA LUSER FUBAR TABLE AAA9yPABZAAChWAAAA AAA9yPABZAAChWIAAA Now, create a new table to hold the good data. Using LUSER.FUBAR as the owner.table_name for the original table:\nCreate table luser.fubar_recover as Select * from luser.fubar Where 1=0; This means we create an empty table with the same structure, we will fill it with data in a few steps. If you had only a few intervals of corrupt blocks, you can create insert statements like the following in order to retrieve the good data:\nInsert into luser.fubar_recover select * from luser.fubar where rowid \u0026lt; ‘AAA9yPABZAAChZ4AAA’; Insert into luser.fubar_recover select * from luser.fubar where rowid \u0026gt;= ‘AAA9yPABZAAChaAAAA’ and rowid \u0026lt; ‘AAA9yPABZAAChY4AAA’; In the first example you get all the rows in blocks up to the corrupt one. In the second you get all the rows between two corrupt blocks. In my case I had a large table with several stretches of corrupt and good blocks. I did not want to handcraft all these insert statements, I wanted one statement that could retrieve all the good data from the table.\nTo achieve that I started with a query that generates interval of corrupt blocks in the order as they are laid out in the table. At this point I was only working on one table and one data file as the query below shows.\nselect FIRST_CORRUPT,FIRST_CLEAN,LAG(FIRST_CORRUPT) over (order by start_corrupt) PREVIOUS_CORRUPT, LEAD(FIRST_CORRUPT) over (order by start_corrupt) NEXT_CORRUPT, rownum rn from( select DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT,0) FIRST_CORRUPT, DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT+C.CORRUPT_BLOCKS,0) FIRST_CLEAN, C.START_CORRUPT from SEGMENTS_WITH_CORRUPTION C, DBA_OBJECTS O where C.OWNER=O.OWNER and C.SEGMENT_NAME=O.OBJECT_NAME and FILE_ID=89 and C.OWNER=’LUSER’ and C.SEGMENT_NAME=\u0026#39;FUBAR’ order by c.start_corrupt ) ; The output looked like this:\nFIRST_CORRUPT\nFIRST_CLEAN\nPREVIOUS_CORRUPT\nNEXT_CORRUPT\nRN\nAAA9yPABZAAChWAAAA\nAAA9yPABZAAChWIAAA\nAAA9yPABZAAChW4AAA\n1\nAAA9yPABZAAChW4AAA\nAAA9yPABZAAChXIAAA\nAAA9yPABZAAChWAAAA\nAAA9yPABZAAChX4AAA\n2\nAAA9yPABZAAChX4AAA\nAAA9yPABZAAChYIAAA\nAAA9yPABZAAChW4AAA\nAAA9yPABZAAChY4AAA\n3\nAAA9yPABZAAChY4AAA\nAAA9yPABZAAChZIAAA\nAAA9yPABZAAChX4AAA\nAAA9yPABZAAChZ4AAA\n4\nAAA9yPABZAAChZ4AAA\nAAA9yPABZAAChaAAAA\nAAA9yPABZAAChY4AAA\n5\nOh, well. Layout of the last table could have been better, I experimented with export html-code from SQL Developer that can be pasted into the HTML-tab. Anyway, it shows stretches of blocks that are corrupt for this file. Now I want to generate WHERE clauses that are used in the select statement that retrieves the good data:\nwith INTERVALS as ( select FIRST_CORRUPT,FIRST_CLEAN,LAG(FIRST_CORRUPT) over (order by start_corrupt) PREVIOUS_CORRUPT, LEAD(FIRST_CORRUPT) over (order by start_corrupt) NEXT_CORRUPT, rownum rn from( select DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT,0) FIRST_CORRUPT, DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT+C.CORRUPT_BLOCKS,0) FIRST_CLEAN, C.START_CORRUPT from SEGMENTS_WITH_CORRUPTION C, DBA_OBJECTS O where C.OWNER=O.OWNER and C.SEGMENT_NAME=O.OBJECT_NAME and FILE_ID=89 and C.OWNER=\u0026#39;LUSER\u0026#39; and C.SEGMENT_NAME=\u0026#39;FUBAR\u0026#39; order by c.start_corrupt ) ) select \u0026#39;where rowid \u0026lt; \u0026#39;\u0026#39;\u0026#39; || FIRST_CORRUPT || \u0026#39;\u0026#39;\u0026#39;\u0026#39; LINE from INTERVALS where RN=1 union select \u0026#39;or (rowid \u0026gt;= \u0026#39;\u0026#39;\u0026#39; || FIRST_CLEAN || \u0026#39;\u0026#39;\u0026#39; and rowid \u0026lt; \u0026#39;\u0026#39;\u0026#39; || NEXT_CORRUPT || \u0026#39;\u0026#39;\u0026#39;)\u0026#39; LINE from INTERVALS where RN\u0026gt;1 and NEXT_CORRUPT is not null union select \u0026#39;or (rowid \u0026gt;= \u0026#39;\u0026#39;\u0026#39; || FIRST_CLEAN || \u0026#39;\u0026#39;\u0026#39;)\u0026#39; LINE from INTERVALS where NEXT_CORRUPT is null order by rn ; This gives the following output:\nLINE where rowid \u0026lt; \u0026#39;AAA9yPABZAAChWAAAA\u0026#39; or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChXIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChX4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChYIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChY4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChZIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChZ4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChaAAAA) I can now amend the insert statement by using these where clauses at the end of the select statement:\ninsert into LUSER.FUBAR_RECOVER select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026lt; \u0026#39;AAA9yPABZAAChWAAAA\u0026#39; or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChXIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChX4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChYIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChY4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChZIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChZ4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChaAAAA\u0026#39;); Also note the ROWID hint that tells Oracle how to scan the table. The biggest problem with this query is that it does not work(!) Yes, I\u0026rsquo;m going to drag you through the learning experience I had when making things automated and slightly more advanced. When executing the SQL above Oracle performs a normal table scan and aborts when it reads the first corrupt block. This can be seen if you look at the execution plan:\nexplain plan for select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026lt; \u0026#39;AAA9yPABZAAChWAAAA\u0026#39; or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChXIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChX4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChYIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChY4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChZIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChZ4AAA\u0026#39;) or (rowid \u0026gt;= \u0026#39;AAA9yPABZAAChaAAAA\u0026#39;); select * from table(dbms_xplan.display); Plan hash value: 3975471795 \\---------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | \\---------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 11M| 2111M| 838K (1)| 02:47:37 | |* 1 | TABLE ACCESS FULL| FUBAR | 11M| 2111M| 838K (1)| 02:47:37 | \\---------------------------------------------------------------------------------- Predicate Information (identified by operation id): \\--------------------------------------------------- 1 - filter(ROWID\u0026lt;\u0026#39;AAA9yPABZAAChWAAAA\u0026#39; OR ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChaAAAA\u0026#39; OR ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChXIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChX4AAA\u0026#39; OR ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChYIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChY4AAA\u0026#39; OR ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChZIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChZ4AAA\u0026#39;) Compare this to the plan for the SQL that was used in the beginning:\nexplain plan for select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;AAA9yPABZAAChaAAAA\u0026#39;; select * from table(dbms_xplan.display); Plan hash value: 3328636578 \\-------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | \\-------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5445K| 1012M| 836K (1)| 02:47:18 | |* 1 | TABLE ACCESS BY ROWID RANGE| FUBAR | 5445K| 1012M| 836K (1)| 02:47:18 | \\-------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): \\--------------------------------------------------- 1 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChaAAAA\u0026#39;) This one uses TABLE ACCESS BY ROWID, but the one that fails uses normal TABLE SCAN. I therefore changed slightly the SQL to generate where clauses:\nwith INTERVALS as ( select FIRST_CORRUPT,FIRST_CLEAN,LAG(FIRST_CORRUPT) over (order by start_corrupt) PREVIOUS_CORRUPT, LEAD(FIRST_CORRUPT) over (order by start_corrupt) NEXT_CORRUPT, rownum rn from( select DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT,0) FIRST_CORRUPT, DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT+C.CORRUPT_BLOCKS,0) FIRST_CLEAN, C.START_CORRUPT from SEGMENTS_WITH_CORRUPTION C, DBA_OBJECTS O where C.OWNER=O.OWNER and C.SEGMENT_NAME=O.OBJECT_NAME and FILE_ID=89 and C.OWNER=\u0026#39;LUSER\u0026#39; and C.SEGMENT_NAME=\u0026#39;FUBAR\u0026#39; order by c.start_corrupt ) ) select \u0026#39;select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026lt; \u0026#39;\u0026#39;\u0026#39; || FIRST_CORRUPT || \u0026#39;\u0026#39;\u0026#39;\u0026#39; LINE from INTERVALS where RN=1 union select \u0026#39;union select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;\u0026#39;\u0026#39; || FIRST_CLEAN || \u0026#39;\u0026#39;\u0026#39; and rowid \u0026lt; \u0026#39;\u0026#39;\u0026#39; || NEXT_CORRUPT || \u0026#39;\u0026#39;\u0026#39;\u0026#39; LINE from INTERVALS where RN\u0026gt;1 and NEXT_CORRUPT is not null union select \u0026#39;union select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;\u0026#39;\u0026#39; || FIRST_CLEAN || \u0026#39;\u0026#39;\u0026#39;;\u0026#39; LINE from INTERVALS where NEXT_CORRUPT is null order by RN ; The resulting SQL used to retrieve good data looks like this:\nselect /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026lt; \u0026#39;AAA9yPABZAAChWAAAA\u0026#39; union select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;AAA9yPABZAAChXIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChX4AAA\u0026#39; union select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;AAA9yPABZAAChYIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChY4AAA\u0026#39; union select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;AAA9yPABZAAChZIAAA\u0026#39; and rowid \u0026lt; \u0026#39;AAA9yPABZAAChZ4AAA\u0026#39; union select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;AAA9yPABZAAChaAAAA\u0026#39;; This one actually runs, but it runs for a long time and it eventually failed later because TEMP ran out of space. The reason for this can also be seen in the execution plan:\nPlan hash value: 3469639691 \\------------------------------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | \\------------------------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 11M| 2177M| | 4680K (78)| 15:36:06 | | 1 | SORT UNIQUE | | 11M| 2177M| 2858M| 4680K (78)| 15:36:06 | | 2 | UNION-ALL | | | | | | | |* 3 | TABLE ACCESS BY ROWID RANGE| FUBAR | 5445K| 1012M| | 836K (1)| 02:47:18 | |* 4 | TABLE ACCESS BY ROWID RANGE| FUBAR | 272K| 50M| | 836K (1)| 02:47:16 | |* 5 | TABLE ACCESS BY ROWID RANGE| FUBAR | 272K| 50M| | 836K (1)| 02:47:16 | |* 6 | TABLE ACCESS BY ROWID RANGE| FUBAR | 272K| 50M| | 836K (1)| 02:47:16 | |* 7 | TABLE ACCESS BY ROWID RANGE| FUBAR | 5445K| 1012M| | 836K (1)| 02:47:18 | \\------------------------------------------------------------------------------------------------------ Predicate Information (identified by operation id): \\--------------------------------------------------- 3 - access(ROWID\u0026lt;\u0026#39;AAA9yPABZAAChWAAAA\u0026#39;) 4 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChXIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChX4AAA\u0026#39;) 5 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChYIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChY4AAA\u0026#39;) 6 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChZIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChZ4AAA\u0026#39;) 7 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChaAAAA\u0026#39;) Look at line 1 and 2 in the plan. I forgot to use UNION ALL instead of UNION and Oracle then goes on to sort this stuff in order to return distinct rows. The table had over 100M rows and sorting this filled the TEMP tablespace completely. Since I know I want all rows, I changed it to UNION ALL (this is finally a good one you can use):\nselect FIRST_CORRUPT,FIRST_CLEAN,LAG(FIRST_CORRUPT) over (order by start_corrupt) PREVIOUS_CORRUPT, LEAD(FIRST_CORRUPT) over (order by start_corrupt) NEXT_CORRUPT, rownum rn from( select DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT,0) FIRST_CORRUPT, DBMS_ROWID.ROWID_CREATE(1,O.DATA_OBJECT_ID,C.FILE_ID,C.START_CORRUPT+C.CORRUPT_BLOCKS,0) FIRST_CLEAN, C.START_CORRUPT from SEGMENTS_WITH_CORRUPTION C, DBA_OBJECTS O where C.OWNER=O.OWNER and C.SEGMENT_NAME=O.OBJECT_NAME and FILE_ID=89 and C.OWNER=\u0026#39;LUSER\u0026#39; and C.SEGMENT_NAME=\u0026#39;FUBAR\u0026#39; order by c.start_corrupt ) ) select \u0026#39;select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026lt; \u0026#39;\u0026#39;\u0026#39; || FIRST_CORRUPT || \u0026#39;\u0026#39;\u0026#39;\u0026#39; LINE from INTERVALS where RN=1 union select \u0026#39;union all select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;\u0026#39;\u0026#39; || FIRST_CLEAN || \u0026#39;\u0026#39;\u0026#39; and rowid \u0026lt; \u0026#39;\u0026#39;\u0026#39; || NEXT_CORRUPT || \u0026#39;\u0026#39;\u0026#39;\u0026#39; LINE from INTERVALS where RN\u0026gt;1 and NEXT_CORRUPT is not null union all select \u0026#39;union all select /*+ ROWID(T) */ * from LUSER.FUBAR T where rowid \u0026gt;= \u0026#39;\u0026#39;\u0026#39; || FIRST_CLEAN || \u0026#39;\u0026#39;\u0026#39;;\u0026#39; LINE from INTERVALS where NEXT_CORRUPT is null order by RN ; The new execution plan looks like this:\nPlan hash value: 439408122 \\--------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | \\--------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 11M| 2177M| 4181K (81)| 13:56:23 | | 1 | UNION-ALL | | | | | | |* 2 | TABLE ACCESS BY ROWID RANGE| FUBAR | 5445K| 1012M| 836K (1)| 02:47:18 | |* 3 | TABLE ACCESS BY ROWID RANGE| FUBAR | 272K| 50M| 836K (1)| 02:47:16 | |* 4 | TABLE ACCESS BY ROWID RANGE| FUBAR | 272K| 50M| 836K (1)| 02:47:16 | |* 5 | TABLE ACCESS BY ROWID RANGE| FUBAR | 272K| 50M| 836K (1)| 02:47:16 | |* 6 | TABLE ACCESS BY ROWID RANGE| FUBAR | 5445K| 1012M| 836K (1)| 02:47:18 | \\--------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): \\--------------------------------------------------- 2 - access(ROWID\u0026lt;\u0026#39;AAA9yPABZAAChWAAAA\u0026#39;) 3 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChXIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChX4AAA\u0026#39;) 4 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChYIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChY4AAA\u0026#39;) 5 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChZIAAA\u0026#39; AND ROWID\u0026lt;\u0026#39;AAA9yPABZAAChZ4AAA\u0026#39;) 6 - access(ROWID\u0026gt;=\u0026#39;AAA9yPABZAAChaAAAA\u0026#39;) To wrap it up:\nIf you have a large table with several corrupt blocks and many good ones, you can save the good data by creating a table that maps the bad blocks and then create a select statements that retrieve the rows from all the good blocks.\nBest is of course to get your backup and recover routines in order, combine it with a decent fault management so you\u0026rsquo;ll discover the corrupt blocks at an early stage, thereby making it possible to recover the blocks before they have been aged out of your backup system.\nIf corrupt blocks happens again and again you should see if it happens only with very few tables or many. If the corruption is restricted to few out of many tables it may indicate that it has something to do with how these tables are created and used (thinking about NOLOGGING operations here). In case of frequent and seemingly random corruption I would start thinking about the underlying storage.\n","permalink":"https://enesi.no/2013/07/corrupt-database-blocks-iii/","summary":"\u003cp\u003eIf you don\u0026rsquo;t know what procrastination is you could start your own blog. Start \u003ca href=\"https://enesi.no/2013/03/corrupt-database-blocks-i/\"\u003ea post\u003c/a\u003e\n with something like \u0026ldquo;I worked with \u0026hellip; and decided to write three posts about it\u0026rdquo;. Procrastination is what you experience before the last post is ready; seemingly simple things take forever to get out the door.\u003c/p\u003e\n\u003cp\u003eThis post is important for my own record and shows how to salvage data from a large table with several corrupt blocks. I also had a learning experience I wanted to remember, explaining the redundancy of information here. \u003ca href=\"https://enesi.no/2013/04/corrupt-database-blocks-ii/\"\u003ePrevious post\u003c/a\u003e\n was about how to map out tables and the corrupt blocks in a temporary table (that is, not a table created as a \u003cem\u003eglobal temporary table\u003c/em\u003e, but a table you\u0026rsquo;ll drop when you\u0026rsquo;re finished.)\u003c/p\u003e","title":"Corrupt database blocks III"},{"content":"Sometimes download from Oracle Technology Network (OTN) is really slow, but it may not be Oracle\u0026rsquo;s fault. Oracle is using Akamai Technologies to distribute the software. You can see this if you do an nslookup on download.oracle.com and see it resolves (finally) to a host under akamai.net.\nThey have servers all over the world and when things work as they are supposed to the box on your end should connect to a host physically near you and not on another continent. If your download is really slow find the IP address that your DNS resolves for download.oracle.com (or the host you are downloading from) and then use a geo-lookup site to see where it is located.\nJust now the DNS server for my local ISP (get.no) resolved download.oracle.com to two servers in California:\n\\[root@rio ~\\]# nslookup download.oracle.com Server: 84.208.20.110 Address: 84.208.20.110#53 Non-authoritative answer: download.oracle.com canonical name = download.oracle.com.edgesuite.net. download.oracle.com.edgesuite.net canonical name = a1961.d.akamai.net. Name: a1961.d.akamai.net Address: 64.209.77.10 Name: a1961.d.akamai.net Address: 64.209.77.18 I used www.infosniper.net to find where this host was located. Me being in Oslo this was obviously not optimal. The only DNS server I can remember is 8.8.8.8 which is one of Google\u0026rsquo;s public DNS servers. I decided to change /etc/resolv.conf to use 8.8.8.8 as the primary DNS server, and download.oracle.com now resolves to two servers in Germany:\n\\[root@rio ~\\]# nslookup download.oracle.com Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: download.oracle.com canonical name = download.oracle.com.edgesuite.net. download.oracle.com.edgesuite.net canonical name = a1961.d.akamai.net. a1961.d.akamai.net canonical name = a1961.d.akamai.net.0.1.cn.akamaitech.net. Name: a1961.d.akamai.net.0.1.cn.akamaitech.net Address: 46.33.72.121 Name: a1961.d.akamai.net.0.1.cn.akamaitech.net Address: 46.33.72.129 Here is the link if you like maps\u0026hellip; I then flushed DNS in Firefox, restarted the download, and confirmed with netstat -an that the connection now went to a server in Germany. The download went much faster, well, at least for a while\u0026hellip; YMMV.\nActually I thought that using Google\u0026rsquo;s DNS server could give a suboptimal response if Google\u0026rsquo;s DNS server was located somewhere far away (like in USA), but clearly they have some magic here that returns an IP address better than the DNS server of my local ISP.\n","permalink":"https://enesi.no/2013/05/download-from-otn-slow-check-dns/","summary":"\u003cp\u003eSometimes download from Oracle Technology Network (OTN) is really slow, but it may not be Oracle\u0026rsquo;s fault. Oracle is using \u003ca href=\"http://en.wikipedia.org/wiki/Akamai_Technologies\" target=\"_blank\" rel=\"noopener\"\u003eAkamai Technologies\u003c/a\u003e\n  to distribute the software. You can see this if you do an nslookup on download.oracle.com and see it resolves (finally) to a host under akamai.net.\u003c/p\u003e\n\u003cp\u003eThey have servers all over the world and when things work as they are supposed to the box on your end should connect to a host physically near you and not on another continent. If your download is really slow find the IP address that your DNS resolves for download.oracle.com (or the host you are downloading from) and then use a geo-lookup site to see where it is located.\u003c/p\u003e","title":"Download from OTN slow? Check DNS"},{"content":"Last week\u0026rsquo;s conference is the second since I joined the board of our user group. Never been more worried, but it turned out better than ever, in my not so humble opinion.\nI was really proud of our agenda, we had several new speakers with top content. Cary Millsap from Method R gave a keynote and two excellent presentations. This was his first time in Norway, but judging by the feedback we got he will be back soon. Actually having him here at our conference was a goal I committed to last year. Thanks, Cary, for excellent content and presentation!\nLast OOW I met Connor McDonald at the blogger\u0026rsquo;s meetup that Pythian organizes at OOW every year and I thought it would be cool if we could have him at our conference. After some severe lobbying with the rest of the board we decided on a plan to make it happen. Actually I had never seen Connor present before, but when he did his Odds and Ends presentation I understood what everybody was talking about. I think it is awesome that we can get the best, even if they come from the other side of the earth. We have not processed the feedback forms yet, but the impression so far is that Connor and Cary is sharing first place on content and presentation.\nTom Kyte, Bryn Llewellyn and Joel Goodman were back too. Always popular, each has his public (overlapping I assume) and genuinely dedicated. I am thankful for all the good publicity they are giving our conference.\nThere was also time for some psycho analysis and after a few beers Doug Burns and me discovered we are more or less soul mates. Always good to see the man and his presentations, he was mildly shocked when I showed up on one of his presos in stead of the one Cary was giving at the same time (confession: I had already seen the one Cary was giving). I learned a lot this time too. Good stuff! And thanks for the feedback, motivation and interests for our user group. You will always be on our MRU list. We have two areas where we have to improve; Java and MySQL. Markus Eisele helped us to get speakers for the Development/Java track in addition to speak himself. We have work to do on Java, and the same goes for MySQL, it is a different group of people. I know Oracle and many in the community have a lot of good stuff on Java and MySQL, but first we need to appear on those users\u0026rsquo; radar and have them come to our table.\nI spend much of my digital social life on Twitter. When I go to OOW or MOW I get to meet people whose Tweets and blogs I\u0026rsquo;ve been reading for a time. It is always a nice experience. This year two Twitter friends came to our conference; Philippe Fierens from Belgium (check out his nice post about the conference) and Jan Karremans from Netherlands. In fact the whole group from Netherlands was quite impressive.\nIt is a lot of hard work to organize a user group conference, the agenda requires a lot to get in place. For me it is the best networking opportunity, it is the place where I get to know what is not written in any manual or book, and meet friends IRL. Here is one, Martin Nash , and the whole Martin cluster .\nThe cruise part of Spring Seminar 2014 will start 2014-04-03, with a possible conference day in Oslo the day before. But first I\u0026rsquo;ll take a short vacation next week, going to MOW2013 .\n","permalink":"https://enesi.no/2013/04/ougn-spring-seminar-2013/","summary":"\u003cp\u003eLast week\u0026rsquo;s conference is the second since I joined the board of our user group. Never been more worried, but it turned out better than ever, in my not so humble opinion.\u003c/p\u003e\n\u003cp\u003eI was really proud of our agenda, we had several new speakers with top content. \u003c/p\u003e\n\u003cp\u003e\u003ca href=\"http://carymillsap.blogspot.no/\" target=\"_blank\" rel=\"noopener\"\u003eCary Millsap\u003c/a\u003e\n from \u003ca href=\"http://www.method-r.com/\" target=\"_blank\" rel=\"noopener\"\u003eMethod R\u003c/a\u003e\n gave a keynote and two excellent presentations. This was his first time in Norway, but judging by the feedback we got he will be back soon. Actually having him here at our conference was a \u003ca href=\"https://enesi.no/2012/03/ougn-2012-day-2-wrap-up/\"\u003egoal\u003c/a\u003e\n I committed to last year. Thanks, Cary, for excellent content and presentation!\u003c/p\u003e","title":"OUGN Spring Seminar 2013 "},{"content":"This is a common error message after upgrade of the database:\nORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance In 11g (last tested with 11.2.0.3) even if your start up the database from an old type pfile without any obsolete parameters and then create an spfile from memory like this:\ncreate spfile from memory; Oracle will insert two obsolete parameters into the spfile. This query will list obsolete parameters in your spfile:\nselect p.name,p.value from v$parameter p, v$spparameter s where s.name=p.name and p.isdeprecated=\u0026#39;TRUE\u0026#39; and s.isspecified=\u0026#39;TRUE\u0026#39;; NAME VALUE \\--------------------- ---------------------------------------------- background_dump_dest /u01/app/oracle/diag/rdbms/hfmtst/HFMTST/trace user_dump_dest /u01/app/oracle/diag/rdbms/hfmtst/HFMTST/trace They can be cleared with:\nalter system reset background_dump_dest; alter system reset user_dump_dest; ","permalink":"https://enesi.no/2013/04/removing-obsolete-parameters-from-spfile/","summary":"\u003cp\u003eThis is a common error message after upgrade of the database:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \nORA-32004: obsolete or deprecated parameter(s)   \nspecified for RDBMS instance  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eIn 11g (last tested with 11.2.0.3) even if your start up the database from an old type pfile without any obsolete parameters and then create an spfile from memory like this:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \ncreate spfile from memory;  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eOracle will insert two obsolete parameters into the spfile. This query will list obsolete parameters in your spfile:\u003c/p\u003e","title":"Removing obsolete parameters from spfile"},{"content":"After you have validated database as shown in previous post any corrupt database blocks will be reported in the view V$DATABASE_BLOCK_CORRUPTION. Now you want to find out the tables that used to have data in these blocks; those data have typically been lost if you have no backup of them or do not posses special tools like DUDE .\nBefore you start I recommend to purge the DBA Recycle Bin if it is in use, as reported here certain queries seem to take longer time when it is not empty. Execute as SYSDBA:\npurge dba_recyclebin; Then create a temporary table with information on the corrupt blocks:\ncreate table segments_with_corruption as select FILE_ID,BLOCK_ID,blocks,OWNER,SEGMENT_NAME, start_corrupt,corrupt_blocks from ( select e.FILE_ID,e.BLOCK_ID,e.blocks,e.OWNER,e.SEGMENT_NAME, c.block# start_corrupt,c.blocks corrupt_blocks from DBA_EXTENTS E, V$DATABASE_BLOCK_CORRUPTION C where E.BLOCK_ID \u0026lt;= C.block# and E.BLOCK_ID + E.blocks - 1 \u0026gt;= C.block# and e.file_id=c.file# union all select e.FILE_ID,e.BLOCK_ID,e.blocks,e.OWNER,e.SEGMENT_NAME, c.block# start_corrupt,c.blocks corrupt_blocks from DBA_EXTENTS E, V$DATABASE_BLOCK_CORRUPTION C where E.BLOCK_ID \u0026gt;= C.block# and E.BLOCK_ID \u0026lt;= C.block# + C.blocks -1 and e.file_id=c.file# ); This query may take a long time to run, which is a reason to materialize it in a table while you are working on this. You can find a more advanced query on MOS in article 472231.1 To create this query I made a drawing for myself with all different scenarios on how interval of corrupt blocks could possible overlap with extents belonging to some object. Then I wrote down the criteria and merged them together.\nUse this query to find owner, name and type of the segments you have to work with:\nselect distinct c.owner,c.segment_name, c.segment_type from segments_with_corruption c, dba_segments s where c.segment_name=c.segment_name and c.owner=s.owner; For segment_type=INDEX you can fix it by simply rebuilding the index (hopefully none of the rows it indexes are in corrupt blocks).\nThe next post will be on how to salvage as much as possibly from the good blocks belonging to a table that has some corrupt blocks. I\u0026rsquo;ll show a query you will run to generate other statements that will be used to extract data from blocks around the corrupt blocks.\n","permalink":"https://enesi.no/2013/04/corrupt-database-blocks-ii/","summary":"\u003cp\u003eAfter you have validated database as shown in \u003ca href=\"https://enesi.no/2013/03/corrupt-database-blocks-i/\"\u003eprevious post\u003c/a\u003e\n any corrupt database blocks will be reported in the view V$DATABASE_BLOCK_CORRUPTION. Now you want to find out the tables that used to have data in these blocks; those data have typically been lost if you have no backup of them or do not posses special tools like \u003ca href=\"http://www.ora600.be/\" target=\"_blank\" rel=\"noopener\"\u003eDUDE\u003c/a\u003e\n.\u003c/p\u003e\n\u003cp\u003eBefore you start I recommend to purge the DBA Recycle Bin if it is in use, as reported \u003ca href=\"https://enesi.no/2013/02/throw-out-the-garbage/\"\u003ehere\u003c/a\u003e\n certain queries  seem to take longer time when it is not empty. Execute as SYSDBA:\u003c/p\u003e","title":"Corrupt database blocks II"},{"content":"I worked with corrupt blocks in a database a few weeks ago and decided to write three posts about it so I don\u0026rsquo;t have to rethink all of it next time it happens. Hardware and the database have improved a lot, because I rarely run into this problem.\nThe first and obvious thing to state is that corrupt blocks in the database should be detected early by proper fault management (if you use Enterprise Manager you are good). When detected it should be fixed by using RMAN, either by command:\nrecover corruption list; or by recovering individual blocks, e.g.\nrecover datafile 42 block 5 to 6; In case you don\u0026rsquo;t have any backup or the error went undetected for a long time and you decide that block recovery is not feasible you need to clean it up some other way. If the RMAN backup detects the corrupt block it will abort at first corrupt block (unless you change the SET MAXCORRUPT FOR DATAFILE). This means that there may be more corrupt blocks in the database and you may just as well find them all while you are working on it.\nThe Database Verification tool is supposed to do find all corrupt blocks, but last time (11.2.0.2 on Linux x64) it failed to find new ones. This tool is documented in the manual Utilities. I looped over all the data files with command dbv, but it was a waste of time. This may have something to do that there are different types of corruptions. Instead I used VALIDATE in RMAN. You can validate individual data files or the entire database:\nVALIDATE DATAFILE 42; VALIDATE DATABASE; This will not consider MAXCORRUPTIONS, but scan the data file / entire database and update the view v$database_block_corruption with info on corrupt blocks found.\nNext post will be on how to find objects that have, or rather, used to have data in these blocks.\n","permalink":"https://enesi.no/2013/03/corrupt-database-blocks-i/","summary":"\u003cp\u003eI worked with corrupt blocks in a database a few weeks ago and decided to write three posts about it so I don\u0026rsquo;t have to rethink all of it next time it happens. Hardware and the database have improved a lot, because I rarely run into this problem.\u003c/p\u003e\n\u003cp\u003eThe first and obvious thing to state is that corrupt blocks in the database should be detected early by proper fault management (if you use Enterprise Manager you are good). When detected it should be fixed by using RMAN, either by command:\u003c/p\u003e","title":"Corrupt database blocks I"},{"content":"The garbage in this case is the recycle bin in the database. Just as in any tidy house you should not wait too long before you empty the garbage bin. I\u0026rsquo;ve come across two reasons to empty it.\nOne is the ORA-3297 that is thrown when you try to downsize a data file you think have lots of free space at the end. I once wrote this SQL to find the lower limit that I can reduce the data file to:\nSelect File_Id,ceil(Hb.Min_Red*8192/1024/1024/1024) Minimal_Size_Gb ,round(Df.Blocks*8/1024/1024,2) Total_Gb, Round((Df.Blocks-Hb.Min_Red)*8/1024/1024) higher_free_GB from dba_data_files df join (Select File_Id, Max(Block_Id+Blocks) Min_Red From Dba_Extents Group By File_Id) Hb Using(File_Id) Order By Higher_Free_Gb Desc; On one occasion when I tried to reduce the size of the data file an ORA-3297 was thrown. Of course I could have checked DBA_RECYCLEBIN, but I was checking DBA_SEGMENTS and noticed several segments with names that started with BIN which immediately reminded me about the recycle bin. I purged the recycle bin and the resize operation went through.\nThe other reason is that certain queries against DBA_FREE_SPACE may actually take much longer when there are segments (tables) in the recycle bin. One third-party monitoring application had a query that took more time to complete than anything else, after a purge dba_recyclebin the query had a reasonable response time.\nSome argue that one should not use the recycle bin in production. I think it depends; if your application creates and drops tables frequently, see if you can add PURGE to the DROP TABLE statement, if not turn the recycle bin off by setting the system parameter recyclebin to off. Also after a successful release of a new version empty the recycle bin when everything is verified OK. Accidents do happen, therefore I like to have the recycle bin enabled just in case. Except for the case already mentioned I think that dropping tables should not happen so often in a production environment that it becomes a problem.\n","permalink":"https://enesi.no/2013/02/throw-out-the-garbage/","summary":"\u003cp\u003eThe garbage in this case is the recycle bin in the database. Just as in any tidy house you should not wait too long before you empty the garbage bin. I\u0026rsquo;ve come across two reasons to empty it.\u003c/p\u003e\n\u003cp\u003eOne is the ORA-3297 that is thrown when you try to downsize a data file you think have lots of free space at the end. I once wrote this SQL to find the lower limit that I can reduce the data file to:\u003c/p\u003e","title":"Throw out the garbage"},{"content":"Flashback database, (active) duplicate with RMAN, and tablespaces that are read-only. You can only have two out of three from my recent testing. One database with a few read-only tablespaces was cloned by using active duplicate in RMAN. Before the testing (using Real Application Testing replay) a restore point with guarantee flashback database was created. After the testing I tried to perform a flashback database, but it failed with:\nSQL\u0026gt; flashback database to restore point b4_test; flashback database to restore point b4_test * ERROR at line 1: ORA-01122: database file 38 failed verification check ORA-01110: data file 38: \u0026#39;+DATA/clonedb/datafile/foo_bar002.443.804434293\u0026#39; ORA-01207: file is more recent than control file - old control file The file in question belongs to a read-only tablespace and is certainly not newer than the control file. It looks to be a bug on 11.2.0.3 and I found a way to work around it if you take some actions before you start testing. I did not find the same error stack mentioned together with flashback, or anything directly relevant on My Oracle Support, but similar symptoms made me think the problem would go away if I set the read-only tablespaces read-write. I generated SQL with this:```\nselect \u0026lsquo;alter tablespace \u0026rsquo; || tablespace_name || \u0026rsquo; read write;\u0026rsquo;\nfrom dba_tablespaces\nwhere status=\u0026lsquo;READ ONLY\u0026rsquo;;\nAfter I executed the generated statements, I created a new restore point with:```\ndrop restore point b4_test; create restore point b4_test guarantee flashback database; For testing databases it is probably no big deal if you set all the tablespaces read only. If you try to set the tablespaces read-write after you have done some changes to the database you want to revert (i.e. testing) the flashback operation will probably fail, possibly with an ORA-600 error, but I have not tested it. In other words, the workaround must be in place before you create the restore point.\nI don\u0026rsquo;t know why this happens for databases that are duplicated with RMAN. Flashback database on the source database works OK even with read-only tablespaces. The source database was created from a restore of an RMAN backup taken on a physical standby database. Also the duplication itself works OK with the read-only tablespace. Two out of three, then.\n","permalink":"https://enesi.no/2013/01/flashback-database-fails-with-read-only-tablespace/","summary":"\u003cp\u003eFlashback database, (active) duplicate with RMAN, and tablespaces that are read-only. You can only have two out of three from my recent testing. One database with a few read-only tablespaces was cloned by using active duplicate in RMAN. Before the testing (using Real Application Testing replay) a restore point with \u003cem\u003eguarantee flashback database\u003c/em\u003e was created. After the testing I tried to perform a flashback database, but it failed with:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \nSQL\u0026gt; flashback database to restore point b4_test;  \nflashback database to restore point b4_test  \n*  \nERROR at line 1:  \nORA-01122: database file 38 failed verification check  \nORA-01110: data file 38: \u0026#39;+DATA/clonedb/datafile/foo_bar002.443.804434293\u0026#39;  \nORA-01207: file is more recent than control file - old control file  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eThe file in question belongs to a read-only tablespace and is certainly not newer than the control file. It looks to be a bug on 11.2.0.3 and I found a way to work around it if you take some actions before you start testing. I did not find the same error stack mentioned together with flashback, or anything directly relevant on My Oracle Support, but similar symptoms made me think the problem would go away if I set the read-only tablespaces read-write. I generated SQL with this:```\u003c/p\u003e","title":"Flashback database fails with read-only tablespace"},{"content":"This is not intended as a complete book review, but a warm recommendation of a book I\u0026rsquo;ve found very useful the last couple of months. That is, highly recommended if you are working with RAC or going to start with a RAC project soon.\nI have been working on and off with RAC for several years, but I don\u0026rsquo;t take any new RAC project lightly. Starting in November I was involved with a proof of concept for a large public customer. Before and during this project I have relied a lot on this book: Pro Oracle Database 11g RAC on Linux written by Steve Shaw and Martin Bach .\nTrue, installation of RAC has become much easier on 11gR2, but the planning part is as important as ever. If you jump into the project without proper planning you are likely to encounter problems later on after the system is installed and running (more or less). Changing network layout or mostly anything else is a hassle later on, though this has also become easier with the added functionality to the commands crsctl and srvctl in 11gR2.\nOne reason I like this book as an important addition to the Oracle documentation is that it is very logical written from a project point of view. It starts out with a good introduction and then explains the important concepts and the arcitechture. This is stuff you need to get right before you start. There are also many decisions you have to make before you start. The book builds a solid ground under your cluster with a good focus on the OS (Linux in this case, but I think I will recommend it for a similar project on Windows as well due to the book\u0026rsquo;s structure). In other words, if you don\u0026rsquo;t have time to read the necessary chapters in this book you will not have time to fix the errors later on either.\nWe\u0026rsquo;re not exactly masters of logistics in this country, and I came in late for this project. I started the preparation for the project by reading this book, when I had read enough and felt prepared for the next day I could put the book aside and get some rest. The next day I would continue by reading and planning. Well, that might read as if this was my first RAC project, it was not, but again I don\u0026rsquo;t imagine I remember every detail, the book serves as a checklist.\nThe project gave us several challenges due to a rather big database that we created from an unconventional RMAN backup, Real Application Testing (RAT), super fast hardware that still needed tweaking on OS level. But not once did I run into problems due to wrong configuration of RAC or errors we could blame the RAC software.\nHave a nice 2013! Go to conferences, user group meetings, and meet all the nice people in the community. Our conference, OUGN 2013, will be better than ever; we will have Oracle experts from Australia, USA and Europe.\n","permalink":"https://enesi.no/2012/12/pro-oracle-database-11g-rac-on-linux/","summary":"\u003cp\u003eThis is not intended as a complete book review, but a warm recommendation of a book I\u0026rsquo;ve found very useful the last couple of months. That is, highly recommended if you are working with RAC or going to start with a RAC project soon.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"http://2.bp.blogspot.com/-U4ohEziAQvY/UOGjDLECL9I/AAAAAAAADUw/AS0b3OhqxJ8/s1600/rac_book.png\" target=\"_blank\" rel=\"noopener\"\u003e\u003cimg loading=\"lazy\" src=\"http://2.bp.blogspot.com/-U4ohEziAQvY/UOGjDLECL9I/AAAAAAAADUw/AS0b3OhqxJ8/s320/rac_book.png\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eI have been working on and off with RAC for several years, but I don\u0026rsquo;t take any new RAC project lightly. Starting in November I was involved with a proof of concept for a large public customer. Before and during this project I have relied a lot on this book: \u003ca href=\"http://www.amazon.com/Pro-Oracle-Database-11g-Linux/dp/1430229586/ref=sr_1_1?s=books\u0026amp;ie=UTF8\u0026amp;qid=1356964351\u0026amp;sr=1-1\" target=\"_blank\" rel=\"noopener\"\u003ePro Oracle Database 11g RAC on Linux\u003c/a\u003e\n written by Steve Shaw and \u003ca href=\"https://twitter.com/martindba\" target=\"_blank\" rel=\"noopener\"\u003eMartin Bach\u003c/a\u003e\n.\u003c/p\u003e","title":"Pro Oracle Database 11g RAC on Linux"},{"content":"Last year was my first time at OOW. I made a write-up every day on my blog, but this year I will maximize, meaning it will be little time for blogging while being there. It is now less than four two one hours before I start on my trip OSL-SFO. Writing about what I plan to do seems like a great way to kill time. Also I\u0026rsquo;m doing some proactive jet-lag prevention research, may be staying up late will make the transition from CET to PT easier (or is it CEST and PST now?).\nI have a schedule packed with interesting stuff, but based on experience from last year, I expect to feel some conference fatigue setting in around noon at Wednesday. I may just as well see Mogens in his office.\nBesides all these good presos I am really looking forward to two events. One is the Oaktable World (formerly called Oracle Closed World; guess what happened). Good thing it is early in the week from Monday to Tuesday. Those guys are not giving out 101-courses, and will demand my attention.\nThe second event is the Bloggers Meetup organized and held by Pythian. This may very well be the best networking opportunity at OOW :-) Last year I met a bunch of people I have known virtually through Twitter and their blogs for years. Nothing beats a conversation in person. You will be surprised how many nice people there are out there that share our somewhat narrow interest.\nI\u0026rsquo;m always interesting to talk to potential speakers at our annual conference. If you have any questions regarding our user group (Oracle User Group Norway) or our conference, please get in touch. Btw, here is our call for paper Paying slightly attention to Twitter, I just noticed a link to the kick-off for the IOUG Big Data SIG. Think I want to check it out too. Not sure what I can make out of this Big Data thing, but it seems to attract a lot of smart brains.\nLuggage is ready, and it includes a cool t-shirt from Method-R, Oaktable World will be a nice occasion to wear it.\n","permalink":"https://enesi.no/2012/09/oracle-open-world-lucky-me/","summary":"\u003cp\u003eLast year was my first time at OOW. I made a write-up every day on my blog, but this year I will maximize, meaning it will be little time for blogging while being there. It is now less than four two one hours before I start on my trip OSL-SFO. Writing about what I plan to do seems like a great way to kill time. Also I\u0026rsquo;m doing some proactive jet-lag prevention research, may be staying up late will make the transition from CET to PT easier (or is it CEST and PST now?).\u003c/p\u003e","title":"Oracle Open World - lucky me"},{"content":"Version 4.2 of VirtualBox was released two days ago (so far for Linux only). Download it from here: https://www.virtualbox.org/wiki/Linux_Downloads Probably not important, but I\u0026rsquo;m using Fedora 16 for the moment, and since I was running VirtualBox 4.1 I had to remove the previous package first and install the 4.2 version. This will not destroy the guest hosts.\nrpm -e VirtualBox-4.1 rpm -ip VirtualBox-4.2-4.2.0_80737_fedora16-1.x86_64.rpm After a new start with the VirtualBox Manager the old VM guests are in place.\n","permalink":"https://enesi.no/2012/09/virtualbox-4.2-released/","summary":"\u003cp\u003eVersion 4.2 of VirtualBox was released two days ago (so far for Linux only). Download it from here: \u003ca href=\"https://www.virtualbox.org/wiki/Linux_Downloads\" target=\"_blank\" rel=\"noopener\"\u003ehttps://www.virtualbox.org/wiki/Linux_Downloads\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eProbably not important, but I\u0026rsquo;m using Fedora 16 for the moment, and since I was running VirtualBox 4.1 I had to remove the previous package first and install the 4.2 version. This will not destroy the guest hosts.\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003erpm -e VirtualBox-4.1  \nrpm -ip VirtualBox-4.2-4.2.0_80737_fedora16-1.x86_64.rpm  \n  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eAfter a new start with the VirtualBox Manager the old VM guests are in place.\u003c/p\u003e","title":"VirtualBox 4.2 released"},{"content":"This is more a note to myself\u0026hellip; Installing Oracle on Linux has become much easier with a package that prepares the OS before installation of the Oracle Server software. In previous versions the rpm package used to be called oracle-validated, but for 11gR2 on OEL6 it is called oracle-rdbms-server-11gR2-preinstall.\nProbably the easiest way to install the Oracle database on Linux is to install Oracle Enterprise Linux 6 and before the installation of Oracle do two things:\n1. Enable the public yum repository at Oracle:\ncd /etc/yum.repos.d wget http://public-yum.oracle.com/public-yum-ol6.repo Edit the file and enable the relevant version by setting enabled=1.\n2. Install the package that prepares the server for Oracle database installation:\nyum install oracle-rdbms-server-11gR2-preinstall No need to run any commands after the yum install command. Continue with runInstaller.\nUpdate 2013-07-12:\nWith the release of database 12c Oracle has released a new package to prepare Oracle Linux 6 for it.\nGet the info from Oracle here . The procedure remains the same as for 11g, only the name of the rpm file changes. I installed it on a fresh new OL6 without any extra configuration of yum repository with:\nyum update yum install oracle-rdbms-server-12cR1-preinstall The outline from Oracle tells you to do step 1 above, but the repo file was included in OL 6.4.\n","permalink":"https://enesi.no/2012/09/easier-installation-of-oracle-on-linux/","summary":"\u003cp\u003eThis is more a note to myself\u0026hellip; Installing Oracle on Linux has become much easier with a package that prepares the OS before installation of the Oracle Server software. In previous versions the rpm package used to be called oracle-validated, but for 11gR2 on OEL6 it is called oracle-rdbms-server-11gR2-preinstall.\u003c/p\u003e\n\u003cp\u003eProbably the easiest way to install the Oracle database on Linux is to install Oracle Enterprise Linux 6 and before the installation of Oracle do two things:\u003c/p\u003e","title":"Easier installation of Oracle on Linux"},{"content":"I learned something very important when doing testing with Hammerora. The documentation is quite good and has a simple, but important point, the importance of planning and preparation. To quote the documentation for Oracle OLTP testing: Planning and Preparation is one of the most crucial stages of successful testing but is often overlooked. Firstly you should fully document the configuration of your entire load testing environment including details such as hardware, operating system versions and settings and Oracle version and parameters. Once you have fully documented your configuration you should ensure that the configuration is not changed for an entire series of measured tests. This takes discipline but is an essential component of conducting accurate and measured tests.\nFrom this I conclude that many tests I\u0026rsquo;ve seen and done myself have not been accurate. As stated earlier the goal for this testing was just to compare performance before and after migration from EVA to 3PAR.\nSince this customer did not have the required license to run AWR I did the following change in the driver scipt to create a Statspack snapshot in place of a AWR snapshot. Search for string containing dbms_workload, replace\nset sql1 \u0026ldquo;BEGIN dbms_workload_repository.create_snapshot(); END;\u0026rdquo;\nwith\nset sql1 \u0026ldquo;BEGIN perfstat.statspack.snap; END;\u0026rdquo;\nWhen testing with Hammerora I decided to run each test three times to see if the numbers where consistent. I recorded the numbers of each run in an spread sheet as shown in the following table for the tests on EVA:\nVusers\nRun\nReport\ntpm\nnopm\nAvg_tpm\n1\n1\n1_2\n7158\n2369\n7634\n1\n2\n11_12\n7874\n2645\n1\n3\n21_22\n7868\n2804\n3\n1\n31_32\n16478\n5765\n17317\n3\n2\n41_42\n17678\n6256\n3\n3\n43_44\n17794\n6130\n10\n1\n45_46\n27847\n9959\n33225\n10\n2\n51_61\n32581\n11600\n10\n3\n71_81\n39248\n13701\n20\n1\n91_101\n47489\n17441\n47075\n20\n2\n111_121\n63062\n22658\n20\n3\n131_141\n30674\n11116\n30\n1\n151_161\n54349\n19756\n44186\n30\n2\n171_181\n45628\n17331\n30\n3\n191_201\n32581\n12733\nVusers is the number of virtual users in Hammerora, Run is 1 - 3 for each new setting of Vusers. Report refers to the Statspack report created on snapshots before and after. Tpm and nopm as reported from Hammerora and finally Avg_tpm is the average in each group. Compare this to the numbers for the 3PAR:\nVusers\nRun\nReport\ntpm\nnopm\nAvg_tpm\n1\n1\n9_10\n8246\n2815\n8262\n1\n2\n11_12\n7983\n2717\n1\n3\n13_14\n8556\n2956\n3\n1\n15_16\n22652\n7854\n22881\n3\n2\n17_18\n22652\n7831\n3\n3\n19_20\n23339\n7994\n10\n1\n21_22\n33539\n11767\n33191\n10\n2\n25_26\n39054\n13729\n10\n3\n27_28\n26981\n9428\n20\n1\n29_30\n47134\n16462\n47356\n20\n2\n31_32\n46436\n16330\n20\n3\n33_34\n48497\n17023\n30\n1\n35_36\n53197\n18902\n50788\n30\n2\n37_38\n44980\n15994\n30\n3\n39_40\n54187\n19033\nThe repeated tests for the same number of virtual users do not vary as much on the 3PAR compared to the EVA. Also the numbers for the EVA seemed to improve for each run, maybe due to some caching taking place.\nThe 3PAR seemed to be more reliable for the same number of virtual users as can be seen in these screen captures, the first for 20 virtual users on EVA:\nThe peformance on the 3PAR does not change much during the test (20 virtual users):\nYou\u0026rsquo;ll see that in one instant the EVA seems to perform better, but I rather have stable and less erratic performance with the 3PAR than a system with occasional good performance.\nAll in all it was very easy to play around with Hammerora, it is very easy to set up so you can spend time on planning and executing the tests. Also I like how you can observe change of performance over time. Clearly Hammerora is a tool I will use more later.\n","permalink":"https://enesi.no/2012/08/oracle-load-testing-part-3-hammerora-results/","summary":"\u003cp\u003eI learned something very important when doing testing with Hammerora. The documentation is quite good and has a simple, but important point, the importance of planning and preparation. To quote the \u003ca href=\"http://hammerora.sourceforge.net/hammerora_oracle_oltp_v2.8.pdf\" target=\"_blank\" rel=\"noopener\"\u003edocumentation for Oracle OLTP testing:\u003c/a\u003e\n\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003ePlanning and Preparation is one of the most crucial stages of successful testing but is often overlooked. Firstly you should fully document the configuration of your entire load testing environment including details such as hardware, operating system versions and settings and Oracle version and parameters. Once you have fully documented your configuration you should ensure that the configuration is not changed for an entire series of measured tests. This takes discipline but is an essential component of conducting accurate and measured tests.\u003c/p\u003e","title":"  Oracle load testing - part 3 Hammerora results"},{"content":"Last week I was running load tests in between other tasks on my agenda. As stated in previous post I wanted to compare performance, as measured by load testing tools, on an old SAN and a new one, EVA and 3PAR respectively. I ran two types of tests, the OLTP test and the SIMPLE test:\norion -run oltp -num_disks 50 orion -run simple -num_disks 50 Following are charts from the ORION OLTP test with \u0026ldquo;num_disks\u0026rdquo; equal to 50.\nTest was running against one disk (LUN) pr execution. Only test-server has been migrated to 3PAR and ORION were run before and after. Not surprisingly the 3PAR can achieve higher number of I/Os per second as the load increases. The EVA seems to behave erratic or breaking down after some point. This is confirmed in the following chart for latency:\nThe wild values from one of the EVA-tests throws the chart out of proportions. But it is still possible to see that the latency is within reasonable limits. By limiting the values from 0 to 50,000 microseconds the chart looks like:\nLooks like I could have expanded the range when testing the 3PAR. Latency is within 10-32 ms the whole range. Latency is the most important measure for databases IMHO, and after testing this repeatedly on 3PAR I expect to have more stable performance after migration.\nNext chart is a histogram for the latency values:\nThe main difference between them is the outliers on the right side for the old EVA.\nThe following charts are from the SIMPLE test. In this test the pattern is less erratic for EVA, but still the 3PAR performs much better and seems to be very happy as the load increases.\nThe SIMPLE test measures throughput with MBPS. Here large I/Os are used to simulate a situation where throughput is more of a concern. The pattern resembles that of the previous picture, and the throughput of the 3PAR is amazingly stable. I\u0026rsquo;m wondering if it will be just as stable when we move production next week.\nThe last chart shows the latency of the SIMPLE test, small I/Os where used:\nIn summary: The ORION tests indicates that the 3PAR is much more stable, and by looking at the results for latency more fit for serving a database server than the EVA. The pattern on the EVA was similar on both production and testing servers, which means that the extra load from other users in production did not explain the oscillations.\nNext post on Hammerora will show a similar pattern. I learned a lot by reading the documentation of Hammerora. It is very educational by giving good advices on how to plan and run tests. I felt my ORION testing was a bit inferior compared to the rounds with Hammerora after I finished them, but I still think the results from ORION are interesting. More on that in a few days.\n","permalink":"https://enesi.no/2012/06/oracle-load-testing-part-2-orion-results/","summary":"\u003cp\u003eLast week I was running load tests in between other tasks on my agenda. As stated in previous post I wanted to compare performance, as measured by load testing tools, on an old SAN and a new one, EVA and 3PAR respectively. I ran two types of tests, the OLTP test and the SIMPLE test:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eorion -run oltp -num_disks 50  \norion -run simple -num_disks 50  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eFollowing are charts from the ORION OLTP test with \u0026ldquo;num_disks\u0026rdquo; equal to 50.\u003c/p\u003e","title":"Oracle load testing - part 2 ORION results"},{"content":"The current customer is migrating from EVA storage to 3PAR, and inspired by some discussions on Twitter between @yvelikanov, @martinberx, @kevinclosson, @martinDBA, and others I decided to try out some free tools for load testing of Oracle servers. I can hardly tune a SQL statement without getting philosophical about it, so especially this first part is a bunch of thoughts I had when starting out.\nThere are a lot of discussions on the usability of such tools; are the measured performance any close to the reality? The load these tools generate is more or less synthetic. One tool that stands out from the crowd is the Real Application Tool (RAT), but RAT requires an extra license from Oracle. With RAT you can capture real load on a production system and replay it on a test system with new or upgraded hardware or configuration. The reports generated in Enterprise Manager lets you compare the performance of old and new system so clearly that even your CEO can understand it. You cannot get much closer to reality than with RAT, but as said, it costs money and time to set it up. [And for a change I will not whine much more on the fact that SMBs around here do not invest in these licensed packages that makes the life of the DBA easier.]\nBack to the question on reliability of these tools; the fact that the load is often quite artificial and likely to be very homogenous (or randomized without the skew you have in your real world) means that they give an upper limit you are not likely to reach in normal production. That was one of the reasons why Kevin Closson wrote the Silly Little Oracle Benchmark (SLOB). After having played with this I have reached the preliminary conclusion that these tools are useful for comparing configurations and behavior more than ranking your system.\nThe last days my mind has started to compare this with the infamous buffer cache hit ratio (CBHR) - see this post over the subject on Jonathan Lewis\u0026rsquo; Blog. Though the BCHR is quite a meaningless measurement you still may know what it use to be for your most important databases since there are several monitoring systems out there that still come with an Oracle module that measure it and sends you an alert if it is too low (too low according to the assumptions Lewis and others have attacked long time ago). But what if your BCHR suddenly changes, what does that mean? It means just that: something changed. And that may be worthwhile investigating. Think of your kid that always make some noise, though silence is welcome, if he suddenly stops his rattling, you want to know why, silence is not always a good sign. Translated back to the BCHR, an increased ratio could be caused by some sessions reading the indexes en masse when a FTS would have performed better.\nAnd what is the connection to load testing? By repeating the tests several times and seeing the same pattern again and again the tool confirmed itself and when running it against another system this pattern changed, but remained consistent on the new system. The measured limit itself is probably off the reality, but I think it is reasonable to assume that when the performance becomes erratic and unstable (that means widely varying results that are not linear or exponential linked to the increasing load, or less fancy, the graph you are looking at goes up and down on the right side) you have found some upper limit of the load the SAN can take. If the performance on the new SAN remains stable for more load compared to the old one, it will probably perform better in real world production also, just don\u0026rsquo;t believe you\u0026rsquo;ve found the correct upper limit for your application. That was pretty much our ambitions, compare old and new SAN, and possibly discover any obvious configuration errors.\nIn my case I wanted to do some measurements of I/O performance so I could compare the old EVA with the new 3PAR SAN. First out was ORION (Oracle IO Numbers). ORION is included with the database software, you\u0026rsquo;ll find it under $ORACLE_HOME/bin. Or you can download it from page I just linked to. The general idea with ORION is that it executes more or less the same code path as the Oracle database does when doing I/O. There is no application testing involved, just a bunch of I/O operations thrown against your storage. You can test small and large I/O to simulate the load from OLTP, DSS, or mixed environments. ORION is supported on several platforms including Windows. ORION reports latency (in microseconds), throughput (megabytes pr second), and I/O operations per second (IOPS). The results are written to text files in CSV format that can easily be imported to Excel for easy visualizations (or much better, imported to your precious database and devoured with Apex).\nI tried to have SLOB work from Windows, but couldn\u0026rsquo;t find an easy way to replace the semaphore mechanism Closson used on Unix to start all sessions at once. I could have written some in Java or other, but I really didn\u0026rsquo;t have the time and it seemed to be an over-kill.\nNext out was Hammerora . It uses Tcl/Tk to run and is also supported on various platforms, but it was very unstable on the Windows 7 32-bit I was using. It would run for a while and suddenly at a random point (sometimes after many minutes of testing) encounter an error with Tcl/Tk needing a restart. Think this was caused by the combination of the OS and Tcl/Tk.\nReally wanting to test both SLOB and Hammerora I installed VirtualBox on the PC and started with Hammerora there. Graphs and numbers coming this week, but in general the results from last week was that ORION indicated stable and consistent better performance on the 3PAR with less erratic pattern. Hammerora actually reported better results from VBox than when running directly from host OS (Windows 7). Depending on how much idle time I\u0026rsquo;ll have the coming weeks I\u0026rsquo;ll try out SLOB, Swingbench and more trying to compare the two SANs.\n","permalink":"https://enesi.no/2012/05/oracle-load-testing-part-1/","summary":"\u003cp\u003eThe current customer is migrating from EVA storage to 3PAR, and inspired by some discussions on Twitter between @yvelikanov, @martinberx, @kevinclosson, @martinDBA, and others I decided to try out some free tools for load testing of Oracle servers. I can hardly tune a SQL statement without getting philosophical about it, so especially this first part is a bunch of thoughts I had when starting out.\u003c/p\u003e\n\u003cp\u003eThere are a lot of discussions on the \u003cstrong\u003eusability\u003c/strong\u003e of such tools; are the measured performance any close to the reality? The load these tools generate is more or less \u003cstrong\u003esynthetic\u003c/strong\u003e. One tool that stands out from the crowd is the Real Application Tool (RAT), but RAT requires an extra license from Oracle. With RAT you can capture real load on a production system and replay it on a test system with new or upgraded hardware or configuration. The reports generated in Enterprise Manager lets you compare the performance of old and new system so clearly that even your CEO can understand it. You cannot get much closer to reality than with RAT, but as said, it costs money and time to set it up. [And for a change I will not whine much more on the fact that SMBs around here do not invest in these licensed packages that makes the life of the DBA easier.]\u003c/p\u003e","title":"Oracle load testing - part 1"},{"content":"\u0026hellip; on a server in the sky\nI had two things on my agenda, test out HP Cloud and ThinkUp. HP Cloud is still in private beta, but I was lucky to get an invitation after registration. They aim to compete with Amazon Web Services (AWS) and so far I think they have a good chance. It is rather easy to just order and set up a new server in the cloud. A simple and easy interface with few buttons. It is tempting to include a few screen shots here, but I\u0026rsquo;m not sure if the agreement I just signed allows it. If they expand the interface with more features I hope they keep this simple version for impatient people who just need a server right now. I\u0026rsquo;ve had great fun with AWS also, but it is crowded with features and things to figure out. For my need of testing a web application the HP Cloud seems just perfect.\nThinkUp is a tool for you to analyze how you stand in your different social networks. It also has the added benefit that it actually does a backup of all your posts to your own private spot. Just go to their web site http://thinkupapp.com/ and check out the features. It does require a web server with php 5.3 and MySQL; a LAMP server in other words.\nIn short this is what you have to do:\nCreate a compute server with CentOS 6\nCreate the ssh-keys and test connection with ssh\nInstall packages for MySQL, Apache and PHP\nConfigure database, web server and php\nDownload ThinkUp and unpack it in the webserver\u0026rsquo;s public html directory\nAccess ThinkUp and start setting up connections to your social networks\nEnjoy the reports on your digital social life.\nI created the smallest version of a compute server (standard.xsmall) at HP Cloud with CentOS 6.2 Server as image. It comes with 1GB of memory and two partitions of 10GB and 30GB disk space. When the server is ready with status \u0026lsquo;running\u0026rsquo; CentOS is already installed with a root partition of 10GB. You don\u0026rsquo;t need more to run ThinkUp, but you have another 30GB partition available if you want to format and mount. First time you need to generate a key pair to be used with SSH when you want to connect to the server. This is the same mechanism that AWS uses and works perfectly on Unix/Linux and on Windows. Once generated you store the private key in a text file, not to be lost unless you want to scratch the servers that depends on them. Since I also have a Windows laptop, I had to make a version of this key that works with Putty. The tool PuTTYgen can be downloaded from the same place as Putty http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html . To convert to a key that Putty can use you select Conversions -\u0026gt; Import key, select the file with the key from HP Cloud and save the converted private key in a new file (do not overwrite the original key). I used SSH-2 RSA which seems to be the default.\nBefore I could connect to the server I had to attach a public IP. This is of course done from the management console at hpcloud.com.\nTo connect with Putty I entered the assigned public IP address, loaded the key file under Connection -\u0026gt; SSH -\u0026gt; Auth. Login as root and the connection is established without a password since you are using a private ssh key.\nFrom my MacBook I connected with:\nssh -i original_key_file ip-address The same works on Linux or most other mature OS.\nBefore ThinkUp can be installed you need an http server that supports php 5.3 or higher, that is, a LAMP server. I used this guide: http://library.linode.com/lamp-guides/centos-6 . It contains a link to a guide on how to set your hostname and timezone. (If you change the timezone, remember to repeat it after running \u0026lsquo;yum update\u0026rsquo;, since it resets the link /etc/localtime).\nDownload the ThinkUp software with\ncd /tmp wget http://thinkupapp.com/download/ This downloads the file zip-file, in my case thinkup_1.0.4.zip. Unpack it under the public_html directory, using foo.example.com as hostname hereafter:\ncd /srv/www/foo.example.com/public_html unzip /tmp/thinkup_1.0.4.zip With web server, PHP and MySQL up you are ready to configure ThinkUp. Open up a browser and go to http://foo.example.com/thinkup . You probably will see this:\nJust do what it says, as root:\nchown -R apache /srv/www/foo.example.com/public_html/thinkup/data/ Then hit F5 to reload page. Next screen will look like:\nHit the link in the page to configure ThinkUp for the first time. Various requirements are checked and in my case I was missing GD and PDO; whatever that is.\nHit the links to continue (See? You get everything almost free of efforts here). OK, maybe it was not obvious what to do get GD installed (hint: click Installation on next page if you want to know how). To install GD for CentOS I did:\nyum install php-gd PDO for PHP and MySQL is installed and configured with:\nyum install php-pdo yum install php-mysql Restart web server and MySQL after this, then reload page, the configuration continues to next page when all requirements are in place. On next page you create your ThinkUp account and configures the connection to the MySQL database. Before you can do that, create the database for ThinkUp and a user. Connect to MySQL as root:\nmysql -u root -p Create database and user with:\ncreate database thinkup; create user \u0026#39;thinkup\u0026#39;@\u0026#39;localhost\u0026#39; identified by \u0026#39;strongpassword\u0026#39; ; grant all on thinkup.* to \u0026#39;thinkup\u0026#39;; Then enter respective information on webpage. On the next page you will be presented by an error message saying that the apache user cannot write to config.inc.php. As usual you are told what to do. Execute this as the root:\nsudo touch /srv/www/foo.example.com/public_html/thinkup/config.inc.php sudo chown apache /srv/www/foo.example.com/public_html/thinkup/config.inc.php This is enough, you don\u0026rsquo;t actually have to copy and paste the script presented in the window below.\nHit next and you are finished with the installation.\nAfter you have verified your ThinkUp account (check your mail) you can login to your ThinkUp web application and start adding accounts for your social networks. Happy analyzing, remember it is about the network.\nWhen playing with this I once again understood that Twitter is the most important social network for me, I\u0026rsquo;m following people who shares a lot of useful information, usually quick quotes and links to good stuff. If you have attention deficit, Twitter is perfect, 140 chars and then time to think. It was through a few tweets I discovered that HP had opened their beta program. In another about ThinkUp as an alternative to Klout. Facebook is less important to me and I often forget to check it, it is just a collection of faces\u0026hellip; Google+ is used by some of my Oracle peers, but it is just much easier to check Twitter when I\u0026rsquo;m on the run. Links to good stuff is either checked out immediately or sent to Read It Later.\nI\u0026rsquo;ve met quite a few that are frustrated by the tardiness of many IT departments, getting a new server takes too much time. Having the possibility to create one in the cloud, in minutes, from a easy to use web interface such as what HP Cloud offers; that is something many will consider.\n","permalink":"https://enesi.no/2012/04/analyze-your-social-networks-with-thinkup/","summary":"\u003cp\u003e\u0026hellip; on a server in the sky\u003c/p\u003e\n\u003cp\u003eI had two things on my agenda, test out HP Cloud and ThinkUp. HP Cloud is still in private beta, but I was lucky to get an invitation after registration. They aim to compete with Amazon Web Services (AWS) and so far I think they have a good chance. It is rather easy to just order and set up a new server in the cloud. A simple and easy interface with few buttons. It is tempting to include a few screen shots here, but I\u0026rsquo;m not sure if the agreement I just signed allows it. If they expand the interface with more features I hope they keep this simple version for impatient people who just need a server right now. I\u0026rsquo;ve had great fun with AWS also, but it is crowded with features and things to figure out. For my need of testing a web application the HP Cloud seems just perfect.\u003c/p\u003e","title":"Analyze your social networks with ThinkUp"},{"content":"Perl is included with the Oracle database software (SE and EE), even on Windows. I wrote this to extract the SQL statements from a trace file (generated with sql_trace=true or setting the 10046 event). A simple indentation - one tab for each level - is used to show recursive statements.\nwhile (\u0026lt;\u0026gt;) { if(/^PARSING IN CURSOR/) { ($level)=$_=~/\\\\s+dep=(\\\\d+)\\\\s+/ ; $line=\u0026lt;\u0026gt;; while ($line!~/END OF STMT/){ for($i=0;$i\u0026lt;$level;$i++) { print \u0026#34;\\\\t\u0026#34; ; } print $line; $line=\u0026lt;\u0026gt;; } print \u0026#34;\\\\n\u0026#34; ; } } You usually have to expand your PATH on Windows to find Perl; you\u0026rsquo;ll find Perl.exe somewhere below %ORACLE_HOME%. The code will of course work on Linux and other OS where you have Perl. Store the code above in a file called xtract.pl and call it with:\nperl xtract.pl your_trace_file.trc | more There is an option (record=filename.sql) in tkprof to extract the SQL, but it does not include recursive statements. I guess the difference is that this is something to build on when you have a more complex task.\n","permalink":"https://enesi.no/2012/04/extract-sql-from-trace-file-with-perl/","summary":"\u003cp\u003ePerl is included with the Oracle database software (SE and EE), even on Windows. I wrote this to extract the SQL statements from a trace file (generated with sql_trace=true or setting the 10046 event). A simple indentation - one tab for each level - is used to show recursive statements.\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \nwhile (\u0026lt;\u0026gt;) {  \n  if(/^PARSING IN CURSOR/) {  \n    ($level)=$_=~/\\\\s+dep=(\\\\d+)\\\\s+/ ;  \n    $line=\u0026lt;\u0026gt;;  \n    while ($line!~/END OF STMT/){  \n      for($i=0;$i\u0026lt;$level;$i++) {  \n           print \u0026#34;\\\\t\u0026#34; ;  \n      }  \n      print $line;  \n      $line=\u0026lt;\u0026gt;;  \n    }  \n    print \u0026#34;\\\\n\u0026#34; ;  \n  }  \n}  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eYou usually have to expand your PATH on Windows to find Perl; you\u0026rsquo;ll find Perl.exe somewhere below %ORACLE_HOME%. The code will of course work on Linux and other OS where you have Perl. Store the code above in a file called xtract.pl and call it with:\u003c/p\u003e","title":"Extract SQL from trace file with Perl"},{"content":" The last day started with a master class Key features of redo by Jonathan Lewis. You need quite a good reputation to gather a large crowd to listen on such a subject. Well prepared with good slides, lots of real-world experience together with questions from the crowd made it to a time well spent in the auditorium. He didn\u0026rsquo;t get through all the slides, but that did not matter since we had two hours filled with entertaining learning.\nAt 11:30am I attended the presentation Falling in Love all over again - OEM 12c Performance Page Enhancements; you can guess by reading the title who gave the presentation. Clearly this was a presentation Doug loved to give, and he praised the new improved functionality and interface. He made some comparison to other methods of tuning/analyzing a performance problem; a subject I still find interesting. The presentation was for the most part a live demo with SwingBench and virtual machines, no Death of PowerPoint after three hours of sleep. The only problem I have with these features is that many customers around here do not purchase the managements packs needed for all the fun. But I hope more people will understand the necessity of having this software. I had to leave the presentation early because of a board meeting, really a lunch with the sponsors. Doug, if you read this, now you know why two of us left early, not because you did less than splendid.\nAfter lunch there was a talkshow with the main sponsors and with Jonathan Lewis, Maria Colgan, Stephan Janssen, and VP of EMEA Andrew Sutherland, hosted by board member Alice Rossman. Alice does this very well, by now it has probably dawned on her that she will have to do this every year. I like to hear stars in the Oracle sphere talk about what matters outside the office. Andrew Sutherland was the other Scottish funny guy. Think we should make sure that we have at least two of them every year, it is something with their humor; their special love for the Brits and that accent.\nStaying up late with good company several nights in a row makes you tired. I attended only one session after lunch. Using PL/SQL Hierarchical Performance Profiler with Bryn Llewelyn. I\u0026rsquo;m not writing much code these days, but knew this is useful and something I want to know about when I need it. Bryn speaks very clearly in his presentations and easily engage in discussions afterward.\nThe feedback we received throughout the conference was vey nice, the level of the sessions and choice of content. Making a program like this is much more than sending emails to a lot of famous people. We need to find a variation on subjects, verify against feedback from last year and also have content that pushes the members forward. Making the program in itself takes a lot of effort, but clearly paid off.\nThis year we had hands-on labs. Not all where successful, others had a great turn out like RAC Attack. We learned a lot from this, and I believe that a one day conference in Oslo before the main conference will be repeated. We also invested in the new Java track; it was a good start, made some new ground we can build on further.\nI had a great conference and feel pride that so many foreign speakers want to come and present at OUGN. Since we are not scaling back I expect to see most of them again together with other genius. We might have some logistic challenges like a larger ship.\nPersonal ambitions for next year is convincing the rest of the board that we need to find some way to have Cary Millsap from Method R, and one or more from Pythian (like Gwen, Alex and Yury). We also need to encourage more Norwegians to present user cases.\nFinally, I want to thank my employer Keystep Consulting for letting me spend many hours otherwise billable in order to prepare for the conference. I am proud of the team there and being part of it.\n","permalink":"https://enesi.no/2012/03/ougn-2012-day-2-wrap-up/","summary":"\u003cp\u003e\u003ca href=\"/images/blogger/ougn-2012-day-2-wrap-up-1.jpg\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/ougn-2012-day-2-wrap-up-1.jpg\"\u003e\u003c/a\u003e\n\u003cbr\u003e\nThe last day started with a master class Key features of redo by Jonathan Lewis. You need quite a good reputation to gather a large crowd to listen on such a subject. Well prepared with good slides, lots of real-world experience together with questions from the crowd made it to a time well spent in the auditorium. He didn\u0026rsquo;t get through all the slides, but that did not matter since we had two hours filled with entertaining learning.\u003c/p\u003e","title":"OUGN 2012 Day 2 - Wrap up"},{"content":" This is my first conference as a board member, and going to a presentation for every slot is not possible. I did go to Doug Burns presentation on SQL Plan Management in 11g though I had actually seen it at OOW 2011, probably because I like Doug\u0026rsquo;s presentation style. This preso does not go into very much detail on SPM, but takes us on a travel through the subject and leave it to us to actually get some experience on it. It does look interesting, and the only thing that keeps me from getting much experience on it is that the current customer is lagging behind on the upgrades to 11g; the few we have on 11g are small and have not presented any problems with plan instability.\nMaria Colgan\u0026rsquo;s presentation on how to collect statistics was a big hit. A colleague stated this was perhaps the best presentation he had been to ever. Very entertaining presentation and packed with useful take aways. Yes, I did go to the same presentation at OOW 2011 as if I\u0026rsquo;m unwilling to expose my brains to something new. But in fact I learned something new, and I still have something to try out, for example next time I encounter a database that uses partitioning, sigh.\nThe RAC Attack team set up in the public area and a couple of enthusiasts insisted to complete the lab from Day 0. The intention was not only to have a lab on board, but also a place where people could come and go, and ask questions on RAC in particular and about life in general. Think it worked well, but obviously the hands-on lab was competing with the sessions, and most concluded that lab was something they could try at home later.\nThe feedback we got after Day 1 was great, but I\u0026rsquo;ll save that to the wrap-up.\n","permalink":"https://enesi.no/2012/03/ougn-2012-day-1-at-sea/","summary":"\u003cp\u003e\u003ca href=\"/images/blogger/ougn-2012-day-1-at-sea-1.jpg\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/ougn-2012-day-1-at-sea-1.jpg\"\u003e\u003c/a\u003e\n This is my first conference as a board member, and going to a presentation for every slot is not possible. I did go to Doug Burns presentation on SQL Plan Management in 11g though I had actually seen it at OOW 2011, probably because I like Doug\u0026rsquo;s presentation style. This preso does not go into very much detail on SPM, but takes us on a travel through the subject and leave it to us to actually get some experience on it. It does look interesting, and the only thing that keeps me from getting much experience on it is that the current customer is lagging behind on the upgrades to 11g; the few we have on 11g are small and have not presented any problems with plan instability.\u003c/p\u003e","title":"OUGN 2012 Day 1 - At sea"},{"content":" 23:48, so tired that a Twitter message would be more appropriate. But, just wanted to make it for the record that today turned out to be a very pleasant day. I had the responsibility to host track #5, which started with Martin Büchi on Information Lifecycle Management in OLTP DBs with Partitioning. ILM is a subject that many postpone because it is too complicated. In Martin\u0026rsquo;s presentation he showed a reference model, illustrated various scenarios with alternative solutions. He then showed how partitioning can be used to archive data that has been aged out. I cannot possible make justice to his presentation, but since I write about it here I\u0026rsquo;m sure I\u0026rsquo;ll remember to come back to it, download it and digest it. If Mr Büchi reads this, he has it on record that he did well. Hope he\u0026rsquo;ll be back again.\nTwo hours later, two other Martins took the stage. Martin Paul Nash and Martin Bach introduced RAC Attack for the first time in Norway. I\u0026rsquo;ve been excited about RAC Attack since I took the courage to recommend it for the OUGN Board, and after getting the approval, inviting them over. They did not disappoint at all! The turnout was quite good; 20 people filled the room and started to hack on VirtualBox, VMware and Oracle. After lunch a few wearied off but the rest continued to the end. We where the last to leave the conference area and we got plenty of positive feedback on this. I think I\u0026rsquo;ve been tweeting enough on #RACATTACK lately, it was a great experience.\nActually OUGN was blessed by a fourth Martin, Martin Widlake. I couldn\u0026rsquo;t go to his presentation, but head counts and feedback showed his presentation went very well. I did have the pleasure to talk to him during the dinner later. The picture of the Martins where taken at Holmenkollen just before the dinner. Martin\u0026rsquo;s smile on his face pretty much sums up the mood after Day 0. Loved it.\n","permalink":"https://enesi.no/2012/03/ougn-2012-day-0-the-martins/","summary":"\u003cp\u003e\u003ca href=\"/images/blogger/ougn-2012-day-0-martins-1.jpg\"\u003e\u003cimg loading=\"lazy\" src=\"/images/blogger/ougn-2012-day-0-martins-1.jpg\"\u003e\u003c/a\u003e\n\u003cbr\u003e\n23:48, so tired that a Twitter message would be more appropriate. But, just wanted to make it for the record that today turned out to be a very pleasant day. I had the responsibility to host track #5, which started with Martin Büchi on Information Lifecycle Management in OLTP DBs with Partitioning. ILM is a subject that many postpone because it is too complicated. In Martin\u0026rsquo;s presentation he showed a reference model, illustrated various scenarios with alternative solutions. He then showed how partitioning can be used to archive data that has been aged out. I cannot possible make justice to his presentation, but since I write about it here I\u0026rsquo;m sure I\u0026rsquo;ll remember to come back to it, download it and digest it. If Mr Büchi reads this, he has it on record that he did well. Hope he\u0026rsquo;ll be back again.\u003c/p\u003e","title":"OUGN 2012 Day 0 - The Martins"},{"content":"Tomorrow is the big day. For the last 4 years our annual conference has been on a cruise ship from Oslo to Kiel and back. That is two days with presentations, fun and stuff. A big success we are not going to change at least this year. But, for this conference, we sent out too many invitations to Very Important Presenters and got an impressive positive response; we run out of slots pretty fast. (Let\u0026rsquo;s say we are not really #1 when it comes to logistics in this country.) So we decided to extend the conference with one day in Oslo before we board the ship on Thursday. Someone called it a pre-conference, which is unfair to the presenters since they are giving real meat tomorrow. Someone called it a whole-day conference, eh, as opposed to the other days? Anyway. tomorrow we\u0026rsquo;ll have lots of hands-on labs, master classes and what I\u0026rsquo;m really excited about, RAC Attack.\nTonight we met at the top floor of the conference hotel as some kind of pre-conference beer meetup. I suspect the guys that showed up didn\u0026rsquo;t really know they took part in a meetup, they just went to the nearest bar with the coolest view in the city. As always, it was nice to meet some of the invited guests: Maria Colgan, Christian Shay, Kuassi Mensah, Bryn Llewelyn, and Holger Friedrich, together with the rest and beers.\nSome came with a late flight from UK, like the two Martins responsible for RAC Attack. Not sure where Doug is, he is not on tomorrow AFAIK, but he will be very welcome when he shows up. Btw, Maria told me they have the same humour in the presentations. I agree, those two can fool anyone into studying statistics.\nMore details on the conference here: http://ougn.no/program-2012 . Let the party begin.\n","permalink":"https://enesi.no/2012/03/ougn-2012-day-1/","summary":"\u003cp\u003eTomorrow is the big day. For the last 4 years our annual conference has been on a cruise ship from Oslo to Kiel and back. That is two days with presentations, fun and stuff. A big success we are not going to change at least this year. But, for this conference, we sent out too many invitations to Very Important Presenters and got an impressive positive response; we run out of slots pretty fast. (Let\u0026rsquo;s say we are not really #1 when it comes to logistics in this country.) So we decided to extend the conference with one day in Oslo before we board the ship on Thursday. Someone called it a pre-conference, which is unfair to the presenters since they are giving real meat tomorrow. Someone called it a whole-day conference, eh, as opposed to the other days? Anyway. tomorrow we\u0026rsquo;ll have lots of hands-on labs, master classes and what I\u0026rsquo;m really excited about, RAC Attack.\u003c/p\u003e","title":"OUGN 2012 Day -1"},{"content":"There is a bug in the export utility on version 11.2.0.1 when running against a 11.2.0.2 (and possiby 11.2.0.1) database that has tables without segments. Deferred segment creation was introduced as a smart feature in 11gR2, but this is causing error EXP-00003 when running the old export utility from lower versions. This is reported on MOS for pre-11g clients, but right now I could not find any reports that this is also a problem with the 11.2.0.1 version when this feature was introduced.\nThis is a simple test to reproduce the error. Create a couple of tables in an 11gR2 database:\ncreate table empty (id number, foo varchar2(2000) ); create table aaa (id number, foo varchar2(10)); insert into aaa values (42,\u0026#39;asdfas\u0026#39;); commit; select table_name,segment_created from user_tables ; TABLE_NAME SEG \\------------------------------ --- EMPTY NO AAA YES Then perform a simple export from a 11.2.0.1 client:\nexp oyise/oracle@orcl file=oyise.dmp log=oyise.log Excerpt from the output / logfile:\n. about to export OYISE\u0026#39;s tables via Conventional Path ... . . exporting table AAA 1 rows exported . . exporting table EMPTY EXP-00003: no storage definition found for segment(0, 0) (For export to succeed with one table, create it with a name alphabetically before the EMPTY table).\nNote, if the X,Y in \u0026ldquo;no storage definition found for segment(X,Y)\u0026rdquo; is non-zero, you have another issue that you probably find a solution for on Oracle Support, e.g. when segment owner \u0026lt;\u0026gt; table owner.\nSolution: Upgrade client or allocate one extent for the table that fails:\nalter table empty allocate extent; You may consider set the parameter DEFERRED_SEGMENT_CREATION to FALSE in case more empty tables will be created.\nAlso this error does not happen on Data Pump export 11.2.0.1.\n","permalink":"https://enesi.no/2011/12/exp-00003/","summary":"\u003cp\u003eThere is a bug in the export utility on version 11.2.0.1 when running against a 11.2.0.2 (and possiby 11.2.0.1) database that has tables without segments. Deferred segment creation was introduced as a smart feature in 11gR2, but this is causing error EXP-00003 when running the old export utility from lower versions. This is reported on MOS for pre-11g clients, but right now I could not find any reports that this is also a problem with the 11.2.0.1 version when this feature was introduced.\u003c/p\u003e","title":"EXP-00003"},{"content":"Three family members invited me to play Wordfeud with them this Christmas. At the end of such a game losing badly to my sister-in-law and with only a few letters left I was thinking about how I could find candidate words using regular expression in SQL.\nFirst I found a list of Norwegian words on the net. I should have taken the effort to get hold of the same word list as Wordfeud is using. The list was loaded into a table using SQL Developer.\nSay you want to see if there is any word that start with N and uses only the letters C, F, S, and A. The following SQL may find it:\nselect word from norwegians where regexp_substr(word,\u0026#39;^n\\[cfsa\\]+$\u0026#39;) is not null ; For those not familiar with regular expressions, well, just Google or check out any good book on Perl. In short, the ^ and $ means the beginning and end of line (word in this case) respectively, [] is used to create a group of letters; [cfsa] means any letter of c, f, s, and a; + means one or more occurrences.\nNow I only need to make a mobile interface to this, and create a function that calculate the score for each word based on Wordfeud\u0026rsquo;s rules. Next Christmas, maybe. I do know there are apps ready to suggest this, but that is not half as fun :-)\n","permalink":"https://enesi.no/2011/12/wordfeud-and-regular-expression-in-sql/","summary":"\u003cp\u003eThree family members invited me to play \u003ca href=\"http://wordfeud.com/\" target=\"_blank\" rel=\"noopener\"\u003eWordfeud\u003c/a\u003e\n with them this Christmas. At the end of such a game losing badly to my sister-in-law and with only a few letters left I was thinking about how I could find candidate words using regular expression in SQL.\u003c/p\u003e\n\u003cp\u003eFirst I found a list of Norwegian words on the net. I should have taken the effort to get hold of the same word list as Wordfeud is using. The list was loaded into a table using SQL Developer.\u003c/p\u003e","title":"Wordfeud and regular expression in SQL"},{"content":"I\u0026rsquo;m a big fan of Method R, both the company and the way to optimize SQL (or any other process that can possibly be measured). The method is explained in details in their classic book Optimizing Oracle Performance . But it is hard to practice method R without a good profiler. In every installation of Oracle database the profiler tkprof is included and it does serve for some problems, but hides a lot of information from you.\nWhen I have a tasks that runs for a few hours I enable extended Oracle trace from start to end, avoiding as best as possible anything outside the time interval for the task itself, meaning I coordinate the tracing with the end user. When I have the trace file ready I check it briefly to make sure I don\u0026rsquo;t have some rubbish before or after the time window. Especially on Windows it is important to check that the tracefile has not already been written to by a previous session, may be it is the same on Unix, I haven\u0026rsquo;t checked since I\u0026rsquo;m in Windows land for the moment. I use two free tools that create a profile in html-format:\nOracle Session Resource Profiler (OraSRP) by Egor Starostin and TVD$XTAT by Christian Antognini , the author of the book Troubleshooting Oracle Performance . In that book he describes his profiler that you can download from his site. Excellent book by the way.\nI often use both profilers because I like to compare the results. They usually agree surprisingly well, the only difference is what is considered unaccounted for time. Whenever unaccounted for time is large I check the tracefile again to see if there is a long period of time spent waiting for the client at the end. Sometimes the missing time is located inside the session, and at one occasion when I was stuck I bought a license of mrls that parse the tracefile and makes it easy to track down where this happens.\nIn addition to create an overall profile for your session they have profiles for individual events, OraSRP is especially good at this. This is interesting because it adds another level. One of the tricks you encounter on the web is to increase the parameter DB_FILE_MULTIBLOCK_READ_COUNT. This parameter decides how much Oracle can try to fetch in each call when reading many blocks at a time, typically db file scattered read or direct path read usually when doing table scans as opposed to single key search in an index. Reading 128 at once must be better than 16 that is often the default, but this profile will in some cases show you that Oracle cannot read a maximum number of blocks for each read call. In one query I was working on the event db file scattered read contributed to 51% of the response time, but a profile on this event showed that the number of blocks pr read call was almost flat with a variation from 1 block pr call (7.1%), 128 blocks pr call (1.0%) to 73 blocks pr call (0.1%). The benefit of increasing the parameter was not as much as I hoped for, and the solution was found elsewhere.\nOne of the important points with profiling is to discover where not to waste time. You might suspect a full table scan to be the problem (because some people believe FTS is always bad), but large tables can be scanned really fast these days. With tracing and profiling you don\u0026rsquo;t have to guess, you measure and draw a conclusion. There are so many things that can go wrong in a complex system, if you can measure and do simple math you are much more likely to reach your goal early and know that you actually have reached it than when you apply all those tricks and guesswork.\n\u0026ldquo;Filter early\u0026rdquo; is important when optimizing, with profiling you filter out early where you don\u0026rsquo;t want to apply the focus of your brain.\n","permalink":"https://enesi.no/2011/11/two-free-profilers-for-10046-trace-files/","summary":"\u003cp\u003eI\u0026rsquo;m a big fan of Method R, both the \u003ca href=\"http://www.method-r.com/\" target=\"_blank\" rel=\"noopener\"\u003ecompany\u003c/a\u003e\n and the way to \u003ca href=\"http://method-r.com/faq/31-method-r/44-can-you-explain-method-r\" target=\"_blank\" rel=\"noopener\"\u003eoptimize\u003c/a\u003e\n SQL (or any other process that can possibly be measured). The method is explained in details in their classic book \u003ca href=\"http://www.amazon.com/gp/product/059600527X?ie=UTF8\u0026amp;tag=methodrcom-20\u0026amp;linkCode=as2\u0026amp;camp=1789\u0026amp;creative=9325\u0026amp;creativeASIN=059600527X\" target=\"_blank\" rel=\"noopener\"\u003eOptimizing Oracle Performance\u003c/a\u003e\n. But it is hard to practice method R without a good profiler. In every installation of Oracle database the profiler tkprof is included and it does serve for some problems, but hides a lot of information from you.\u003c/p\u003e","title":"Two free profilers for 10046 trace files"},{"content":"OOW has been great and I\u0026rsquo;m reaching the limit of how much information I can consume for a while. I\u0026rsquo;ll probably spend some weeks or months reading and understanding presentations. Time to go home and practice what I\u0026rsquo;ve learned. I\u0026rsquo;m not even going to try to report dutifully on every session today. Just say that I\u0026rsquo;ve been to Real-World Performance Questions and Answers, Oracle Database Optimizer: Tips for Preventing Suboptimal Execution Plans, Looking Under the Hood at Oracle Clusterware 11g Release 2, and Under the Hood of Oracle Automatic Storage Management: Fault Tolerance. The first one was great fun, 3 of 4 had more than 20 years experience in this field. The youngest \u0026ldquo;only\u0026rdquo; 8. Greg Rahn revealed they have a \u0026ldquo;secret\u0026rdquo;: they try to solve complex problems with simple solutions. Graham Woods advised people to ask Why? and understand what you are doing before you actually change anything in production. Simple guidelines, but since many are not following them it needs to be repeated.\nI\u0026rsquo;m not going to say much about the two following, again useful content and I have to look again at the mysteries of Clusterware. Also Maria Colgan showed as she did yesterday she is an excellent speaker. She is invited to next year\u0026rsquo;s OUGN conference, which according to Bryn is the world\u0026rsquo;s best user group conference. Looking forward to it.\nBack at home I\u0026rsquo;ve had many discussions regarding ASM, it is surprising what people can get religious about. One former coworker stated that Oracle has no right to implement a volume manager\u0026hellip; I\u0026rsquo;ve never had any good arguments for migrating to ASM. If the DBAs at the site like it, I chime in with things I like, simplicity to name one. If not, because \u0026ldquo;any datafile should be verifiable with ls or dir\u0026rdquo;, then I usually back off from any discussion. In the last presentation of the conference by Alex Gorbatchev from Pythian I learned quite a few reasons to go for ASM, like you have more integrity checks and efficiency if you use ASM. I\u0026rsquo;ll probably stay away from heated discussion until I\u0026rsquo;ve read everyting from James Morle, baarf.org, ASM-gurus like Alex; SAN- and Unix-admins don\u0026rsquo;t give up easily.\nThis was first time at OOW. Was it worth it ? Absolutely. I got served more than I could eat, met a lot of smart people whose blog posts and tweets I\u0026rsquo;ve been reading for years; i.e. great networking opportunity, and had great fun. Learned a lot, now I know what other SSF-travelers are talking about.\nThanks to everybody who stopped for a talk or sent a smile in my general direction. Going home looking forward to see wife \u0026amp; jr.\n","permalink":"https://enesi.no/2011/10/oow-2011-day-5/","summary":"\u003cp\u003eOOW has been great and I\u0026rsquo;m reaching the limit of how much information I can consume for a while. I\u0026rsquo;ll probably spend some weeks or months reading and understanding presentations. Time to go home and practice what I\u0026rsquo;ve learned. I\u0026rsquo;m not even going to try to report dutifully on every session today. Just say that I\u0026rsquo;ve been to \u003cem\u003eReal-World Performance Questions and Answers\u003c/em\u003e, \u003cem\u003eOracle Database Optimizer: Tips for Preventing Suboptimal Execution Plans\u003c/em\u003e, \u003cem\u003eLooking Under the Hood at Oracle Clusterware 11g Release 2\u003c/em\u003e, and \u003cem\u003eUnder the Hood of Oracle Automatic Storage Management: Fault Tolerance\u003c/em\u003e. The first one was great fun, 3 of 4 had more than 20 years experience in this field. The youngest \u0026ldquo;only\u0026rdquo; 8. Greg Rahn revealed they have a \u0026ldquo;secret\u0026rdquo;: they try to solve complex problems with simple solutions. Graham Woods advised people to ask \u003cem\u003eWhy?\u003c/em\u003e and understand \u003cem\u003ewhat\u003c/em\u003e you are doing before you actually change anything in production. Simple guidelines, but since many are not following them it needs to be repeated.\u003c/p\u003e","title":"OOW 2011 Day 5"},{"content":"Day four started well with Oracle Optimizer: Best Practices for Managing Optimizer Statistics presented by Maria Colgan. Then followed SQL Tuning Expert Roundtable with Jagan Athreya, Benoit Dageville, Tom Kyte and Graham Wood. These two presentations had a lot in common, correct statistics should be checked if plan is wrong, note correct, but not necessarily updated statistics. If an automatic routine for stats gathering cannot be found the metadata for the table and columns should be set manually and locked. More people talk about skews this day, it was mentioned here also, but I guess James Morle and Cary has more to say on that. The problem with histograms, bind variables and bind peeking is somewhat easier in 11g with adaptive cursor sharing, but not a 100% solution according to the panel. One 11g feature I\u0026rsquo;m going to look more into is extended statistics.\nThe third and last presentation for me was Powerful New Ways to Use Oracle Data Guard for Planned Maintenance with Larry Carpenter. This stuff requires a lot of consideration and planning. Downtime, complexity and risk. Finding a solution that lowers especially the last two is important. Another presentation I have to digest.\nAt 17:00 I went to Pythian\u0026rsquo;s bloggers meetup . Met a lot of guys IRL that I\u0026rsquo;ve been following on Twitter and whose blogs I\u0026rsquo;ve been reading for years. Impressive event and very social. Pythian just ranked even higher. Tim Hall won the prize for having collected most signatures, my collection in the picture. I was too busy talking to people, but got a fair share of signatures. Loved it all.\n","permalink":"https://enesi.no/2011/10/oow-2011-day-4-and-bloggers-meetup/","summary":"\u003cp\u003eDay four started well with \u003cem\u003eOracle Optimizer: Best Practices for Managing Optimizer Statistics\u003c/em\u003e presented by Maria Colgan. Then followed \u003cem\u003eSQL Tuning Expert Roundtable\u003c/em\u003e with Jagan Athreya, Benoit Dageville, Tom Kyte and Graham Wood. These two presentations had a lot in common, correct statistics should be checked if plan is wrong, note correct, but not necessarily updated statistics. If an automatic routine for stats gathering cannot be found the metadata for the table and columns should be set manually and locked. More people talk about skews this day, it was mentioned here also, but I guess James Morle and Cary has more to say on that. The problem with histograms, bind variables and bind peeking is somewhat easier in 11g with adaptive cursor sharing, but not a 100% solution according to the panel. One 11g feature I\u0026rsquo;m going to look more into is extended statistics.\u003c/p\u003e","title":"OOW 2011 Day 4 and Blogger's meetup"},{"content":"Went to five presentations, two of them stood out. Doug Burn spoke about SQL plan management in 11g and I learned some new tricks. The material and the content was quite good, also I like these comments that may not be in the slides, e.g. he quoted Jonathan Lewis who said that if you have a narrow problem you should apply a narrow solution, an advice against changing a system wide parameter just because you have one SQL statement gone astray.\nThe other presentation I liked a lot was about mining the AWR data with SQL, by Yury Velikanov from Pythian. The presentation was packed with useful examples and based on experience he had with customers. When he started almost all seats were taken, and then some people left, maybe it was over their head, but for me, I like these kind of presentations. Yes, it is cool with fancy colorful slides where rule number one is max six words pr slide, but I don\u0026rsquo;t mind seeing code in the slides once in a while. I think that if you strive to understand the AWR repository and analyze it the way Yury did you will have a better understanding on how things are connected, in a way you cannot have when looking on nice graphs in EM. Also you see the limitations in AWR, most data are aggregated which creates other limitations; see Cary\u0026rsquo;s presentation about skew. He made a point in the beginning that there are different concepts to use when attacking optimization and troubleshooting, he gave pointers to tools and methods. All in all a presentation that I have to download and study for a while.\nAnother great day.\n","permalink":"https://enesi.no/2011/10/oow-2011-day-3/","summary":"\u003cp\u003eWent to five presentations, two of them stood out. Doug Burn spoke about SQL plan management in 11g and I learned some new tricks. The material and the content was quite good, also I like these comments that may not be in the slides, e.g. he quoted Jonathan Lewis who said that if you have a narrow problem you should apply a narrow solution, an advice against changing a system wide parameter just because you have one SQL statement gone astray.\u003c/p\u003e","title":"OOW 2011 Day 3"},{"content":"Went to the keynote and five presentations. Keynote was a lot of show, but not much to bring home. Two presentations regarding database migration assistant for unicode, and Real Application Testing were relevant, but so uninspiring and lacked the extra stuff that it was like attending someone reading from the manual. Not much of a take away there either. Also went to a presentation on PL/SQL - Divide and Conquer, modularizing your code. I had hoped to hear more from Bryn Llewellyn, Oracle, but it was really Martin Büchi from Avaloq that had the whole show. Lots of usefull info, interesting real life case on how to structure enormous amount of code. Will remember this next time I get a chance to work with developers and have to look into refactoring of PL/SQL. It does seem to be something one can apply elsewhere in any large environment. Bryn is an excellent speaker, by the way. The two were clear that they did not want to sell consulting or anything else, just to share some goals and passion. I liked that, Bryn talks in way that could fool any DBA into write some code; wish he had a bigger part of it.\nNext presentation was of course the highlight of the day. Cary Millsap talked about skew. I\u0026rsquo;ve heard some of this before, but again there was something new to learn and reconsider. A reminder of how skew occurs in many levels and groupings and is really the reason why he and most of us has a job. (Just that Method R gets the job done sooner, I think). Cary is the best presenter I can think of in the Oracle world, and made the room laugh many times at a potentially boring subject. Two of my friends heard him speak for the first time and was impressed. One of them considered to skip the remaining presentations for the day so the day would have a good ending. Couldn\u0026rsquo;t have made a better compliment myself.\nI finished of with a presentation regarding best practices for Oracle on Windows. I\u0026rsquo;m not much a fan of best practices, but this was a full packed check-list on what can go wrong on Windows; a unix/linux guy would suspect that there is enough to keep you busy. Very useful to use later since I have been sent to windows land on an assignment for 6-12 months.\nI also met briefly Dan Norris from Oracle and Alex Gorbatchev from Pythian; two guys I\u0026rsquo;ve been following on Twitter for a while. Always great to see people IRL once in a while (MOW 2011 and Hotsos 2008 in that order). These guys remind me why I like working with this (the database), people outside, especially the open source java cool coding guys, wonder how can one have a life at all working with a comercial database that is anything than simple and inexpensive. Lots of smart people work with this and create an atmosphere beside the work - e.g. OOW has become a great event. Since they don\u0026rsquo;t compete in my market it does not cost much to write this; but Method R and Pythian are two companies I admire; they create and share a lot with others, and appearantly have lots of fun at work and outside.\nNow breakfast, no time to spell check. Love my new android tablet, by the way.\nUpdate:\n-Breakfast over, some spell checking done.\n-Power tip for the Android tablet: Check the Wi-Fi disconnection policy, set it to disconnect when screen is turned off; battery lasts much longer.\n-I remember that I met a smart guy on Rac Attack yesterday; Martin Nash. From UK, good for us we have such experts we can ask to come over to OUGN.\n","permalink":"https://enesi.no/2011/10/oow-2011-day-2/","summary":"\u003cp\u003eWent to the keynote and five presentations. Keynote was a lot of show, but not much to bring home. Two presentations regarding database migration assistant for unicode, and Real Application Testing were relevant, but so uninspiring and lacked the extra stuff that it was like attending someone reading from the manual. Not much of a take away there either. Also went to a presentation on PL/SQL - Divide and Conquer, modularizing your code. I had hoped to hear more from Bryn Llewellyn, Oracle, but it was really Martin Büchi from Avaloq that had the whole show. Lots of usefull info, interesting real life case on how to structure enormous amount of code. Will remember this next time I get a chance to work with developers and have to look into refactoring of PL/SQL. It does seem to be something one can apply elsewhere in any large environment. Bryn is an excellent speaker, by the way. The two were clear that they did not want to sell consulting or anything else, just to share some goals and passion. I liked that, Bryn talks in way that could fool any DBA into write some code; wish he had a bigger part of it.\u003c/p\u003e","title":"OOW 2011 Day 2"},{"content":"First day at OOW 2011 started with interesting presentations arranged by IOUG, very usefull stuff and I have to read the presentations again to get all the details regarding Oracle on VMware, and deploying APEX with security in mind.\nHighlight of the day was OUGN\u0026rsquo;s encounter for Norwegians at OOW 2011 and invited guests (invited VIP speakers for next years OUGN 2012 conference). This took place at The Norwegian Church Abroad . The church has a spectacular view over the city. Among the guests were Jonathan Lewis and Cary Millsap, two of my heroes in the Orace World. I had a long conversation with Cary regarding work, relationship between DBAs and developers, something we both have thought a lot about . I also tried to convince him to come next year and speak at OUGN 2012. Hopefully we can arrange for one of his one day classes a day before the conference.\n","permalink":"https://enesi.no/2011/10/oow-2011-day-1/","summary":"\u003cp\u003eFirst day at OOW 2011 started with interesting presentations arranged by IOUG, very usefull stuff and I have to read the presentations again to get all the details regarding Oracle on VMware, and deploying APEX with security in mind.\u003c/p\u003e\n\u003cp\u003eHighlight of the day was OUGN\u0026rsquo;s encounter for Norwegians at OOW 2011 and invited guests (invited VIP speakers for next years OUGN 2012 conference). This took place at \u003ca href=\"http://www.sjomannskirken.no/english\" target=\"_blank\" rel=\"noopener\"\u003eThe Norwegian Church Abroad\u003c/a\u003e\n. The church has a spectacular view over the city. Among the guests were Jonathan Lewis and Cary Millsap, two of my heroes in the Orace World. I had a long conversation with Cary regarding work, relationship between DBAs and developers, something we both have thought a lot about . I also tried to convince him to come next year and speak at OUGN 2012. Hopefully we can arrange for one of his one day classes a day before the conference.\u003c/p\u003e","title":"OOW 2011 Day 1"},{"content":"This year when I was on my way to MOW 2011 in Denmark, a few months after I started at Keystep, my manager called me and asked if I wanted to go to Oracle Open World. That has not happened before in any other job I\u0026rsquo;ve had and of course I said yes, so here I am in San Francisco. Arrived late Friday, and spent Saturday adjusting to new timezone and walk around. First observations: really nice city, want to bring my family and see more. Also lots of nice people, with some exceptions from those driving car through the crossing of Lombart street and Hyde Street; they were pointing to the sky with wrong finger.\n","permalink":"https://enesi.no/2011/10/oow-2011-day-0/","summary":"\u003cp\u003eThis year when I was on my way to MOW 2011 in Denmark, a few months after I started at Keystep, my manager called me and asked if I wanted to go to Oracle Open World. That has not happened before in any other job I\u0026rsquo;ve had and of course I said yes, so here I am in San Francisco. Arrived late Friday, and spent Saturday adjusting to new timezone and walk around. First observations: really nice city, want to bring my family and see more. Also lots of nice people, with some exceptions from those driving car through the crossing of Lombart street and Hyde Street; they were pointing to the sky with wrong finger.\u003c/p\u003e","title":"OOW 2011 Day 0"},{"content":"\u0026hellip;tomorrow to meet a lot of fellow Oracle nerds at Oracle Open World. Looking especially forward to Pythian\u0026rsquo;s bloggers meetup on Wednesday, though my blog has been quite dormant for a while and I might not deserve it that much. We are planning for next year\u0026rsquo;s OUGN conference, and hopefully we\u0026rsquo;ll have a chance to meet some of them we want to invite. All I know is that we\u0026rsquo;ll have more of what has been a success so far. I might bring a cuddly toy to hand over to that guy who knows so much about partioning\u0026hellip;. Until another time zone.\n","permalink":"https://enesi.no/2011/09/going-to-san-francisco.../","summary":"\u003cp\u003e\u0026hellip;tomorrow to meet a lot of fellow Oracle nerds at Oracle Open World. Looking especially forward to Pythian\u0026rsquo;s \u003ca href=\"http://www.pythian.com/news/26437/oracle-openworld-2011-bloggers-meetup/\" target=\"_blank\" rel=\"noopener\"\u003ebloggers meetup\u003c/a\u003e\n on Wednesday, though my blog has been quite dormant for a while and I might not deserve it that much.  We are planning for next year\u0026rsquo;s OUGN conference, and hopefully we\u0026rsquo;ll have a chance to meet some of them we want to invite. All I know is that we\u0026rsquo;ll have more of what has been a success so far. I might bring a cuddly toy to hand over to that guy who knows so much about partioning\u0026hellip;. Until another time zone.\u003c/p\u003e","title":"Going to San Francisco..."},{"content":"The job portal.wwv_context.sync stopped working with the following error message:\nORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at \u0026#34;PORTAL.WWERR_API_EXCEPTION\u0026#34;, line 71 ORA-06512: at \u0026#34;PORTAL.WWV_CONTEXT\u0026#34;, line 2530 ORA-20000: Oracle Text error: DRG-10502: index WWSBR_CORNER_CTX_INDX does not exist We opened a Service Request at Oracle support but never figured out how the index disappeared, if that was the case and not a corruption somewhere. Support suggested that we executed the procedure given in Doc ID 340439.1; reinstall Oracle Text, then connect as the portal user from Portal server and execute the three scripts from middle tier home:\n@?/portal/admin/plsql/wws/ctxdrind.sql @?/portal/admin/plsql/wws/inctxgrn.sql @?/portal/admin/plsql/wws/ctxcrind.sql But the last script failed with:\nSQL\u0026gt; @?/portal/admin/plsql/wws/ctxcrind.sql Creating Context Preferences and Indexes... PL/SQL procedure successfully completed. declare * ERROR at line 1: ORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at \u0026#34;PORTAL.WWERR_API_EXCEPTION\u0026#34;, line 164 ORA-06512: at \u0026#34;PORTAL.WWV_CONTEXT\u0026#34;, line 2189 ORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at \u0026#34;PORTAL.WWERR_API_EXCEPTION\u0026#34;, line 71 ORA-06512: at \u0026#34;PORTAL.WWV_CONTEXT\u0026#34;, line 2118 ORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at \u0026#34;PORTAL.WWERR_API_EXCEPTION\u0026#34;, line 164 ORA-06512: at \u0026#34;PORTAL.WWV_CONTEXT\u0026#34;, line 2077 ORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at \u0026#34;PORTAL.WWERR_API_EXCEPTION\u0026#34;, line 71 ORA-06512: at \u0026#34;PORTAL.WWV_CONTEXT\u0026#34;, line 1705 ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine ORA-20000: Oracle Text error: DRG-50857: oracle error in drvxtab.create_index_tables ORA-00955: name is already used by an existing object ORA-06512: at line 6 The Application Server Portal Configuration Guide section 8.3.4.1 shows that the last script executes wwv_context.createindex and the error was isolated first to this procedure. The same section shows that wwv_context.createindex is equivalent to:\nwwv_context.drop_prefs; /* Drop all Oracle Text preferences for the indexes, except Lexer preferences */ wwv_context.drop_invalid_indexes; /* Drop all invalid indexes */ wwv_context.create_prefs; /* Create all Oracle Text preferences,except Lexer preferences */ wwv_context.create_missing_indexes(l_indexes); /* Create missing indexes and record them in l_indexes */ wwv_context.touch_index(l_indexes); /* Mark all rows for created indexes as requiring synchronization */ wwv_context.sync; /* Synchronize indexes */ wwv_context.optimize; /* Optimize indexes */ By executing each procedure in steps the error was isolated to wwv_context.create_missing_indexes. All of the indexes in question are listed in 8.3.4.2 and can be created one by one with procedure wwv_context.create_index.\nTo get around the error I executed the following procedure in stead of the script ctxcrind.sql, that is, after I executed the scripts inctxgrn.sql and ctxcrind.sql:\nEXEC WWV_CONTEXT.CREATE_PREFS EXEC WWV_CONTEXT.CREATE_INDEX(WWV_CONTEXT.PAGE_TEXT_INDEX) EXEC WWV_CONTEXT.CREATE_INDEX(WWV_CONTEXT.DOC_TEXT_INDEX) exec WWV_CONTEXT.CREATE_INDEX(wwv_context.PERSPECTIVE_TEXT_INDEX) EXEC WWV_CONTEXT.CREATE_INDEX(WWV_CONTEXT.ITEM_TEXT_INDEX) EXEC WWV_CONTEXT.CREATE_INDEX(WWV_CONTEXT.CATEGORY_TEXT_INDEX) EXEC WWV_CONTEXT.CREATE_INDEX(WWV_CONTEXT.URL_TEXT_INDEX) EXEC WWV_CONTEXT.TOUCH_INDEX(WWV_CONTEXT.PAGE_TEXT_INDEX) EXEC WWV_CONTEXT.TOUCH_INDEX(WWV_CONTEXT.DOC_TEXT_INDEX) EXEC WWV_CONTEXT.TOUCH_INDEX(WWV_CONTEXT.PERSPECTIVE_TEXT_INDEX) EXEC WWV_CONTEXT.TOUCH_INDEX(WWV_CONTEXT.ITEM_TEXT_INDEX) EXEC WWV_CONTEXT.TOUCH_INDEX(WWV_CONTEXT.CATEGORY_TEXT_INDEX) exec wwv_context.touch_index(WWV_CONTEXT.URL_TEXT_INDEX) exec wwv_context.sync; exec wwv_context.optimize; We did not verified if it was really necessary to reinstall Oracle Text, you may try the procedure above before reinstall Oracle Text if you encounter the same error with wwv_context.sync.\n","permalink":"https://enesi.no/2011/07/rebuild-oracle-text-indexes-for-portal/","summary":"\u003cp\u003eThe job portal.wwv_context.sync stopped working with the following error message:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \nORA-06510: PL/SQL: unhandled user-defined exception  \nORA-06512: at \u0026#34;PORTAL.WWERR_API_EXCEPTION\u0026#34;, line 71  \nORA-06512: at \u0026#34;PORTAL.WWV_CONTEXT\u0026#34;, line 2530  \nORA-20000: Oracle Text error:  \nDRG-10502: index WWSBR_CORNER_CTX_INDX does not exist  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eWe opened a Service Request at Oracle support but never figured out how the index disappeared, if that was the case and not a corruption somewhere. Support suggested that we executed the procedure given in Doc ID 340439.1; reinstall Oracle Text, then connect as the portal user from Portal server and execute the three scripts from middle tier home:\u003c/p\u003e","title":"Rebuild Oracle Text indexes for Portal"},{"content":"You may get error ORA-32012 if you are on 11g, but have the compatible parameter set to pre-11g in the database you are cloning from, and the spfile for the source database is stored in ASM when you do an RMAN duplicate from active database.\nTo get around this error, I have found two workarounds:\nSet the value for compatible to at least 11.1 Skip transferring of the spfile during the clone process. Option 1 is a big change for the database (it affects the CBO among other things), but say you need a clone and are going to change this parameter anyway it is the simplest one since you don\u0026rsquo;t have to create the parameterfile manually. Option 2 means you create the spfile before the cloning starts and remove the clause with spfile from the duplicate command.\nAn example of duplicate for option 1 is:\nrun { set newname for datafile 1 to NEW ; set newname for datafile 2 to NEW ; set newname for datafile 3 to NEW ; set newname for datafile 4 to NEW ; set newname for datafile 5 to NEW ; set newname for datafile 6 to NEW ; set newname for tempfile 2 to NEW ; duplicate target database to FOOTEST from active database spfile parameter_value_convert=\u0026#39;FOOPROD\u0026#39;,\u0026#39;FOOTEST\u0026#39; set compatible=\u0026#39;11.1.0.0\u0026#39; set diagnostic_dest=\u0026#39;/u01/app/oracle\u0026#39; ; } An example for option 2:\nrun { set newname for datafile 1 to NEW ; set newname for datafile 2 to NEW ; set newname for datafile 3 to NEW ; set newname for datafile 4 to NEW ; set newname for datafile 5 to NEW ; set newname for datafile 6 to NEW ; set newname for tempfile 2 to NEW ; duplicate target database to WINTIDST from active database; } It looks as the transfer of spfile from ASM to normal file system is perceived as a downgrade, though there is no change when compatible is 10.2.3 on both databases. The format of the spfile in ASM seems to be different when stored in ASM as explained in Database Administrator\u0026rsquo;s Guide 11g . The change from HARD-enabled storage to normal is not supported then if you are below 11g. Makes sense when one thinks about it, but I did a few tests back and forth to understand the connection, and it looks as if this new format was introduced in 11g. Exactly what format is used for the spfile in ASM is not clear. The ASM installation in this case is on version 11.2.0.2 and the database is on 11.1.0.7 (with the compatible left on 10.2.0.3 for some unknown reason). Maybe the new format is used when the database is on version 11g even though the compatible parameter is pre-11g, if so it looks like a bug when RMAN duplicate looks only at the compatible parameter and not on version of Oracle.\nWhen RMAN creates the spfile it will be created in $ORACLE_HOME/dbs, I could not find any way of having it created in ASM directly.\nAnyway, probably not a frequent error, but decided to write it up since I didn\u0026rsquo;t find much when googling.\n","permalink":"https://enesi.no/2011/07/ora-32012/","summary":"\u003cp\u003eYou may get error ORA-32012 if you are on 11g, but have the compatible parameter set to pre-11g in the database you are cloning from, and the spfile for the source database is stored in ASM when you do an RMAN duplicate from active database.\u003c/p\u003e\n\u003cp\u003eTo get around this error, I have found two workarounds:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eSet the value for compatible to at least 11.1\u003c/li\u003e\n\u003cli\u003eSkip transferring of the spfile during the clone process.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eOption 1 is a big change for the database (it affects the CBO among other things), but say you need a clone and are going to change this parameter anyway it is the simplest one since you don\u0026rsquo;t have to create the parameterfile manually. Option 2 means you create the spfile before the cloning starts and remove the clause with spfile from the duplicate command.\u003c/p\u003e","title":"ORA-32012"},{"content":"An insert or update statement that is taking a long time to complete is often working on the indexes belonging to the table rather than the table itself. If you look at the plan and finds nothing wrong, it is easy to forget that the indexes have to be updated as well.\nThe solution as always when something takes time is to enable trace. When updating the indexes the trace file will usually show waits of type db file sequential read, and usually one block at the time.\nFull table scan normally causes db file scattered read with many blocks at the time. The number of blocks it fetches is limited, but not exclusively, by the parameter db_file_multiblock_read_count.\nIn other words, with lots of full table scans you expect db file scattered read but your GUI gives you db file sequential read. The trace file will show you what objects are causing waits and may remind you that the table has an index or two that you can drop and add later.\nThis command counts the number of waits of one type grouped by object_id (as in dba_objects):\ngrep \u0026#34;db file sequential read\u0026#34; FOOPROD_ora_6237.trc |awk \u0026#39;{ print $12}\u0026#39; |sort | uniq –c You can find total elapsed time (in seconds) for a specific object_id (106836 in this example) with:\nawk \u0026#39;/106836/ { ela += $8} END {print ela/1000000}\u0026#39; FOOPROD_ora_6237.trc Repeat this for the object ids from the first list to find where most time is spent.\nThen you can find the type and name of the object:\nselect object_type, object_name from dba_objects where object_id=106836; Of course a decent profiler will do this for you. I wrote this post to remind myself that the trace file will tell me what the fancy GUI hides for me. And it is kind of cool that on a mature OS a string of simple commands allow me to make my own ad-hoc profiler.\n","permalink":"https://enesi.no/2011/06/hidden-updates-another-reason-to-trace/","summary":"\u003cp\u003eAn insert or update statement that is taking a long time to complete is often working on the indexes belonging to the table rather than the table itself. If you look at the plan and finds nothing wrong, it is easy to forget that the indexes have to be updated as well.\u003c/p\u003e\n\u003cp\u003eThe solution as always when something takes time is to enable trace. When updating the indexes the trace file will usually show waits of type db file sequential read, and usually one block at the time.\u003c/p\u003e","title":"Hidden updates - another reason to trace"},{"content":"This is what I did to install Oracle VM VirtualBox on a new Fedora 15 installation.\nThe documentation at the bottom of download page VirtualBox for Linux gives you the repos-file for yum on Fedora. Find the repos-file and save it to /etc/yum.repos.d/virtualbox.repo, it should look like this:\n\\[virtualbox\\] name=Fedora $releasever - $basearch - VirtualBox baseurl=http://download.virtualbox.org/virtualbox/rpm/fedora/$releasever/$basearch enabled=1 gpgcheck=1 gpgkey=http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc Then you can search for VirtualBox:\n\\[root@favela ~\\]# yum search VirtualBox Loaded plugins: langpacks, presto, refresh-packagekit \\=========================== N/S Matched: VirtualBox ============================ VirtualBox-4.0.x86_64 : Oracle VM VirtualBox Name and summary matches only, use \u0026#34;search all\u0026#34; for everything. To install:\nyum install VirtualBox-4.0.x86_64 Users need to be included in the OS group vboxusers before they can run it:\nusermod -a -G vboxusers oisene VirtualBox is now ready to run from menu.\nIn fact this was not what I did first time, because I didn\u0026rsquo;t read the fine manual to the end. I download it from here and installed the missing packages something like this:\nyum install qt qt-x11 SDL yum install kernel-headers-$(uname -r) kernel-devel-$(uname -r) yum install binutils gcc patch glibc-headers glibc-devel yum install dkms rpm -ip /home/oisene/Downloads/VirtualBox-4.0-4.0.8_71778_fedora15-1.x86_64.rpm Without the dkms package VirtualBox will run, but it is recommended to ensure VirtualBox host kernel modules will be update during next kernel update.\n","permalink":"https://enesi.no/2011/05/virtualbox-on-fedora-15/","summary":"\u003cp\u003eThis is what I did to install Oracle VM VirtualBox on a new Fedora 15 installation.\u003c/p\u003e\n\u003cp\u003eThe documentation at the bottom of download page \u003ca href=\"http://www.virtualbox.org/wiki/Linux_Downloads\" target=\"_blank\" rel=\"noopener\"\u003eVirtualBox for Linux\u003c/a\u003e\n gives you the repos-file for yum on Fedora. Find the \u003ca href=\"http://download.virtualbox.org/virtualbox/rpm/fedora/virtualbox.repo\" target=\"_blank\" rel=\"noopener\"\u003erepos-file\u003c/a\u003e\n and save it to /etc/yum.repos.d/virtualbox.repo, it should look like this:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \n\\[virtualbox\\]  \nname=Fedora $releasever - $basearch - VirtualBox  \nbaseurl=http://download.virtualbox.org/virtualbox/rpm/fedora/$releasever/$basearch  \nenabled=1  \ngpgcheck=1  \ngpgkey=http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eThen you can search for VirtualBox:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \n\\[root@favela ~\\]# yum search VirtualBox  \nLoaded plugins: langpacks, presto, refresh-packagekit  \n\\=========================== N/S Matched: VirtualBox ============================  \nVirtualBox-4.0.x86_64 : Oracle VM VirtualBox  \n  \n  Name and summary matches only, use \u0026#34;search all\u0026#34; for everything.  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eTo install:\u003c/p\u003e","title":"VirtualBox on Fedora 15"},{"content":"I have made a few changes to my blog\u0026hellip; think I want to blog again.\nSo far this year in a few words; new job at Keystep - smart move, if you try two years in the wrong company it is time to move on; met quite a few smart oracle folks at the yearly conference of Oracle User Group Norway - got elected as a board member; met more smart guys at Miracle Open World in Denmark the week after. And been thinking about moving back to Brazil quite a lot lately.\n","permalink":"https://enesi.no/2011/05/today/","summary":"\u003cp\u003eI have made a few changes to my blog\u0026hellip; think I want to blog again.\u003c/p\u003e\n\u003cp\u003eSo far this year in a few words; new job at Keystep - smart move, if you try two years in the wrong company it is time to move on; met quite a few smart oracle folks at the yearly conference of Oracle User Group Norway - got elected as a board member; met more smart guys at Miracle Open World in Denmark the week after. And been thinking about moving back to Brazil quite a lot lately.\u003c/p\u003e","title":"Today"},{"content":"Say you define a DP job from the API (DBMS_DATAPUMP) you may end up with jobs with status NOT RUNNING until you get it right. Verify if you have such a job with\nselect job_name,state from user_datapump_jobs; Then you may try to remove it with:\ndeclare l_h number; begin l_h:=dbms_datapump.attach(\u0026#39;YOUR_JOB\u0026#39;); dbms_datapump.stop_job(l_h,immediate=\u0026gt;1); commit; end; / If the query above still reports the job with the same status it can be removed by dropping the master table, according to Note 336014.1:\ndrop table YOUR_JOB; If the database is using a recycle bin the table has been renamed to a name starting with \u0026lsquo;BIN\u0026rsquo;; if you rerun the query above the job is renamed to reflect this, but if you purge the recycle bin the job will be finally removed:\npurge recyclebin; If your PL/SQL code fails during definitions before you do a detach you cannot do a second attach. Since you have no reference to the job the only solution I\u0026rsquo;m aware of is to reconnect (i.e. create another database session). Better yet is to define an exception on ORA-39001 and retrieve the error message with dbms_datapump.get_status, and dbms_datapump.detach(l_handle) in the exception handler:\ndeclare invalid_arg exception; pragma exception_init(invalid_arg,-39001); : : begin : : exception when invalid_arg then dbms_datapump.detach(l_h); end; If you drop the master table that is in the defining state, you will not be able to attach to the job, but if you try to attach to such a job the job will be removed for the user_datapump_jobs table.\nIf you try to add a metadata filter of type NAME_EXPR and you add a subquery in the expression, make sure that any table name in the subquery is prefixed with table owner or else you may receive ORA-00942 in the logfile:\ndbms_datapump.metadata_filter(handle =\u0026gt; l_h, name =\u0026gt; \u0026#39;NAME_EXPR\u0026#39;, value =\u0026gt;\u0026#39;IN (SELECT TABLE_NAME FROM SCOTT.CHANGE_TABLE)\u0026#39; ); Typical error in logfile when schema is missing:\nEstimate in progress using BLOCKS method... Processing object type TABLE_EXPORT/TABLE/TABLE_DATA ORA-39125: Worker unexpected fatal error in KUPW$WORKER.GET_TABLE_DATA_OBJECTS while calling DBMS_METADATA.FETCH_XML_CLOB \\[\\] ORA-00942: table or view does not exist ORA-06512: at \u0026#34;SYS.DBMS_SYS_ERROR\u0026#34;, line 95 ORA-06512: at \u0026#34;SYS.KUPW$WORKER\u0026#34;, line 6307 \\----- PL/SQL Call Stack ----- object line object handle number name 0x7bfb80b0 15032 package body SYS.KUPW$WORKER : : ","permalink":"https://enesi.no/2010/11/removing-orphan-datapump-jobs-and-filtering-included-tables-with-a-query/","summary":"\u003cp\u003eSay you define a DP job from the API (DBMS_DATAPUMP) you may end up with jobs with status NOT RUNNING until you get it right. Verify if you have such a job with\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \nselect job_name,state   \nfrom user_datapump_jobs;  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eThen you may try to remove it with:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \ndeclare   \n l_h number;  \nbegin  \n  l_h:=dbms_datapump.attach(\u0026#39;YOUR_JOB\u0026#39;);  \n  dbms_datapump.stop_job(l_h,immediate=\u0026gt;1);  \n  commit;  \nend;  \n/  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eIf the query above still reports the job with the same status it can be removed by dropping the master table, according to Note 336014.1:\u003c/p\u003e","title":"Removing orphan DataPump jobs and filtering included tables with a query"},{"content":"In version 10.2.0.5 of the database there is a bug when using Transparent Data Encryption (TDE) on a column with the CHAR datatype. In the following test a table is created where the primary key is defined as CHAR(11) and then encrypted with NO SALT. A simple lookup on this PK works as expected by using the respective index on 10.2.0.4. But on 10.2.0.5 a trace on event 10053 shows that an INTERNAL_FUNCTION is wrapped around the PK-column and therefore impedes use of the index.\nThe following test was performed on an OVM guest created from a template downloaded from edelivery.oracle.com. It contains Oracle Server EE 10.2.0.4 running on Oracle Enterprise Linux 5, 64-bit. Configuration of TDE and wallet is not shown.\ncreate table foo ( id char(11) encrypt no salt primary key , msg varchar2(1000) ); insert into foo ( select * from ( with generator as ( select rownum rn from dual connect by rownum \u0026lt;= 100 ) select dbms_random.string(\u0026#39;x\u0026#39;,11) id, dbms_random.string(\u0026#39;x\u0026#39;,1000) msg from generator g1, generator g2 where rownum \u0026lt;= 10000 ) ); commit; exec dbms_stats.gather_table_stats(user,\u0026#39;FOO\u0026#39;, cascade=\u0026gt;true,method_opt=\u0026gt;\u0026#39;for all indexed columns size 1\u0026#39;,estimate_percent=\u0026gt;100) variable lid char(11) exec :lid:=\u0026#39;abc123def45\u0026#39;; alter session set events=\u0026#39;10053 trace name context forever, level 1\u0026#39;; select msg from foo where id=:lid; The query returns immediately since the index is used and the trace file shows the good plan (a simple explain plan will show the same):\n\\--------------------------------------------------+-----------------------------------+ | Id | Operation | Name | Rows | Bytes | Cost | Time | \\--------------------------------------------------+-----------------------------------+ | 0 | SELECT STATEMENT | | | | 2 | | | 1 | TABLE ACCESS BY INDEX ROWID | FOO | 1 | 514 | 2 | 00:00:01 | | 2 | INDEX UNIQUE SCAN | SYS_C007236| 1 | | 1 | 00:00:01 | \\--------------------------------------------------+-----------------------------------+ Predicate Information: \\---------------------- 2 - access(\u0026#34;ID\u0026#34;=:LID) Now, the database is patched to version 10.2.0.5 (patch 8202632). There are no invalid objects in the database after the upgrade. Then repeating the last query from above together with the trace on event 10053, the query takes a long time and the trace file shows this plan:\n\\-------------------------------------+-----------------------------------+ | Id | Operation | Name | Rows | Bytes | Cost | Time | \\-------------------------------------+-----------------------------------+ | 0 | SELECT STATEMENT | | | | 388 | | | 1 | TABLE ACCESS FULL | FOO | 100 | 50K | 388 | 00:00:05 | \\-------------------------------------+-----------------------------------+ Predicate Information: \\---------------------- 1 - filter(INTERNAL_FUNCTION(\u0026#34;ID\u0026#34;)=:LID) The INTERNAL_FUNCTION is used among other situations when oracle has to perform an implicit data conversion. This may happen when the data type of a bind variable is different from the data type used in the table.\nThis kind of mismatch may happen during upgrades of JDBC drivers where the mapping may change; Java had support for TimeStamp before it was introduced in the Oracle database. Later when Oracle supported TimeStamp this mapping changed, leading to this kind of errors. There are other reasons for such mismatches; a trace on event 10053 will reveal the data type of the bind variable:\n******************************************* Peeked values of the binds in SQL statement ******************************************* kkscoacd Bind#0 oacdty=96 mxl=32(11) mxlc=00 mal=00 scl=00 pre=00 oacflg=03 fl2=1000000 frm=01 csi=31 siz=32 off=0 kxsbbbfp=2b19acc082a0 bln=32 avl=11 flg=05 value=\u0026#34;abc123def45\u0026#34; The part \u0026ldquo;oacdty=96\u0026rdquo; shows the type is CHAR or NCHAR. This did not change between the tests, so there has to be another reason why the INTERNAL_FUNCTION is used. Also in this case the only change is the upgrade, even the parameter compatible remained the same (10.2.0.3). The tests where performed from sqlplus on the database server.\nThis is bug 10296606 on MOS.\nUpdate: It was discovered that if you use a variable of VARCHAR type the INTERNAL_FUNCTION is not applied, meaning that columns of CHAR type are converted to VARCHAR after encryption, but that USER_TAB_COLUMNS continues to report CHAR for the column.\n","permalink":"https://enesi.no/2010/11/slow-performance-with-tde-and-char-data-type-on-10.2.0.5/","summary":"\u003cp\u003eIn version 10.2.0.5 of the database there is a bug when using Transparent Data Encryption (TDE) on a column with the CHAR datatype. In the following test a table is created where the primary key is defined as CHAR(11) and then encrypted with NO SALT. A simple lookup on this PK works as expected by using the respective index on 10.2.0.4. But on 10.2.0.5 a trace on event 10053 shows that an INTERNAL_FUNCTION is wrapped around the PK-column and therefore impedes use of the index.\u003c/p\u003e","title":"Slow performance with TDE and CHAR data type on 10.2.0.5"},{"content":"This is an old post I kept as draft 8 months before I published it\u0026hellip;\nI\u0026rsquo;ve been playing around with Oracle Virtual Manager (OVM) version 2.2 lately. This is collection of a few notes, not all directly related to OVM. The project started with building the server from parts I hoped would play along (they do). The CPU is an Intel i7-930, 12GB of RAM and two Western Digital 1.5TB disks (WD15EARS-00Z5B1). Current version of OVM (2.2) uses update 3 of OEL 5. The kernel lags behind a bit and support for certain wirless card is not included. The AR5008 Wireless chip from Atheros (a Dlink card) was not supported; in order to avoid more cables in the house I created a bridge between the server and another PC nearby.\nThe installation of OVM went smooth, but the disks are a disgrace with a horrible I/O performance until you do the partitioning right (each partition should start on a sector divisible by 4096 bytes). Meaning I had to choose a custom layout of the partitions and file systems during installation.\nThe OVM manager was installed in a guest based on a OVM template downloaded from Oracle.\nConfiguration of OVM Manager fails because host names cannot be resolved:\nThe server resides on a private network (192.168.x.x) without any DNS; if you get error messages indicating connection problems to the OVM server then make sure the hostname is defined in /etc/hosts, also make sure the hostname is not resolved to localhost (127.0.0.1); which was the case on a fresh installation of OVM Server. This should be done both on the OVM Server and in the guest hosting OVM Manager. (If the initial configuration of OVM Manager fails, connect with ssh and fix /etc/hosts before restarting).\nConnecting to the Manager database from afar:\nThe listener binds to standard port 1521 on localhost. In case you want to connect to the database over SQL*Net you need to change listener.ora and replace localhost with a public IP address/name of the Manager\u0026rsquo;s host. Then you need to add a line to /etc/sysconfig/iptables allowing remote access to the listener:\n\\-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 1521 -j ACCEPT Configuring yum:\nI added the following lines to the file /etc/yum.repos.d/public-yum-el5.repo\n\\[el5_u3_base\\] name=Enterprise Linux $releasever U3 - $basearch - base baseurl=http://public-yum.oracle.com/repo/EnterpriseLinux/EL5/3/base/$basearch/ gpgkey=http://public-yum.oracle.com/RPM-GPG-KEY-oracle-el5 gpgcheck=1 enabled=1 \\[ovm22_2.2.1_base\\] name=OracleVM 2 - 2.2.1 - $basearch - base baseurl=http://public-yum.oracle.com/repo/OracleVM/OVM2/2.2.1/base/$basearch/ gpgkey=http://public-yum.oracle.com/RPM-GPG-KEY-oracle-el5 gpgcheck=1 enabled=1 Then gcc and others can be installed with\nyum install gcc Tuning network\nAfter the disk performance was improved I started to get annoyed about the slow refresh of VNC. The following command reported an increasing queue length when I interacted with a GUI through VNC:\nnetstat -o -t -c This Guide was helpful.\nUsing physical devices for storage\nI could not find any way to add a physical device to a guest in the Manager. From the documentation though it is clear how one goes about to do this. Disadvantage is that the added physical device does not show up in Manager after. Anyway, the performance gain outweighs that. I had one disk that I gave entirely to a volume group and then created a logical device with pvcreate, vgcreate and lvcreate.\nTo add the logical volume as a physical volume to the guest:\nxm block-attach 24 phy:/dev/vg00/oradb1_asm1 xvdc \u0026#39;w\u0026#39; 24 is the id of the guest as reported by xm list, xvdc is the device name at the guest (/dev/xvdc), and \u0026lsquo;w\u0026rsquo; means read/write. At the guest this disk device needs to partitioned with fdisk or similar as with any other physical device.\nTo make the change permanent the following line (in bold) was added to the disk configuration in vm.cfg:\ndisk = [\u0026lsquo;file:/var/ovs/mount/3FEB9655D88E4177B8571AFCF81C5B8A/running_pool/50_oel2/System.img,xvda,w\u0026rsquo;,\n\u0026lsquo;phy:/dev/vg00/oel2_1,xvdb,w\u0026rsquo;,\n\u0026lsquo;file:/var/ovs/mount/3FEB9655D88E4177B8571AFCF81C5B8A/sharedDisk/software.img,xvdc,w!\u0026rsquo;,\n]\nMonitoring with sar -d showed that the service time substantially lower (1/10) with the physical device, even when the physical device had almost max utilization on writing.\nVNC and maximum resolution\nAny changes made to the screen resolution was quietly ignored and the connection to the vnc server that is started from dom0 was unreliable. After a while I gave up and started a new vnc server from inside the guest.\nUpdate: Using OVM for my own lab was not optimal, I installed Fedora 15 which among other things supports the wireless card without any hassle. I plan to install Virtual Box on top of it for my lab needs.\n","permalink":"https://enesi.no/2010/10/a-few-notes-on-oracle-vm/","summary":"\u003cp\u003eThis is an old post I kept as draft 8 months before I published it\u0026hellip;\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;ve been playing around with Oracle Virtual Manager (OVM) version 2.2 lately. This is collection of a few notes, not all directly related to OVM. The project started with building the server from parts I hoped would play along (they do). The CPU is an Intel i7-930, 12GB of RAM and two Western Digital 1.5TB disks (WD15EARS-00Z5B1). Current version of OVM (2.2) uses update 3 of OEL 5. The kernel lags behind a bit and support for certain wirless card is not included. The AR5008 Wireless chip from Atheros (a Dlink card) was not supported; in order to avoid more cables in the house I created a bridge between the server and another PC nearby.\u003c/p\u003e","title":"A few notes on Oracle VM"},{"content":"This is a simple one, a reminder till next time I forget this, since stuff encountered after a Google-search led me astray\u0026hellip;\nJust in case you try to add disks to an existing ASM disk group using syntax like this:\nalter diskgroup data add disk \u0026#39;/dev/xvdd1\u0026#39; add disk \u0026#39;/dev/xvde1\u0026#39;; You may get error messages like these:\nORA-15032: not all alterations performed ORA-15031: disk specification \u0026#39;/dev/xvde1\u0026#39; matches no disks ORA-15025: could not open disk \u0026#39;/dev/xvde1\u0026#39; ORA-15056: additional error message Linux-x86_64 Error: 13: Permission denied Additional information: 42 Additional information: -1614995373 Additional information: 1730318408 ORA-15031: disk specification \u0026#39;/dev/xvdd1\u0026#39; matches no disks ORA-15025: could not open disk \u0026#39;/dev/xvdd1\u0026#39; ORA-15056: additional error message Linux-x86_64 Error: 13: Permission denied Additional information: 42 Additional information: 87328232 Additional information: 8192 Solution: Use the ASM disk names as given when creating them with oracleasm:\nalter diskgroup data add disk \u0026#39;ORCL:VOL3\u0026#39; add disk \u0026#39;ORCL:VOL4\u0026#39;; Here VOL3 and VOL4 are the names given when creating the ASM-disk with oracleasm:\noracleasm createdisk VOL3 /dev/xvdd1 oracleasm createdisk VOL4 /dev/xvde1 The initialization parameter asm_diskstring is empty as is default.\nThe error messages from OS (Linux in this case) may lead you to think that there is something wrong with the owner or attributes of the devices you\u0026rsquo;ve just created, even they have the same permissions as the devices previously allocated to ASM\u0026hellip; Then when you against reason set the owner and group to be like \u0026lsquo;oracle:dba\u0026rsquo; the error message from Linux does not show up, but you will still get error messages like this:\nORA-15032: not all alterations performed ORA-15031: disk specification \u0026#39;/dev/xvde1\u0026#39; matches no disks ORA-15014: location \u0026#39;/dev/xvde1\u0026#39; is not in the discovery set ORA-15031: disk specification \u0026#39;/dev/xvdd1\u0026#39; matches no disks ORA-15014: location \u0026#39;/dev/xvdd1\u0026#39; is not in the discovery set A note on MOS with id 603210.1 is one example of this confusion, it contains this command indicating that it is OK to use the physical device name (which it may be in certain environments for what I know):\ncreate diskgroup DATADG external redundancy disk \u0026#39;/dev/sda14\u0026#39;; I think this is one of several reasons why certain un*x guys used to old and mature volume managers and file systems do not easily fall in love with ASM\u0026hellip; Once one has passed these initial obstacles I find ASM OK to work with, even for non-RAC databases. It usually shortens the path from getting a LUN allocated on the SAN or whatever until more free space is available to the tablespace in the database.\n[This testing was done on a Oracle VM guest, created from an OVM template downloaded from Oracle. It comes with OEL 5 64-bit and version 10.2.0.4 of the database. Oracle calls this OS for JeOS (Just enough OS), but for some reason it does not come with the \u0026lsquo;man\u0026rsquo; command, so for me it is JneOS.]\n","permalink":"https://enesi.no/2010/10/permission-denied-when-adding-asm-disk/","summary":"\u003cp\u003eThis is a simple one, a reminder till next time I forget this, since stuff encountered after a Google-search led me astray\u0026hellip;\u003c/p\u003e\n\u003cp\u003eJust in case you try to add disks to an existing ASM disk group using syntax like this:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \nalter diskgroup data  \nadd disk \u0026#39;/dev/xvdd1\u0026#39;  \nadd disk \u0026#39;/dev/xvde1\u0026#39;;  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eYou may get error messages like these:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \nORA-15032: not all alterations performed  \nORA-15031: disk specification \u0026#39;/dev/xvde1\u0026#39; matches no disks  \nORA-15025: could not open disk \u0026#39;/dev/xvde1\u0026#39;  \nORA-15056: additional error message  \nLinux-x86_64 Error: 13: Permission denied  \nAdditional information: 42  \nAdditional information: -1614995373  \nAdditional information: 1730318408  \nORA-15031: disk specification \u0026#39;/dev/xvdd1\u0026#39; matches no disks  \nORA-15025: could not open disk \u0026#39;/dev/xvdd1\u0026#39;  \nORA-15056: additional error message  \nLinux-x86_64 Error: 13: Permission denied  \nAdditional information: 42  \nAdditional information: 87328232  \nAdditional information: 8192  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eSolution: Use the ASM disk names as given when creating them with oracleasm:\u003c/p\u003e","title":"Permission denied when adding ASM disk"},{"content":"The only reason to read this post is if you have googled for the error \u0026ldquo;ORA-00904: \u0026ldquo;POLTYP\u0026rdquo;: invalid identifier\u0026rdquo;. This error occurs if you try the old export command from an 11g client against a database on version 10g or lower. The export command runs a query against a table called EXU9RLS in the SYS schema. On 11g this table was expanded with the column POLTYP and the export command (exp) expects to find this column. This should not be much of a problem since Data Pump export can be used.\n","permalink":"https://enesi.no/2010/06/error-when-using-11g-export-client-on-a-10g-database/","summary":"\u003cp\u003eThe only reason to read this post is if you have googled for the error \u0026ldquo;ORA-00904: \u0026ldquo;POLTYP\u0026rdquo;: invalid identifier\u0026rdquo;. This error occurs if you try the old export command from an 11g client against a database on version 10g or lower. The export command runs a query against a table called EXU9RLS in the SYS schema. On 11g this table was expanded with the column POLTYP and the export command (exp) expects to find this column. This should not be much of a problem since Data Pump export can be used.\u003c/p\u003e","title":"Error when using 11g export client on a 10g database"},{"content":"The following code is from a migration project I\u0026rsquo;m working on. It is part of a package named PERFORMANCE and is run every 15 minutes:\nDeclare test number; Cursor c_worklist Is select * from v_worklist where id = v_id; Begin FOR rec IN c_worklist LOOP test := 0; -- does nothing END LOOP; end; A nice little wtf-snippet. The purpose? According to \u0026ldquo;the vendor\u0026rdquo; the cursor that loops through every row in a view is supposed to keep the data in Oracle\u0026rsquo;s cache, and if not in place some web service will time out. I believe we have better tools to keep stuff in cache.\n","permalink":"https://enesi.no/2010/05/someone-trying-to-outperform-oracles-cache/","summary":"\u003cp\u003eThe following code is from a migration project I\u0026rsquo;m working on. It is part of a package named PERFORMANCE and is run every 15 minutes:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \n  Declare  \n  test number;  \n  Cursor c_worklist  \n  Is  \n     select *  \n     from v_worklist  \n     where id = v_id;  \n  Begin  \n  \n  FOR rec IN c_worklist  LOOP  \n    test := 0;  -- does nothing  \n  END LOOP;  \n  end;  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eA nice little wtf-snippet. The purpose? According to \u0026ldquo;the vendor\u0026rdquo; the cursor that loops through every row in a view is supposed to keep the data in Oracle\u0026rsquo;s cache, and if not in place some web service will time out. I believe we have better tools to keep stuff in cache.\u003c/p\u003e","title":"Someone trying to outperform Oracle's cache"},{"content":"A simple implementation of Collatz conjecture in PL/SQL:\ncreate or replace type int_tab_typ is table of integer; / create or replace function collatz(p_n in integer) RETURN int_tab_typ PIPELINED as n integer; BEGIN if p_n \u0026lt; 1 or mod(p_n,1)\u0026gt;0 then RETURN ; end if; n:=p_n; while n \u0026gt; 1 loop pipe row (n); if mod(n,2)=1 then n:=3*n+1; else n:=n / 2; end if; end loop; pipe row(n); end; / select * from table(collatz(101)); More on Collatz Conjecture (Wikipedia).\nThe point? Just for fun. Showed it to my 11 years old kid and asked him to figure out the 17 steps sequence of collatz(7) in his head. \u0026ldquo;Is this going to stop?\u0026rdquo;, he asked half through. \u0026ldquo;That is what we are going to find out, nobody really knows\u0026rdquo;, I told him.\nIf you have absolutely nothing useful to do, but have CPU and storage to waste, you may create a table containing n and the number of steps for collatz(n), with values of n ranging from 1 to 231-1 or until storage is exhausted, whichever comes first. Before doing so you may consider changing the function above to just count the steps and look up previous calculations from the table.\n","permalink":"https://enesi.no/2010/03/collatz-conjecture-in-pl-sql/","summary":"\u003cp\u003eA simple implementation of Collatz conjecture in PL/SQL:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e  \ncreate or replace type int_tab_typ is table of integer;  \n/  \n  \ncreate or replace function collatz(p_n in integer)   \n   RETURN int_tab_typ PIPELINED as  \n   n integer;  \nBEGIN  \n  if p_n \u0026lt; 1 or mod(p_n,1)\u0026gt;0 then   \n    RETURN ;  \n  end if;  \n  n:=p_n;  \n  while n \u0026gt; 1   \n  loop  \n    pipe row (n);  \n    if mod(n,2)=1 then   \n      n:=3*n+1;  \n    else   \n      n:=n / 2;  \n    end if;  \n  end loop;  \n  pipe row(n);  \nend;  \n/  \n  \nselect * from table(collatz(101));  \n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eMore on \u003ca href=\"http://en.wikipedia.org/wiki/Collatz_conjecture\" target=\"_blank\" rel=\"noopener\"\u003eCollatz Conjecture\u003c/a\u003e\n (Wikipedia).\u003c/p\u003e","title":"Collatz conjecture in PL/SQL"},{"content":"Not talking about a lost wallet with money, but the wallet used for encryption in the Oracle database and with encrypted backup.\nA wallet is used for storing keys used to encrypt/decrypt data to/from the database, as with Transparent Data Encryption. The documentation on Advanced Security says that Oracle searches the parameter ENCRYPTION_WALLET_LOCATION in sqlnet.ora, or if not found it searches for WALLET_LOCATION in the same file. If none of them are given it searches in the default location of the wallet for the database. I found different references to where this default location is, depending on underlying OS and version. After a few rounds of trial and error (always receiving ORA-28368) I gave up and resorted to tracing my process:\nselect p.spid from v$session s join v$process p on(s.paddr=p.addr) where s.sid in (select sid from v$mystat); Then using strace on the process:\nstrace -p 42 Among the output from strace something like this can be found:\naccess(\u0026#34;/u01/app/oracle/admin/FOO/wallet/ewallet.p12\u0026#34;, F_OK) = -1 ENOENT (No such file or directory) open(\u0026#34;/u01/app/oracle/admin/FOO/wallet/ewallet.p12\u0026#34;, O_RDWR|O_CREAT|O_TRUNC, 0666) = -1 ENOENT (No such file or directory) In this case the directory did not exist. It needs to be created with proper privileges for the OS user running the instance (usually oracle).\nOn Windows I imagine Process Monitor can be used to track down the location.\nMorgans Library has a nice reference on wallets .\n","permalink":"https://enesi.no/2010/03/using-trace-to-find-wallet-location/","summary":"\u003cp\u003eNot talking about a lost wallet with money, but the wallet used for encryption in the Oracle database and with encrypted backup.\u003c/p\u003e\n\u003cp\u003eA wallet is used for storing keys used to encrypt/decrypt data to/from the database, as with Transparent Data Encryption. The documentation on Advanced Security says that Oracle searches the parameter \u003ccode\u003eENCRYPTION_WALLET_LOCATION\u003c/code\u003e in \u003ccode\u003esqlnet.ora\u003c/code\u003e, or if not found it searches for \u003ccode\u003eWALLET_LOCATION\u003c/code\u003e in the same file. If none of them are given it searches in the default location of the wallet for the database. I found different references to where this default location is, depending on underlying OS and version. After a few rounds of trial and error (always receiving ORA-28368) I gave up and resorted to tracing my process:\u003c/p\u003e","title":"Using trace to find wallet location"},{"content":"If you have a lousy database model, you can get around it with triggers. If you need to hide some business logic from everybody else, you can do it with triggers. If you need a mechanism that bypass the consistency in the database, you can do it with triggers. They live a life on their own, seemingly autonomous from everything else and one easily forgets they exist. That is why I hate them. What matters if you do an database export in consistent mode if some active triggers make the data inconsistent during import? Triggers are like secret police, they are threat to the democracy if they take over.\nOh, just a rant.\nNote to myself: next time import fails with ORA-1 or ORA-2298, check for evil triggers.\n","permalink":"https://enesi.no/2010/01/database-triggers-are-evil/","summary":"\u003cp\u003eIf you have a lousy database model, you can get around it with triggers. If you need to hide some business logic from everybody else, you can do it with triggers. If you need a mechanism that bypass the consistency in the database, you can do it with triggers. They live a life on their own, seemingly autonomous from everything else and one easily forgets they exist. That is why I hate them. What matters if you do an database export in consistent mode if some active triggers make the data inconsistent during import? Triggers are like secret police, they are threat to the democracy if they take over.\u003c/p\u003e","title":"Database triggers are evil"},{"content":"If your database is using anything else than UTF8 as database character set you may consider to migrate from it. Oracle states in the Globalization Guide (10gR2):\nAt the top of the list of character sets Oracle recommends for all new system deployment is the Unicode character set AL32UTF8.\nDepending on what characters you actually have in your database you have three options on how to do this:\nChanging the character set with the package CSALTER. Only data dictionary is migrated.\nUsing CSALTER and convert application data using export/import for all users\nUsing CSALTER and convert a subset of application data.\nOption 1 migrates the data dictionary, that is, the meta data regarding your application data. Nothing else is changed and Oracle, after successfully executing CSALTER, behaves as if your database was created with the new character set. That works OK if the characters used are actually encoded the same way in the new character set as before, if not funny letters will show up when querying the database, meaning you have to go for option 2 (or possibly 3).\nCharacter expansion happens when a symbol changes from one character set to another with more bytes being used for storage in the latter. This is the case for the euro sign (€) in WE8MSWIN, where it requires one byte, in AL32UTF8 it takes three bytes. This requires a conversion of application data using export/import.\nOracle has made an utility for this, csscan. It checks the data dictionary and application data and gives you an analysis telling you what to do. The following procedure was executed on Oracle 10.2.0.4.\nFirst create these two directories are defined in the database:\nlog_file_dir data_file_dir I don\u0026rsquo;t think they are really necessary, they don\u0026rsquo;t seem to be used, but the following script grants access to them for the CSMIG user. Directories are listed in DBA_DIRECTORIES and you may create these anywhere you like with:\ncreate directory log_file_dir as \u0026lsquo;/tmp/csscan\u0026rsquo;;\ncreate directory data_file_dir as \u0026lsquo;/tmp/csscan\u0026rsquo;;\nRun the following script as SYS in sqlplus to install the tool:\n@?/rdbms/admin/csminst.sql\nIt will ask for a password which you\u0026rsquo;ll use later (on 9iR2 it uses standard password CSMIG). Then start the scanning with:\ncsscan csmig/your_password full=y tochar=al32utf8 process=3 array=1024000\nThe csscan command is documented in chapter 12 in the Globalization Guide. The above analyzes the entire database. Otherwise owner, tables or columns may be specified. The process parameter selects the number of simultaneous scanning processes.\nWhen scanning is completed csscan create a database scan report and individual exception reports as scan.txt and scan.err respectively. The file scan.out contains the output from the command above. These files are created in the directory that was current when the command was executed.\n","permalink":"https://enesi.no/2010/01/changing-character-set-in-database/","summary":"\u003cp\u003eIf your database is using anything else than UTF8 as database character set you may consider to migrate from it. Oracle states in the Globalization Guide (10gR2):\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAt the top of the list of character sets Oracle recommends for all new system deployment is the Unicode character set AL32UTF8.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eDepending on what characters you actually have in your database you have three options on how to do this:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003eChanging the character set with the package CSALTER. Only data dictionary is migrated.\u003c/p\u003e","title":"Changing character set in database"},{"content":"Using the CONVERT command in RMAN can be an efficient way to migrate a database from one OS to another, if they have the same endian.\nWe migrated 9 databases on 10.2.0.4 from HP-UX (on Itanium) to AIX. The databases where distributed on three servers with not exactly the same hardware configuration. On one of the servers we encountered an error described in bug 888530 on Metalink or My Oracle Support. The process terminated prematurely with ORA-7445 when converting an undo tablespace. The other tablespaces were migrated fine, but the undo tablespace ended up with a missing datafile. After the error occurred the RMAN session was left hanging and after killing it the migrated datafile belonging to undo was removed; may be that had something to do with the file system being NFS mounted. Anyway, our solution was to create another undo tablespace and change the undo_tablespace parameter, and then dropping the corrupted one. In one of the tests we did, we managed to do recovery on the migrated undo tablespace, but some funny error messages showed up in the alert log, probably because it was not finally migrated to the new platform. Meaning, if you get this error, create a new tablespace for undo and drop the old one.\n","permalink":"https://enesi.no/2009/11/rman-convert-database-and-ora-7445/","summary":"\u003cp\u003eUsing the CONVERT command in RMAN can be an efficient way to migrate a database from one OS to another, if they have the same endian.\u003c/p\u003e\n\u003cp\u003eWe migrated 9 databases on 10.2.0.4 from HP-UX (on Itanium) to AIX. The databases where distributed on three servers with not exactly the same hardware configuration. On one of the servers we encountered an error described in bug 888530 on Metalink or My Oracle Support. The process terminated prematurely with ORA-7445 when converting an undo tablespace. The other tablespaces were migrated fine, but the undo tablespace ended up with a missing datafile. After the error occurred the RMAN session was left hanging and after killing it the migrated datafile belonging to undo was removed; may be that had something to do with the file system being NFS mounted. Anyway, our solution was to create another undo tablespace and change the undo_tablespace parameter, and then dropping the corrupted one. In one of the tests we did, we managed to do recovery on the migrated undo tablespace, but some funny error messages showed up in the alert log, probably because it was not finally migrated to the new platform. Meaning, if you get this error, create a new tablespace for undo and drop the old one.\u003c/p\u003e","title":"RMAN Convert database and ORA-7445"},{"content":"Case: I wanted to reinstall the Enterprise Manager agent on a host. I went on to delete the host in the list of targets in EM. Then after a new installation and synchronization I somehow managed to push the list of targets from GC to the agent and thereby removing the host as a target in the local targets.xml file. Result: The monitored host did never show up in the target list of hosts even after a successful installation of the agent.\nSolution: Create a temporary file /tmp/newtarget with one line:\n\u0026lt;Target TYPE=\u0026quot;host\u0026quot; NAME=\u0026quot;foo42.bar.com\u0026quot;/\u0026gt; Where foo42.bar.com is the host name. Then add the target with:\nemctl config agent addtarget -f /tmp/newtarget ","permalink":"https://enesi.no/2009/10/manually-add-a-host-target-to-grid-control/","summary":"\u003cp\u003eCase: I wanted to reinstall the Enterprise Manager agent on a host. I went on to delete the host in the list of targets in EM. Then after a new installation and synchronization I somehow managed to push the list of targets from GC to the agent and thereby removing the host as a target in the local targets.xml file. Result: The monitored host did never show up in the target list of hosts even after a successful installation of the agent.\u003c/p\u003e","title":"Manually add a host target to Grid Control"},{"content":"I\u0026rsquo;ll probably need this in the not-so-distant future. Here is the link to Noon\u0026rsquo;s write-up when I need it. His blog is recommended, by the way.\n","permalink":"https://enesi.no/2009/06/aix-and-large-pages/","summary":"\u003cp\u003eI\u0026rsquo;ll probably need this in the not-so-distant future. Here is the link to \u003ca href=\"http://dbasrus.blogspot.com/2009/06/size-sometimes-does-matter.html\" target=\"_blank\" rel=\"noopener\"\u003eNoon\u0026rsquo;s write-up\u003c/a\u003e\n when I need it. His blog is recommended, by the way.\u003c/p\u003e","title":"AIX and large pages"},{"content":"Friday I passed the OCP exam (1Z0-043). In two months I have studied for, and taken three tests in order to achieve OCP certification. Before I started on this in January I had noe previous experience with Prometric testing or certification at all. But I knew I had read a post about it earlier and started looking up an old post by Chen Shapira about how she prepared for the OCA exam . I ordered the same book as well as another book for the test that was added as an extra requirement for the OCA level December last year. I more or less made the same experience that she did, in my words:\nWorking experience with Oracle is very usefull. It helps you remember it all.\nThese exam preparation books are not perfect, but the authors seem to know what is important and not. verifying against the Oracle documentation, other books, and your own experience is important, though.\nThe cd-rom that come bundled with the book is not that usefull. In one of the question the answer is given, the questions seem mixed up between OCA and OCP, and overall did not impress with quality. But it managed to raise my stress level the night before the OCA exam and keeping me alert. Usefull if you\u0026rsquo;ve never taken a Prometric test before. But before the OCP test I skipped the test software in order to avoid confusion.\nEspecially when reading for the OCA tests I had the feeling that I knew 95% of the stuff, but the questions in the book revealed to me which concepts that where not completely clear to me. Also one may read it too lightly and not capture important details for trick questions.\nCheck the exam topics at the Oracle site (as of now: OCA topics and OCP topics ) and compare it with the preparation book you are reading (if you do). For instance, globalization is not much of a topic for the OCA exam (except that you have to know about setting a few NLS-variables), but it is a topic for the OCP exam. In the book by Watson and Bersinic Globalization is covered in chapter 21 in the OCA part. The topic list also served as check list of what to study the last days before the exam when I had finished reading the preparation book.\nGetting used to the Prometric testing in itself is important. Pay attention to details in the questions and check the timer once in a while, if you spend too much time in the beginning you may not have enough time at the end. Now that OCA requires an extra test it may be the best opportunity to get used to these tests if you have never taken them before. Exam 051 11g SQL Fundamentals has only 65 questions to be answered in 120 minutes and a passing score of 60% (again, check oracle.com for changes).\nOne example of where statements in the preparation book does not agree with other books and documentation I\u0026rsquo;ve read can be found on page 812. The exam tip at the end of the page reads:\nmany people think that a log switch triggers a checkpoint. It doesn\u0026rsquo;t. It used to, but that behaviour changed with release 8i, when incremental checkpointing was introduced\u0026hellip;\nHowever on page 100 in Tom Kyte\u0026rsquo;s book \u0026ldquo;Expert Oracle Database Architecture\u0026rdquo; second-last paragraph:\nMany things can cause a checkpoint to occur, the most common event being a redo log switch.\nI\u0026rsquo;m inclined to believe that Kyte got it right. One is not allowed to report from these exams, but I think it is safe to say that the issue was somehow touched.\nThe main reason I went for certification was a requirement from at least one of our customers. Now that I\u0026rsquo;m through it all I am of course contend that I was pushed into it. The certification helps to identify gaps in your knowledge, gaps one may not be aware of even if you have worked with the database for years. One may question what is the use of knowing how to solve the same problem in five different tools.\nBeing an OCP does not equal being a good DBA, but it looks nice on the CV and quite a few of those requesting a certification do not know what it is all about.\n","permalink":"https://enesi.no/2009/03/passed-the-ocp-exam/","summary":"\u003cp\u003eFriday I passed the OCP exam (1Z0-043). In two months I have studied for, and taken three tests in order to achieve OCP certification. Before I started on this in January I had noe previous experience with Prometric testing or certification at all. But I knew I had read a post about it earlier and started looking up an old post by \u003ca href=\"http://prodlife.wordpress.com/\" target=\"_blank\" rel=\"noopener\"\u003eChen Shapira\u003c/a\u003e\n about how \u003ca href=\"http://prodlife.wordpress.com/2007/09/28/how-i-prepared-for-oracles-oca-exam/\" target=\"_blank\" rel=\"noopener\"\u003eshe prepared for the OCA exam\u003c/a\u003e\n. I ordered the same book as well as another book for the test that was added as an extra requirement for the OCA level December last year. I more or less made the same experience that she did, in my words:\u003c/p\u003e","title":"Passed the OCP exam"},{"content":"This year started off fine with a new job. I quit the company I joined just 25 months earlier, in itself a defeat. Though I gained much useful experience there, some signals convinced me to move on, and now I am quite happy that I did so. I joined Steria Norway to work as a senior system consultant, and so far it looks quite promising. The company is internationally focused and one is encouraged to build networks outside one\u0026rsquo;s country, something I find interesting. There is constant pressure to improve; yesterday I finished the second exam in order to achieve my OCA certification (Last December Oracle introduced a second exam as a requirement for the OCA). And I won\u0026rsquo;t stop there.\nWhat else; two weeks ago I gave another thought about using Facebook; understanding that social networking on the net is important I concluded that I should give it another try, and reactivated the account after more than a year off. What matters is probably what one publishes there, and not being there or not. This post has been an exception to my rule; I should get back to writing about the database.\n","permalink":"https://enesi.no/2009/02/new-job/","summary":"\u003cp\u003eThis year started off fine with a new job. I quit the company I joined just 25 months earlier, in itself a defeat. Though I gained much useful experience there, some signals convinced me to move on, and now I am quite happy that I did so. I joined Steria Norway to work as a senior system consultant, and so far it looks quite promising. The company is internationally focused and one is encouraged to build networks outside one\u0026rsquo;s country, something I find interesting. There is constant pressure to improve; yesterday I finished the second exam in order to achieve my OCA certification (Last December Oracle introduced a second exam as a requirement for the OCA). And I won\u0026rsquo;t stop there.\u003c/p\u003e","title":"New job"},{"content":"A couple of months ago I had an incident on a live production system. The obvious error was to execute a DDL statement at 9 am that should have been performed during off-business hours. On the other hand I guess you cannot avoid any ad hoc change on a live system forever. That will require you to be aware of any problem and issue that may strike your system. Say you have an issue right now, users are complaining, or to put it bluntly, someone is losing money. You think you have the solution, but before you actually apply it you want to know if it will actually work or make matters worse. Most people will run to their test system, at least to check that the syntax is right. I had verified it long time ago and knew it would work. However, there is one problem with test systems. Though we strive to make them as similar to the original production system as possible, they never are. Even if you can afford the seemingly waste of hardware and space, something is likely to be missing in the test system. In my case it was the users\u0026hellip; With Oracle 11g and Real Application Testing (which I have not tried yet) the load can be better mirrored in the test system, but I can\u0026rsquo;t imagine it will behave as real users.\nThe mistake I did was to change a rather unimportant index that was supposed to speed up a query from one specific app. What I forgot in the rush was that we had other pl/sql packages that depended on the underlying table; at the moment the DDLs were executed several connections had exactly one failed transaction. The package state became invalid and the transactions failed with ORA-4061. Slightly embarrassing, and it reinforced three principles:\nPick the right moment for any change.\nWhen someone is pressing on for an immediate change, take a break.\nThere is no substitute for a peer review.\nWe have redundancy on every level, from mirrored disks to a standby database in another city. But no second DBA.\n","permalink":"https://enesi.no/2008/10/side-effect/","summary":"\u003cp\u003eA couple of months ago I had an incident on a live production system. The obvious error was to execute a DDL statement at 9 am that should have been performed during off-business hours. On the other hand I guess you cannot avoid any ad hoc change on a live system forever. That will require you to be aware of any problem and issue that may strike your system. Say you have an issue right now, users are complaining, or to put it bluntly, someone is losing money. You think you have the solution, but before you actually apply it you want to know if it will actually work or make matters worse. Most people will run to their test system, at least to check that the syntax is right. I had verified it long time ago and knew it would work. However, there is one problem with test systems. Though we strive to make them as similar to the original production system as possible, they never are. Even if you can afford the seemingly waste of hardware and space, something is likely to be missing in the test system. In my case it was the users\u0026hellip; With Oracle 11g and Real Application Testing (which I have not tried yet) the load can be better mirrored in the test system, but I can\u0026rsquo;t imagine it will behave as real users.\u003c/p\u003e","title":"Side effect"},{"content":"A simple configuration with Oracle Connection Manager (CMAN) did not work, I constantly received ORA-12529 when trying to connect with sqlplus to a database through CMAN. After checking cman.ora over and over and browsing all the trace files I finaly dived into one and found a problem with host name resolving. Turned out that the host name defined in /etc/hosts on the database server was not known outside the server. The database, I assume, did a lookup on the ip number in local /etc/hosts and sent the host name to CMAN when registering. The fact that I had used ip-numbers in the local listener.ora and tnsnames.ora did not change that behavior. A properly DNS would have saved me this, the solution was to define the database server name in local host file on server where CMAN is running.\nThis means that receiving ORA-12529 does not necessarily mean that there is something wrong with your rules. 12529 seems to be a kind of \u0026lsquo;catch all error\u0026rsquo; when rule filtering fails. Look for nprffilter: entry in the trace file and start reading from there. I found this:\nsnlinGetAddrInfo: entry snlinGetAddrInfo: Invalid IP address string FOO snlinFreeAddrInfo: entry snlinFreeAddrInfo: exit snlinGetAddrInfo: exit snlinGetAddrInfo: entry snlinGetAddrInfo: Name resolution failed for FOO snlinFreeAddrInfo: entry snlinFreeAddrInfo: exit snlinGetAddrInfo: exit nprffilter: Unable to resolve to IP nprffilter: exit nsglbfok: no rule match or an error from nprffilter nsglbfok: exit (Real hostname replaced and timestamps removed). Looking for 12529 or whatever code you receive in the trace-file and start reading backwards is also an option.\n","permalink":"https://enesi.no/2008/08/connection-manager-and-ora-12529/","summary":"\u003cp\u003eA simple configuration with Oracle Connection Manager (CMAN) did not work, I constantly received ORA-12529 when trying to connect with sqlplus to a database through CMAN. After checking cman.ora over and over and browsing all the trace files I finaly dived into one and found a problem with host name resolving. Turned out that the host name defined in /etc/hosts on the database server was not known outside the server. The database, I assume, did a lookup on the ip number in local /etc/hosts and sent the host name to CMAN when registering. The fact that I had used ip-numbers in the local listener.ora and tnsnames.ora did not change that behavior. A properly DNS would have saved me this, the solution was to define the database server name in local host file on server where CMAN is running.\u003c/p\u003e","title":"Connection Manager and ORA-12529"},{"content":"Today I was looking at an SQL statement with a slow response. The statement does a lookup on a person using \u0026lsquo;upper(lastname) and upper(firstname)\u0026rsquo;. There is a functioned-based index (fbi) on the lastname column, but by checking the statistics it turned out to have a rather low selectivity and the CBO went for a full table scan. I don\u0026rsquo;t know why it has not occurred to me before, but today it dawned upon me that it may be possible to create an index like this:\ncreate index foo_idx on foo(bar(a), bar(b)) ; There is an example in Tom Kyte\u0026rsquo;s last book, but I guess the tidbit got lost on its way to or inside my hippocampus. I have always thought about the fbi as a way to index the result of a function (which could take many arguments, but only return one value). By looking at the fbi as an index on a shadow-column it becomes obvious that more than one column can be indexed this way, just like any other multi-columned index.\nPoint is of course that each of lastname and firstname has a very low density (at least in this part of the world), but the combination of them is not too bad.\nOr may be it was something else that went right, one should always supply a proof these days. I reapplied the old index with the same slow response as a result. Also I gathered statistics again every time an index was created, and the results where consistent.\n","permalink":"https://enesi.no/2008/07/multi-column-function-based-index/","summary":"\u003cp\u003eToday I was looking at an SQL statement with a slow response. The statement does a lookup on a person using \u0026lsquo;upper(lastname) and upper(firstname)\u0026rsquo;. There is a functioned-based index (fbi) on the lastname column, but by checking the statistics it turned out to have a rather low selectivity and the CBO went for a full table scan. I don\u0026rsquo;t know why it has not occurred to me before, but today it dawned upon me that it may be possible to create an index like this:\u003cbr\u003e\n\u003ccode\u003e  create index foo_idx on foo(bar(a), bar(b)) ;  \u003c/code\u003e\u003cbr\u003e\nThere is an example in Tom Kyte\u0026rsquo;s last book, but I guess the tidbit got lost on its way to or inside my hippocampus. I have always thought about the fbi as a way to index the result of a function (which could take many arguments, but only return one value). By looking at the fbi as an index on a shadow-column it becomes obvious that more than one column can be indexed this way, just like any other multi-columned index.\u003c/p\u003e","title":"Multi-column function based index"},{"content":"In a earlier post about legacy databases I mentioned that dirty data are more likely to enter the database on an earlier stage. One day while I was looking at the selectivity for a few columns I found the values a bit strange. After some digging I noticed that for most of the time the data was OK, even though a proper foreign key constraint was missing for the columns. But at some time many years ago somebody had inserted something completely different , which had clearly affected the selectivity (the columns are indexed).\nTable partitioning is better to assist in the management of the data more than for tuning purposes, and I imagined that this would be another example. By range partitioning on transaction time dirty data would be kept on a partition separate from more recent history and thus improve the column stats. Though I haven\u0026rsquo;t set up a test case (for the lack of a real problem) I believe that a correct selectivity will help the CBO and avoid these border problems where the CBO is tricked to throw off a wrong plan based on wrong statistics. That would be another example where the CBO is blamed for something that is not its fault.\nThis is why I don\u0026rsquo;t like hints, it feels like cheating and being lazy. The CBO may have bugs, but I think you have to be as good as the author of this book to prove it; usually there is an underlying problem, like the one I encountered.\nAnyway, I did a quick test, by copying the table in question into a partioned version and ran dbms_stats.gather_table_stats on it. In fact the selectivity did not change much between partitions, probably not enough to make an impact on CBO choice of plan. But I had used other parameters on the new table (most importantly I think, estimate_percent=\u0026gt;100), the columns in the two tables had very different values for selectivity. Which reiterated to me previous knowledge; checking the stats and the way they are collected is important, and sometimes a simple solution will do.\n","permalink":"https://enesi.no/2008/06/table-partitioning-to-improve-stats/","summary":"\u003cp\u003eIn a earlier post about \u003ca href=\"https://enesi.no/2008/04/understanding-a-legacy-database/\"\u003elegacy databases\u003c/a\u003e\n I mentioned that dirty data are more likely to enter the database on an earlier stage. One day while I was looking at the selectivity for a few columns I found the values a bit strange. After some digging I noticed that for most of the time the data was OK, even though a proper foreign key constraint was missing for the columns. But at some time many years ago somebody had inserted \u003ca href=\"http://en.wikipedia.org/wiki/And_Now_For_Something_Completely_Different\" target=\"_blank\" rel=\"noopener\"\u003esomething completely different\u003c/a\u003e\n, which had clearly affected the selectivity (the columns are indexed).\u003c/p\u003e","title":"Table partitioning to improve stats"},{"content":"I like great tools. An efficient tool will help you do a piece of work faster than when you are on your own. If the tool is made by someone who is close to the material chances are the tool is efficient. The tool I use most is SQL Developer . It is easy to use and I promote it to any developer where I work, especially if they only need a quick access to the data dictionary, run some simple (or complex) SQL statements. It is not invasive, contrary to other tools it does not provide all the DBA tools that may be useful in some cases, but also may be rather intrusive and put a severe load on oracle instance as it queries the internal v$-tables.\nInstallation of SQL Developer can hardly be easier; unpack a zip-file, and execute the program. It asks for the java.exe if you downloaded the version without java and asks you if you would like to migrate from a previous version. Connection can be done through JDBC, meaning no oracle client is necessary, but in case you have the client software installed, it lets you pick a connection from a list of known tns-connections.\nLatest version 1.5 adds support for CVS and Subversion, and lets you generate documentation in html-format. Very useful for offline-browsing or add to some intra net server.\nAnother tool I use is Schemester. Simple and efficient to capture a data model from the database and visualize the foreign keys. Schemester used to be located at www.schemester.co.uk , but that site seems to have been abandoned, and I have not seen much activity around it for some time now. As you upgrade or change the data model in Schemester it can generate \u0026ldquo;delta DB DDL script\u0026rdquo;. On top of my wish list for SQL Developer are these features from Schemester. If the status quo sets the standard they are likely to be implemented without some bugs that Schemester has.\nSue Harper posts updates about SQL Developer on her blog , you won\u0026rsquo;t have too many posts to catch up, but the tidbits are absolutely useful (like when I can look forward to a new version.)\n","permalink":"https://enesi.no/2008/05/another-great-tool-sql-developer/","summary":"\u003cp\u003eI like great tools. An efficient tool will help you do a piece of work faster than when you are on your own. If the tool is made by someone who is close to the material chances are the tool is efficient. The tool I use most is \u003ca href=\"http://www.oracle.com/technology/products/database/sql_developer/index.html\" target=\"_blank\" rel=\"noopener\"\u003eSQL Developer\u003c/a\u003e\n. It is easy to use and I promote it to any developer where I work, especially if they only need a quick access to the data dictionary, run some simple (or complex) SQL statements. It is not invasive, contrary to other tools it does not provide all the DBA tools that may be useful in some cases, but also may be rather intrusive and put a severe load on oracle instance as it queries the internal v$-tables.\u003c/p\u003e","title":"Another great tool: SQL Developer"},{"content":"I have been working on a small scale data warehouse project for a few months along with many other tasks. Following Kimball\u0026rsquo;s ideas on dimensional modeling and by keeping it simple I had more or less an idea on how to proceed with the construction of the database, data model and how to create the reports (but only in SQL). One guy in management actually wanted to learn SQL and went ahead with SQL Developer (fantastic product, so easy to use that anybody familiar with the layout of the keyboard can use it). But I did not expect the same attitude from the rest. I have to say that I\u0026rsquo;ve seen many products for making glorified reports, but all of them gave me the impression of being cumbersome to install and not exactly bug free. I did not know any product very well and neither had I the time to start testing them. Earlier I\u0026rsquo;ve played with Application Express and thought some reports could be created in ApEx, but I had a nagging feeling this would not cover all our needs.\nAfter a presentation delivered by a guy from Bicon at OUGN\u0026rsquo;s (Oracle User Group Norway) yearly conference I decided to try out Oracle BI Publisher (formerly called XML Publisher). BIP is clearly an option for us. After installation which is plain sailing, you define a few simple reports for immediate publishing. BIP comes with its own web server and can be installed on any windows box lying around (which you then baptize The Application Server). Anybody with a net browser can indulge themselves with the reports. The security model is intuitive with roles and users; I can easily write a report and decide who will have access to it. The BIP Desktop add-on for Windows does a good job with defining a layout. The ability to export the report in different formats guarantees that reports can be published in different ways and even processed further elsewhere (e.g. Excel). BIP has a scheduler (requires access to an oracle database for storing a repository) so that long-running reports (i.e. the sofar-not-converted batch job reports that used to run on the OLTP database) can be run during the night and be ready for the early morning bird at 07 hs.\nIf you are a DBA with limited experience outside the world of sqlplus and alike, I recommend checking out BIP. Later on when demand for more analyzing tools increases I will dig into some other BI tools, but for a while I will spend time creating more reports in BIP.\n","permalink":"https://enesi.no/2008/05/praise-for-oracle-bi-publisher/","summary":"\u003cp\u003eI have been working on a small scale data warehouse project for a few months along with many other tasks. Following Kimball\u0026rsquo;s ideas on dimensional modeling and by keeping it simple I had more or less an idea on how to proceed with the construction of the database, data model and how to create the reports (but only in SQL). One guy in management actually wanted to learn SQL and went ahead with \u003ca href=\"http://www.oracle.com/technology/products/database/sql_developer/index.html\" target=\"_blank\" rel=\"noopener\"\u003eSQL Developer\u003c/a\u003e\n (fantastic product, so easy to use that anybody familiar with the layout of the keyboard can use it). But I did not expect the same attitude from the rest. I have to say that I\u0026rsquo;ve seen many products for making glorified reports, but all of them gave me the impression of being cumbersome to install and not exactly bug free. I did not know any product very well and neither had I the time to start testing them. Earlier I\u0026rsquo;ve played with \u003ca href=\"http://www.oracle.com/technology/products/database/application_express/index.html\" target=\"_blank\" rel=\"noopener\"\u003eApplication Express\u003c/a\u003e\n and thought some reports could be created in ApEx, but I had a nagging feeling this would not cover all our needs.\u003c/p\u003e","title":"Praise for Oracle BI Publisher"},{"content":"Just a short post on this error code. Happened last time I recreated a physical standby . I could not find more than the reference note on this error on Metalink (a note that is just a copy of the output from command oerr ora 16047):\nCause: The DB_UNIQUE_NAME specified for the destination does not match the DB_UNIQUE_NAME at the destination.\nAction: Make sure the DB_UNIQUE_NAME specified in the LOG_ARCHIVE_DEST_n parameter defined for the destination matches the DB_UNIQUE_NAME parameter defined at the destination.\nThe error message was misleading in my case; my bad was that the log_archive_config on primary and the standby did not agree (actually it was nothing on standby). This error may show up in alert-log (didn\u0026rsquo;t every time with me), and will also show up in v$archive_dest:\nselect dest_id,status,error from v$archive_dest; If this is your case, check log_archive_config on all nodes.\n","permalink":"https://enesi.no/2008/05/ora-16047/","summary":"\u003cp\u003eJust a short post on this error code. Happened last time I \u003ca href=\"https://enesi.no/2008/04/create-a-standby-database-with-rman/\"\u003erecreated a physical standby\u003c/a\u003e\n. I could not find more than the reference note on this error on Metalink (a note that is just a copy of the output from command oerr ora 16047):\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eCause: The DB_UNIQUE_NAME specified for the destination does not match the DB_UNIQUE_NAME at the destination.\u003cbr\u003e\nAction: Make sure the DB_UNIQUE_NAME specified in the LOG_ARCHIVE_DEST_n parameter defined for the destination matches the DB_UNIQUE_NAME parameter defined at the destination.\u003c/p\u003e","title":"ORA-16047"},{"content":" After the incident with missing forced logging in primary database the logical standby had to be created again. Compared to what is normal these days the database is not huge, but the standby database remains on a server in a different city and the link between isn\u0026rsquo;t exactly a digital super highway. Also I had to restrict the consumed bandwidth during transfer with secure copy (scp -l 2500, restricts to 2500 kbs) or else connections to the primary database might suffer.\nRMAN provides the command duplicate target database for standby, but you don\u0026rsquo;t have to use this in order to create a physical standby. Actually some prefer not to use the duplicate command; the target database RMAN needs to be connected to while the duplicate process executes is the source database, which may be your main production database. If it takes some time to finish and you forget that your target is not the new database (the clone), but the primary, and you then do a shutdown immediate\u0026hellip; well, you probably do that mistake only once.\nAnyway you need a backup of the database and of the controlfile in standby format. In RMAN:\nrun { backup current controlfile for standby format \u0026#39;/s01/backup/foo/standby.ctl\u0026#39;; allocate channel c1 device type disk format \u0026#39;/s01/backup/foo/%U\u0026#39; maxpiecesize 10G backup database ; } I use the MAXPIECESIZE-option in order to limit the size of each file, it makes it all more manageable. Another useful option is RATE, which limits how much is read pr second from the database files. I did not use it since RMAN did not put a severe load on our SAN while the backup was running. The next three days where spent copying the files to the other server.\nThe instance for the physical standby was started in nomount and with RMAN connected to the soon to be physical database as target, restore was done like this:\nrestore standby controlfile from \u0026#39;/s01/backup/bar/standby.ctl\u0026#39;; alter database mount; catalog start with \u0026#39;/s01/backup/bar/ld\u0026#39;; crosscheck backup; restore database; The path in the catalog-command is just a common part for the backup pieces, the files started with ld. The restore actually finished several days after the backup was started on primary and physical standby would have to apply many archivelogs to catch up with primary. Here is where another useful feature of RMAN comes to rescue, incremental backup from a specific SCN. With the database at least in mount run the following query to find SCN of the physical standby:\nselect current_scn from v$database; Then on the primary start an incremental backup in RMAN:\nbackup incremental from scn 42 database format \u0026#39;/s01/backup/foo/incr%U\u0026#39;; 42 might not be your SCN (wasn\u0026rsquo;t our either), I used the SCN from the query minus one (a small overlap, didn\u0026rsquo;t bother to check the manual). The incremental backup was of course much smaller and took less time than a full backup to complete and transfer. After the backup was transferred to the other server I executed an incomplete recovery:\ncatalog start with \u0026#39;/s01/backup/bar/incr\u0026#39;; recover database noredo; This does not take much time to complete and the physical standby is not far behind primary. Normal apply can now be started with:\nalter database recover managed standby database disconnect; ","permalink":"https://enesi.no/2008/04/create-a-standby-database-with-rman/","summary":"\u003cp\u003e\u003ca href=\"http://bp3.blogger.com/_nF0DWc61z1k/SBYsc4u48oI/AAAAAAAAAA4/yDC-WErY5Js/s1600-h/DSC00076.JPG\" target=\"_blank\" rel=\"noopener\"\u003e\u003cimg loading=\"lazy\" src=\"http://bp3.blogger.com/_nF0DWc61z1k/SBYsc4u48oI/AAAAAAAAAA4/yDC-WErY5Js/s320/DSC00076.JPG\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003eAfter the incident with missing \u003ca href=\"https://enesi.no/2008/04/data-guard-and-forced-logging/\"\u003eforced logging in primary\u003c/a\u003e\n database the logical standby had to be created again. Compared to what is normal these days the database is not huge, but the standby database remains on a server in a different city and the link between isn\u0026rsquo;t exactly a digital super highway. Also I had to restrict the consumed bandwidth during transfer with secure copy (scp -l 2500, restricts to 2500 kbs) or else connections to the primary database might suffer.\u003c/p\u003e","title":"Create a standby database with RMAN"},{"content":" Any database that has gone into production is a legacy database according to Tom Kyte . I agree, but to me some databases give me associations to legacy more often than others. In a post at the OraStory blog Dom complains about companies that hire contractors for a maximum of one year, and due to the fact that it takes half a year to understand the business and the data, you hardly get to show what you can deliver.\nUnderstanding the business is one thing, but how it is projected into the database is another matter. With a crash introduction to SQL any developer can create a few tables and start dumping data. Everyone working with the code knows the connections, but lack of documentation, no constraints, and apparently inconsistency in the data makes it difficult for the newly hired DBA to understand what is going on or what is supposed to be going on. And this agile movement doesn\u0026rsquo;t make it easier, documentation is not exactly given high priority (\u0026hellip;the primary goal of software development is to create software, not documents- Ambler; Agile Database Techniques 2003, p8). In places where more attention has been given to the applications than the database the database can easily become a hard to manage legacy system. The characteristics are tables with few or no constraints (besides the primary key on an id column with values from a sequence, of course), tables with similar column names but with different content and meaning for the same column name, same data stored twice or more (and not in sync), same column used again in another table but with different type, missing data (not null-constraint missing) and so on.\nIf you have a tuning problem, adding constraints can help, but how can you add a constraint if you don\u0026rsquo;t know the data model or the connections in the data? Adding an unique index can certainly speed things up, but what if a column has 99% unique data, is that an error in the data or just coincidental? I have found the following techniques and strategies useful for understanding the data model:\nAsk questions - some like to talk about the code they wrote, but there is a limit to all interruptions.\nTalk to staff likely to be in need for reports and find out if they use reports from the database, then you trace the reports back to the database, i.e. the actual SQL used. Sometimes there is an intermediate data dump that is later imported to Excel for further processing. Looking at each step, asking for an interpretation of the report (if it is not obvious) you may see some new connections.\nViews in the database are useful to store complex queries, hide access to columns, create a standard interface to a set of tables, and more. But I have seen views defined as select col1 newname1, col2 newname2, \u0026hellip; from foo; i.e. the only change from the underlying table is new column names that relates to reports or a term that is used outside the database.\nStored procedure and packages are more likely to have comments than tables and views. If some strange logic is applied in the source, it motivates a question and may reveal something about the data model.\nAnticipate dirty data (if constraints are few). Say the quality of the applications are reasonable good, most of the data may be fine, but there will be some orphan rows, illegal values, etc. Doing some digging with simple select statements counting and looking for seemingly missing parent values (as per an imagined business rule, since the constraint is not defined) may show a set of dirty data that should be looked into. I have used this as a pretext to ask about the tables (\u0026ldquo;Is it not so that \u0026hellip; is supposed to have \u0026hellip;\u0026lsquo;cause I have seen some \u0026hellip; that does not have a corresponding \u0026hellip;\u0026rdquo;). This may be an opportunity to suggest to clean up the data and add constraints (because management understands the value of the data and the danger of not be able to trust them). Invalid data are more likely to have entered early in the history of the database (\u0026ldquo;well, some testing was done on the production database\u0026rdquo;) or because the data model evolved with new columns added or business rules changed at an early stage.\nBesides finding corrupt data, researching the statistical distribution is useful for tuning purposes and identifying other constraints. When a column with a discrete set of distinct values have relative few nulls I suspect a candidate for a not null constraint.\nI wish the oracle blogosphere would discuss database refactoring more often. The biggest challenge is to identify what can break elsewhere when you try to fix wrongdoings (impact analysis is what they call it) It is not trivial. Do people undertake this, or is it sometimes a relief when the contracts expires after twelve months so you can move on?\n","permalink":"https://enesi.no/2008/04/understanding-a-legacy-database/","summary":"\u003cp\u003e\u003ca href=\"http://bp2.blogger.com/_nF0DWc61z1k/SAtYFV2U95I/AAAAAAAAAAM/_mFPtl_JZGA/s1600-h/memstick.png\" target=\"_blank\" rel=\"noopener\"\u003e\u003cimg loading=\"lazy\" src=\"http://bp2.blogger.com/_nF0DWc61z1k/SAtYFV2U95I/AAAAAAAAAAM/_mFPtl_JZGA/s320/memstick.png\"\u003e\u003c/a\u003e\n\u003cbr\u003e\nAny database that has gone into production is a legacy database according to \u003ca href=\"http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:2168279457908#55724114487361\" target=\"_blank\" rel=\"noopener\"\u003eTom Kyte\u003c/a\u003e\n. I agree, but to me some databases give me associations to legacy more often than others. In a \u003ca href=\"http://orastory.wordpress.com/2008/03/19/the-dearth-of-oracle-rdbms-and-contracting/\" target=\"_blank\" rel=\"noopener\"\u003epost\u003c/a\u003e\n at the OraStory blog Dom complains about companies that hire contractors for a maximum of one year, and due to the fact that it takes half a year to understand the business and the data, you hardly get to show what you can deliver.\u003c/p\u003e","title":"Understanding a legacy database"},{"content":"When you start with Oracle Data Guard you have three sources of useful information: The Data Guard Concepts and Administration manual, which is quite good, other peoples experience, and after some time your own. The manual does not include all the mistakes one can do, nor what many really don\u0026rsquo;t use (like the data guard broker; not everybody would leave switchover, or worse failover to a robot).\nAfter a planned switchover to the physical standby the following ora-600 error message was received on our logical standby:\nORA-00600: internal error code, arguments: \\[krvxbpx20\\], \\[1\\], \\[34735\\], \\[60\\], \\[16\\], \\[\\], \\[\\],\\[\\] SQL Apply terminated after this error and neither restart of sql apply or bouncing the instance helped. On Metalink the only relevant information I found was Bug 6022014 , which has status \u0026ldquo;Closed, not a bug\u0026rdquo;. An ora-600 error that is not a bug\u0026hellip;well, if it\u0026rsquo;s my fault a decent error message would help. An SR was opened with Oracle Support and it was suspected quite right that supplemental logging was not enabled on primary and asked me to run the following query on the new primary:\nselect FORCE_LOGGING, SUPPLEMENTAL_LOG_DATA_PK, SUPPLEMENTAL_LOG_DATA_UI, SUPPLEMENTAL_LOG_DATA_FK, SUPPLEMENTAL_LOG_DATA_MIN, SUPPLEMENTAL_LOG_DATA_ALL from v$database; Which returned:\nFOR SUP SUP SUP SUPPLEME SUP \\--- --- --- --- -------- --- NO NO NO NO NO NO Suddenly our logical standby was not my biggest concern. How come the primary does not run with forced logging? Following the manual this is one of the first step when preparing for data guard, mentioned in section 3.1.1 (10g) and from table 3-1 looks like a requirement. The manual doesn\u0026rsquo;t tell you to execute it again for the new physical standby, neither before or after the first switchover. It surprised me that it is possible to successfully execute a switchover without having the new primary running with forced logging. Though I have not tested it, I imagine that the clone for the first physical standby was made before forced logging was enabled in the production database, and if I had enabled it in the order given in the manual the setting would follow the copy of the controlfile (in standby format) used to make the physical standby. At least that is the only explanation I have.\nThe solution was simple, just turn on forced logging with:\nalter database force logging; This is not recommended in an OLTP database with many ongoing transactions and can take some time to complete, but will finish soon when activity is low.\nJust in case you don\u0026rsquo;t know, forced logging makes sure nobody can create a table, index or other object without logging everything to the redo log, i.e. the NOLOGGING option in create statements is ignored. Checking that every node runs with forced logging should be part of a regular review of your data guard configuration.\nOur logical standby must be recreated since many transactions have been lost, but that\u0026rsquo;s OK now that a potential bigger problem have been avoided. Why do we have a physical and a logical standby ? The former serves the fastest and safest switchover or failover if necessary. The latter is used for off-site recovery options and as a source for our data warehouse. Remarkably stable and easy to setup once you have data guard with physical standby in place.\n","permalink":"https://enesi.no/2008/04/data-guard-and-forced-logging/","summary":"\u003cp\u003eWhen you start with Oracle Data Guard you have three sources of useful information: The Data Guard Concepts and Administration manual, which is quite good, other peoples experience, and after some time your own. The manual does not include all the mistakes one can do, nor what many really don\u0026rsquo;t use (like the data guard broker; not everybody would leave switchover, or worse failover to a robot).\u003c/p\u003e\n\u003cp\u003eAfter a planned switchover to the physical standby the following ora-600 error message was received on our logical standby:\u003c/p\u003e","title":"Data Guard and forced logging"},{"content":"A Sunday morning is a perfect moment for starting your blog. Not sure where this is going, or if it\u0026rsquo;s going. Here the other day I found a solution to a problem I thought others may have encountered. I figured that my own blog would be a place to write it down. At least I can find the solution again if I make the same mistake twice.\nThis blog will concern mostly about stuff related to Oracle, the database, that is, not the company. Posts for fellow DBAs and maybe developers (I\u0026rsquo;m a hybrid now, working as a semi-DBA among developers\u0026hellip;)\nUntil more, take care.\n","permalink":"https://enesi.no/2008/04/first-post.../","summary":"\u003cp\u003eA Sunday morning is a perfect moment for starting your blog. Not sure where this is going, or if it\u0026rsquo;s going. Here the other day I found a solution to a problem I thought others may have encountered. I figured that my own blog would be a place to write it down. At least I can find the solution again if I make the same mistake twice.\u003c/p\u003e\n\u003cp\u003eThis blog will concern mostly about stuff related to Oracle, the database, that is, not the company. Posts for fellow DBAs and maybe developers (I\u0026rsquo;m a hybrid now, working as a semi-DBA among developers\u0026hellip;)\u003c/p\u003e","title":"First post..."},{"content":"Hi, my name is Øyvind Isene, and I run Enesi Consulting.\nI live in Arendal, Norway, and work as an independent consultant specializing in Oracle Database on Linux and other Unix-like platforms. My work spans classical Oracle DBA responsibilities, high-performance SQL and PL/SQL development, and an increasing focus on data-centric development, data science, and integration between systems.\nWhat I do As an Oracle DBA and database developer, I help organizations design, build, and operate robust database platforms that are secure, performant, and easy to maintain. Typical engagements include:\nInstallation, configuration, and lifecycle management of Oracle databases on Linux and in virtualized or containerized environments. Architecture, implementation, and optimization of solutions on Oracle Cloud Infrastructure (OCI), including databases, networking, and related services. Performance diagnostics and optimization across the stack: from SQL and PL/SQL, via schema and index design, to storage and operating system. Database development: schema design, SQL and PL/SQL programming, batch processing, ETL-style workloads, and integration with applications and APIs. Exploratory and diagnostic data analysis to understand system behavior, identify trends, and support better technical and business decisions. High availability, backup, and recovery strategies to keep critical systems running and data safe. Modern features such as JSON, analytics, and automation to get more value from existing Oracle investments. How I work Optimization (aka tuning) has been one of my longest-standing interests. Databases often get blamed when systems are slow, but Oracle is very well instrumented and provides rich runtime information about what is actually happening. This makes it possible to pinpoint errors and bottlenecks, whether they originate inside the database or in surrounding application logic, infrastructure, or design. In practice, that often means uncovering issues in application code, data models, or integration patterns and helping teams fix the real root cause instead of just adding more hardware.\nI enjoy working closely with developers, operations, and business stakeholders to translate real-world requirements into clear, measurable technical solutions. Engagements can range from short troubleshooting assignments and performance reviews to longer-term architecture, implementation, and mentoring roles.\nCommunity and background I have been active in the Oracle community for many years and enjoy connecting with other professionals through conferences, user groups, and online channels. Travel is a natural part of my work, and combining assignments with the opportunity to meet new people, cultures, and technical challenges is something I value highly.\nIn February 2021 I started Enesi as my own company and have since focused on helping customers improve the reliability, performance, and maintainability of their data platforms. For many clients, this means getting the most out of existing systems before investing in new ones.\nAbout the name “Enesi” Enesi is simply my surname reversed. I also like prime numbers.\nIf you have a relevant challenge or would like to discuss how your database platform can be improved, feel free to get in touch at oyvind@enesi.no .\nCheers! Oracle ACE Pro Symposium 42 Enesi Consulting AS ","permalink":"https://enesi.no/about/","summary":"\u003cp\u003eHi, my name is Øyvind Isene, and I run Enesi Consulting.\u003c/p\u003e\n\u003cp\u003eI live in Arendal, Norway, and work as an independent consultant specializing in Oracle Database on Linux and other Unix-like platforms. My work spans classical Oracle DBA responsibilities, high-performance SQL and PL/SQL development, and an increasing focus on data-centric development, data science, and integration between systems.\u003c/p\u003e\n\u003ch2 id=\"what-i-do\"\u003eWhat I do\u003c/h2\u003e\n\u003cp\u003eAs an Oracle DBA and database developer, I help organizations design, build, and operate robust database platforms that are secure, performant, and easy to maintain. Typical engagements include:\u003c/p\u003e","title":"About me"},{"content":"Email: oyvind@enesi.no · LinkedIn ","permalink":"https://enesi.no/contact/","summary":"\u003cp\u003eEmail: \u003ca href=\"mailto:oyvind@enesi.no\"\u003eoyvind@enesi.no\u003c/a\u003e\n · \u003ca href=\"https://www.linkedin.com/in/oyvindisene/\" target=\"_blank\" rel=\"noopener\"\u003eLinkedIn\u003c/a\u003e\n\u003c/p\u003e","title":"Contact"}]