Twitter

Python venv

A lot of application running on our VMs use python. As python evolves a lot and is extensible through modules, it is very frequent that a user do not have the correct modules / modules versions he needs for his project.
To keep it clean and flexible, we do not want to install python packages at the system level, we want each non-privileged user to be using python virtual environment. Note that the base python version requested by the user will need to be installed at the system level.


Proxy

Depending on your company, you may need a proxy to be able to install any python version or module:
export https_proxy=http://www-proxy-your_company.com:80


Example

Below example will be using a non-privileged user "frdenis" on the myvm VM.
[frdenis@myvm ~]$ id
uid=1003(frdenis) gid=1003(frdenis) groups=1003(frdenis) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[frdenis@myvm ~]$
The default python of the system is 3.6.8:
[opc@myvm ~]$ python -V
Python 3.6.8
[opc@myvm ~]$
Python 3.11 will need to be installed as the example will showcase a python3.11 virtual environment:
[opc@myvm ~]$ sudo dnf install -y python3.11 python3.11-pip
. . .
[opc@myvm ~]$ python3.11 -V
Python 3.11.13
[opc@myvm ~]$


Python venv


Check the system python

[frdenis@myvm ~]$ type python
python is /usr/bin/python
[frdenis@myvm ~]$ /usr/bin/python --version
Python 3.6.8
[frdenis@myvm ~]$

Create and activate a venv

We will name this virtual environment "venv" in this example which is the usage.
[frdenis@myvm ~]$ python -V
Python 3.6.8
[frdenis@myvm ~]$ python3.11 -m venv venv             <== the second "venv" is the name of your venv
[frdenis@myvm ~]$ source venv/bin/activate
(venv) [frdenis@myvm ~]$ python -V            <== note that starting "(venv)" showing you are IN the virtual env
Python 3.11.13
(venv) [frdenis@myvm ~]$

Install packages in the venv

Once the venv activated, we can install any module we want in it and as our non-privileged user:
(venv) [frdenis@myvm ~]$ python
Python 3.11.13 (main, Apr 27 2026, 16:44:16) [GCC 8.5.0 20210514 (Red Hat 8.5.0-28.0.1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy                               <== numpy is not default
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>> quit()
(venv) [frdenis@myvm ~]$ export https_proxy=http://www-proxy-hqdc.us.oracle.com:80
(venv) [frdenis@myvm ~]$ pip install numpy
Collecting numpy
  Downloading numpy-2.4.6-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.9/16.9 MB 9.2 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-2.4.6
[notice] A new release of pip available: 22.3.1 -> 26.2.1
[notice] To update, run: pip install --upgrade pip
(venv) [frdenis@myvm ~]$ python
Python 3.11.13 (main, Apr 27 2026, 16:44:16) [GCC 8.5.0 20210514 (Red Hat 8.5.0-28.0.1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>>

Showcase

Let's write a simple python script using numpy:
[frdenis@myvm ~]$ cat test_env.py
import sys
import numpy as np
print("Python Version :", sys.version.split()[0])
print("Numpy Version  :", np.__version__)
print("Numpy Test     :", np.array([10, 20, 30]) * 2)
[frdenis@myvm ~]$
It works in the venv:
(venv) [frdenis@myvm ~]$ python test_env.py
Python Version : 3.11.13
Numpy Version  : 2.4.6
Numpy Test     : [20 40 60]
(venv) [frdenis@myvm ~]$
But not outside of the venv (as numpy is not default) and has only been installed in the venv:
(venv) [frdenis@myvm ~]$ deactivate
[frdenis@myvm ~]$ python test_env.py
Traceback (most recent call last):
  File "test_env.py", line 2, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'
[frdenis@myvm ~]$


Automatically use the venv

As it may not be nice to source the venv each time we want to run a python script (from a shell for example), 2 solutions exist for that:

Run the script using the venv python

[frdenis@myvm ~]$ ~/venv/bin/python test_env.py
Python Version : 3.11.13
Numpy Version  : 2.4.6
Numpy Test     : [20 40 60]
[frdenis@myvm ~]$

Use a shebang which points to the venv

[frdenis@myvm ~]$ cat test_env.py
#!/home/frdenis/venv/bin/python                             <== here, this is the shebang, adapt to your path
import sys
import numpy as np
print("Python Version :", sys.version.split()[0])
print("Numpy Version  :", np.__version__)
print("Numpy Test     :", np.array([10, 20, 30]) * 2)
[frdenis@myvm ~]$ chmod u+x test_env.py                   <== has to be done only once when setting it up
[frdenis@myvm ~]$ ./test_env.py
Python Version : 3.11.13
Numpy Version  : 2.4.6
Numpy Test     : [20 40 60]
[frdenis@myvm ~]$


Transfer a venv to another user/machine

Note: Do NOT copy the "venv" directory to another user or machine, it contains hardcoded absolute paths and machine specifics which wont make it work.

Export the module list

[frdenis@myvm ~]$ source venv/bin/activate
(venv) [frdenis@myvm ~]$ pip freeze > requirements.txt
(venv) [frdenis@myvm ~]$ cat requirements.txt
numpy==2.4.6
(venv) [frdenis@myvm ~]$

Re-apply to another virtual env

Once this requirements.txt file has been transferred:
[new_user@another_vm ~]$ python3.11 -m venv new_venv             <== create a new venv
[new_user@another_vm ~]$ source new_venv/bin/activate            <== activate the new venv
[new_user@another_vm ~]$ pip install -r requirements.txt         <== reinstall the modules from the original venv
. . .
[new_user@another_vm ~]$

OCI: Datapump between 23ai ADB and 19c ADB using database link

Now that we know how to manually create a 23ai ADB in OCI, that we also know how to create a database link between a 23ai ADB and a 19C ADB, let's now try a datapump between a 19c ADB and a 23ai ADB (oh, this is also a migration method, right?) using a database link.
First, let's remind our environment:
[fred@myvm ~]$ cd wallet_23ai
[fred@myvm wallet_23ai]$ export TNS_ADMIN=$(pwd)
[fred@myvm wallet_23ai]$ sqlplus admin/"Iw0ntt3lly0u"@sandbox01_high
SQL> select banner_full from v$version;
BANNER_FULL
--------------------------------------------------------------------------------
Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - Production
Version 23.6.0.24.10
SQL> select banner_full from v$version@TO_19C;
BANNER_FULL
--------------------------------------------------------------------------------
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.25.0.1.0
SQL>
Let's datapump that ~ 50 million rows table from our 19C ADB:
SQL> select count(*) from USER_19C.TABLE_19C@TO_19C;
COUNT(*)
----------
 48930604
SQL>
Let's create a simple datapump parfile (nothing specific compared to a "classic" Oracle database):
[fred@myvm ~]$ cat imp.par
TABLES=USER_19C.TABLE_19C
TABLE_EXISTS_ACTION=append
REMAP_SCHEMA=USER_19C:USER_23C
network_link=TO_19C
encryption_pwd_prompt=no
directory=DATA_PUMP_DIR
logfile=imp.log
[fred@myvm ~]$
And run it:
[fred@myvm ~]$ impdp admin/"Iw0ntt3lly0u"@sandbox01_high parfile=imp.par
Import: Release 21.0.0.0.0 - Production on Thu Nov 21 15:47:06 2024
Version 21.14.0.0.0
Copyright (c) 1982, 2024, Oracle and/or its affiliates.  All rights reserved.
Connected to: Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - Production
Starting "ADMIN"."SYS_IMPORT_TABLE_01":  admin/********@sandbox01_high parfile=imp.par
Processing object type TABLE_EXPORT/TABLE/TABLE
. . imported "USER_23C"."TABLE_19C"               48930604 rows
Processing object type TABLE_EXPORT/TABLE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Job "ADMIN"."SYS_IMPORT_TABLE_01" completed successfully at Thu Nov 21 16:09:50 2024 elapsed 0 00:22:41
[fred@myvm ~]$
That was very easy and pretty fast: 22 minutes for 48 million lines (with no parallel, this example table was not partitioned).

OCI: ADB 23ai to ADB 19c Database link

This one is very similiar to OCI: Database Link between 2 Autonomous Databases 19c but this time it is the creation of a database link from a 23ai ADB to a 19c ADB. The only difference on top of the version is that I will be creating a private DB Link and not a public DB Link. This is more to ensure that everything is working as expected. I'll put very less comment here aseach step has already been detailed in OCI: Database Link between 2 Autonomous Databases 19c.
Get the 19c DB wallet:
[fred@myvm ~]$ mkdir adb19c_wallet
[fred@myvm ~]$ oci db autonomous-database generate-wallet --autonomous-database-id ocid1.autonomousdatabase.oc1.iad.l4pq7nwusgtxorjiszblek2aqqe2b4jusqtcw5fuev6bwcntifaaq5lna3bl --file adb19c_wallet/adb19c_wallet.zip --password "abc123456"
Downloading file  [####################################]  100%
[fred@myvm ~]$ cd adb19c_wallet
[fred@myvm adb19c_wallet]$ ls -ltr
-rw-rw-r--. 1 frdenis frdenis 21964 Nov 20 12:21 adb19c_wallet.zip
[fred@myvm adb19c_wallet]$ unzip adb19c_wallet.zip
Archive:  adb19c_wallet.zip
  inflating: ewallet.pem
  inflating: README
  inflating: cwallet.sso
  inflating: tnsnames.ora
  inflating: truststore.jks
  inflating: ojdbc.properties
  inflating: sqlnet.ora
  inflating: ewallet.p12
  inflating: keystore.jks
[fred@myvm adb19c_wallet]$
Create a directory into the 23ai database:
[fred@myvm adb19c_wallet]$ cd ../wallet_23ai/
[fred@myvm wallet_23ai]$ export TNS_ADMIN=$(pwd)
[fred@myvm wallet_23ai]$ sqlplus admin/"Iw0ntt3lly0u"@sandbox01_high
SQL>*Plus: Release 21.0.0.0.0 - Production on Wed Nov 20 12:22:22 2024
Version 21.14.0.0.0
Copyright (c) 1982, 2022, Oracle.  All rights reserved.
Last Successful login time: Tue Nov 19 2024 15:35:47 +00:00
Connected to:
Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - Production
Version 23.6.0.24.10
SQL> create directory wallet19c as 'wallet19c';
Directory created.
SQL> select DIRECTORY_NAME, DIRECTORY_PATH from dba_directories where DIRECTORY_NAME = 'WALLET19C';
DIRECTORY_NAME        DIRECTORY_PATH
----------------   -----------------------------------------------------------------
WALLET19C            /u03/dbfs/27409389D44E5891E063D911000A6123/data/wallet19c
SQL> select * from dbms_cloud.list_files('WALLET19C');
no rows selected
SQL>
Move the cwallet.sso file into a directory into the 23ai database (long journey to copy a file):
[fred@myvm ~]$ oci os bucket create --name wallet --compartment-id $COMP_DEV
[fred@myvm ~]$ oci os object put --bucket-name wallet --file adb19c_wallet/cwallet.sso
[fred@myvm ~]$ oci iam auth-token create --user-id ocid1.user.oc1..4p2paj5vy5ghhx7ayavkqoiqarqaqomx3jroqabkl32aze7qpamp7wfkfayo --description fred_cred_for_wallet_dblink
. . .
    "token": "9m>1x0heLR4bDe:{8lnT",
. . .
[fred@myvm ~]$ sqlplus admin/"Iw0ntt3lly0u"@sandbox01_high
Connected to:
Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - Production
Version 23.6.0.24.10
SQL> show user
USER is "ADMIN"
SQL> begin
  2  DBMS_CLOUD.CREATE_CREDENTIAL(credential_name => 'fred_cred_for_wallet_dblink', username => 'fred', password => '9m>1x0heLR4bDe:{8lnT') ;
  3  end;
  4  /
PL/SQL procedure successfully completed.
SQL>
[fred@myvm ~]$ NAMESPACE=$(oci os ns get | jq -r '.data')
[fred@myvm ~]$ REGION="us-ashburn-1"
[fred@myvm ~]$ BUCKET_NAME="wallet"
[fred@myvm ~]$ FILE="cwallet.sso"
[fred@myvm ~]$ echo "https://${NAMESPACE}.objectstorage.${REGION}.oci.customer-oci.com/n/${NAMESPACE}/b/${BUCKET_NAME}/o/${FILE}"
https://qdci7zudoklt.objectstorage.us-ashburn-1.oci.customer-oci.com/n/qdci7zudoklt/b/wallet/o/cwallet.sso
[fred@myvm ~]$ 
SQL> begin
  2  DBMS_CLOUD.GET_OBJECT(
  3  object_uri => 'https://qdci7zudoklt.objectstorage.us-ashburn-1.oci.customer-oci.com/n/qdci7zudoklt/b/wallet/o/cwallet.sso',
  4  directory_name => 'WALLET19C',
  5  credential_name => 'fred_cred_for_wallet_dblink');
  6  end;
  7  /
PL/SQL procedure successfully completed.
SQL> SELECT OBJECT_NAME, BYTES, CREATED from DBMS_CLOUD.LIST_FILES('WALLET19C');
OBJECT_NAME           BYTES CREATED
---------------- ---------- -----------------------------------
cwallet.sso            5349 20-NOV-24 02.35.52.642528 PM +00:00
SQL>
Create some credentials on the 23ai DB (a user and password from the target 19c database are needed):
SQL> show user
USER is "ADMIN"
SQL> begin
  2  DBMS_CLOUD.CREATE_CREDENTIAL(credential_name => 'TO_19C', username => 'ADMIN', password => 'xxxx');
  3  end;
  4  /
PL/SQL procedure successfully completed.
SQL>
And create the database link:
SQL> BEGIN
 DBMS_CLOUD_ADMIN.CREATE_DATABASE_LINK(
 db_link_name => 'TO_19C',
 hostname => '19cDB_host.adb.us-ashburn-1.oraclecloud.com',
 port => '1522',
 service_name => '1a0e3d6a347c8g0_19cDB_high.adb.oraclecloud.com',
 credential_name => 'TO_19C',
 directory_name => 'WALLET19C',
 private_target => TRUE,
 public_link => FALSE);
 END;
 /
 PL/SQL> procedure successfully completed.
SQL>
And finally time to test!
SQL> select banner_full from v$version;
BANNER_FULL
--------------------------------------------------------------------------------
Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - Production
Version 23.6.0.24.10
SQL> select banner_full from v$version@TO_19C;
BANNER_FULL
--------------------------------------------------------------------------------
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.25.0.1.0
SQL>
And you can drop a database link as below:
SQL> begin
  2  DBMS_CLOUD_ADMIN.DROP_DATABASE_LINK(db_link_name => 'MY_PUBLIC_DBLINK', public_link => FALSE);     # Private DB Link
  3  end;
  4  /
PL/SQL procedure successfully completed.
SQL> begin
  2  DBMS_CLOUD_ADMIN.DROP_DATABASE_LINK(db_link_name => 'MY_PRIVATE_DB_LINK', public_link => TRUE);     # Public DB Link
  3  end;
  4  /
PL/SQL procedure successfully completed.
SQL>

Python venv

A lot of application running on our VMs use python. As python evolves a lot and is extensible through modules, it is very frequent that a u...