Tutorial 04: Multiple wrappers

This tutorial introduces the use of multiple data sources, and shows how can one exchange information between them. The code given here is based on this example and builds on the introduction on the CUDS API.

Background

One of the main strengths of CUDS objects is their ability to share information between different underlying data sources interchangeably. Using OSP-core’s innerworkings a data source can be represented as a CUDS object. A data source can be in turn a database, a simulation engine, or any other software package, which is able to either generate or store information.

We refer to a CUDS object, which represents an underlying data source as a wrapper, as it wraps around the data source. Wrappers use the CUDS API with the addition to some wrapper-specific methods, which will be discussed later on in this tutorial.

For a wrapper to be initialized, one needs some context for the underlying data source (e.g. location, credentials, etc.) for this we introduce an object called session. Conceptually a session can be thought as an interoperability level, or in simple terms it handles the transition from the user-friendly CUDS API to the more task-specific syntax data sources tend to have.

Let’s get hands on

We start by importing the example namespace of osp-core

[1]:
# If you did not install the CITY ontology (pico install city), you have to execute these commands first:
# from osp.core import Parser
# p = Parser()
# p.parse("city")

from osp.core import CITY

The pretty_print function is part of our utilities module and is a convinient way to output the tree-like structure of a CUDS object.

[2]:
from osp.core.utils import pretty_print

The getpass function is used to retrieve an input from an user. It prints a prompt, then reads input from the user until they press return.

[3]:
from getpass import getpass

The next statements imports the second data source session we will use in this example, namely a database, or more precisely the ORM toolkit SQLAlchemy which we will use in turn to connect to a PostgreSQL database. It will throws an error if it cannot find the SQLAlchemyWrapperSession as it needs to be installed from a separate repository. Please refer here for installation instructions.

[4]:
try:
    from osp.wrappers.sqlalchemy_wrapper_session import \
        SqlAlchemyWrapperSession
except ImportError as e:
    raise ImportError("For this example, the SQLAlchemy "
                      "wrapper for SimPhoNy is required!") from e

Next we import a session object, which will provide the context to a simple simulation engine we developed for demonstrational purposes. To install it refer to this repo.

[5]:
try:
    from osp.wrappers.dummy_simulation_wrapper import DummySimWrapperSession
except ImportError as e:
    raise ImportError("For this example, the dummy simulation "
                      "wrapper for SimPhoNy is required!") from e

The following lines prompt the user to enter the information needed to create a connection to a running instance of a PostgreSQL database, where the data of our simulation will be stored. Please make sure to point to an existing and running instance of PostgreSQL. To install PostgreSQL on your machine, please refer to their documentation.

The information is stored in a string postgres_url, which will later be passed to the SQLAlchemyWrapperSession object to initiate a connection with the data base.

[6]:
print("Input data to connect to Postgres table!")
user = input("User: ")
pwd = getpass("Password: ")
db_name = input("Database name: ")
host = input("Host: ")
port = int(input("Port [5432]: ") or 5432)
postgres_url = 'postgres://%s:%s@%s:%s/%s' % (user, pwd, host, port, db_name)
Input data to connect to Postgres table!
User: postgres
Password: ········
Database name: osp_core
Host: 127.0.0.1
Port [5432]:

In the next lines we create our small ontology-loving **EMMO town** example. Please pay a closer attention to the fourth line (this line in the multiple_wrappers_example.py file) as there you can see the power of the add method and how with one statement one can add multiple CITIZEN CUDS objects simultaneously.

[7]:
emmo_town = CITY.CITY(name='EMMO town')

emmo_town.add(CITY.CITIZEN(name='Emanuele Ghedini'), rel=CITY.HAS_INHABITANT)
emmo_town.add(CITY.CITIZEN(name='Adham Hashibon'), rel=CITY.HAS_INHABITANT)
emmo_town.add(CITY.CITIZEN(name='Jesper Friis'),
              CITY.CITIZEN(name='Gerhard Goldbeck'),
              CITY.CITIZEN(name='Georg Schmitz'),
              CITY.CITIZEN(name='Anne de Baas'),
              rel=CITY.HAS_INHABITANT)

emmo_town.add(CITY.NEIGHBOURHOOD(name="Ontology"))
emmo_town.add(CITY.NEIGHBOURHOOD(name="User cases"))
[7]:
<CITY.NEIGHBOURHOOD: bee623d8-fba2-4106-bfd4-c6a659950966,  CoreSession: @0x7fcdcdf94490>

Next we grow the Ontology neighbourhood by adding some streets to it: namely relationships and entities (puns are intended).

[8]:
ontology_uid = None
for neighbourhood in emmo_town.get(oclass=CITY.NEIGHBOURHOOD):
    if neighbourhood.name == "Ontology":
        ontology_uid = neighbourhood.uid
        neighbourhood.add(CITY.STREET(name="Relationships"), rel=CITY.HAS_PART)
        neighbourhood.add(CITY.STREET(name="Entities"), rel=CITY.HAS_PART)

Whenever you add a relationship to a CUDS object, OSP-core will always automatically add the inverse relationship. In our example case we can now retrieve from the “Ontology” neighbourhood onto, which town it belongs to.

[9]:
onto = emmo_town.get(ontology_uid)
print(onto.get(rel=CITY.IS_PART_OF)[0].name + ' is my city!')
EMMO town is my city!

Let’s now store our small city persistently in the PostgreSQL database. Working with session objects is similar to the way one is used to work with files in Python. And when you ponder a bit about it, it makes kind of sense, as what one intends to do is store some information in a data storage.

Using Python’s with statement the connection to the database will be maintained only within the scope of the with statement. After exiting its scope the connection will be closed automatically. We advise the use of the with statement as it automatically manages opening and closing of database connections.

First we open a connection to a PostgreSQL database through our SqlAlchemyWrapperSession and assign it to the session variable. Then we assign to the wrapper variable a CITY_WRAPPER and pass it the needed context with the session=session. From there on, we treat the wrapper variable as a normal CUDS object with the only difference that its internal state is managed by the SqlAlchemyWrapperSession behind the scenes. On the last line, we see the benefits of that by simply executing the commit command onto the session object. This will trigger a series of events, with the end result being that our EMMO town will be stored in the database.

[10]:
with SqlAlchemyWrapperSession(postgres_url) as session:
    wrapper = CITY.CITY_WRAPPER(session=session)
    wrapper.add(emmo_town)
    session.commit()

Next we show how one can use multiple data source wrappers simultaneously. First we open a connection to the PostgreSQL database through SqlAlchemyWrapperSession as shown above and assign it to the db_session variable. Then we retrieve the information we have previously stored to it using CUDS’ get method and use the pretty_print function to output the information.

Then we open a connection to our demonstrational simulation engine through DummySimWrapperSession and assign it to the sim_session variable. As you can see opening the second simulation session is within the scope of the SqlAlchemyWrapperSession.

In the scope of DummySimSession, we initialize a CitySimWrapper, pass it the context from the sim_session and assign it to the sim_wrapper variable. The CitySimWrapper consumes a city and a person. The magic happens in the following run method, which recognise that Peter is a person and it then transforms him into a citizen of the EMMO town. This new information is then stored in the sim_emmo_town automatically within the run method. We then output the information about Peter, who is now a citizen of EMMO town.

Finally we update our now outdated EMMO town in the database by using the update command. It checks for any inconsistencies between the EMMO town stored in the database, db_emmo_town, and the modified by our simulation engine town, sim_emmo_town. In our case it will find its new citizen Peter and it will add it to the EMMO town in the database instance of the town, db_emmo_town. This change is then in turn made persistent by calling the commit method, which will actually store the information the database.

[11]:
with SqlAlchemyWrapperSession(postgres_url) as db_session:
    db_wrapper = CITY.CITY_WRAPPER(session=db_session)
    db_emmo_town = db_wrapper.get(emmo_town.uid)
    print("The database contains the following information about the city:")
    pretty_print(db_emmo_town)

    # Working with a Simulation wrapper
    with DummySimWrapperSession() as sim_session:
        sim_wrapper = CITY.CITY_SIM_WRAPPER(num_steps=1,
                                            session=sim_session)
        new_inhabitant = CITY.PERSON(age=31, name="Peter")
        sim_emmo_town, _ = sim_wrapper.add(db_emmo_town, new_inhabitant)
        sim_session.run()
        print("The city has a new inhabitant:")
        pretty_print(sim_emmo_town.get(new_inhabitant.uid))

    # update database
    db_wrapper.update(sim_emmo_town)
    db_session.commit()

The database contains the following information about the city:
- Cuds object named <EMMO town>:
  uuid: 3954ba3f-27db-4cff-9453-619d2ee6435f
  type: CITY.CITY
  superclasses: CITY.CITY, CITY.POPULATED_PLACE, CITY.GEOGRAPHICAL_PLACE, CUBA.ENTITY
  values: coordinates: [0. 0.]
  description:
    To Be Determined

   |_Relationship CITY.HAS_INHABITANT:
   | -  CITY.CITIZEN cuds object named <Emanuele Ghedini>:
   | .  uuid: 89deb6b7-eaac-4db1-a573-53beb132a183
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Adham Hashibon>:
   | .  uuid: 121e73f3-35a9-4b49-8b32-6a58fd1375f1
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Jesper Friis>:
   | .  uuid: d077246d-105d-429f-b041-27dfb170ae31
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Gerhard Goldbeck>:
   | .  uuid: 0c1e5ca6-1279-465c-ab0a-701eee118343
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Georg Schmitz>:
   | .  uuid: 0d7630f0-6965-4ff0-a40b-c1851f354308
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Anne de Baas>:
   |    uuid: adc48ce9-264a-49cf-813d-377825228c49
   |    age: 25
   |_Relationship CITY.HAS_PART:
     -  CITY.NEIGHBOURHOOD cuds object named <Ontology>:
     .  uuid: d8e6cf00-0747-42c9-8c54-f09a532df7e0
     .  coordinates: [0. 0.]
     .   |_Relationship CITY.HAS_PART:
     .     -  CITY.STREET cuds object named <Relationships>:
     .     .  uuid: a54f76dc-1b2c-4899-9ae3-ca48ddb05811
     .     .  coordinates: [0. 0.]
     .     -  CITY.STREET cuds object named <Entities>:
     .        uuid: 092d17b5-37f1-4f54-ad71-d208218a1851
     .        coordinates: [0. 0.]
     -  CITY.NEIGHBOURHOOD cuds object named <User cases>:
        uuid: bee623d8-fba2-4106-bfd4-c6a659950966
        coordinates: [0. 0.]
The city has a new inhabitant:
- Cuds object named <Peter>:
  uuid: c8d11136-d88b-4cff-9508-240678947ce2
  type: CITY.CITIZEN
  superclasses: CITY.CITIZEN, CITY.PERSON, CITY.LIVING_BEING, CUBA.ENTITY
  values: age: 32
  description:
    To Be Determined

Finally to ensure our database has successfully interpreted our addition to Emmo town, we check its contents and print them using pretty_print.

[12]:
with SqlAlchemyWrapperSession(postgres_url) as db_session:
    db_wrapper = CITY.CITY_WRAPPER(session=db_session)
    db_emmo_town = db_wrapper.get(emmo_town.uid)
    print("The database contains the following information about the city:")
    pretty_print(db_emmo_town)
The database contains the following information about the city:
- Cuds object named <EMMO town>:
  uuid: 3954ba3f-27db-4cff-9453-619d2ee6435f
  type: CITY.CITY
  superclasses: CITY.CITY, CITY.POPULATED_PLACE, CITY.GEOGRAPHICAL_PLACE, CUBA.ENTITY
  values: coordinates: [0. 0.]
  description:
    To Be Determined

   |_Relationship CITY.HAS_INHABITANT:
   | -  CITY.CITIZEN cuds object named <Emanuele Ghedini>:
   | .  uuid: 89deb6b7-eaac-4db1-a573-53beb132a183
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Adham Hashibon>:
   | .  uuid: 121e73f3-35a9-4b49-8b32-6a58fd1375f1
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Jesper Friis>:
   | .  uuid: d077246d-105d-429f-b041-27dfb170ae31
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Gerhard Goldbeck>:
   | .  uuid: 0c1e5ca6-1279-465c-ab0a-701eee118343
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Georg Schmitz>:
   | .  uuid: 0d7630f0-6965-4ff0-a40b-c1851f354308
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Anne de Baas>:
   | .  uuid: adc48ce9-264a-49cf-813d-377825228c49
   | .  age: 25
   | -  CITY.CITIZEN cuds object named <Peter>:
   |    uuid: c8d11136-d88b-4cff-9508-240678947ce2
   |    age: 32
   |_Relationship CITY.HAS_PART:
     -  CITY.NEIGHBOURHOOD cuds object named <Ontology>:
     .  uuid: d8e6cf00-0747-42c9-8c54-f09a532df7e0
     .  coordinates: [0. 0.]
     .   |_Relationship CITY.HAS_PART:
     .     -  CITY.STREET cuds object named <Relationships>:
     .     .  uuid: a54f76dc-1b2c-4899-9ae3-ca48ddb05811
     .     .  coordinates: [0. 0.]
     .     -  CITY.STREET cuds object named <Entities>:
     .        uuid: 092d17b5-37f1-4f54-ad71-d208218a1851
     .        coordinates: [0. 0.]
     -  CITY.NEIGHBOURHOOD cuds object named <User cases>:
        uuid: bee623d8-fba2-4106-bfd4-c6a659950966
        coordinates: [0. 0.]
[ ]: