{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Tutorial: Multiple wrappers\n", "This tutorial introduces the use of multiple data sources, and shows how can one exchange information between them. The code given here is based on [this example](https://github.com/simphony/osp-core/blob/master/examples/multiple_wrappers_example.py) and builds on the [introduction on the CUDS API](./cuds_api.ipynb).\n", "\n", "## Background\n", "One of the main strengths of CUDS objects is their ability to share information between different underlying data sources interchangeably.\n", "Using OSP-core's inner workings a data source can be represented as a CUDS object.\n", "A data source can be in turn a database, a simulation engine, or any other software package, which is able to either generate or store information.\n", "\n", "We refer to a CUDS object, which represents an underlying data source as a **wrapper**, as it wraps around the data source.\n", "Wrappers use the CUDS API with the addition to some wrapper-specific methods, which will be discussed later on in this tutorial.\n", "\n", "For a wrapper to be initialized, one needs some context for the underlying data source (e.g. location, credentials, etc.) for this we introduce an object called **session**.\n", "Conceptually a session can be thought as an interoperability level, or in simple terms it handles the transition from the user-friendly CUDS API to the more task-specific syntax data sources tend to have." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Let's get hands on\n", "We start by importing the example namespace of OSP-core. If you haven't already, you should install the city ontology before:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO 2020-12-02 11:54:26,244 [osp.core.ontology.installation]: Will install the following namespaces: ['city']\n", "INFO 2020-12-02 11:54:26,280 [osp.core.ontology.yml.yml_parser]: Parsing YAML ontology file /mnt/c/Users/dea/Documents/Projects/simphony/osp-core/osp/core/ontology/docs/city.ontology.yml\n", "INFO 2020-12-02 11:54:26,331 [osp.core.ontology.yml.yml_parser]: You can now use `from osp.core.namespaces import city`.\n", "INFO 2020-12-02 11:54:26,333 [osp.core.ontology.parser]: Loaded 403 ontology triples in total\n", "INFO 2020-12-02 11:54:26,374 [osp.core.ontology.installation]: Installation successful\n" ] } ], "source": [ "!pico install city" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from osp.core.namespaces import city" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `pretty_print` function is part of our utilities module and is a convinient way to output the tree-like structure of a CUDS object." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from osp.core.utils import pretty_print" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `getpass` function is used to retrieve an input from an user. It prints a prompt, then reads input from the user until they press return." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from getpass import getpass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The next statements imports the second data source session we will use in this example, namely a database, or more precisely the ORM toolkit [SQLAlchemy](https://www.sqlalchemy.org/) which we will use in turn to connect to a PostgreSQL database.\n", "It will throws an error if it cannot find the `SQLAlchemyWrapperSession` as it needs to be installed from a separate repository. Please refer [here](https://github.com/simphony/simphony-sqlalchemy) for installation instructions." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from osp.wrappers.sqlalchemy import SqlAlchemySession" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we import a session object, which will provide the context to a simple simulation engine we developed for demonstrational purposes. It is already included in OSP-core." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "tags": [] }, "outputs": [], "source": [ "from osp.wrappers.simdummy import SimDummySession" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following lines prompt the user to enter the information needed to create a connection to a running instance of a PostgreSQL database, where the data of our simulation will be stored. Please make sure to point to an existing and running instance of PostgreSQL. To install PostgreSQL on your machine, please refer to their [documentation](https://www.postgresql.org/download/).\n", "\n", "The information is stored in a string `postgres_url`, which will later be passed to the `SQLAlchemyWrapperSession` object to initiate a connection with the data base." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Input data to connect to Postgres table!\n" ] } ], "source": [ "print(\"Input data to connect to Postgres table!\")\n", "user = input(\"User: \")\n", "pwd = getpass(\"Password: \")\n", "db_name = input(\"Database name: \")\n", "host = input(\"Host: \")\n", "port = int(input(\"Port [5432]: \") or 5432)\n", "postgres_url = 'postgres://%s:%s@%s:%s/%s' % (user, pwd, host, port, db_name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the next lines we create our small ontology-loving [**EMMO town** example](https://emmc.info/emmo-info/). Please pay a closer attention to the fourth line as there you can see the power of the `add` method and how with one statement one can add multiple `CITIZEN` CUDS objects simultaneously." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emmo_town = city.City(name='EMMO town')\n", "\n", "emmo_town.add(city.Citizen(name='Emanuele Ghedini'), rel=city.hasInhabitant)\n", "emmo_town.add(city.Citizen(name='Adham Hashibon'), rel=city.hasInhabitant)\n", "emmo_town.add(city.Citizen(name='Jesper Friis'),\n", " city.Citizen(name='Gerhard Goldbeck'),\n", " city.Citizen(name='Georg Schmitz'),\n", " city.Citizen(name='Anne de Baas'),\n", " rel=city.hasInhabitant)\n", "\n", "emmo_town.add(city.Neighborhood(name=\"Ontology\"))\n", "emmo_town.add(city.Neighborhood(name=\"User cases\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we grow the `Ontology` neighbourhood by adding some streets to it: namely *relationships* and *entities* (puns are intended)." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "ontology_uid = None\n", "for neighbourhood in emmo_town.get(oclass=city.Neighborhood):\n", " if neighbourhood.name == \"Ontology\":\n", " ontology_uid = neighbourhood.uid\n", " neighbourhood.add(city.Street(name=\"Relationships\"), rel=city.hasPart)\n", " neighbourhood.add(city.Street(name=\"Entities\"), rel=city.hasPart)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Whenever you add a relationship to a CUDS object, OSP-core will always automatically add the `inverse` relationship. In our example case we can now retrieve from the \"Ontology\" neighbourhood `onto`, which town it belongs to." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "EMMO town is my city!\n" ] } ], "source": [ "onto = emmo_town.get(ontology_uid)\n", "print(onto.get(rel=city.isPartOf)[0].name + ' is my city!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's now store our small city persistently in the PostgreSQL database. Working with session objects is similar to the way one is used to work with files in Python. And when you ponder a bit about it, it makes kind of sense, as what one intends to do is store some information in a data storage.\n", "\n", "Using Python's `with` statement the connection to the database will be maintained only within the scope of the `with` statement. After exiting its scope the connection will be closed automatically. We advise the use of the `with` statement as it automatically manages opening and closing of database connections.\n", "\n", "First we open a connection to a PostgreSQL database through our `SqlAlchemyWrapperSession` and assign it to the session variable. Then we assign to the `wrapper` variable a `CITY_WRAPPER` and pass it the needed context with the `session=session`. From there on, we treat the `wrapper` variable as a normal CUDS object with the only difference that its internal state is managed by the `SqlAlchemyWrapperSession` behind the scenes. On the last line, we see the benefits of that by simply executing the `commit` command onto the `session` object. This will trigger a series of events, with the end result being that our **EMMO town** will be stored in the database." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "with SqlAlchemySession(postgres_url) as session:\n", " wrapper = city.CityWrapper(session=session)\n", " wrapper.add(emmo_town)\n", " session.commit()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we show how one can use multiple data source wrappers simultaneously. First we open a connection to the PostgreSQL database through `SqlAlchemyWrapperSession` as shown above and assign it to the `db_session` variable. Then we retrieve the information we have previously stored to it using CUDS' `get` method and use the `pretty_print` function to output the information.\n", "\n", "Then we open a connection to our demonstrational simulation engine through `DummySimWrapperSession` and assign it to the `sim_session` variable. As you can see opening the second simulation session is within the scope of the `SqlAlchemyWrapperSession`. \n", "\n", "In the scope of `DummySimSession`, we initialize a `CitySimWrapper`, pass it the context from the `sim_session` and assign it to the `sim_wrapper` variable. The `CitySimWrapper` consumes a city and a person. The magic happens in the following `run` method, which recognise that **Peter** is a person and it then transforms him into a citizen of the **EMMO town**. This new information is then stored in the `sim_emmo_town` automatically within the `run` method. We then output the information about Peter, who is now a citizen of **EMMO town**.\n", "\n", "Finally we `update` our now outdated **EMMO town** in the database by using the `update` command. It checks for any inconsistencies between the **EMMO town** stored in the database, `db_emmo_town`, and the modified by our simulation engine town, `sim_emmo_town`. In our case it will find its new citizen **Peter** and it will add it to the **EMMO town** in the database instance of the town, `db_emmo_town`. This change is then in turn made persistent by calling the `commit` method, which will actually store the information the database." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The database contains the following information about the city:\n", "- Cuds object named :\n", " uuid: 96a2f49c-8009-456a-ad69-1cb1dea7f128\n", " type: city.City\n", " superclasses: city.City, city.GeographicalPlace, city.PopulatedPlace, cuba.Entity\n", " values: coordinates: [0 0]\n", " description: \n", " To Be Determined\n", "\n", " |_Relationship city.hasInhabitant:\n", " | - city.Citizen cuds object named :\n", " | . uuid: 08e05374-0187-4114-a114-085431aeebde\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: a084c4ba-c054-4afd-b194-b4e983743706\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: 209926bc-6bcd-466c-a86a-807780e4789c\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: 0a0a8488-7400-4f4e-9e44-dbf56782cd3c\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: cdfe72ec-fc15-4a14-a516-88b576bf0a27\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | uuid: 4ec1b1af-341f-4aa5-845d-fb3f9c502c9c\n", " | age: 25\n", " |_Relationship city.hasPart:\n", " - city.Neighborhood cuds object named :\n", " . uuid: e1632cf5-24e3-4afb-b72b-757ef62d5bef\n", " . coordinates: [0 0]\n", " . |_Relationship city.hasPart:\n", " . - city.Street cuds object named :\n", " . . uuid: 19a6a5a4-c22c-4be7-98c4-c69b3ee613d2\n", " . . coordinates: [0 0]\n", " . - city.Street cuds object named :\n", " . uuid: 681f7441-e711-4375-ab70-074234206faa\n", " . coordinates: [0 0]\n", " - city.Neighborhood cuds object named :\n", " uuid: 574e4a68-92c8-4b0e-b41f-e0341b42b00b\n", " coordinates: [0 0]\n", "The city has a new inhabitant:\n", "- Cuds object named :\n", " uuid: 7a263d53-9c4d-47ac-bb31-fa54d7920bcd\n", " type: city.Citizen\n", " superclasses: city.Citizen, city.LivingBeing, city.Person, cuba.Entity\n", " values: age: 32\n", " description: \n", " To Be Determined\n", "\n" ] } ], "source": [ "with SqlAlchemySession(postgres_url) as db_session:\n", " db_wrapper = city.CityWrapper(session=db_session)\n", " db_emmo_town = db_wrapper.get(emmo_town.uid)\n", " print(\"The database contains the following information about the city:\")\n", " pretty_print(db_emmo_town)\n", "\n", " # Working with a Simulation wrapper\n", " with SimDummySession() as sim_session:\n", " sim_wrapper = city.CitySimWrapper(numSteps=1,\n", " session=sim_session)\n", " new_inhabitant = city.Person(age=31, name=\"Peter\")\n", " sim_emmo_town, _ = sim_wrapper.add(db_emmo_town, new_inhabitant)\n", " sim_session.run()\n", " print(\"The city has a new inhabitant:\")\n", " pretty_print(sim_emmo_town.get(new_inhabitant.uid))\n", "\n", " # update database\n", " db_wrapper.update(sim_emmo_town)\n", " db_session.commit()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally to ensure our database has successfully interpreted our addition to **Emmo town**, we check its contents and print them using `pretty_print`." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The database contains the following information about the city:\n", "- Cuds object named :\n", " uuid: 96a2f49c-8009-456a-ad69-1cb1dea7f128\n", " type: city.City\n", " superclasses: city.City, city.GeographicalPlace, city.PopulatedPlace, cuba.Entity\n", " values: coordinates: [0 0]\n", " description: \n", " To Be Determined\n", "\n", " |_Relationship city.hasInhabitant:\n", " | - city.Citizen cuds object named :\n", " | . uuid: 08e05374-0187-4114-a114-085431aeebde\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: a084c4ba-c054-4afd-b194-b4e983743706\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: 209926bc-6bcd-466c-a86a-807780e4789c\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: 0a0a8488-7400-4f4e-9e44-dbf56782cd3c\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: cdfe72ec-fc15-4a14-a516-88b576bf0a27\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | . uuid: 4ec1b1af-341f-4aa5-845d-fb3f9c502c9c\n", " | . age: 25\n", " | - city.Citizen cuds object named :\n", " | uuid: 7a263d53-9c4d-47ac-bb31-fa54d7920bcd\n", " | age: 32\n", " |_Relationship city.hasPart:\n", " - city.Neighborhood cuds object named :\n", " . uuid: e1632cf5-24e3-4afb-b72b-757ef62d5bef\n", " . coordinates: [0 0]\n", " . |_Relationship city.hasPart:\n", " . - city.Street cuds object named :\n", " . . uuid: 19a6a5a4-c22c-4be7-98c4-c69b3ee613d2\n", " . . coordinates: [0 0]\n", " . - city.Street cuds object named :\n", " . uuid: 681f7441-e711-4375-ab70-074234206faa\n", " . coordinates: [0 0]\n", " - city.Neighborhood cuds object named :\n", " uuid: 574e4a68-92c8-4b0e-b41f-e0341b42b00b\n", " coordinates: [0 0]\n" ] } ], "source": [ "with SqlAlchemySession(postgres_url) as db_session:\n", " db_wrapper = city.CityWrapper(session=db_session)\n", " db_emmo_town = db_wrapper.get(emmo_town.uid)\n", " print(\"The database contains the following information about the city:\")\n", " pretty_print(db_emmo_town)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 2 }