Bulk data loading API
grano has a Python-based bulk loading API that can be used to implement custom data loaders (e.g. as part of an ETL process). The interface uses a factory pattern to generate projects, entities and graphs. The following tutorial assumes that you have grano installed and configured to use a local database.
Creating a loader
As a first step, you need to create a
Loader object. A loader is related to one particular
Project, which will be created if it isn’t already present in the database.
For the moment, the loader itself does not support creating schemata or loading them from a file. This can be done, though, using the command line:
For more details, see the schema documentation.
Note that all data loaded via the loader API will be created by the
_system user. To allow further accounts to manage and modify the data, you may have to grant them permissions explicitly.
Creating entities and relations
Once the loader (and project) have been initalized, you can begin to create entities and relations. Both entities and relations have so-called properties associated with them, each of which can be set individually.
In order to update existing entities and relations (rather than duplicating them when a process is executed multiple times), the loader uses a set of properties of the entity or relation which will be considered unique. This means that if an entity or relation exist with a given value, that record will be updated - otherwise, a new one is created.
For entities, the default unique property is
name (i.e. there can only be one entity with each name, which will be updated when the same name is used again). Relations, on the other hand, will always include the source and target entities. Other properties can be added as necessary.
Let’s create an entity:
Once we create a second entity, we can then establish a relation between both:
Finally, we must tell the loader to actually commit the data we have given to the database:
The interface in detail
Visit the grano API documentationfor a detailed overview of the available classes and modules.