API Reference

Admin

The admin module hosts all functionality that is necessary to connect to the grid administration file. That is, the data can be read, altered, serialized and exported by the classes and methods implemented here.

Breaches

Models

Breaches are better described as possible breach locations as they represent the intersection between threedicore 1d-2d flow lines and levee geometries from the models spatialite database. That is, once the kcu (the type classifier, see Line types) of the 1d-2d flow line has the value 56 there actually will be an active link between the 1d and the 2d calculation nodes.

To query the Breaches model:

>>> from threedigrid.admin.breaches.models import Breaches
>>> from threedigrid.admin.gridadmin import GridH5Admin

>>> f = 'gridadmin.h5'
>>> ga = GridH5Admin(f)
>>> # get all active breaches
>>> ga.breaches.filter(kcu__eq=56).data
class threedigrid.admin.breaches.models.Breaches(*args, **kwargs)[source]

fields originating from threedicore:

  • levbr (breach width)
  • levmat (material code)
  • levl (exchange level)
  • kcu (type field 55 or 56)

added fields from spatialite database:

  • content_pk (primary key database)
  • seq_ids (sequence ids generated during input file generation)

Exporters

At this moment there is one exporter for breach data, called BreachesOgrExporter. For an overview of supported drivers call:

>>> from threedigrid.admin.breaches.exporters import BreachesOgrExporter
>>> from threedigrid.admin.gridadmin import GridH5Admin

>>> f = 'gridadmin.h5'
>>> ga = GridH5Admin(f)

>>> # get all active breaches
>>> active_breaches = ga.breaches.filter(kcu__eq=56)

>>> exporter = BreachesOgrExporter(active_breaches)
>>> exporter.supported_drivers
>>> {u'ESRI Shapefile', u'GPKG'}
class threedigrid.admin.breaches.exporters.BreachesOgrExporter(breaches)[source]

ogr exporter for breaches. See <instance>.supported_drivers to get a list of supported drivers

save(file_name, breach_data, target_epsg_code, **kwargs)[source]

save breaches to file

Parameters:
  • file_name – file name including full path
  • breach_data – queryset
  • target_epsg_code – desired epsg code for coords
  • kwargs – does not take extra kwargs

Lines

Models

In the threedicore lines are called flow lines and represent links between calculation points. They can exist between 1d and 2d points, between surface and groundwater nodes etc. The line type is described by the kcu attribute For an overview of the kcu types see Line types.

class threedigrid.admin.lines.models.Channels(*args, **kwargs)[source]
class threedigrid.admin.lines.models.Culverts(*args, **kwargs)[source]
class threedigrid.admin.lines.models.Lines(*args, **kwargs)[source]

fields originating from threedicore:

  • kcu (line type)
  • lik ()
  • line (index of node representing start/end point)

added fields from spatialite database:

  • content_pk (primary key database)
  • content_type (type of object as stored in database, e.g v2_channel, v2_weir etc)
custom fields
  • line_coords (coordinate pairs start/end point)
  • line_geometries (geometries from content_type)
class threedigrid.admin.lines.models.Orifices(*args, **kwargs)[source]
class threedigrid.admin.lines.models.Pipes(*args, **kwargs)[source]
class threedigrid.admin.lines.models.Weirs(*args, **kwargs)[source]

Exporters

class threedigrid.admin.lines.exporters.LinesOgrExporter(lines)[source]

Exports to ogr formats. You need to set the driver explicitly before calling save()

save(file_name, line_data, target_epsg_code, **kwargs)[source]

save to file format specified by the driver, e.g. shapefile

Parameters:
  • file_name – name of the outputfile
  • line_data – dict of line data

Nodes

Models

The Nodes models represent the actual calculation points of the threedicore and derived information like the cells they lie in.

For examples how to query the Nodes model see API documentation

class threedigrid.admin.nodes.models.Cells(*args, **kwargs)[source]

Model that represents the threedicore calculation cells. Cells are a sub-class of Nodes because they share the same attributes. Cells, however, also have a z_coordinate, the bottom level of the grid cell and cell_coords, the lower left and upper right coordinates of the cells extent.

bounds
Returns:coordinates of the cell bounds (counter clockwise): minx, miny, maxx, miny, maxx, maxy, minx, maxy
get_id_from_xy(x, y, xy_epsg_code=None, subset_name=None)[source]
Parameters:
  • x – the x coordinate in xy_epsg_code
  • y – the y coordinate in xy_epsg_code
  • subset_name – filter on a subset of cells
Returns:

numpy array with cell id’s for x, y

class threedigrid.admin.nodes.models.Grid(*args, **kwargs)[source]

Implemented fields:

  • nodm
  • nodn
  • nodk

They all have the same size as nodes and cells, which is why this model lives in the nodes/model module. In fact they are attributes of the cell coordinates

get_pixel_map(dem_pixelsize, dem_shape)[source]

get the node grid to pixel map

Parameters:
  • dem_pixelsize – pixelsize of the geo tiff
  • dem_shape – shape of the numpy representation of the geo tiff
Returns:

flipped array of the dem_shape that matches the geotiff

Model Meta options

A model can have a Meta class that accepts several options as input. Meta options are used at this point mainly in the models that use the ResultMixin.

field_attrs

Attributes for the given fields. ‘units’ for instance could yield ‘m’ [meters]. The attribute will be avaible throught the models _meta property. For example:

>>> gr.nodes._meta.s1
>>> s1(units=u'm', long_name=u'waterlevel', standard_name=u'water_surface_height_above_reference_datum')

s1 is a namedtuple so you can retrieve the units attribute by the the . notation gr.nodes._meta.s1.units or using the _as_dict method gr.nodes._meta.s1._asdict()['units']

composite_fields

If a single model field is composed of multiple sources, you can use the composite_fields option to join those sources into a single model field. The threedicore result netCDF file, for instance, is split into 1D and 2D subsets:

>>> gr.nodes._datasource.keys()
>>> ['Mesh2D_s1', 'Mesh1D_s1',...]

To map those source fields simply define a composite_fields dictionary:

>>> composite_fields = {'s1': ['Mesh2D_s1', 'Mesh1D_s1'],}

This will add the field s1 to your nodes model.

Note

fields starting with ‘_’ are private and will not be added to fields property

lookup_fields

A pair of fields of search_array and index_array. The search_array can be used to lookup the indices from index_array.

class threedigrid.orm.base.options.ModelMeta[source]

Metaclass for defining model meta classes.

If you have a complexer setup you can make use of the ModelMeta meta class that provides some field constructors behind the scenes. Aggregation files potentially contain a lot of different aggregation variables resulting in a lot of different source fields. You don’t manually have to define all those combinations simply define a base_composition and a composition_vars dictionary. This computes all necessary composite_fields automatically.

static combine_vars(prod_a, prod_b, join_str='_')[source]

Return the cartesian product of prod_a with prod_b, combined together with join_str.

>>> combine_vars({'a', 'b'}, {'c', 'd'})
['a_c', 'a_d', 'b_c', 'b_d']
Parameters:
  • prod_a – (iterable)
  • prod_b – (iterable)
  • join_str – (string)
Returns:

(list) ∏ (set_a, set_b)

Example

The NodesAggregateResultsMixin makes use of a base_composition and composition_vars setup.

The base_composition is equal to the fields in the result netcdf file. Those are:

BASE_COMPOSITE_FIELDS = {
    's1': ['Mesh2D_s1', 'Mesh1D_s1'],
    'vol': ['Mesh2D_vol', 'Mesh1D_vol'],
    'su': ['Mesh2D_su', 'Mesh1D_su'],
    'rain': ['Mesh2D_rain', 'Mesh1D_rain'],
    'q_lat': ['Mesh2D_q_lat', 'Mesh1D_q_lat'],
    '_mesh_id': ['Mesh2DNode_id', 'Mesh1DNode_id'],  # private
}

The composition_vars

composition_vars = {
    's1': ['min', 'max', 'avg'],
    'vol': ['max', 'cum'],
    'su': ['min'],
    'rain': ['avg'],
    'q_lat': ['cum'],
}

define all possible combinations with those field names. s1 and vol alone result in 10 field names.

>>> Mesh2D_s1_min
>>> Mesh1D_s1_min
>>> Mesh2D_s1_max
>>> Mesh1D_s1_max
>>> Mesh2D_s1_avg
>>> Mesh1D_s1_avg
>>> Mesh2D_vol_max
>>> Mesh1D_vol_max
>>> Mesh2D_vol_cum
>>> Mesh1D_vol_cum

Using the __metaclass__ = ModelMeta hook will compute all those combinations automatically.

Note

The field names that are being added to the model instance will be a combination of the base_composition keys and the composition_vars keys like s1_min, s1_max etc

class NodesAggregateResultsMixin(ResultMixin):

    class Meta:
        __metaclass__ = ModelMeta

        base_composition = BASE_COMPOSITE_FIELDS

        # attributes for the given fields
        field_attrs = ['units', 'long_name']

        # extra vars that will be combined with the
        # composite fields, e.g.
        # s1 --> s1_min [Mesh2D_s1_min + Mesh1D_s1_min]
        #    --> s1_max  [Mesh2D_s1_max + Mesh1D_s1_max]
        composition_vars = {
            's1': ['min', 'max', 'avg'],
            'vol': ['max', 'cum'],
            'su': ['min'],
            'rain': ['avg'],
            'q_lat': ['cum'],
        }

        lookup_fields = ('id', '_mesh_id')

ORM

Fields

Base fields

class threedigrid.orm.base.fields.ArrayField[source]

Generic field that can be used to describe values to be retrieved from a Datasource.

static get_value(datasource, name, **kwargs)[source]
Returns: the data from the datasource
or None if ‘name’ is not in the datasource

Optional transforms can be done here.

class threedigrid.orm.base.fields.BooleanArrayField[source]

Generic array field for boolean values.

Because HDF5 does not support boolean datatype. No data fields are interpreted as False.

static get_value(datasource, name, **kwargs)[source]
Returns: the data from the datasource
or None if ‘name’ is not in the datasource

Optional transforms can be done here.

class threedigrid.orm.base.fields.IndexArrayField(to=None)[source]

Simple pointer

class threedigrid.orm.base.fields.TimeSeriesArrayField[source]
static get_value(datasource, name, **kwargs)[source]
Returns: the data from the datasource
or None if ‘name’ is not in the datasource

Optional transforms can be done here.

class threedigrid.orm.base.fields.TimeSeriesCompositeArrayField(meta=None)[source]

Field for composite arrays.

Result (netCDF) files split their data into subsets, e.g. 1D and 2D. A composite field can be used to combine several data source fields into a single model field by specifying a composition dict. Example:

LINE_COMPOSITE_FIELDS = {
    'au': ['Mesh1D_au', 'Mesh2D_au'],
    'u1': ['Mesh1D_u1', 'Mesh2D_u1'],
    'q': ['Mesh1D_q', 'Mesh2D_q']
}
get_value(datasource, name, **kwargs)[source]
Parameters:
  • datasource – a datasource object that can return data on __getitem__()
  • name – the name of the section to read, e.g a HF5 group or netCDF variable
  • kwargs
    timeseries_filter (optional): read only a slice of
    the time dimension
    model_name: name of the model the field belongs to.
    Is used for a reverse lookup of the composite fields
    lookup_index (optional): a numpy array that will be used
    to sort the values by this lookup index
Returns: the data from the datasource
or None if ‘name’ is not in the datasource

Optional transforms can be done here.

class threedigrid.orm.base.fields.TimeSeriesSubsetArrayField(source_name=None, size=None)[source]

Field for subset arrays (for example only spanning the 2d section)

get_value(datasource, name, **kwargs)[source]
Parameters:
  • datasource – a datasource object that can return data on __getitem__()
  • name – the name of the section to read, e.g a HF5 group or netCDF variable
  • kwargs
    timeseries_filter (optional): read only a slice of
    the time dimension
    subset_index: index array where to store the subset
    values
    lookup_index (optional): a numpy array that will be used
    to sort the values by this lookup index
Returns: the data from the datasource or None if ‘name’ is not
in the datasource

Optional transforms can be done here.

Model _meta API

class threedigrid.orm.base.options.Options(inst)[source]

class that adds meta data to a model instance. Accessible through the _meta API. To retrieve meta data of a field, say s1:

>>> from threedigrid.admin.gridresultadmin import GridH5ResultAdmin
>>> f = "/code/tests/test_files/results_3di.nc"
>>> ff = "/code/tests/test_files/gridadmin.h5"
>>> gr = GridH5ResultAdmin(ff, f)
>>> gr.nodes._meta.s1
>>> s1(units=u'm', long_name=u'waterlevel',
...     standard_name=u'water_surface_height_above_reference_datum')

s1 is a namedtuple so you can retrieve the units attribute by the . notation gr.nodes._meta.s1.units or using the _as_dict method gr.nodes._meta.s1._asdict()['units']

add_field(field_name, field, hide_private=True)[source]

add field names to the model instance

Parameters:
  • field_names – iterable of field names
  • hide_private – fields starting with ‘_’ will be added to the instance but excluded from the list of field names
add_fields(fields, hide_private=True)[source]

add fields to the models instance

Parameters:
  • fields – dict of field names and fields
  • hide_private – fields starting with ‘_’ will be added to the instance but excluded from the list of field names
get_field(field_name)[source]
Returns:the ArrayField with field_name on this instance.
get_fields(only_names=False)[source]
Parameters:only_names – omit the field instances and return a list of field names instead
Returns:a dict of the models {field name: Field instance}. If the only_names flag is given a list of the models field names is returned