andyfrench.info

I have been using the ArcGIS Silverlight API to create a mapping application. To provide some context, the application had to show a pipe network together with valves and other associated assets. The pipes were to be selectable in the interface so that pipe asset IDs could be used to drive queries and other processing.
In order to render the valves etc. I chose to use the ESRI FeatureLayer. I also used a FeatureLayer with a Mode of SelectionOnly for pipe selection.
One of the requirements of the system was that background imagery be used. This was created using an ArcGISDynamicMapServiceLayer. The feature layers and the background layer were taking their data from different ArcGIS services.
Although my code was using MVVM the scenario could be replicated in XAML like this (this is the map control XAML only with a number of layers omitted):

<esri:Map x:Name="AWMap">
    <esri:Map.Layers>
        <esri:ArcGISDynamicMapServiceLayer 
                 ID="BaseLayerStreets" 
                 Url="http://servername/ArcGIS/rest/services/projectname/backgroundservicename/MapServer" />
        
        <esri:FeatureLayer ID="Hydrants" 
            Url="http://servername/ArcGIS/rest/services/projectname/featureservicename/MapServer/0"
            Where="1 = 1"
            Mode="OnDemand"
            Renderer="{StaticResource ValveRenderer}">
            <esri:FeatureLayer.Clusterer>
                <esri:FlareClusterer 
                    FlareBackground="Red" 
                    FlareForeground="White" 
                    MaximumFlareCount="9" />
            </esri:FeatureLayer.Clusterer>
        </esri:FeatureLayer>
    
    </esri:Map.Layers>
</esri:Map>

The problem

The problem was that as soon as the background layer was included the feature layers simply didn’t render. Handling the feature LayerInitialized and InitializationFailed events showed that the feature layers were initialised and that no errors were reported.
So what was going on?

The solution

After hours of head-scratching I reread the ESRI documentation and this popped out:

“By default, the first layer with a valid spatial reference defines the spatial reference for the map. Dynamic ArcGIS Server map and image services as well as feature layers (FeatureLayer) will be reprojected to the map's spatial reference if necessary.” - http://help.arcgis.com/en/webapi/silverlight/help/index.html#/Creating_a_map/016600000011000000/

When I checked the metadata for the 2 services in the ArcGIS Services Directory I noticed that the Spatial Reference was different for the 2 services. So, I changed the XAML to something like this (note lines 2 to 8):

<esri:Map x:Name="AWMap">
    <esri:Map.Extent>
        <esri:Envelope XMin="111111" YMin="222222" XMax="333333" YMax="444444" >
            <esri:Envelope.SpatialReference>
                <esri:SpatialReference WKID="27700"/>
            </esri:Envelope.SpatialReference>
        </esri:Envelope>
    </esri:Map.Extent>
    <esri:Map.Layers>
        <esri:ArcGISDynamicMapServiceLayer 
                 ID="BaseLayerStreets" 
                 Url="http://servername/ArcGIS/rest/services/projectname/servicename/MapServer" />
        
        <esri:FeatureLayer ID="Hydrants" 
            Url="http://servername/ArcGIS/rest/services/projectname/servicename/MapServer/0"
            Where="1 = 1"
            Mode="OnDemand"
            Renderer="{StaticResource ValveRenderer}">
            <esri:FeatureLayer.Clusterer>
                <esri:FlareClusterer 
                    FlareBackground="Red" 
                    FlareForeground="White" 
                    MaximumFlareCount="9" />
            </esri:FeatureLayer.Clusterer>
        </esri:FeatureLayer>
    
    </esri:Map.Layers>
</esri:Map>

The result was that the missing feature layer magically appeared.
So, if you are having problems with missing feature layers check your spatial references. My guess is you should change the spatial references at the server to prevent re-projection on the client; performance may be better.

Read More

I’ve been working on a piece of code that updates data in an attribute table in ArcGIS. As part of integration testing it was necessary to setup test data using ArcMap and then run the code which would modify other data. Unfortunately, even though the code appeared to be working, the data in ArcMap didn’t look like it had changed. Even closing the ArcMap attribute table view and reopening it didn’t have any effect. I recalled that ArcGIS can use versioning so it occurred to me that this might be part of the problem. I realised I needed to refresh the data but couldn’t remember how to do it. It turns out I needed to use the Versioning toolbar.

• Right-click on the toolbar area at the top of the ArcMap screen.
• In the pop-up menu scroll to the bottom and tick Versioning to open the Versioning tool bar.
• Click the Refresh icon on the Versioning toolbar.

Figure 1 – Right-click on the toolbar area.


Figure 2 – The Versioning tools (the Refresh icon circled).

Read More

Following on from a recent post on Relationship classes (and other things) in ArcGIS I thought I’d take a look at a confusing term that’s banded around in ArcGIS documentation, namely the attributed relationship.
Firstly I have noticed that there are 2 variations on the term: attribute relationship and attributed relationship. These 2 terms, although painfully similar, refer to quite different concepts.
The book Modeling Our World (Second Edition) draws a distinction between spatial relationships and attribute relationships:

“Modeling with spatial relationships
Your first choice for modeling relationships is to use the GIS to manage the spatial relationships inherent among features… [snip] …
Modeling with attribute relationships
There are also many associations that require attribute relationships to be defined. You can have an association between a geographic feature, such as a parcel of land, and a non-geographic entity, such as one or more parcel owners… [snip] …
Features in feature classes and rows in tables can have attribute relationships established between them through a common field called a key… [snip] … You can make these associations in several ways, including joining or relating tales temporarily in your map or by creating relationship classes in your geodatabase that maintains more permanent associations.” *

So here the definition of an attribute relationship is quite simple; an attribute relationship refers to a relationship between a feature class and a table. That’s it.
A few pages on you get this:

“Attributed relationship classes
An attributed relationship class is a type of relationship class that uses a table to store the key values for each individual relationship. Because of this, attributes can be optionally added to each relationship.” **

So, the definition of an attributed relationship is also simple but more specific; an attributed relationship is a relationship class with additional attributes other than just the keys.
I also came across this explanation in an old ESRI PDF:

“If a relationship class is not attributed and does not have a cardinality of many-to-many, it is stored as a set of foreign keys on the feature or object classes. Attributed relationships are stored in tables.”

Again this seems to confirm that an attributed relationship is just a relationship class (which I visualise as just a database link table) with some extra columns for additional attributes.
NB: If you are programming against ArcGIS you will find that there is an AttributedRelationship class, so from an API point of view the attributed relationship is a first class entity. I’m afraid I can’t comment on this aspect because I’ve yet to encounter it.
If you search ESRI documentation you’ll see the two terms coming up quite frequently. They express the notion that there can be relationships other than pure spatial ones between entities in the geodatabase and that relationship classes specifically can have attributes other than key values.

References

* p.78 – p79, Modeling Our World (Second Edition), Michael Zeiler, ISBN 978-1-58948-278-4
** p.85, Modeling Our World (Second Edition), Michael Zeiler, ISBN 978-1-58948-278-4

Read More

When modelling networks in ArcGIS there are 2 basic options you can choose:

• Network datasets
• Geometric networks

Network datasets are good for modelling undirected networks because they can allow flow in any direction. Network datasets are based on junctions, edges, and turn sources, which are simple feature classes. Network datasets are best suited for modelling transportation networks such as roads. Given that the work I do is not really concerned with road networks I’m going to put network datasets on the back burner.
Geometric datasets are better suited to modelling utilities (e.g. gas or oil pipelines), and natural resources (e.g. river systems). This is of much greater interest to me at the moment, hence this post. In essence a geometric network is a set of features that form a connected system of edges and junctions. An important characteristic of geometric networks is that features are regarded as being connected if they exist at the same x,y coordinate (geometric coincidence). 
So, geometric networks are based on 2 main elements: edges and junctions.

Edges

Edges are features that have a length and through which some commodity flows.
There are 2 types of edge:

• Simple – A line between two adjacent junctions. Resources enter one end of an edge and exit at the other end.
• Complex – A set of connected lines with 2 or more junctions. Resources flow from one end to another but may be ‘siphoned off’ along the edge without having to split the edge feature.

Note that in the diagram above the complex edge is intersected by 2 simple edges but it is not split by them. The complex edge is one continuous feature. This could represent a gas main (complex edge) intersected by service mains (simple edges).

Connectivity

Connectivity for simple edges occurs only at the end points of the feature. Connectivity for complex edges occurs at the ends of features but also at mid-span intersections.
Connections between coincident junctions is not permitted. You can only connect junctions through edges.
You can specify rules to constrain connectivity between features. For example you can specify edge-junctions rules to stipulate that only edges of a certain type can connect to junctions of a certain type. Similarly you can specify edge-edge rules to constrain what types of edge can connect to each other.

Junctions

Junctions represent the the location of endpoints of edges or where multiple edges are connected. A junction can connect 2 or more edges and facilitates flow of a commodity (traffic, water, etc.) from one edge to another.

Logical network

A geometric network will have a corresponding logical network. The logical is not concerned with coordinate values but rather data about how the network is connected (connectivity, weights, and other information). The logical network is a graph stored as tables in the geodatabase and is used for tracing and flow operations. This is primarily acheived through the use of a connectivity table that lists, for every junction, adjacent junctions and edges. 

Sources and Sinks

Any of the features in the geometric dataset can take on the role of a source (where a commodity flows from) or a sink (where a commodity terminates). These roles are used to define flow in the network.

Weights

Features can have a weight which represents the cost of traversing the edge or passing through a junction. Any number of attributes can be used as weights, for example lengths, capacity or slope.

Read More

I’ve started dipping my toes in the ESRI ArcGIS APIs and wanted to get something clear in my mind - What is a relationship class?
Firstly, I’ve made the somewhat obvious observation that the ESRI concepts around classes, tables, joins etc. are really a layer over the top of relational database concepts. It seems that the ArcGIS tools provide a non-SQL way to design or access the data in the geodatabase – sensible if you want a tool for users not familiar with databases or SQL. However, if you are a developer who is familiar with SQL and databases the ArcGIS interface gets a bit confusing until you can draw a mapping between well understood relational database concepts and what the ArcGIS tools give you.

Tables

Tables in ArcGIS are just the same as tables in an RDBMS. They have fields (columns if you prefer) and rows.
Note that in ArcGIS a table will always have an ObjectID field. This is the unique identifier for a row and is maintained by ArcGIS. If the requirements of your application mean that you need to use your own identifiers then you will have to add a new field to the table (in addition to the ObjectID) and use that to store your identifiers.

Subtypes

Tables can have subtypes. A subtype is just an integer value that is used as a discriminator to group rows together (this is somewhat similar to the discriminator column when using a single table to store a class hierarchy in NHibernate). Note that it becomes a design decision as to whether you represent different feature types as separate feature classes or by using subtypes.

Domains

The term domain pops up all over the place but the explanation for what they are is quite simple.

“Attribute domains are rules that describe the legal values for a field type, providing a method for enforcing data integrity.” ***

That looks very much like a way to maintain constraints to me. There are 2 types of domain:

• Coded-value domains – where you specify a set of valid values for an attribute.
• Range domains – where you specify a range of valid values for an attribute.

Feature classes

First up, a feature class is just a table. The way I think about it is that feature class could have been named feature table. The difference between a feature class and a plain table is that a feature class has a special field for holding shape data – the feature type.
All features in a feature class have the same feature type.

Relationship classes

This is what started me off on this investigation. I visualise a relationship class as just a link table in a database (i.e. the way you would normally manage many-to-many relationships in an RDBMS).
Here are some quotes from ESRI documentation:

“A relationship class is an object in a geodatabase that stores information about a relationship between two feature classes, between a feature class and a nonspatial table, or between two nonspatial tables. Both participants in a relationship class must be stored in the same geodatabase.” *

“In addition to selecting records in one table and seeing related records in the other, with a relationship class you can set rules and properties that control what happens when data in either table is edited, as well as ensure that only valid edits are made. You can set up a relationship class so that editing a record in one table automatically updates related records in the other table.” *

Relationship classes have a cardinality which is used to specify the number of objects in the origin class that can relate to a number of objects in the destination class. A relationship class can have one of three cardinalities:

• One-to-one
• One-to-many
• Many-to-many

No surprises there; we are following a relational data model. Cardinality can be set using the relationship cardinality constants provided by the ESRI API. Note that once a relationship class is created, the cardinality parameter cannot be altered. If you need to change the cardinality the relationship class must be deleted and recreated.
In a relationship one class will act as the origin and another as the destination. Different behaviour is associated with the origin and destination classes so it is important to define them correctly.
In code, relationships are expressed as instances of classes that implement the IRelationshipClass interface.
Note that relationships can have additional attributes (this is like adding extra columns to a link table to store data specific to an instance of the relationship). See The mythical ‘attributed relationship’ in ArcGIS for more details.

Simple relationships

In a simple relationship, related objects can exist independently of each other. The cascade behaviour is restricted; when deleting an origin object the key field value for the matching destination object is set to null. Deleting a destination object has no effect on the origin object.

Composite relationships

In a composite relationship, the destination objects are dependent on the lifetime of the origin objects. Deletes are cascaded from an origin object to all of the related destination objects.
The IRelationshipClass interface has an IsComposite parameter; a Boolean. Setting it to true indicates the relationship will be composite and setting it to false indicates that the relationship will be simple.

Relates

A relate is distinct from a relationship and is a mechanism for defining a relationship between two datasets based on a key. In other words this very much like an RDBMS foreign-key relationship (where one table has a foreign-key column that references the primary key of another table). So, a relate can be used to represent one-to-one and one-to-many relationships.

Joins

Although regarded as a separate construct in ArcGIS the join is really just what you’d expect a join to be in an RDBMS:

“Joining datasets in ArcGIS will append attributes from one table onto the other based on a field common to both. When an attribute join is performed the data is dynamically joined together, meaning nothing is actually written to disk.” ***

 

References

* Relates vs. Relationship Classes
** How to create relationship classes in the geodatabase
*** Modeling Our World, Michael Zeiler, ISBN 978-1-58948-278-4

Read More

I’m getting started with ArcGIS so I need to keep some notes. NB: This post is just an aide-mémoire for me as I get started so nothing is covered in any detail.

What is Esri?

Esri is a company providing Geographic Information System (GIS) software and geodatabase management applications. They are based in California and have about 30% of the GIS software market (see http://en.wikipedia.org/wiki/Esri).

What is the APDM?

APDM (ArcGIS Pipeline Data Model) is an open standard for storing geographical data associated with pipelines:

“The ArcGIS Pipeline Data Model is designed for storing information pertaining to features found in gathering and transmission pipelines, particularly gas and liquid systems. The APDM was expressly designed for implementation as an ESRI geodatabase for use with ESRI's ArcGIS and ArcSDE® products. A geodatabase is an object-relational construct for storing and managing geographic data as features within an industry-standard relational database management system (RDBMS).” - http://www.apdm.net/

What is ArcSDE?

“ArcSDE technology is a core component of ArcGIS Server. It manages spatial data in a relational database management system (RDBMS) and enables it to be accessed by ArcGIS clients.” - http://www.esri.com/software/arcgis/arcsde/index.html

“ArcSDE technology serves as the gateway between GIS clients and the RDBMS. It enables you to easily store, access, and manage spatial data within an RDBMS package…

ArcSDE technology is critical when you need to manage long transactions and versioned-based workflows such as

* Support for multiuser editing environments
* Distributed editing
* Federated replicas managed across many RDBMS architectures
* Managing historical archives

The responsibility for defining the specific RDBMS schema used to represent geographic data and for application logic is retained in ArcGIS, which provides the behavior, integrity, and utility of the underlying records.” - http://www.esri.com/software/arcgis/geodatabase/storage-in-an-rdbms.html

What is a geodatabase?

“The geodatabase is the common data storage and management framework for ArcGIS. It combines "geo" (spatial data) with "database" (data repository) to create a central data repository for spatial data storage and management.” - http://www.esri.com/software/arcgis/geodatabase/index.html

Basic terms and concepts

There are four fundamental types upon which geographic representations in a GIS are based:

• Features (collections or points, lines, and polygons)
  • Representations of things located on or near the surface of the earth.
  • Can be natural (rivers, vegetation, etc).
  • Can be constructions (roads, pipelines, buildings, etc.).
  • Can be subdivisions of land (counties, political divisions, land parcels, etc.).
  • Most commonly represented as points, lines, and polygons.
• Attributes (descriptive attributes of features)
  • Managed in tables based on simple relational database concepts.
• Imagery
  • Imagery is managed as a raster data type composed of cells organized in a grid of rows and columns.
  • In addition to the map projection, the coordinate system for a raster dataset includes its cell size and a reference coordinate (usually the upper left or lower left corner of the grid).
  • These properties enable a raster dataset to be described by a series of cell values starting in the upper left row.
  • Each cell location can be located using the reference coordinate, the cell size, and the number of rows and columns.
• Continuous surfaces (such as elevation)
  • A surface describes an occurrence that has a value for every point on the earth.
  • Surface elevation is a continuous layer of values for ground elevation above mean sea level.
  • Other surface type examples include rainfall, pollution concentration, and sub-surface representations of geological formations.

See the ArcGIS Desktop Help file for further details.

GIS data structures

Features, rasters, attributes, and surfaces are managed using three primary GIS data structures:

• Feature classes
• Attribute tables
• Raster datasets

Map Layer Types	GIS Datasets
Features (points, lines, and polygons)	Feature classes
Attributes	Tables
Imagery	Raster datasets
Surfaces	Both features and rasters can be used to provide a number of alternative surface representations: Feature classes (such as contours) Raster-based elevation datasets TINs built from XYZ points and 3D line feature classes

In a GIS datasets hold data about a particular feature collection (for example, roads) that is geographically referenced to the earth's surface. A dataset is a collection of homogeneous features. Most datasets are collections of simple geographic elements.

Users work with geographic data in two fundamental ways:

• As datasets (homogeneous collections of features, rasters, or attributes)
• As individual elements (e.g. individual features, rasters, and attribute values) contained within each dataset

Datasets are:

• The primary inputs and outputs for geoprocessing.
• Datasets are the primary means for data sharing.

 

See also

There’s some good basic information on GIS systems on the Ordinance Survey website: http://www.ordnancesurvey.co.uk/oswebsite/gisfiles/index.html

Read More

It looks like I’m going to have to get to grips with shapefiles and GIS data so here’s a post to give me a heads-up on shapefiles.

What is a shapefile?

“A shapefile stores nontopological geometry and attribute information for the spatial features in a data set. The geometry for a feature is stored as a shape comprising a set of vector coordinates.

Because shapefiles do not have the processing overhead of a topological data structure, they have advantages over other data sources such as faster drawing speed and edit ability. Shapefiles handle single features that overlap or that are noncontiguous. They also typically require less disk space and are easier to read and write.

Shapefiles can support point, line, and area features. Area features are represented as closed loop, double-digitized polygons. Attributes are held in a dBASE® format file. Each attribute record has a one-to-one relationship with the associated shape record.” *

Note that a geographic element forming part of a shapefile is referred to as a feature.

“A representation of a geographic feature that has both a spatial representation referred to as a "shape" and a set of attributes.” ***

So, key points:

• The data is nontopographical geometry it faster to search.
• Shapefiles can support point, line, and area features.

A shapefile is capable of storing a mixture of different shape types but this is prevented by the specification: "All the non-Null shapes in a shapefile are required to be of the same shape type."

Parts of a shapefile

Firstly, a shapefile actually consists of several files, not one, and a shapefile can contain a combination of mandatory and optional files. A shapefile will contain 3 mandatory files:

• .shp - the shape file containing the feature geometry
• .shx - the shape index containing a positional index of the feature geometry (facilitates quick searching)
• .dbf - the attribute file containing attributes for each shape (dBase IV format)

“An ESRI shapefile consists of a main file, an index file, and a dBASE table. The main file is a direct access, variable-record-length file in which each record describes a shape with a list of its vertices. In the index file, each record contains the offset of the corresponding main file record from the beginning of the main file. The dBASE table contains feature attributes with one record per feature. The one-to-one relationship between geometry and attributes is based on record number. Attribute records in the dBASE file must be in the same order as records in the main file.” **

There are a number of optional files but I am currently interested in:

• .prj – the projection file containing the coordinate system and projection information in plain text
• .sbn and .sbx – both containing the spatial index of the features
• .shp.xml - metadata in XML format

 

References

* http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf, p.5
** http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf, p.6
*** http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf, p.31

Read More