Field Search API Description

While APIs such as Field Facts, Field Trends and Field Forecasts provide detailed information about a single field, the Field Search API allows you to retrieve information for a wide area. With the Field Search API you can:

  • Specify exactly the information you want to see
  • Limit a search to a specific geographical area
  • Filter the results.

If you wish to experiment with searches without writing any code, you can use the Field Search UI here.

Search Format

A search consists of a HTTP GET request to https://api.agrimetrics.co.uk/field-search with query parameters that specify the information you wish to receive and any filters you wish to apply:

  • $select is used to specify the information you wish to see
  • $filter is used to specify any filters you wish to apply.

URL Encoding

Query parameter values should be URL encoded, and multiple parameters separated by &, as per standard URL requirements. For ease of reading, filter and select expressions in this page are shown without URL encoding.

For example, the filter expression written in this document as:

$filter=Field/hasAltitude/value le 100 and Field/hasAltitude/value ge 10

should be written in a URL as:

$filter=Field%2FhasAltitude%2Fvalue%20le%20100%20and%C2%A0Field%2FhasAltitude%2Fvalue%20ge%2010

Some tools and libraries will encode parameters for you; check how your chosen tool behaves, or how you can easily build URLs using your tool or library.

Results format

Matching fields are returned in a JSON-LD document. This will have:

  • A top-level results property (an array)
  • A totalResults property (integer) which tells you how many results matched the query
  • A HAL _links property
  • A link to the JSON-LD context in a @context property

As an example, a result-set matching a single field might look like:

{
  "results":[
      {
        "@id":"http://data.agrimetrics.co.uk/fields/gcx9tj1ry",
        "@type":"Field",
        "hasAltitude":{
            "value": 47,
            "@type": "Altitude",
            "hasUnit": "http://data.agrimetrics.co.uk/units/metre"
        }
      }
  ],
  "totalResults":1384,
  "_links":{
      "self":{
        "href":"http://api.agrimetrics.co.uk/field-search?$select=Field/hasAltitude"
      },
      "ag:context":{
        "href":"http://api.agrimetrics.co.uk/field-search/context"
      }
  },
  "@context":"http://api.agrimetrics.co.uk/field-search/context"
}

The result-level @id property, which is a Field id, is always returned for each entry in results.

Note: unless your query matches only a small number of fields, the results will contain a subset of matches. Use the Pagination facility to retrieve more results, or the Batch mode to retrieve all results.

$select parameter

Choose the data you want to return using the $select parameter.

If $select is not specified, then only the Field id is returned for each of the matching fields. This id can be used with other Agrimetrics APIs.

To return additional data for each field in the results, specify a comma-separated list of top-level property paths. For example:

$select=Field/hasAltitude,Field/hasForecastDailyMinimumTemperature

The $select properties are the same as those returned by the Field Facts, Field Trends and Field Forecasts APIs, with a Field/ prefix:

From Field Facts:

  • Field/hasSownCrop
  • Field/hasPriorityHabitat
  • Field/hasSoilLayer
  • Field/hasWaterCatchment
  • Field/centroid
  • Field/hasAltitude
  • Field/hasLongTermAverageMonthlyMaximumTemperature
  • Field/hasLongTermAverageMonthlyMinimumTemperature
  • Field/hasLongTermAverageMonthlyMeanTemperature
  • Field/hasLongTermAverageMonthlyTotalRainfall
  • Field/hasLongTermAverageMonthlyDaysOfAirFrost
  • Field/hasLongTermAverageMonthlyDaysOfGroundFrost
  • Field/hasLongTermAverageMonthlyHoursOfSunshine
  • Field/hasLongTermAverageMonthlyDaysOfRainfallAbove1mm
  • Field/hasLongTermAverageMonthlyDaysOfRainfallAbove10mm
  • Field/hasLongTermAverageMonthlyGrowingDegreeDays

From Field Trends:

  • Field/hasMonthlyMaximumTemperature
  • Field/hasMonthlyMinimumTemperature
  • Field/hasMonthlyMeanTemperature
  • Field/hasMonthlyDaysOfAirFrost
  • Field/hasMonthlyDaysOfGroundFrost
  • Field/hasMonthlyTotalSunshine
  • Field/hasMonthlyTotalRainfall
  • Field/hasMonthlyGrowingDegreeDays

From Field Forecasts:

  • Field/hasDailyMaximumTemperature
  • Field/hasDailyMinimumTemperature
  • Field/hasDailyMeanTemperature
  • Field/hasDailyMeanWindSpeed
  • Field/hasDailyMeanRelativeHumidity
  • Field/hasDailyTotalRainfall
  • Field/hasForecastDailyTotalRainfall
  • Field/hasForecastDailyMaximumTemperature
  • Field/hasForecastDailyMinimumTemperature
  • Field/hasForecastDailyMeanTemperature
  • Field/hasForecastDailyMeanWindSpeed
  • Field/hasForecastDailyMeanRelativeHumidity

$filter parameter

Filters are sent using the $filter query parameter, in the following format: $filter={expression}, where {expression} is an expression in OData syntax. See references below.

There are different types of filter expressions which may be combined using the boolean operators and, and or.

Geographical filters

Geographical filers allow you to limit a search to a specific area. Without a geographical filter, all fields are searched. You can limit a search to a specific area in two ways.

Circle

Using the geo.distance function, you can define a filter that includes only fields with central points that are within a given distance from a point.

For example, the following filter selects fields within 1500m of 51°48'33.7"N 0°21'17.4"W (longitude -0.354829 latitude 51.809346); this currently matches 41 fields:

$filter=geo.distance(Field/centroid, geography'SRID=4326;Point(-0.354829 51.809346)') lt 1500

In the geo.distance function:

  • The Field/centroid property holds the central point of fields
  • geography is made up of an SRID and a Point
  • Point(-0.354829 51.809346) is the point from which the distance is calculated, defined using the Point(longitude latitude) syntax,
  • SRID=4326 specifies the coordinate type as WGS84 co-ordinates
  • The distance is specified in metres (m)

The comparison uses the operator lt (less than).

Polygon

Using the geo.intersects function, you can define a filter that includes only fields with boundaries that intersect a custom shape. The shape is defined using a set of points defining the vertices of single closed polygon. For example: (-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797)

Note: To be a valid closed polygon, the last point is always the same as the first point.

Given the rectangle above, the following geo.intersects filter currently matches 94 fields:

$filter=geo.intersects(Field/hasGeometry, geography'SRID=4326;MultiLineString((-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797))')

In the geo.intersects function

  • The boundary of the fields for the intersect is matched using the Field/hasGeometry property
  • geography is made is of an SRID and a MultiLineString containing a list of points
  • SRID=4326 specifies the coordinate system as WGS84 co-ordinates
  • MultiLineString([points list]) contains the coordinates of the polygon, where [points list] is a comma-separated list of points defined by 'longitude latitude' pairs. Because the polygon must be closed, the last point is always the same as the first point

Data filters

Data filters are boolean expressions using property comparisons, and may be combined with geographical filters.

For example, to find fields with altitude greater than or equal to 150m:

$filter=Field/hasAltitude/value ge 150

In a data filter:

  • The property selector (for example: Field/hasAltitude/value) must appear on the left-hand side of the expression
  • The right-hand side of the expression should be a literal (number, string)

Multiple expressions are joined using boolean operators. For example, to find fields at altitudes of 150m to 250m:

$filter=Field/hasAltitude/value ge 150 and Field/hasAltitude/value le 250

Multiple properties can be queried simultaneously, using parentheses as necessary to override natural operator precedence. For example, to find fields at altitudes of 150m to 250m where wheat or barley was grown in previous years

$filter=Field/hasAltitude/value ge 150 and Field/hasAltitude/value le 250 and (Field/hasSownCrop/label eq 'Wheat' or Field/hasSownCrop/label eq 'Barley')

Operators

The following boolean operators are available:

  • not
  • and
  • or

and has higher precedence than or, so that a and b or c is interpreted as: (a and b) or c. Use parentheses to override operator precedence.

The following operators may be used in expressions:

  • lt - less than
  • le - less than or equal to
  • gt - greater than
  • ge - greater than or equal
  • eq - equal to
  • ne - not equal to

Functions

  • substringof - property has a substring, e.g. substringof(Field/hasPriorityHabitat/hasHabitatType/preferredLabel, 'woodland') filters the field where 'woodland' is a substring of their priority habitat types
  • startswith - property starts with string, e.g. startswith(Field/hasPriorityHabitat/hasHabitatType/preferredLabel, 'Deciduous') filters the field whose priority habitat types starts with 'Deciduous'
  • endswith - property ends with string, e.g. endswith(Field/hasPriorityHabitat/hasHabitatType/preferredLabel, 'woodland') filters the field whose priority habitat types ends with 'woodland'

The text operators, including eq and ne are case-sensitive.

Simple vs. Complex Properties

Some properties, such as Field/hasAltitude have only a single value per field, in this case value; others have multiple sets of values per field.

For example, Field/hasSoilLayer contains the multiple soil layers in a field, each layer with its own properties. The structure of Field/hasSownCrop is an array of crops, seasons and years; years may be repeated because multiple crops can be grown in a year, and crops are repeated because the same crop may be grown in more than one year:

{
  "hasSownCrop":[
      {
        "hasSowingSeason":"http://data.agrimetrics.co.uk/seasons/winter",
        "@type":"http://data.agrimetrics.co.uk/ontologies/agricultural-plant/YOXebXpkakF6D1",
        "harvestYear":2016,
        "label":"Wheat"
      },
      {
        "hasSowingSeason":"http://data.agrimetrics.co.uk/seasons/spring",
        "@type":"http://data.agrimetrics.co.uk/ontologies/agricultural-plant/Gad8kKje3ZcgP4",
        "harvestYear":2017,
        "label":"Barley"
      }
  ]
}

If you wanted to write a query to match fields that grew wheat in 2017, you might try;

$filter=Field/hasSownCrop/harvestYear eq 2017 and Field/hasSownCrop/label eq 'Wheat'

However, because hasSownCrop has multiple crops per field, you will get a hit for every field with a crop that has a harvestYear of 2017, and a hit for every field that is recorded as having grown wheat, but only some of those hits will be only where wheat was grown in 2017.

For these kind of queries, use the any function, after specifying the path to the parent property:

$filter=Field/hasSownCrop/any(f: f/harvestYear eq 2017 and f/label eq 'Wheat')

This ensures that the same hasSownCrop child (referred to as f in the query) has both harvestYear of 2017 and label of 'Wheat'.

Complex properties where this sort of query may be necessary are:

  • Field/hasSownCrop
  • Field/hasPriorityHabitat
  • Field/hasSoilLayer
  • Field/hasWaterCatchment

and all FieldTrends and FieldForecast properties.

Further Examples

Simple Searches

Tell me the id's of the fields in a specified area

$filter=geo.intersects(Location, geography'SRID=4326;MultiLineString((-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797))')

The same search, but also return me soil information for those fields

$filter=geo.intersects(Location, geography'SRID=4326;MultiLineString((-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797))')&$select=Field/hasSoilLayer

Searches with filters

This query returns the soil information for those fields in a given circular area where the crop grown in 2017 is wheat:

$filter=(Field/hasSownCrop/any(c: c/harvestYear eq 2017 and c/label eq 'Wheat')) and geo.distance(Field/centroid,geography'SRID=4326;Point(-0.933838 51.862156)') lt 60164&$select=Field/hasSoilLayer

This query returns the soil information for those fields in a given circular area where the long term maximum temperature in August is greater than 20 degrees:
$filter=(Field/hasLongTermAverageMonthlyMaximumTemperature/any(d: d/value gt 20 and d/month eq 8)) and geo.distance(Field/centroid,geography'SRID=4326;Point(-0.933838 51.862156)') lt 60164&$select=Field/hasSoilLayer

Batching and Pagination

The total number of results matched by your filter is returned in the totalResults property of the response.

The response body itself may only return a subset of these results, by default the first 100 of them. To retrieve more results, depending on your application, use either the pagination or batching functions of the API.

Pagination

The pagination facility allows you to skip directly to any numbered result in a result set, and return results from that point. Using the following request parameters:

  • Specify $skip to locate the first result in the result set that you wish to return. Defaults to 0
  • Specify $top to state the number of results to return, starting at the result specified by $skip. Defaults to 100

Pagination is convenient for sampling the results, but may be slow when used to skip across multiple pages in a larger result set, and cannot be used to retrieve very large retrieve result-sets (more than 10,000 results); use batch queries for that.

Batch Queries

A batch query allows you to reliably and efficiently return all records in result sets of any size.

To perform a batch query:

  • Specify batch=true as a request parameter along with your $filter and $select parameters. The response will then contain a unique Batch ID in the X-BatchId response header, as well as the first batch of results.
    • For example: $filter=Field/hasAltitude/value gt 200&batch=true
  • To return the next batch of results, call the API with the same URL again, but specify a X-BatchId header containing the header value from the previous response.
    • Note that the response for an empty result set omits the X-BatchId header.
  • Continue calling until the returned result-set is empty.
  • To start a new batch query, do not supply a X-BatchId header.

Batch queries should be used when you need to return all results matching a query, or a significant proportion of them.

The entire result-set captured in the batch query expires 1 minute after the last use of the Batch ID; you will get a 400 response on subsequent calls. For reliable processing, either retrieve all results from the query before processing them, or ensure you can process each batch of results, and request the next batch, before the query expires.

Recommendations

  • Use pagination when experimenting with your queries; use batching to retrieve large result sets
  • Specify smaller geographical areas in queries for faster results
  • Use fewer clauses in your $filter for faster results
  • Specify fewer properties in your $select for faster results, as well as smaller data sizes
  • The API supports gzip encoding of the response body: use the compression option in your client to enable this
  • Use the Field Search UI to experiment with queries, and export them for use with the API.

Versioning

Field Search API is versioned through URL path: https://api.agrimetrics.co.uk/field-search/{version}/. By specifying a version, you will remain isolated from potential breaking changes introduced in newer versions.

The latest version of Field Search API is v1, avaliable at https://api.agrimetrics.co.uk/field-search/v1/.

If the version is not specified in the path, the latest available version will be used.

.

More Information

The Search API's select and filter options use the OData format. For more some more general OData usage examples, see OData tutorial.