Field Search API Description

While APIs such as Field Facts, Field Trends and Field Forecasts provide detailed information about a single field, the Field Search API allows you to retrieve information for a wide area. With the Field Search API you can:

Specify exactly the information you want to see
Limit a search to a specific geographical area
Filter the results.

For specific examples see Examples later in this page.The API Examples GitHub repository also contains a code example in Python

If you wish to experiment with searches without writing any code, you can use the Field Explorer Demo.

Search Format

A search consists of a HTTP GET request to https://api.agrimetrics.co.uk/field-search with query parameters that specify the information you wish to receive and any filters you wish to apply:

$select is used to specify the information you wish to see
$filter is used to specify any filters you wish to apply.

URL Encoding

Query parameter values should be URL encoded, and multiple parameters separated by &, as per standard URL requirements. For ease of reading, filter and select expressions in this page are shown without URL encoding.

For example, the filter expression written in this document as:

$filter=Field/hasAltitude/value le 100 and Field/hasAltitude/value ge 10

should be written in a URL as:

$filter=Field%2FhasAltitude%2Fvalue%20le%20100%20and%C2%A0Field%2FhasAltitude%2Fvalue%20ge%2010

Some tools and libraries will encode parameters for you; check how your chosen tool behaves, or how you can easily build URLs using your tool or library.

Results format

Matching fields are returned in a JSON-LD document. This will have:

A top-level results property (an array)
A totalResults property (integer) which tells you how many results matched the query
A HAL _links property
A link to the JSON-LD context in a @context property

As an example, a result-set matching a single field might look like:

{
  "results": [
      {
        "@id":"http://data.agrimetrics.co.uk/fields/gcx9tj1ry",
        "@type":"Field",
        "hasAltitude":{
            "value": 47,
            "@type": "Altitude",
            "hasUnit": "http://data.agrimetrics.co.uk/units/metre"
        }
      }
  ],
  "totalResults":1384,
  "_links":{
      "self":{
        "href":"http://api.agrimetrics.co.uk/field-search?$select=Field/hasAltitude"
      },
      "ag:context":{
        "href":"http://api.agrimetrics.co.uk/field-search/context"
      }
  },
  "@context":"http://api.agrimetrics.co.uk/field-search/context"
}

The result-level @id property, which is a Field id, is always returned for each entry in results.

Note: unless your query matches only a small number of fields, the results will contain a subset of matches. Use the Pagination facility to retrieve more results, or the Batch mode to retrieve all results.

`$select` parameter

Choose the data you want to return using the $select parameter.

If $select is not specified, then only the Field id is returned for each of the matching fields. This id can be used with other Agrimetrics APIs.

To return additional data for each field in the results, specify a comma-separated list of top-level property paths. For example:

$select=Field/hasAltitude,Field/hasForecastDailyMinimumTemperature

The $select properties are the same as those returned by the Field Facts, Field Trends and Field Forecasts APIs, with a Field/ prefix:

From Field Facts:

Field/hasSownCrop
Field/hasPriorityHabitat
Field/hasSoilLayer
Field/hasWaterCatchment
Field/centroid
Field/hasAltitude
Field/hasLongTermAverageMonthlyMaximumTemperature
Field/hasLongTermAverageMonthlyMinimumTemperature
Field/hasLongTermAverageMonthlyMeanTemperature
Field/hasLongTermAverageMonthlyTotalRainfall
Field/hasLongTermAverageMonthlyDaysOfAirFrost
Field/hasLongTermAverageMonthlyDaysOfGroundFrost
Field/hasLongTermAverageMonthlyHoursOfSunshine
Field/hasLongTermAverageMonthlyDaysOfRainfallAbove1mm
Field/hasLongTermAverageMonthlyDaysOfRainfallAbove10mm
Field/hasLongTermAverageMonthlyGrowingDegreeDays

From Field Trends:

Field/hasMonthlyMaximumTemperature
Field/hasMonthlyMinimumTemperature
Field/hasMonthlyMeanTemperature
Field/hasMonthlyDaysOfAirFrost
Field/hasMonthlyDaysOfGroundFrost
Field/hasMonthlyTotalSunshine
Field/hasMonthlyTotalRainfall
Field/hasMonthlyGrowingDegreeDays

From Field Forecasts:

Field/hasDailyMaximumTemperature
Field/hasDailyMinimumTemperature
Field/hasDailyMeanTemperature
Field/hasDailyMeanWindSpeed
Field/hasDailyMeanRelativeHumidity
Field/hasDailyTotalRainfall
Field/hasForecastDailyTotalRainfall
Field/hasForecastDailyMaximumTemperature
Field/hasForecastDailyMinimumTemperature
Field/hasForecastDailyMeanTemperature
Field/hasForecastDailyMeanWindSpeed
Field/hasForecastDailyMeanRelativeHumidity

`$filter` parameter

Filters are sent using the $filter query parameter, in the following format: $filter={expression}, where {expression} is an expression in OData syntax. See references below.

There are different types of filter expressions which may be combined using the boolean operators and, and or.

Geographical filters

Geographical filers allow you to limit a search to a specific area. Without a geographical filter, all fields are searched. You can limit a search to a specific area in two ways.

Circle

Using the geo.distance function, you can define a filter that includes only fields with central points that are within a given distance from a point.

For example, the following filter selects fields within 1500m of 51°48'33.7"N 0°21'17.4"W (longitude -0.354829 latitude 51.809346); this currently matches 41 fields:

$filter=geo.distance(Field/centroid, geography'SRID=4326;Point(-0.354829 51.809346)') lt 1500

In the geo.distance function:

The Field/centroid property holds the central point of fields
geography is made up of an SRID and a Point
Point(-0.354829 51.809346) is the point from which the distance is calculated, defined using the Point(longitude latitude) syntax,
SRID=4326 specifies the coordinate type as WGS84 co-ordinates
The distance is specified in metres (m)

The comparison uses the operator lt (less than).

Polygon

Using the geo.intersects function, you can define a filter that includes only fields with boundaries that intersect a custom shape. The shape is defined using a set of points defining the vertices of single closed polygon. For example: (-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797)

Note: To be a valid closed polygon, the last point is always the same as the first point.

Given the rectangle above, the following geo.intersects filter currently matches 94 fields:

$filter=geo.intersects(Field/hasGeometry, geography'SRID=4326;MultiLineString((-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797))')

In the geo.intersects function

The boundary of the fields for the intersect is matched using the Field/hasGeometry property
geography is made is of an SRID and a MultiLineString containing a list of points
SRID=4326 specifies the coordinate system as WGS84 co-ordinates
MultiLineString([points list]) contains the coordinates of the polygon, where [points list] is a comma-separated list of points defined by 'longitude latitude' pairs. Because the polygon must be closed, the last point is always the same as the first point

Note: Searches will be faster if the polygon is a rectangle aligned with the coordinate grid, particularly if its large

Data filters

Data filters are boolean expressions using property comparisons, and may be combined with geographical filters.

For example, to find fields with altitude greater than or equal to 150m:

$filter=Field/hasAltitude/value ge 150

In a data filter:

The property selector (for example: Field/hasAltitude/value) must appear on the left-hand side of the expression
The right-hand side of the expression should be a literal (number, string)

Multiple expressions are joined using boolean operators. For example, to find fields at altitudes of 150m to 250m:

$filter=Field/hasAltitude/value ge 150 and Field/hasAltitude/value le 250

Multiple properties can be queried simultaneously, using parentheses as necessary to override natural operator precedence. For example, to find fields at altitudes of 150m to 250m where wheat or barley was grown in previous years

$filter=Field/hasAltitude/value ge 150 and Field/hasAltitude/value le 250 and (Field/hasSownCrop/label eq 'Wheat' or Field/hasSownCrop/label eq 'Barley')

Operators

The following boolean operators are available:

not
and
or

and has higher precedence than or, so that a and b or c is interpreted as: (a and b) or c. Use parentheses to override operator precedence.

The following operators may be used in expressions:

lt - less than
le - less than or equal to
gt - greater than
ge - greater than or equal
eq - equal to
ne - not equal to

Functions

substringof - property has a substring, e.g. substringof(Field/hasPriorityHabitat/hasHabitatType/preferredLabel, 'woodland') filters the field where 'woodland' is a substring of their priority habitat types
startswith - property starts with string, e.g. startswith(Field/hasPriorityHabitat/hasHabitatType/preferredLabel, 'Deciduous') filters the field whose priority habitat types starts with 'Deciduous'
endswith - property ends with string, e.g. endswith(Field/hasPriorityHabitat/hasHabitatType/preferredLabel, 'woodland') filters the field whose priority habitat types ends with 'woodland'

The text operators, including eq and ne are case-sensitive.

Simple vs. Complex Properties

Some properties, such as Field/hasAltitude have only a single value per field, in this case value; others have multiple sets of values per field.

For example, Field/hasSoilLayer contains the multiple soil layers in a field, each layer with its own properties. The structure of Field/hasSownCrop is an array of crops, seasons and years; years may be repeated because multiple crops can be grown in a year, and crops are repeated because the same crop may be grown in more than one year:

{
  "hasSownCrop": [
      {
        "hasSowingSeason":"http://data.agrimetrics.co.uk/seasons/winter",
        "@type":"http://data.agrimetrics.co.uk/ontologies/agricultural-plant/YOXebXpkakF6D1",
        "harvestYear":2016,
        "label":"Wheat"
      },
      {
        "hasSowingSeason":"http://data.agrimetrics.co.uk/seasons/spring",
        "@type":"http://data.agrimetrics.co.uk/ontologies/agricultural-plant/Gad8kKje3ZcgP4",
        "harvestYear":2017,
        "label":"Barley"
      }
  ]
}

If you wanted to write a query to match fields that grew wheat in 2017, you might try;

$filter=Field/hasSownCrop/harvestYear eq 2017 and Field/hasSownCrop/label eq 'Wheat'

However, because hasSownCrop has multiple crops per field, you will get a hit for every field with a crop that has a harvestYear of 2017, and a hit for every field that is recorded as having grown wheat, but only some of those hits will be only where wheat was grown in 2017.

For these kind of queries, use the any function, after specifying the path to the parent property:

$filter=Field/hasSownCrop/any(f: f/harvestYear eq 2017 and f/label eq 'Wheat')

This ensures that the same hasSownCrop child (referred to as f in the query) has both harvestYear of 2017 and label of 'Wheat'.

Complex properties where this sort of query may be necessary are:

Field/hasSownCrop
Field/hasPriorityHabitat
Field/hasSoilLayer
Field/hasWaterCatchment

and all FieldTrends and FieldForecast properties.

Further Examples

Simple Searches

Tell me the id's of the fields in a specified area

$filter=geo.intersects(Location, geography'SRID=4326;MultiLineString((-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797))')

The same search, but also return me soil information for those fields

$filter=geo.intersects(Location, geography'SRID=4326;MultiLineString((-0.142994 51.972797,-0.142994 52.001249,-0.090466 52.001249,-0.090466 51.972797,-0.142994 51.972797))')&$select=Field/hasSoilLayer

Searches with filters

This query returns the soil information for those fields in a given circular area where the crop grown in 2017 is wheat:

$filter=(Field/hasSownCrop/any(c: c/harvestYear eq 2017 and c/label eq 'Wheat')) and geo.distance(Field/centroid,geography'SRID=4326;Point(-0.933838 51.862156)') lt 60164&$select=Field/hasSoilLayer

This query returns the soil information for those fields in a given circular area where the long term maximum temperature in August is greater than 20 degrees:
$filter=(Field/hasLongTermAverageMonthlyMaximumTemperature/any(d: d/value gt 20 and d/month eq 8)) and geo.distance(Field/centroid,geography'SRID=4326;Point(-0.933838 51.862156)') lt 60164&$select=Field/hasSoilLayer

Batching and Pagination

The total number of results matched by your filter is returned in the totalResults property of the response.

The response body itself may only return a subset of these results, by default the first 100 of them.To retrieve more results, depending on your application, use either the pagination or batchingfunctions of the API.

Pagination

The pagination facility allows you to skip directly to any numbered result in a result set, and return results from that point. Using the following request parameters:

Specify $skip to locate the first result in the result set that you wish to return. Defaults to 0
Specify $top to state the number of results to return, starting at the result specified by $skip. Defaults to 100

Pagination is convenient for sampling the results, but may be slow when used to skip acrossmultiple pages in a larger result set, and cannot be used to retrieve very large retrieveresult-sets (more than 10,000 results); use batch queries for that.

Batch Queries

A batch query allows you to reliably and efficiently return all records in result sets of any size. Batch queries should be used when you need to return all results matching a query, or a significant proportion of them.

To perform a batch query:

Specify batch=true as a request parameter along with your $filter and $select parameters. The response will then contain a unique Batch ID in the X-BatchId response header, as well as the first batch of results.
- For example: $filter=Field/hasAltitude/value gt 200&batch=true
To return the next batch of results, call the API again with the same URL, but additionally specify a X-BatchId header containing value of the X-BatchId header returned with the previous response.
Note that the response for an empty result set omits the X-BatchId header.
Continue calling in the same manner until the returned result-set is empty.

The unfetched results associated with a batch query expire 1 minute after the last call was madeusing the Batch ID; you will get a 400 response on subsequent calls. To avoid errors, you must callthe API to fetch the next batch of results within 1 minute of the results of the last batch being returned.

For reliable processing, either retrieve all results from the batch query before processing them,or ensure that you can process each batch of results, and request the next batch, before the query expires.

Recommendations

Use pagination with a small page size when experimenting with your queries; use batching to retrieve large result sets
Specify smaller geographical areas in queries for faster results
Use fewer clauses in your $filter for faster results
Specify fewer properties in your $select for faster results, as well as smaller data sizes
The API supports gzip encoding of the response body: use the compression option in your client to enable this

Versioning

Field Search API is versioned through URL path: https://api.agrimetrics.co.uk/field-search/{version}/. By specifying a version, you will remain isolated from potential breaking changes introduced in newer versions.

The latest version of Field Search API is v1, avaliable at https://api.agrimetrics.co.uk/field-search/v1/.

If the version is not specified in the path, the latest available version will be used.

More Information

The Search API's select and filter options use the OData format. For more some more general OData usage examples, see OData tutorial.

Field Search API Description