diff --git a/docs/Algorithm/Overview/Overview.md b/docs/Algorithm/Overview/Overview.md
index 8efc1a3d..e4224f31 100644
--- a/docs/Algorithm/Overview/Overview.md
+++ b/docs/Algorithm/Overview/Overview.md
@@ -10,30 +10,29 @@ Developed by ECMWF - the European Centre for Medium-Range Weather Forecasts - it
### Traditional Extraction Techniques
-Traditional data extraction techniques only allow users to access datacubes "orthogonally" by selecting specific values or ranges along datacube dimensions.
-Such data access mechanisms can be seen as extracting so-called "bounding boxes" of data.
-These mechanisms are quite limited however as many user requests cannot be formulated using bounding boxes.
+Traditional data extraction techniques only allow users to access boxes of data from datacubes.
+These techniques are quite restrictive however as many user requests cannot be formulated using such boxes.
!!!note "Example"
- Imagine for example someone interested in extracting temperature data over the shape of France.
- France is not a box shape over latitude and longitude.
- Using current extraction techniques, this exact request would therefore be impossible and users would instead need to request a bounding box around France.
+ Imagine for example someone interested in extracting wind data over the Mediterranean sea.
+ The Mediterranean is not a box shape over latitude and longitude.
+ Using current extraction techniques, this exact request would therefore be impossible and users would instead need to request a bounding box around the Mediterranean.
The user would thus get back much more data than he truly needs.
In higher dimensions, this becomes an even bigger challenge with only tiny fractions of the extracted data being useful to users.
### Polytope Extraction Technique
-As an alternative, Polytope enables users to access datacubes "non-orthogonally".
-Instead of extracting bounding boxes of data, Polytope has the capability of querying high-dimensional "polytopes" along several axes of a datacube.
-This is much less restrictive than the popular bounding box approach described before.
+Instead, Polytope enables users to access high-dimensional "polytopes" from datacubes, rather than only boxes of data.
+
+
!!!note "Example"
- Using Polytope, extracting the temperature over just the shape of France is now trivially possible by specifying the right polytope.
+ Using Polytope, extracting the temperature over just the shape of the Mediterranean is now trivially possible by specifying the right polytope.
This returns much less data than by using a bounding box approach.
These polytope-based requests do in fact allow Polytope to fulfill its two main aims.
-Indeed, because polytope requests return only the exact data users need, they significantly reduce I/O usage as less data has to be transmitted.
+Indeed, because polytope requests return only the data users need, they significantly reduce I/O usage as less data has to be transmitted.
Moreover, because only the data inside the requested polytope is returned, this method completely removes the challenge of post-processing on the user side, as wanted.
\ No newline at end of file
diff --git a/docs/Algorithm/User_Guide/Example.md b/docs/Algorithm/User_Guide/Example.md
index 10ea2a51..8e5b903f 100644
--- a/docs/Algorithm/User_Guide/Example.md
+++ b/docs/Algorithm/User_Guide/Example.md
@@ -1,7 +1,7 @@
# Example
Here is a step-by-step example of how to use the Polytope software.
-1. In this example, we first specify the data which will be in our Xarray datacube. Note that the data here comes from the GRIB file called "winds.grib", which is 3-dimensional with dimensions: step, latitude and longitude.
+1. In this example, we first specify the data which will be in our XArray datacube. Note that the data here comes from the GRIB file called "winds.grib", which is 3-dimensional with dimensions: step, latitude and longitude.
import xarray as xr
diff --git a/docs/Algorithm/User_Guide/Getting_started.md b/docs/Algorithm/User_Guide/Getting_started.md
index fcaf042b..57f76b24 100644
--- a/docs/Algorithm/User_Guide/Getting_started.md
+++ b/docs/Algorithm/User_Guide/Getting_started.md
@@ -26,13 +26,13 @@ or from PyPI with the command
Polytope's tests and examples require some additional dependencies compared to the main Polytope software.
-- **Git Large File Storage**
+
- **Additional Dependencies**
diff --git a/docs/Service/Design_doc.md b/docs/Service/Design_doc.md
index 37e0f6aa..545f4aa7 100644
--- a/docs/Service/Design_doc.md
+++ b/docs/Service/Design_doc.md
@@ -4,7 +4,7 @@
### Feature Keyword
-Feature extraction expands existing mars requests to include a `feature` keyword that includes a json dictionary taht describes the given feature. This feature is then extracted using the Polytope feature extraction algoithm and only points within the given feature are returned.
+Feature extraction expands existing mars requests to include a `feature` keyword that includes a json dictionary that describes the given feature. This feature is then extracted using the Polytope feature extraction algorithm and only points within the given feature are returned.
```python
"feature" : {
@@ -15,7 +15,7 @@ Feature extraction expands existing mars requests to include a `feature` keyword
#### Type
-An example of a minimal feature of `type` : `timeseries` can be seen above. A feature dictionary must always contain a `type`. The `type` in this case refers to what feature is being requested, the `type` of feaature requested will then determine the format of the output returned, what other keys can go in the feature and suitable defaults if they are not available. In some cases it may also affect keys outside of the feature dictionary that come from the traditional mars request. For example if `type` : `verticalprofile` and `levtype` : `sfc`, this request wont be sent as a vertical profile expects either `levtype` : `pl/ml`. Other exceptions will be given for each seperate feature `type`.
+An example of a minimal feature of `type` : `timeseries` can be seen above. A feature dictionary must always contain a `type`. The `type` in this case refers to what feature is being requested, the `type` of feature requested will then determine the format of the output returned, what other keys can go in the feature and suitable defaults if they are not available. In some cases it may also affect keys outside of the feature dictionary that come from the traditional mars request. For example if `type` : `verticalprofile` and `levtype` : `sfc`, this request will not be sent as a vertical profile expects either `levtype` : `pl/ml`. Other exceptions will be given for each separate feature `type`.
The value available for `type` currently are as follows:
@@ -34,7 +34,7 @@ A feature dictionary must also contain the requested geometry in some form. For
#### Axis
-A non mandatory field that is available for each feature that isnt present in the above example is `axis`. `axis` determines what field that the data should be enumerated along. In the case of a `timeseries` this will default to `step` meaning the timeseries will be along the `step` axis, however there are other available `axis` such as `datetime`, this would be for climate data which contains no `step` `axis`.
+A non-mandatory field that is available for each feature that is not present in the above example is `axis`. `axis` determines what field that the data should be enumerated along. In the case of a `timeseries` this will default to `step` meaning that the timeseries will be along the `step` axis, however there are other available `axes` such as `datetime`, this would be for climate data which contains no `step` `axis`.
#### Range
@@ -48,13 +48,13 @@ A non mandatory field that is available for each feature that isnt present in th
}
```
-If this range was included in the above feature dictionary for a `timeseries` it would ask for `step` (due to it being the default axis for timeseries) starting at `0` and ending at `10` with an interval of `2`, the returned steps would be `0,2,4,6,8,10`. Or equivilent to asking for the following in a mars request.
+If this range was included in the above feature dictionary for a `timeseries` it would ask for `step` (due to it being the default axis for timeseries) starting at `0` and ending at `10` with an interval of `2`, the returned steps would be `0,2,4,6,8,10`. This is equivalent to asking for the following in a mars request:
```python
"step" : "0/to/10/by/2"
```
-The above can also be put in the not feature key however it must then be mutually exclusive with `range`. If both or neither are in the request an error is thrown.
+The above can also be put in the body of the request. However it must then be mutually exclusive with `range`. If both or neither are in the request an error is thrown.
`range` can also appear in the following form:
@@ -62,18 +62,18 @@ The above can also be put in the not feature key however it must then be mutuall
"range" : [0,1,4,7,10]
```
-This will only return the asked steps similar to in a mars request where a user asks for the following:
+This will only return the asked steps similar to in a MARS request where a user asks for the following:
```python
"step" : "0/1/4/7/10"
```
-Again either a `range` within the feature or an explicit `step` within the main body of the request can be used but not both or neither as there is no suitable default value unlike mars.
+Again either a `range` within the feature or an explicit `step` within the main body of the request can be used but not both or neither as there is no suitable default value unlike MARS.
### MARS Fields
-The non `feature` elements of the polytope-mars request act similar to the way one would expect when creating a mars request with a few differences.
+The non `feature` elements of the polytope-mars request act similar to the way one would expect when creating a MARS request with a few differences.
* Most fields do not have a default value that will be tried if the field is not in the request.
* If a user makes a request and data is only available for some of the fields requested an error will be returned. Users will either receive all the data they requested or none.
@@ -152,11 +152,11 @@ request = {
}
```
-The above would throw an error that `step` has been over subscribed.
+The above would throw an error that `step` has been over-subscribed.
Ideally an valid mars request should be able to accept a valid `feature` and the polytope-mars request be valid but this may not always be true.
-Users can include the `format` key however initally the only value available will be `covjson` or `application/json+covjson`, these will be the default values if `format` is not included. Further formats may be added in the future.
+Users can include the `format` key. However, initially the only value available will be `covjson` or `application/json+covjson`, these will be the default values if `format` is not included. Further formats may be added in the future.
### Features
@@ -164,7 +164,7 @@ The following features will be available for use in polytope-mars.
#### Timeseries
-A timeseries request has a `feature` with `type` : `timeseries` and a geomtry in the form of `points` containing a single point with latitude and longitude values. It also requires atleast one time dimension with the default being `step` however `datetime` is also accepted. The following is an example of a timeseries request:
+A timeseries request has a `feature` with `type` : `timeseries` and a geometry in the form of `points` containing a single point with latitude and longitude values. It also requires at least one time dimension with the default being `step`, although `datetime` is also accepted. The following is an example of a timeseries request:
```python
request = {
@@ -213,7 +213,7 @@ request = {
}
```
-In this case the user is requesting `step` `0-360` on `20241006` for the point `[-9.10, 38.78]`. As the user doesnt specify `interval` all steps between `0-360` that are available. If the datacube is a climate dataset that does not contain step, an error would be thrown as `step` is not in the datacube. In this case the user would have to provide a request like the following:
+In this case the user is requesting `step` `0-360` on `20241006` for the point `[-9.10, 38.78]`. As the user does not specify `interval` all steps between `0-360` that are available. If the datacube is a climate dataset that does not contain step, an error would be thrown as `step` is not in the datacube. In this case the user would have to provide a request like the following:
```python
request = {
@@ -252,7 +252,7 @@ CoverageJSON output type: PointSeries
#### Vertical Profile
-A vertical profile request has a `feature` with `type` : `verticalprofile` and a geomtry in the form of `points` containing a single point with latitude and longitude values. It also requires a `levtype` that is not `sfc` and a `levelist` in the request or as part of the `feature`. The following is an example of a vertical profile request:
+A vertical profile request has a `feature` with `type` : `verticalprofile` and a geometry in the form of `points` containing a single point with latitude and longitude values. It also requires a `levtype` that is not `sfc` and a `levelist` in the request or as part of the `feature`. The following is an example of a vertical profile request:
```python
request = {
@@ -302,19 +302,19 @@ request = {
}
```
-`levtype` can either be `ml` or `pl` but atleast one must be present.
+`levtype` can either be `ml` or `pl` but at least one must be present.
`levelist` can either be in the main body of the request or in `range` as described in the `range` section. If no `interval` is provided all values in from `start` to `end` will be requested.
Currently the default for `axes` is `levelist` and is the only valid value for this key. This may change in the future. Users can include this in the request but it is not necessary.
-In the above case if a range is provided for a field such as `number` a vertical profile as described above will be provided per `number` or any other range field.
+In the above case if a range is provided for a field such as `number`, a vertical profile as described above will be provided per `number` or any other range field.
CoverageJSON output type: VerticalProfile
#### Trajectory
-A trajectory request has a `feature` with `type` : `trajectory` and a geomtry in the form of `points` containing atleast two points with latitude and longitude, a level value, and a time value if no `axes` is provided. This is because the default `axes` are as follows:
+A trajectory request has a `feature` with `type` : `trajectory` and a geometry in the form of `points` containing at least two points with latitude and longitude, a level value, and a time value if no `axes` is provided. This is because the default `axes` are as follows:
```python
"axes" : ["lat", "long", "level", "step"]
@@ -417,7 +417,7 @@ CoverageJSON output type: Trajectory
#### Polygon
-A polygon request has a `feature` with `type` : `poylgon` and a geomtry in the form of `shape` containing atleast one list containing three points with latitude and longitude with the first and final point being the same to complete the polygon. The user can provide multiple lists of points forming polygons in the same request. An example of the `polygon` feature is seen below:
+A polygon request has a `feature` with `type` : `poylgon` and a geometry in the form of `shape` containing at least one list containing three points with latitude and longitude with the first and final point being the same to complete the polygon. The user can provide multiple lists of points forming polygons in the same request. An example of the `polygon` feature is seen below:
```python
request = {
@@ -471,7 +471,7 @@ Returned coverages as polygons:
Each of these will be an individual coverage with the 3 requested parameters.
-The `polygon` feature currently has limits on the size of a returned polygon and the maximum number of points allowed for a requsted polygon.
+The `polygon` feature currently has limits on the size of a returned polygon and the maximum number of points allowed for a requested polygon.
CoverageJSON output type: MultiPoint
@@ -481,4 +481,4 @@ CoverageJSON output type: MultiPoint
CoverageJSON has a number of different output features. Depending on the feature selected the output type will vary.
A coverageCollection is always returned even if there is only a single coverage.
-A new coverage is created for each ensemble number and depending on the feature type each new date (except in timeseries). The only grouped field is param which will be in the same coverage.
\ No newline at end of file
+A new coverage is created for each ensemble number and depending on the feature type each new date (except in timeseries). The only grouped field is `param` which will be in the same coverage.
\ No newline at end of file
diff --git a/docs/Service/Examples/boundingbox_example.ipynb b/docs/Service/Examples/boundingbox_example.ipynb
new file mode 100644
index 00000000..c8700db7
--- /dev/null
+++ b/docs/Service/Examples/boundingbox_example.ipynb
@@ -0,0 +1,567 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Bounding Box Example"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "2024-11-04 13:49:02 - INFO - Key read from /Users/maaw/.polytopeapirc\n",
+ "2024-11-04 13:49:02 - INFO - Sending request...\n",
+ "{'request': 'class: od\\n'\n",
+ " \"date: '20241103'\\n\"\n",
+ " 'domain: g\\n'\n",
+ " 'expver: 0079\\n'\n",
+ " 'feature:\\n'\n",
+ " ' points:\\n'\n",
+ " ' - - -0.1\\n'\n",
+ " ' - -0.1\\n'\n",
+ " ' - - 0.1\\n'\n",
+ " ' - 0.1\\n'\n",
+ " ' type: boundingbox\\n'\n",
+ " 'levtype: sfc\\n'\n",
+ " \"number: '1'\\n\"\n",
+ " 'param: 164/166/167/169\\n'\n",
+ " \"step: '0'\\n\"\n",
+ " 'stream: enfo\\n'\n",
+ " \"time: '0000'\\n\"\n",
+ " 'type: pf\\n',\n",
+ " 'verb': 'retrieve'}\n",
+ "2024-11-04 13:49:02 - INFO - Polytope user key found in session cache for user maaw\n",
+ "2024-11-04 13:49:03 - INFO - Request accepted. Please poll ./7cbd9977-c3eb-4b06-955d-eca8fdf331b0 for status\n",
+ "2024-11-04 13:49:03 - INFO - Polytope user key found in session cache for user maaw\n",
+ "2024-11-04 13:49:03 - INFO - Checking request status (./7cbd9977-c3eb-4b06-955d-eca8fdf331b0)...\n",
+ "2024-11-04 13:49:04 - INFO - The current status of the request is 'queued'\n",
+ "2024-11-04 13:49:05 - INFO - The current status of the request is 'processing'\n",
+ "2024-11-04 13:49:06 - INFO - The current status of the request is 'processed'\n"
+ ]
+ },
+ {
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "8fc93bfe74b24b6dab9cb51f248b0216",
+ "version_major": 2,
+ "version_minor": 0
+ },
+ "text/plain": [
+ "7cbd9977-c3eb-4b06-955d-eca8fdf331b0.grib: 0%| | 0.00/5.63k [00:00\n",
+ "
"
+ ],
+ "text/plain": [
+ "\n",
+ "Dimensions: (datetimes: 1, number: 1, steps: 1, points: 6)\n",
+ "Coordinates:\n",
+ " * datetimes (datetimes) Timeseries
* Polygon
-* Vertical Profile
\ No newline at end of file
+* Vertical Profile
+* Bounding Box
+* Trajectory
\ No newline at end of file
diff --git a/docs/Service/Examples/polygon_example.ipynb b/docs/Service/Examples/polygon_example.ipynb
index 62937e73..776ee2c4 100644
--- a/docs/Service/Examples/polygon_example.ipynb
+++ b/docs/Service/Examples/polygon_example.ipynb
@@ -1,260 +1,17 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Polygon Example"
+ ]
+ },
{
"cell_type": "code",
- "execution_count": 8,
+ "execution_count": null,
"metadata": {},
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "DEBUG:polytope.api.Client.MCIRN2:Gathered Polytope client configuration:\n",
- "{'address': 'polytope-test.ecmwf.int',\n",
- " 'insecure': False,\n",
- " 'key_path': PosixPath('/home/maaw/.polytopeapirc'),\n",
- " 'log_file': None,\n",
- " 'log_level': 'DEBUG',\n",
- " 'password': None,\n",
- " 'port': None,\n",
- " 'quiet': False,\n",
- " 'skip_tls': False,\n",
- " 'user_email': None,\n",
- " 'user_key': None,\n",
- " 'username': 'maaw',\n",
- " 'verbose': False}\n",
- "DEBUG:polytope.api.Client.MCIRN2:Creating Polytope client...\n",
- "2024-09-19 13:50:24 - INFO - Key read from /home/maaw/.polytopeapirc\n",
- "INFO:polytope.api.Client.MCIRN2:Key read from /home/maaw/.polytopeapirc\n",
- "2024-09-19 13:50:24 - INFO - Sending request...\n",
- "{'request': 'class: od\\n'\n",
- " \"date: '20240917'\\n\"\n",
- " 'domain: g\\n'\n",
- " 'expver: 0079\\n'\n",
- " 'feature:\\n'\n",
- " ' shape:\\n'\n",
- " ' - - 41.870881288\\n'\n",
- " ' - -8.879136\\n'\n",
- " ' - - 41.694339317422646\\n'\n",
- " ' - -8.824238614026456\\n'\n",
- " ' - - 40.171924585721314\\n'\n",
- " ' - -8.902386975546364\\n'\n",
- " ' - - 38.75694209400925\\n'\n",
- " ' - -9.493088042617785\\n'\n",
- " ' - - 38.42424252381525\\n'\n",
- " ' - -9.171674240710018\\n'\n",
- " ' - - 38.49907333213173\\n'\n",
- " ' - -8.676525850529856\\n'\n",
- " ' - - 37.057269459205145\\n'\n",
- " ' - -8.971873318897366\\n'\n",
- " ' - - 37.162874354643776\\n'\n",
- " ' - -7.406745406502978\\n'\n",
- " ' - - 38.19776118392036\\n'\n",
- " ' - -6.931663452624974\\n'\n",
- " ' - - 38.4280922170291\\n'\n",
- " ' - -7.321584397020473\\n'\n",
- " ' - - 39.011852875635526\\n'\n",
- " ' - -6.9787177479519755\\n'\n",
- " ' - - 39.66227871551288\\n'\n",
- " ' - -7.5393956904523804\\n'\n",
- " ' - - 39.66568774825791\\n'\n",
- " ' - -7.03915852435145\\n'\n",
- " ' - - 40.0019453234905\\n'\n",
- " ' - -6.883203763416162\\n'\n",
- " ' - - 40.20373392742229\\n'\n",
- " ' - -7.035724907677206\\n'\n",
- " ' - - 40.350463990828985\\n'\n",
- " ' - -6.8135246275213035\\n'\n",
- " ' - - 41.030499770212515\\n'\n",
- " ' - -6.905947651233703\\n'\n",
- " ' - - 41.593647729084154\\n'\n",
- " ' - -6.22847017956974\\n'\n",
- " ' - - 41.67712153119277\\n'\n",
- " ' - -6.544984134823352\\n'\n",
- " ' - - 41.949682257268876\\n'\n",
- " ' - -6.567927092516641\\n'\n",
- " ' - - 41.96960294343674\\n'\n",
- " ' - -7.1747800681640115\\n'\n",
- " ' - - 41.88337981339092\\n'\n",
- " ' - -7.196871678410446\\n'\n",
- " ' - - 41.81334515396762\\n'\n",
- " ' - -8.156666519264604\\n'\n",
- " ' - - 42.14242723772878\\n'\n",
- " ' - -8.205142297350534\\n'\n",
- " ' - - 41.870881288\\n'\n",
- " ' - -8.879136\\n'\n",
- " ' type: polygon\\n'\n",
- " 'levtype: sfc\\n'\n",
- " \"number: '1'\\n\"\n",
- " 'param: 167/169\\n'\n",
- " \"step: '0'\\n\"\n",
- " 'stream: enfo\\n'\n",
- " \"time: '1200'\\n\"\n",
- " 'type: pf\\n',\n",
- " 'verb': 'retrieve'}\n",
- "INFO:polytope.api.Client.MCIRN2:Sending request...\n",
- "{'request': 'class: od\\n'\n",
- " \"date: '20240917'\\n\"\n",
- " 'domain: g\\n'\n",
- " 'expver: 0079\\n'\n",
- " 'feature:\\n'\n",
- " ' shape:\\n'\n",
- " ' - - 41.870881288\\n'\n",
- " ' - -8.879136\\n'\n",
- " ' - - 41.694339317422646\\n'\n",
- " ' - -8.824238614026456\\n'\n",
- " ' - - 40.171924585721314\\n'\n",
- " ' - -8.902386975546364\\n'\n",
- " ' - - 38.75694209400925\\n'\n",
- " ' - -9.493088042617785\\n'\n",
- " ' - - 38.42424252381525\\n'\n",
- " ' - -9.171674240710018\\n'\n",
- " ' - - 38.49907333213173\\n'\n",
- " ' - -8.676525850529856\\n'\n",
- " ' - - 37.057269459205145\\n'\n",
- " ' - -8.971873318897366\\n'\n",
- " ' - - 37.162874354643776\\n'\n",
- " ' - -7.406745406502978\\n'\n",
- " ' - - 38.19776118392036\\n'\n",
- " ' - -6.931663452624974\\n'\n",
- " ' - - 38.4280922170291\\n'\n",
- " ' - -7.321584397020473\\n'\n",
- " ' - - 39.011852875635526\\n'\n",
- " ' - -6.9787177479519755\\n'\n",
- " ' - - 39.66227871551288\\n'\n",
- " ' - -7.5393956904523804\\n'\n",
- " ' - - 39.66568774825791\\n'\n",
- " ' - -7.03915852435145\\n'\n",
- " ' - - 40.0019453234905\\n'\n",
- " ' - -6.883203763416162\\n'\n",
- " ' - - 40.20373392742229\\n'\n",
- " ' - -7.035724907677206\\n'\n",
- " ' - - 40.350463990828985\\n'\n",
- " ' - -6.8135246275213035\\n'\n",
- " ' - - 41.030499770212515\\n'\n",
- " ' - -6.905947651233703\\n'\n",
- " ' - - 41.593647729084154\\n'\n",
- " ' - -6.22847017956974\\n'\n",
- " ' - - 41.67712153119277\\n'\n",
- " ' - -6.544984134823352\\n'\n",
- " ' - - 41.949682257268876\\n'\n",
- " ' - -6.567927092516641\\n'\n",
- " ' - - 41.96960294343674\\n'\n",
- " ' - -7.1747800681640115\\n'\n",
- " ' - - 41.88337981339092\\n'\n",
- " ' - -7.196871678410446\\n'\n",
- " ' - - 41.81334515396762\\n'\n",
- " ' - -8.156666519264604\\n'\n",
- " ' - - 42.14242723772878\\n'\n",
- " ' - -8.205142297350534\\n'\n",
- " ' - - 41.870881288\\n'\n",
- " ' - -8.879136\\n'\n",
- " ' type: polygon\\n'\n",
- " 'levtype: sfc\\n'\n",
- " \"number: '1'\\n\"\n",
- " 'param: 167/169\\n'\n",
- " \"step: '0'\\n\"\n",
- " 'stream: enfo\\n'\n",
- " \"time: '1200'\\n\"\n",
- " 'type: pf\\n',\n",
- " 'verb': 'retrieve'}\n",
- "2024-09-19 13:50:24 - INFO - Polytope user key found in session cache for user maaw\n",
- "INFO:polytope.api.Client.MCIRN2:Polytope user key found in session cache for user maaw\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP POST https://polytope-test.ecmwf.int:443/api/v1/requests/ecmwf-mars\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': {'verb': 'retrieve', 'request': \"class: od\\ndate: '20240917'\\ndomain: g\\nexpver: 0079\\nfeature:\\n shape:\\n - - 41.870881288\\n - -8.879136\\n - - 41.694339317422646\\n - -8.824238614026456\\n - - 40.171924585721314\\n - -8.902386975546364\\n - - 38.75694209400925\\n - -9.493088042617785\\n - - 38.42424252381525\\n - -9.171674240710018\\n - - 38.49907333213173\\n - -8.676525850529856\\n - - 37.057269459205145\\n - -8.971873318897366\\n - - 37.162874354643776\\n - -7.406745406502978\\n - - 38.19776118392036\\n - -6.931663452624974\\n - - 38.4280922170291\\n - -7.321584397020473\\n - - 39.011852875635526\\n - -6.9787177479519755\\n - - 39.66227871551288\\n - -7.5393956904523804\\n - - 39.66568774825791\\n - -7.03915852435145\\n - - 40.0019453234905\\n - -6.883203763416162\\n - - 40.20373392742229\\n - -7.035724907677206\\n - - 40.350463990828985\\n - -6.8135246275213035\\n - - 41.030499770212515\\n - -6.905947651233703\\n - - 41.593647729084154\\n - -6.22847017956974\\n - - 41.67712153119277\\n - -6.544984134823352\\n - - 41.949682257268876\\n - -6.567927092516641\\n - - 41.96960294343674\\n - -7.1747800681640115\\n - - 41.88337981339092\\n - -7.196871678410446\\n - - 41.81334515396762\\n - -8.156666519264604\\n - - 42.14242723772878\\n - -8.205142297350534\\n - - 41.870881288\\n - -8.879136\\n type: polygon\\nlevtype: sfc\\nnumber: '1'\\nparam: 167/169\\nstep: '0'\\nstream: enfo\\ntime: '1200'\\ntype: pf\\n\"}}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n"
- ]
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"POST /api/v1/requests/ecmwf-mars HTTP/11\" 202 49\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "2024-09-19 13:50:25 - INFO - Request accepted. Please poll ../requests/1fcf6c16-538c-4387-a809-332e92c461a8 for status\n",
- "INFO:polytope.api.Client.MCIRN2:Request accepted. Please poll ../requests/1fcf6c16-538c-4387-a809-332e92c461a8 for status\n",
- "DEBUG:polytope.api.Client.MCIRN2:Server message: Request queued\n",
- "2024-09-19 13:50:25 - INFO - Polytope user key found in session cache for user maaw\n",
- "INFO:polytope.api.Client.MCIRN2:Polytope user key found in session cache for user maaw\n",
- "2024-09-19 13:50:25 - INFO - Checking request status (../requests/1fcf6c16-538c-4387-a809-332e92c461a8)...\n",
- "INFO:polytope.api.Client.MCIRN2:Checking request status (../requests/1fcf6c16-538c-4387-a809-332e92c461a8)...\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 202 49\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "2024-09-19 13:50:25 - INFO - The current status of the request is 'queued'\n",
- "INFO:polytope.api.Client.MCIRN2:The current status of the request is 'queued'\n",
- "DEBUG:polytope.api.Client.MCIRN2:Server message: Request queued\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 202 57\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "2024-09-19 13:50:26 - INFO - The current status of the request is 'processing'\n",
- "INFO:polytope.api.Client.MCIRN2:The current status of the request is 'processing'\n",
- "DEBUG:polytope.api.Client.MCIRN2:Server message: Request processing\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 202 57\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 202 57\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 202 57\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 202 57\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 202 57\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (202)\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client attempting HTTP GET https://polytope-test.ecmwf.int:443/api/v1/requests/../requests/1fcf6c16-538c-4387-a809-332e92c461a8\n",
- "{'headers': {'Authorization': 'EmailKey adam.warde@ecmwf.int:026098cc447ef4ee65be50caa10b9fdd'}, 'json': None}\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://polytope-test.ecmwf.int:443 \"GET /api/v1/requests/1fcf6c16-538c-4387-a809-332e92c461a8 HTTP/11\" 303 179\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): downloads.polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://downloads.polytope-test.ecmwf.int:443 \"GET /default/1fcf6c16-538c-4387-a809-332e92c461a8.grib HTTP/11\" 200 72919\n",
- "DEBUG:polytope.api.Client.MCIRN2:Polytope client received HTTP SUCCESS (200)\n",
- "2024-09-19 13:50:30 - INFO - The current status of the request is 'processed'\n",
- "INFO:polytope.api.Client.MCIRN2:The current status of the request is 'processed'\n",
- "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): downloads.polytope-test.ecmwf.int:443\n",
- "DEBUG:urllib3.connectionpool:https://downloads.polytope-test.ecmwf.int:443 \"HEAD /default/1fcf6c16-538c-4387-a809-332e92c461a8.grib HTTP/11\" 200 0\n"
- ]
- },
- {
- "data": {
- "application/vnd.jupyter.widget-view+json": {
- "model_id": "2e8d39a14d4a4567a36efc6d2a6ef563",
- "version_major": 2,
- "version_minor": 0
- },
- "text/plain": [
- "1fcf6c16-538c-4387-a809-332e92c461a8.grib: 0%| | 0.00/71.2k [00:00
-An example bounding box requested via Earthkit-data:
+An example bounding box requested via earthkit-data:
```python
import earthkit.data
@@ -68,28 +68,24 @@ ds = earthkit.data.from_source("polytope", "ecmwf-mars", request, stream=False,
This request will return a bounding box with forecast date of `20240930T000000` for the three requested parameters for the points within a bounding box with top left coordinate at latitude -1 and longitude -1, and bottom right point at latitude 1 and longitude 1.
-`"polytope"` refers to the underlying service being used to return the data. `"emcwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
+`"polytope"` refers to the underlying service being used to return the data. `"ecmwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
-Notes:
-* The data has to exist in the fdb on the polytope server.
-* No config is required to be passed when using this method, it is generated on the server side.
-* Further details on the `from_source` method can be found here: https://earthkit-data.readthedocs.io/en/latest/guide/sources.html
## Required Fields
-For a boundingbox within the `feature` dictionary two fields are required
+For a bounding box two fields are required within the `feature` dictionary
* `type`
* `points`
For a bounding box `type` must be `boundingbox`.
-`points` must contain two points, the first corresponding to the top left of the requested box, and the second correspongin to the bottom right coordinate. By default they should only contain a latitude and longitude. However as seen below this can be changed with the `axes` key.
+`points` must contain two points, the first corresponding to the top left of the requested box, and the second corresponding to the bottom right coordinate. By default they should only contain a latitude and longitude. However as seen below this can be changed with the `axes` key.
## Optional Fields
-`axes` refers to the axes on which to generate the bounding box. As stated above the minimum default `axes` contains `lat` and `long` meaning if `axes` is not included these values must be provided per point. By default the level is taken from the main body of the request.
+`axes` refers to the axes on which to generate the bounding box. As stated above the minimum default `axes` contains `lat` and `lon` meaning if `axes` is not included these values must be provided per point. By default the level is taken from the main body of the request.
However `axes` can also be provided by the user and with a value for level. Such as here:
@@ -97,7 +93,7 @@ However `axes` can also be provided by the user and with a value for level. Such
"axes" : ["lat", "long", "level"]
```
-In this case the user must provide a `lat`, `long` and `level`. `level` should not be included in the main body of the request in this case. An example can be seen here:
+In this case the user must provide a `lat`, `lon` and `level`. `level` should not be included in the main body of the request in this case. An example can be seen here:
```python
@@ -115,12 +111,12 @@ request = {
"feature" : {
"type" : "boundingbox",
"points" : [[-1, -1, 1000], [1, 1, 500]],
- "axes" : ["lat", "long", "level"],
+ "axes" : ["lat", "lon", "level"],
},
"format" : "covjson",
}
```
-For this request a bounding box with top left corner at lat -1, long -1 and pressure level 1000, and bottom right corner at lat 1, long 1, and pressure level 500.
+For this request, a bounding box with top left corner at lat -1, lon -1 and pressure level 1000, and bottom right corner at lat 1, lon 1, and pressure level 500.
Without level in the `axes` this will be taken from the main body of the request. In the case of `levtype` = `sfc`, no levelist is required.
\ No newline at end of file
diff --git a/docs/Service/Features/feature.md b/docs/Service/Features/feature.md
index fb808ea7..bc866f54 100644
--- a/docs/Service/Features/feature.md
+++ b/docs/Service/Features/feature.md
@@ -12,4 +12,11 @@ Follow the links below to see how to request different types of features.
- [Vertical Profile](vertical_profile.md)
- [Polygon](polygon.md)
- [Bounding Box](boundingbox.md)
-- [Trajectory](trajectory.md)
\ No newline at end of file
+- [Trajectory](trajectory.md)
+
+## Notes
+
+Some important notes that hold for all features are that:
+
+* The data has to exist in the fdb on the polytope server.
+* Further details on the `from_source` method can be found here: [https://earthkit-data.readthedocs.io/en/latest/guide/sources.html](https://earthkit-data.readthedocs.io/en/latest/guide/sources.html)
\ No newline at end of file
diff --git a/docs/Service/Features/polygon.md b/docs/Service/Features/polygon.md
index 99ce9b72..74131870 100644
--- a/docs/Service/Features/polygon.md
+++ b/docs/Service/Features/polygon.md
@@ -39,10 +39,10 @@ Notes:
### Earthkit-data -->
-An example polygon requested via Earthkit-data:
+An example polygon requested via earthkit-data:
```python
-from polytope_mars.api import PolytopeMars
+import earthkit.data
request = {
"class" : "od",
@@ -58,35 +58,31 @@ request = {
"step": "0",
"feature" : {
"type" : "polygon",
- "shape" : [[-1, 1], [-1, 0], [0, 1], [-1, 1]],
+ "shape" : [[-1, 1], [-1, 0], [0, 1]],
},
"format" : "covjson",
}
-result = PolytopeMars().extract(request)
+ds = earthkit.data.from_source("polytope", "ecmwf-mars", request, stream=False, address='polytope.ecmwf.int')
```
This request will return all points contained in `shape` for with forecast date of `20240930T000000` for `step` `0`, ensemble `number` `1` and the three prvoided parameters.
-`"polytope"` refers to the underlying service being used to return the data. `"emcwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
+`"polytope"` refers to the underlying service being used to return the data. `"ecmwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
-Notes:
-* The data has to exist in the fdb on the polytope server.
-* No config is required to be passed when using this method, it is generated on the server side.
-* Further details on the `from_source` method can be found here: https://earthkit-data.readthedocs.io/en/latest/guide/sources.html
## Required Fields
-For a polygon within the `feature` dictionary two fields are required
+For a polygon two fields are required within the `feature` dictionary
* `type`
* `shape`
For a polygon `type` must be `polygon`.
-The values in `points` must correspond to a latitude and a longitude. The first point and last point must also be the same to complete the polygon.
+The values in `points` must correspond to a latitude and a longitude. The first point is assumed to be the last point, however users can also provide the last point in the request to complete the polygon.
-The polygon feature also has a max number of points that can be requested in the perimeter of the polygon, and the max area of the polygon is also constrained based on the vonfig provided.
+The polygon feature also has a max number of points that can be requested in the perimeter of the polygon, and the max area of the polygon is also constrained based on the config provided.
`shape` can also take multiple polygons in a single request in the following form:
@@ -97,8 +93,6 @@ The polygon feature also has a max number of points that can be requested in the
User can also request ranges for other keys such as `number`. In this case the polygon cutout will be returned for each of the values requested.
```python
-from polytope_mars.api import PolytopeMars
-
request = {
"class" : "od",
"stream" : "enfo",
@@ -117,8 +111,6 @@ request = {
},
"format" : "covjson",
}
-
-result = PolytopeMars().extract(request)
```
The returned values will be:
@@ -128,6 +120,3 @@ The returned values will be:
* `number: 2, step: 0, Points within shape`
* `number: 2, step: 1, Points within shape`
-
-## Optional Fields
-
diff --git a/docs/Service/Features/timeseries.md b/docs/Service/Features/timeseries.md
index e82ebe01..8b448719 100644
--- a/docs/Service/Features/timeseries.md
+++ b/docs/Service/Features/timeseries.md
@@ -43,7 +43,7 @@ Notes:
### Earthkit-data -->
-An example of a time-series requested via Earthkit-data:
+An example of a time-series requested via earthkit-data:
```python
import earthkit.data
@@ -62,7 +62,7 @@ request = {
"feature" : {
"type" : "timeseries",
"points": [[-9.10, 38.78]],
- "axes": "step",
+ "axis": "step",
"range" : {
"start" : 0,
"end" : 360,
@@ -75,12 +75,7 @@ ds = earthkit.data.from_source("polytope", "ecmwf-mars", request, stream=False,
```
The following will return a timeseries starting on `2024-10-06 00:00:00` with steps from `0` to `360` including all steps available in between, for the parameters `164/167/169` at the point given. This data will be returned for each ensemble number requested.
-`"polytope"` refers to the underlying service being used to return the data. `"emcwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
-
-Notes:
-* The data has to exist in the fdb on the polytope server.
-* No config is required to be passed when using this method, it is generated on the server side.
-* Further details on the `from_source` method can be found here: https://earthkit-data.readthedocs.io/en/latest/guide/sources.html
+`"polytope"` refers to the underlying service being used to return the data. `"ecmwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
## Required Fields
@@ -88,31 +83,31 @@ For a timeseries within the `feature` dictionary three fields are required
* `type`
* `points`
-* `axes`
+* `axis`
For a timeseries `type` must be `timeseries`.
-`points` has to be a nested list with two points corresponding to a latitude and a longitude.
+`points` must be a nested list with a points containing a latitude and a longitude.
-`axes` refers to the axes on which to generate the timeseries. In this case the timeseries is generated accross `step` based on the inputted `range`. However if the data requested was a climate dataset the `axes` may be `datetime` denoting that the timeseries is generated accross that axes.
+`axis` refers to the axis on which to generate the timeseries. In this case the timeseries is generated across `step` based on the inputted `range`. However if the data requested was a climate dataset the `axis` may be `datetime` denoting that the timeseries is generated across that axis.
## Optional Fields
-`range` is an optional field within `feature`. It refers to the extent of the `axes` on which the timeseries will be generated. In the above case where:
+`range` is an optional field within `feature`. It refers to the extent of the `axis` on which the timeseries will be generated. In the above case where:
```python
- "axes": "step",
+ "axis": "step",
"range" : {
"start" : 0,
"end" : 360,
}
```
-A timerseries accross `step` will start at step `0` and end at step `360` with all steps found in between being included. `range` can also contain `interval`.
+A timeseries across `step` will start at step `0` and end at step `360` with all steps found in between being included. `range` can also contain `interval`.
```python
- "axes": "step",
+ "axis": "step",
"range" : {
"start" : 0,
"end" : 360,
@@ -121,7 +116,7 @@ A timerseries accross `step` will start at step `0` and end at step `360` with a
```
In this case every second step will be returned if it exists.
-As `range` is an optional field it can be left out, however there is not a default value. Instead the user has to include the timeseries `axes` in the main body of the request like below:
+As `range` is an optional field it can be left out, however there is not a default value. Instead the user has to include the timeseries `axis` in the main body of the request like below:
```python
request = {
@@ -139,7 +134,7 @@ request = {
"feature" : {
"type" : "timeseries",
"points": [[-9.10, 38.78]],
- "axes": "step",
+ "axis": "step",
},
"format": "covjson",
}
@@ -147,8 +142,6 @@ request = {
This is equivalent to the first request presented.
-Atleast one of `range` or `step` must be included in the request, but not both. In this case an error will be provided telling the user that `step` is overspecified.
+At least one of `range` or `step` must be included in the request, but not both. In this case an error will be provided telling the user that `step` is overspecified.
Conversely at least one of `range` or `step` must be included.
-
-## Alternate Examples
\ No newline at end of file
diff --git a/docs/Service/Features/trajectory.md b/docs/Service/Features/trajectory.md
index b67c9700..7ba7c2cf 100644
--- a/docs/Service/Features/trajectory.md
+++ b/docs/Service/Features/trajectory.md
@@ -44,7 +44,7 @@ Notes:
### Earthkit-data -->
-An example trajectory requested via Earthkit-data:
+An example trajectory requested via earthkit-data:
```python
import earthkit.data
@@ -72,22 +72,17 @@ ds = earthkit.data.from_source("polytope", "ecmwf-mars", request, stream=False,
This request will return a trajectory with forecast date of `20240930T000000` for the three requested parameters for the points:
-* `lat: -1, long: -1, pressure level: 1000, step: 0`
-* `lat: 0, long: 0, pressure level: 1000, step: 12`
-* `lat: 1, long: 1, pressure level: 250, step: 24`
+* `lat: -1, lon: -1, pressure level: 1000, step: 0`
+* `lat: 0, lon: 0, pressure level: 1000, step: 12`
+* `lat: 1, lon: 1, pressure level: 250, step: 24`
The `trajectory` `feature` also contains another field called `padding` with a default of 1. This is the radius of the circle swept around the trajectory where points within this radius are returned to the user.
-`"polytope"` refers to the underlying service being used to return the data. `"emcwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
-
-Notes:
-* The data has to exist in the fdb on the polytope server.
-* No config is required to be passed when using this method, it is generated on the server side.
-* Further details on the `from_source` method can be found here: https://earthkit-data.readthedocs.io/en/latest/guide/sources.html
+`"polytope"` refers to the underlying service being used to return the data. `"ecmwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
## Required Fields
-For a trajectory within the `feature` dictionary two fields are required
+For a trajectory two fields are required within the `feature` dictionary
* `type`
* `points`
@@ -98,25 +93,25 @@ For a trajectory `type` must be `trajectory`.
The values in `points` can change depending on the `axes`. The default for `axes` is:
```python
-"axes" : ["lat", "long", "level", "step"]
+"axes" : ["lat", "lon", "level", "step"]
```
-In this default case a nested list of at least two points with values for `lat`, `long`, `level`, and `step` must be provided.
+In this default case, a nested list of at least two points with values for `lat`, `lon`, `level`, and `step` must be provided.
Another required field that is within the `feature` dictionary is `padding`. This refers to the radius of the circle swept around the trajectory along which points will be included.
## Optional Fields
-`axes` refers to the axes on which to generate the trajectory. As stated above the minimum default `axes` contains `lat`, `long`, `level`, and `step` meaning if `axes` is not included these values must be provided per point.
+`axes` refers to the axes on which to generate the trajectory. As stated above the minimum default `axes` contains `lat`, `lon`, `level`, and `step` meaning if `axes` is not included these values must be provided per point.
However `axes` can also be provided by the user and with less values. The minimum values of `axes` are:
```python
-"axes" : ["lat", "long"]
+"axes" : ["lat", "lon"]
```
-In this case only `lat` and `long` must be provided in the requested points but a level and time axis must be provided in the main body of the request. These values will be propogated for each set of `lat`, `long` points. For example in the following request:
+In this case only `lat` and `lon` must be provided in the requested points but a level and time axis must be provided in the main body of the request. These values will be propagated for each set of `lat`, `lon` points. For example in the following request:
```python
request = {
@@ -135,22 +130,20 @@ request = {
"feature" : {
"type" : "trajectory",
"points" : [[-1, -1], [0, 0], [-1, -1]],
- "axis" : ['lat', 'long']
+ "axis" : ['lat', 'lon']
},
}
```
The following points would be returned:
-* `lat: -1, long: -1, pressure level: 500, step: 0`
-* `lat: 0, long: 0, pressure level: 500, step: 0`
-* `lat: 1, long: 1, pressure level: 500, step: 0`
-* `lat: -1, long: -1, pressure level: 500, step: 1`
-* `lat: 0, long: 0, pressure level: 500, step: 1`
-* `lat: 1, long: 1, pressure level: 500, step: 1`
+* `lat: -1, lon: -1, pressure level: 500, step: 0`
+* `lat: 0, lon: 0, pressure level: 500, step: 0`
+* `lat: 1, lon: 1, pressure level: 500, step: 0`
+* `lat: -1, lon: -1, pressure level: 500, step: 1`
+* `lat: 0, lon: 0, pressure level: 500, step: 1`
+* `lat: 1, lon: 1, pressure level: 500, step: 1`
The user does not have to give `step` as the time axis. In the case of a climate dataset `datetime` can also be used.
-Combinations such as `"axis" : ['lat', 'step']` will return an error.
-
-If `step` is included as an `axis` and also in the main body of teh request. An error that the request is overspecified will also be thrown.
\ No newline at end of file
+Combinations such as `"axis" : ['lat', 'step']` will return an error if `step` is included as an `axis` and also in the main body of the request. An error that the request is overspecified will also be thrown.
\ No newline at end of file
diff --git a/docs/Service/Features/vertical_profile.md b/docs/Service/Features/vertical_profile.md
index 41c28347..93a0d320 100644
--- a/docs/Service/Features/vertical_profile.md
+++ b/docs/Service/Features/vertical_profile.md
@@ -44,7 +44,7 @@ Notes:
### Earthkit-data -->
-An example vertical profile requested via Earthkit-data:
+An example vertical profile requested via earthkit-data:
```python
import earthkit.data
@@ -64,7 +64,7 @@ request = {
"feature" : {
"type" : "verticalprofile",
"points": [[-9.10, 38.78]],
- "axes": "levelist",
+ "axis": "levelist",
"range" : {
"start" : 0,
"end" : 1000,
@@ -77,43 +77,38 @@ ds = earthkit.data.from_source("polytope", "ecmwf-mars", request, stream=False,
```
The following will return a vertical profile on `2024-10-06 00:00:00` with levels from `0` to `1000` including all levels available in between, for the parameters `164/167/169` at the point given. This data will be returned for each ensemble number requested.
-`"polytope"` refers to the underlying service being used to return the data. `"emcwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
-
-Notes:
-* The data has to exist in the fdb on the polytope server.
-* No config is required to be passed when using this method, it is generated on the server side.
-* Further details on the `from_source` method can be found here: https://earthkit-data.readthedocs.io/en/latest/guide/sources.html
+`"polytope"` refers to the underlying service being used to return the data. `"ecmwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
## Required Fields
-For a vertical profile within the `feature` dictionary three fields are required
+For a vertical profile three fields are required within the `feature` dictionary
* `type`
* `points`
For a vertical profile `type` must be `verticalprofile`.
-`points` has to be a nested list with two points corresponding to a latitude and a longitude.
+`points` must be a nested list with two points corresponding to a latitude and a longitude.
## Optional Fields
-`axes` refers to the axes on which to generate the vertical profile. In this case the vertical profile is generated accross `levelist` based on the inputted `range`. In the vertical profile this field is optional as the default is assumed to be `levelist` if not given.
+`axis` refers to the axis on which to generate the vertical profile. In this case the vertical profile is generated across `levelist` based on the inputted `range`. In the vertical profile this field is optional as the default is assumed to be `levelist` if not given.
-`range` is an optional field within `feature`. It refers to the extent of the `axes` on which the vertical profile will be generated. In the above case where:
+`range` is an optional field within `feature`. It refers to the extent of the `axis` on which the vertical profile will be generated. In the above case where:
```python
- "axes": "levelist",
+ "axis": "levelist",
"range" : {
"start" : 0,
"end" : 1000,
}
```
-A vertical profile accross `levelist` will start at level `0` and end at level `1000` with all levels found in between being included. `range` can also contain `interval`.
+A vertical profile across `levelist` will start at level `0` and end at level `1000` with all levels found in between being included. `range` can also contain `interval`.
```python
- "axes": "levelist",
+ "axis": "levelist",
"range" : {
"start" : 0,
"end" : 1000,
@@ -122,7 +117,7 @@ A vertical profile accross `levelist` will start at level `0` and end at level `
```
In this case every second level will be returned if it exists.
-As `range` is an optional field it can be left out, however there is not a default value. Instead the user has to include the vertical profile `axes` in the main body of the request like below:
+As `range` is an optional field it can be left out, however there is not a default value. Instead the user has to include the vertical profile `axis` in the main body of the request like below:
```python
request = {
@@ -140,7 +135,7 @@ request = {
"feature" : {
"type" : "timeseries",
"points": [[-9.10, 38.78]],
- "axes": "levelist",
+ "axis": "levelist",
},
"format": "covjson",
}
@@ -148,7 +143,7 @@ request = {
This is equivalent to the first request presented.
-Atleast one of `range` or `levelist` must be included in the request, but not both. In this case an error will be provided telling the user that `levelist` is overspecified.
+At least one of `range` or `levelist` must be included in the request, but not both. In this case an error will be provided telling the user that `levelist` is overspecified.
Conversely at least one of `range` or `levelist` must be included.
diff --git a/docs/Service/Installation.md b/docs/Service/Installation.md
index 13f1914a..07ec6bfc 100644
--- a/docs/Service/Installation.md
+++ b/docs/Service/Installation.md
@@ -1,22 +1,15 @@
# Installation
-The script automatically places your token in `~/.polytopeapirc` where the client will pick it up. The token is a long-lived ("offline_access") token.
+Install **earthkit-data** with python3 (>= 3.10) and ``pip`` as follows:
-Install **earthkit-data** with python3 (>= 3.8) and ``pip`` as follows:
+ python3 -m pip install earthkit-data[polytope]
- python3 -m pip install earthkit-data
-
-The package installed like this is **minimal** supporting only GRIB and NetCDF data and cannot access remote services other than URLs. If you want to use more data types or remote services you need to install the optional Python packages.
-
-To use polytope also install its dependencies.
-
-You can install **earthkit-data** with all the optional Polytope packages in one go by using:
-
+Installing like this gives you a **minimal** package which can talk to Polytope. If you want to use more data types or remote services you need to install other optional features of earthkit-data, or just install all of them:
- python3 -m pip install earthkit-data[polytope]
+ python3 -m pip install earthkit-data[all]
-For further details on earthkit-data installation you can visit this page: https://earthkit-data.readthedocs.io/en/latest/install.html.
+For further details on earthkit-data installation you can visit this page: https://earthkit-data.readthedocs.io/en/latest/install.html.
We recommend to create a conda environment for your earthkit installation. This can be done as follows:
@@ -34,9 +27,12 @@ python3 -m ipykernel install --user --name=$envname
# Authentication
-To access ECMWF data a user needs an ECMWF account. This can be created https://www.ecmwf.int/. Once created a user can find their key at https://api.ecmwf.int/v1/key/.
+To access ECMWF data you need an ECMWF account. This can be created https://www.ecmwf.int/. Once created, you can find your key at https://api.ecmwf.int/v1/key/.
+
+**DISCLAIMER**
+> *Polytope is currently available for users at the national meteorological services of ECMWF’s Member and Co-operating States.*
-This should then be placed in their home directory in a file called `~/.polytopeapirc`. Ths file should have the following format:
+Copy your API key into your home directory, in a file called `~/.polytopeapirc`. Ths file should have the following format:
```
@@ -46,6 +42,6 @@ This should then be placed in their home directory in a file called `~/.polytope
}
```
-You should now be automatically authenticated when using Polytope Feature Extraction via Earthkit-data.
+You should now be automatically authenticated when using Polytope feature extraction via earthkit-data.
-Once installed and with an api key in place you can follow the Quick Start guide to begin making requests.
\ No newline at end of file
+After following these steps, go to the Quick Start guide to begin making requests.
\ No newline at end of file
diff --git a/docs/Service/Overview.md b/docs/Service/Overview.md
index 8014833c..e41ba406 100644
--- a/docs/Service/Overview.md
+++ b/docs/Service/Overview.md
@@ -1,22 +1,30 @@
# Polytope Feature Extraction
-Polytope is an open-source web service designed to provide efficient access to hypercubes of data in scientific analysis workflows and is able to federate access between hypercubes in distributed computing resources. It is designed to couple data-centric workflows operating across multiple platforms (HPC, cloud) and across multiple distributed sites.
+Polytope is a web service designed to provide efficient access to datacubes. Polytope's key feature is the ability to directly extract **features** from a datacube, as well as whole fields, without any intermediate copies.
-Polytope feature extraction allows users instead of extracting global fields to only extract data of interest to them, whether this is a time-series, a vertical profile, or a custom region. This approach offers a number of different advantages:
+Features currently includes time-series, vertical profiles, a custom polygon region, bounding box or spatio-temporal trajectory.
-- Reduce I/O usage when requesting data from large datacubes and
+Extracting features directly offers two main advantages:
-- Reduce post-processing needs for users after extraction.
+- Reduced I/O usage when requesting data from large datacubes, which means less data downloaded.
-## Polytope Feature Extraction vs Web-Mars
+- Reduced post-processing needs for users after extraction, making the data more analysis-ready.
-Feature Extraction differs from web-mars in the fact that it allows users to request specific features rather than only global fields, as mentioned above this provides a number of benefits. Polytope also allows users to request global fields by simply omitting the `feature` keyword from the request.
+
+
+
+
+
-Feature extraction is also integrated into the earthkit ecosystem allowing users to request and retireve data using Polytope and then immediately use it with other earthkit tools for mapping, plotting, regridding, and transforming.
+## Polytope Feature Extraction vs Web MARS
-## Feature Extraction Frontend
+Feature Extraction differs from Web-MARS by allowing users to request specific features rather than only global fields. However, Polytope does also allow users to request global fields by simply omitting the `feature` keyword from the request.
-The recommended front-end for Polytope Feature Extraction is earthkit-data. A guide on how to install earthkit-data can be found here, a quick start user guide is also provided here. This allows users to quickly install earthkit-data and to begin making requests.
+Both Polytope and Web-MARS are integrated into the earthkit ecosystem allowing users to request and retrieve data using either service. Earthkit tools for mapping, plotting, regridding, and transforming are available for working with both whole fields and specific features.
+
+## Feature Extraction Client
+
+The recommended client for Polytope Feature Extraction is earthkit-data. A guide on how to install earthkit-data can be found here, a quick start user guide is also provided here. This allows users to quickly install earthkit-data and to begin making requests.
For more in-depth information about the various features see the following pages:
@@ -27,6 +35,6 @@ For more in-depth information about the various features see the following pages
- Bounding Box
- Trajectory
-A design document on the general principles of how requests can be generated can also be found here.
+
A set of example notebooks can also be found in the Examples page along with some examples of integration with other Earthkit libraries.
\ No newline at end of file
diff --git a/docs/Service/Quick_Start.md b/docs/Service/Quick_Start.md
index 13f28bf3..b9c7c5a2 100644
--- a/docs/Service/Quick_Start.md
+++ b/docs/Service/Quick_Start.md
@@ -1,8 +1,8 @@
# Quick Start
-Once a user has installed Earthkit-data and has their credentials in place, it is very easy to make a simple request.
+Once a user has installed earthkit-data and has their credentials in place, you can make a simple request.
-An example of a time-series requested via Earthkit-data:
+An example of a time-series requested via earthkit-data:
```python
import earthkit.data
@@ -34,7 +34,7 @@ ds = earthkit.data.from_source("polytope", "ecmwf-mars", request, stream=False,
```
The following will return a timeseries starting on `2024-10-06 00:00:00` with steps from `0` to `360` including all steps available in between, for the parameters `164/167/169` at the point given. This data will be returned for each ensemble number requested.
-`"polytope"` refers to the underlying service being used to return the data. `"emcwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
+`"polytope"` refers to the underlying service being used to return the data. `"ecmwf-mars"` is the dataset we are looking to retrieve from. Setting `stream=False` returns all the requested data to us once it is available. `address` points to the endpoint for the polytope server.
To view the returned covjson run:
@@ -42,17 +42,10 @@ To view the returned covjson run:
ds._json()
```
-To convert your covjson into an xarray the following can be done:
+To convert your covjson into an xarray the following can be done:
```
ds.to_xarray()
```
-For more information about each feature see the following pages:
-
-### Features
- - Timeseries
- - Vertical Profile
- - Polygon
- - Bounding Box
- - Trajectory
\ No newline at end of file
+For more information about each feature see the Features page.
\ No newline at end of file
diff --git a/docs/images/polytope_feature.png b/docs/images/polytope_feature.png
new file mode 100644
index 00000000..4f6cfab6
Binary files /dev/null and b/docs/images/polytope_feature.png differ
diff --git a/docs/index.md b/docs/index.md
index 375fb23b..00677abc 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -37,10 +37,10 @@ Note that Polytope reads only the user-requested data, instead of whole fields.
!!! important
Note that Polytope reads only the user-requested data, instead of whole fields. Importantly, this implies a significant decrease of the I/O usage when reading data from a datacube.
-Polytope feature extraction consists of the algorithm itself, and the service that uses the algorithm to extract features from ECMWF datacubes. Details on the service can be found the Polytope service Overview. While details on the algorithm can be found on the polytope algorithm Overview.
+Polytope feature extraction consists of the algorithm itself, and the service that uses the algorithm to extract features from ECMWF datacubes. Details on the service can be found in the Polytope service overview, while details on the feature extraction algorithm can be found on the Polytope algorithm overview.
-To learn more about how to use Polytope, refer to the Quick Start. In particular, see the Quickstart page for a step-by-step example of how to use the Polytope software.
-For a more in-depth explanation of how Polytope achieves its feature exctraction, refer to the Developer Guide.
+To learn more about how to use Polytope, refer to the Quickstart page. In particular, see the Quickstart page for a step-by-step example of how to use the Polytope software.
+For a more in-depth explanation of how Polytope achieves its feature extraction, refer to the Developer Guide.
!!! Warning
This project is BETA and will be experimental for the foreseeable future. Interfaces and functionality are likely to change. DO NOT use this software in any project/software that is operational.
@@ -63,7 +63,7 @@ For a more in-depth explanation of how Polytope achieves its feature exctraction
# License
*Polytope* is available under the open source [Apache License](http://www.apache.org/licenses/LICENSE-2.0).
- In applying this licence, ECMWF does not waive the privileges and immunities granted to it by virtue of its status as an intergovernmental organisation nor does it submit to any jurisdiction.
+ In applying this license, ECMWF does not waive the privileges and immunities granted to it by virtue of its status as an intergovernmental organisation nor does it submit to any jurisdiction.
diff --git a/docs/requirements.txt b/docs/requirements.txt
index 710d7792..c3e04038 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,2 +1,3 @@
jinja2>=3.1.3
-Markdown
\ No newline at end of file
+Markdown
+mkdocs-jupyter
\ No newline at end of file
diff --git a/mkdocs.yml b/mkdocs.yml
index 3026749a..33f92ae3 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -17,9 +17,14 @@ nav:
- Overview: Service/Overview.md
- Installation: Service/Installation.md
- Quick Start: Service/Quick_Start.md
- - Design Doc: Service/Design_doc.md
- - Features: Service/Features/feature.md
+ - Features:
+ - Time Series: Service/Features/timeseries.md
+ - Vertical Profile: Service/Features/vertical_profile.md
+ - Polygon: Service/Features/polygon.md
+ - Bounding Box: Service/Features/boundingbox.md
+ - Trajectory: Service/Features/trajectory.md
- Examples: Service/Examples/examples.md
+ # - Design Doc: Service/Design_doc.md
- Algorithm:
- Overview: Algorithm/Overview/Overview.md
- Polytope at ECMWF: Algorithm/Overview/Polytope_at_ECMWF.md
@@ -35,3 +40,6 @@ nav:
Algorithm/Developer_Guide/API.md
# - API levels: Developer_Guide/API.md
# - Mid-level Shapes: Developer_Guide/shapes.md
+plugins:
+ - search
+ - mkdocs-jupyter
+