Skip to content

Latest commit

 

History

History
346 lines (186 loc) · 18.1 KB

README.md

File metadata and controls

346 lines (186 loc) · 18.1 KB

Climate Change event : 7-9th Feb 2020

Where the IBM team will be

Saturday/Sunday : We will be downstairs - please book a formal 15 min mentoring session or just catch one of us if you see we are free.

0.1 Datasets and DataLayers

We have arranged enhanced access to some Datasets/DataLayers for this event. Datasets contain one or more DataLayers. Datasets have an ID number and short names and DataLayers have the same (ID + short names). We are providing a spreadsheet which decribes the mapping and availability of Datasets vs DataLayers.

Spreadsheet file is here -> Spreadsheet : It has 410 datalayers for you to look at

0.2 TWC API-keys

TWC API-keys are In this github

APIs : https://docs.google.com/document/d/15Ru_3wdMgpbM4aOCm-4qNAnRfjx2w-Ruw3lnr8Hnodk/edit

0.3 New Notebooks

Please got to https://github.com/IBM/ibmpairs/tree/master/examples where we have published some Notebooks very relevant to the Climate Change theme of the weekend. Please see Notebook file ending in .ipynb in each example.

NOTE : Do not cut/paste from this github README since there are control characters that get copied across that you can't see and the Python notebook does not like and so execution stop. Go here for tested version of the text you need.

Where you see the following line in any of the Notebooks :

PAIRS_SERVER='https://pairs.res.ibm.com'
PAIRS_USER=<username>'
PAIRS_CREDENTIALS = (
    PAIRS_USER, paw.get_pairs_api_password(PAIRS_SERVER, PAIRS_USER, passFile= os.path.expanduser('~/ibmpairspass.txt'))
)

Replace it with the following lines from here :

PAIRS_USER=‘<your IBM ID>’
PAIRS_SERVER=’https://oxaihack2020.eu-gb.mybluemix.net'
PAIRS_PASSWORD=‘pairs’
BASE_URI=‘/’
PAIRS_CREDENTIALS=(PAIRS_USER, PAIRS_PASSWORD)

Important : must do by 9pm on Friday 7th Feb

For people who wish to use the PAIRS GUI or PAIRS APIs we need you to have carried out steps 1 to 3 by 9pm on Fri 7th Feb 2020 so that we are able to add you to a special access group that has been arranged for this event. This cannot be carried out on Saturday or Sunday.

1. MUST DO : Get an IBM ID and IBM Cloud account

  • Go to https://ibm.biz/Bdzwwi and fill in the form using your Oxford University email address ending in ox.ac.uk and a password of your choice
  • An IBM ID and IBM Cloud account will be automatically provisioned for you. The IBM ID is your email
  • Verify your account through the email you receive in your Inbox
  • Login to the https://cloud.ibm.com/login to check that you see the initial Dashboard page - you will use this account when using Watson Studio

2. MUST DO : Register your IBM ID to get access to PAIRS GUI

You must have done step 1 above before doing this step, an active IBM ID is required.

  • Go to https://ibmpairs.mybluemix.net/
  • NOTE we have seen issues in the Safari browser that a blank page is returned the first time you go to this link. Just Refresh and all should be OK
  • Click "Get Started"
  • Click "Sign in using IBM ID"
  • Enter your IBM ID and then click Continue
  • Login with your IBMid password
  • Read the Terms and conditions and click "I accept the terms"
  • You will be presented with the GUI of PAIRS
  • Click the circular symbol to the top right of the screen and check your username is shown
  • You are now part of the Trial group. The PAIRS team will monitor all users with ox.ac.uk in their email and move them to the Oxford group to give you access to a large number of datalayers in PAIRS

3. MUST DO : Register your IBM ID to get access to PAIRS APIs

You must have done step 1 above before doing this step, an active IBM ID is required.

In order to allow your IBM ID access to the PAIRS APIs you will need to register your IBM ID.

NOTE When you click on the link below you will be asked to fill in your IBM ID and a password. Do NOT enter your IBM ID password in the Password field - put 'pairs' (lowercase, without quotes) instead.

  • Use Firefox or Safari browsers and go to https://oxaihack2020.eu-gb.mybluemix.net/api-auth

  • At the username/password prompt, enter your IBM ID. DO NOT use your IBM password for password field, just type 'pairs' (lowercase, without quotes). Click OK

  • You will see long string of characters returned (example 80889d27-e42a-4e13-8f37-4829659620cc, your text will be different to this example), save the reply to a text editor of your choice

  • Refresh your browser to go to the same link and you should get a reply which contains your IBM ID email (example : {"_id":"pairs-your IBM ID","_rev":"1-c7a7500961d845a7ce0f8ea8a430295a","token":"80889d27-e42a-4e13-8f37-4829659620cc"}), this confirms you are registered to call the PAIRS APIs initerface

  • Save away the reply to a text editor of your choice

OVERVIEW

Architecture

An overview of the architecture is shown below.

For this event, we have created a PAIRS Proxy to provide more rights than a standard User would be given. In a normal architecture this Proxy is not present and an end users access rights are controlled by the PAIRS group they are assigned to.

Useful background links

PAIRS

There are two modes for using PAIRS, Graphical User Interface (GUI) and via API calls. There are "Datasets" and "DataLayers". There is a one to many relationship (a dataset contains one or more datalayers). A CSV file showing the datasets/datalayers will be shared with you during the Event (see Step 2 above). We recommend you take some time looking at the contents before using the GUI and/or APIs to determine if some DataLayers are relevant to your idea.

PAIRS GUI

We recommend quickly going through the videos (they are each very short) to become familiar with the interface and then go to the step by step instructions while also looking at the online tutorial :

PAIRS APIs

Documentation for PAIRS APIs. We recomend quickly going becoming familiar with the APIs and then go to the step by step instructions :

TWC

APIs

We have arranged for you to have access to Historic and Forecast APIs. The full set of TWC APIs is shown here

Some additional useful information on using TWC APIs is contained in this PDF

Watson Studio

Documentation :

DETAILS

PAIRS GUI

This section shows how to use the PAIRS GUI functionality on a datalayer over Winchester in Hampshire. It shows how we can use a satellite image to identify an abmormally high value of heat being generated at a specific date in 2019.

Make sure you are logged in

Click on the "Data Explorer" option

Enter 49687 and click Search. On the Search result click "Data Layers"

Scroll down the pop up and find 49687 and then click "New query using this layer"

Click Next

Zoom the map to the Winchester area (north of Southampton in the South of the UK). Make sure Rectangle is select and click your mouse once (and then let go) on the map to indicate the start of the rectangle and then drag a rectangle similar to what you see below

Enter the exact date shown below. You need to click on each date item to change it. NOTE : there is a hint on page indicating for what date ranges this datalayer has data for. This is important since not all datalayers have data for all dates. We (IBM) asked for specific data to be created to cover IBM Hurlsey which is close to Wiinchester. Click Next.

Do not change an item. Click Next

Give youe Query a meaningful name so you can remember what it was doing. Click "Run query"

In the Query tab your request is shown running. Time to get a drink or a bite to eat !! Come back after a few minutes to check on the status

Once completed your query will be shown in the "Recent Queries" section of the Queries tab/. Click on the image

You might just be able to see a red dot in the centre of the image. Zoom in. to take a closer look. In the below image, we have clicked on the map just to the left of the red dot - a graph is shown of some historic values at this point. This shows the value of data at this point as defined in the Legend. Note the value.

Click on the red area with your mouse and look at the value again - you will see that it is higher then the previous click that was carried out

Zoom further into the map and change the Opacity so you can see the map a little more clearly

Click on the image and press the "Download CSV" button

A CSV file is created with the values of the point for a range of dates

From the PAIRS GUI there are a range of option when you click the Actions option on the top left of the page

Click "Download data". This will download a zip file to your local machine and this can be viewed in applications like QGIS.

Click on "Developer tools". This gives the curl command to create exactly the same query you created manually and another curl command to check the query has finished

Calling PAIRS and TWC APIs within Watson Studio

We will be using IBM Watson Studio and Jupyter Notebooks to call both PAIRS and TWC APIs for this event.

We are now going to Import a Project file that contains two Notebooks :

  • Winchester-datalayer-sample : Replicates the same data as shown in the PAIRS GUI steps above
  • External-github-pairs-example : A wide range of calls to multiple datalayers

The Project zip file is located here. Please click on this link and download it to your local machine and then follow the steps below.

Setting up Watson Studio for the first time

Go to https://cloud.ibm.com/login and login with your IBM ID. You should see the Dashboard page

Type 'watson studio' in the Search field, the icon with auto filter. Click on the Watson Studio icon

Make sure the Lite Plan is picked and click Create

Click the Get Started button

Click the Get Started button again

Click the. New. Project button

Click on Create a project from a sample or file

We now need to create a Cloud Object Store for Watson Studio to use. On the right of the screen in the Definie storage section click Add

You will see the Cloud Object Storage page

Scroll down and click Create

Click Confirm

Click Refresh and make sure ann entry appears under the Storage section

In the Upload file section click on browse and locate the zip file you downloaded

Make sure the zip filename is shown. Name the project (a suggestion is to use your team name in the name somewhere in the project name). Click Create

Wait for the Import

Wait for the successfully created prompt. Click View all projects

Click on Assets. You should see two Notebooks

Click on Winchester-datalayer-sample. And then click on the Pencil icon

Check that "1vCPU and 4 GB RAM and is free" is shown. This is important since if anything else is shown then you will start consuming resources in the Lite account and could possibly run out of the 50 vCPU hours allocated to free accounts.

If you create a new Notebook always pick "1vCPU and 4 GB RAM and is free" for the environment otherwise your Lite IBM Cloud account will use it's resources and there will become a point where the Notebook will be locked unless you revert back to the "1vCPU and 4 GB RAM and is free" level.

VERY IMPORTANT Change to your IBM ID leaving all other parameters As-is

Save the Notebook. IMPORTANT Please regularly click this icon when making changes so that any changes are saved

Click the Restart Kernel and run the whole Notebook icon

Click Restart and Run All Cells

The page will jump to the end of the Notebook to show that all Cells will be run in sequence

Scroll to the top and you will see various Cells being executed. Ones that have a " * " indicate that the Cell is still running. Once the Notebook has completed then you can see the outputs from each Cell

Look at the Notebook to see how various PAIRS APIs are used. Good luck on your teams activities !!!