notebooks/SPARQLEndpoints.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Using SPARQL Endpoints with PyShEx\n",
    "\n",
    "The '-ss' option tells `shex_evaluator` to reference SPARQL endpoint instead of a local RDF graph.  As ab example:\n",
    "\n",
    "```bash\n",
    "shexeval -ss  https://query.wikidata.org/sparql test_shape.shex\n",
    "```\n",
    "\n",
    "Would treat the *entire* contents of the Wikidata sparql endpoint as one huge graph.\n",
    "\n",
    "This is probably not what you want to do.  There are several options that can be used in its place:\n",
    "1.  Evaluate every non-bnode subject:\n",
    "\n",
    "    `shexeval -ss -A http://wifo5-04.informatik.uni-mannheim.de/drugbank/sparql test.shex`\n",
    "    \n",
    "    \n",
    "2. Use the RDF type (\"-ut\") to identify the shape expressions(s) to evaluate the nodes against\n",
    "\n",
    "    `shexeval -ss -ut http://wifo5-04.informatik.uni-mannheim.de/drugbank/vocab/resource/class/Offer http://wifo5-04.informatik.uni-mannheim.de/drugbank/sparql test.shex`\n",
    "    \n",
    "    \n",
    "3. The shape to evaluate is the object of a specific predicate (\"-sp\")\n",
    "\n",
    "    `shexeval -ss -sp http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugbank/  http://wifo5-04.informatik.uni-mannheim.de/drugbank/sparql test.shex`\n",
    "    \n",
    "The most workable approach, however, is to use a SPARQL query to generate the focus nodes.  The query must yield a list of elements named `?item`:\n",
    "\n",
    "```bash\n",
    "> cat q1.sparql\n",
    "PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\n",
    "PREFIX vocabClass: <http://wifo5-04.informatik.uni-mannheim.de/drugbank/vocab/resource/class/>\n",
    "\n",
    "SELECT DISTINCT ?item WHERE {\n",
    "  ?item rdf:type vocabClass:Offer\n",
    "}\n",
    "LIMIT 10\n",
    ">\n",
    "```\n",
    "\n",
    "This query can then be used as input to the ShEx evaluator:\n",
    "    \n",
    "    `shexeval -se -ss --stopafter -sq q1.sparql http://wifo5-04.informatik.uni-mannheim.de/drugbank/sparql test.shex`\n",
    "    \n",
    "    \"-se\" option says stop on the first error\n",
    "    \n",
    "    \"--stopafter\" option says stop after N (5) nodes are evaluated\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SPARQL Endpoints in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example assumes that PyShEx has been installed in jupyter environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00001: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00002: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00003: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00004: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00005: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00006: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00007: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00008: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00009: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00010: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00011: FAIL:   Testing <http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00011> against shape http://example.org/t1/S1\n",
      "    Triples:\n",
      "      <http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00011> foaf:page <http://www.drugbank.ca/drugs/DB00011> .\n",
      "   1 triples less than {3,3}\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00012: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00013: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00014: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00015: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00016: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00017: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00018: FAIL:   Testing <http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00018> against shape http://example.org/t1/S1\n",
      "    Triples:\n",
      "      <http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00018> foaf:page <http://www.drugbank.ca/drugs/DB00018> .\n",
      "   1 triples less than {3,3}\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00019: PASS\n",
      "http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugs/DB00020: PASS\n"
     ]
    }
   ],
   "source": [
    "from pyshex.shex_evaluator import ShExEvaluator\n",
    "from pyshex.user_agent import SlurpyGraphWithAgent\n",
    "from pyshex.utils.sparql_query import SPARQLQuery\n",
    "\n",
    "# SPARQL Endpoint\n",
    "endpoint = 'http://wifo5-04.informatik.uni-mannheim.de/drugbank/sparql'\n",
    "\n",
    "# SPARQL Query\n",
    "sparql = \"\"\"\n",
    "PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\n",
    "PREFIX vocabClass: <http://wifo5-04.informatik.uni-mannheim.de/drugbank/vocab/resource/class/>\n",
    "\n",
    "SELECT DISTINCT ?item WHERE {\n",
    "  ?item rdf:type vocabClass:Offer\n",
    "}\n",
    "LIMIT 20\n",
    "\"\"\"\n",
    "\n",
    "# ShEx Expression\n",
    "shex = \"\"\"\n",
    "PREFIX drugbank: <http://wifo5-04.informatik.uni-mannheim.de/drugbank/resource/drugbank/>\n",
    "PREFIX foaf: <http://xmlns.com/foaf/0.1/>\n",
    "PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>\n",
    "PREFIX : <http://example.org/t1/>\n",
    "\n",
    "START=@:S1\n",
    "\n",
    ":S1 {foaf:page IRI{3} ;                     # one or more foaf pages\n",
    "     drugbank:limsDrugId xsd:string       # ane exactly one drug id\n",
    "}\"\"\"\n",
    "\n",
    "\n",
    "result = ShExEvaluator(SlurpyGraphWithAgent(endpoint),\n",
    "                       shex,\n",
    "                       SPARQLQuery(endpoint, sparql).focus_nodes()).evaluate()\n",
    "for r in result:\n",
    "    print(f\"{r.focus}: \", end=\"\")\n",
    "    if not r.result:\n",
    "        print(f\"FAIL: {r.reason}\")\n",
    "    else:\n",
    "        print(\"PASS\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}