Update civic API #4775

leexgh · 2023-11-10T03:31:17Z

Fix: cBioPortal/cbioportal#9521
Civic stops supporting v1 API and switches to v2 graphql API. This PR changes civic rest API to graphql API, other things remain the same.
Now we only need to call one API to get civic gene information and all variants associated with the gene, not need to keep two calls for gene and variant separately.

inodb · 2023-11-10T18:52:01Z

@leexgh Thanks for fixing this! Looks like it's not yet working for copy number data:

https://cbioportal.mskcc.org/patient?studyId=ucec_tcga_pub&caseId=TCGA-BK-A0CC

onursumer · 2023-11-10T19:14:17Z

Looks like we are querying the civic API separately for each gene in the CNA and mutation tables on patient view. Maybe that's why the CNA table is failing?

src/shared/lib/CivicUtils.ts

onursumer · 2023-11-10T19:58:01Z

packages/cbioportal-utils/src/civic/CivicUtils.ts

+    hugoSymbols.forEach(hugoSymbol =>
+        promises.push(civicClient.getCivicGenesBatch(hugoSymbol))


This is not a batch query anymore. We are making a separate API call for each hugo symbol. This may cause a rate limit issue.

Yes that's the problem right now. I don't see a way to send "POST" queries that can accept a list of hugo gene symbols. As far as I can see it can only accept single gene or giving the "start civic id" and "end civic id", the response will then include all genes between this range, but we still need to send extra queries to get the civic id, so it's still not ideal.
For results view it only sends one gene, but for copy number alterations on patients view it sends too many requests. I already asked civic team to see if they have a better way to handle it.

This is solved. Civic team just made a new release yesterday, they can support a list of hugo symbols now in their genes API.

onursumer

Looks good. Thanks!

Just added a few minor suggestions.

packages/cbioportal-utils/src/civic/CivicDataFetcher.ts

netlify · 2024-01-22T05:24:37Z

✅ Deploy Preview for cbioportalfrontend ready!

Name	Link
🔨 Latest commit	`ff5fb8b`
🔍 Latest deploy log	https://app.netlify.com/sites/cbioportalfrontend/deploys/65bb560dddaf730009af4938
😎 Deploy Preview	https://deploy-preview-4775--cbioportalfrontend.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

onursumer

Added a few minor suggestions

packages/cbioportal-utils/src/civic/CivicDataFetcher.ts

packages/react-mutation-mapper/src/component/civic/CivicCard.tsx

alisman · 2024-01-26T17:39:20Z

@leexgh i'd like to suggest that we not use Apollo client for this. It's a rather heavy dependency to bring into the project just for one fetch and brings a lot of unecessary code/complication. We should be able to fetch data from a graphql endpoint using fetch. We just have to find the data format the endpoint expects and generate/serialize it. I'm happy to help with this and it shouldn't take more than an hour i wouldn't think.

fetch(<graphql_endpoint>, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ query: '{ posts { title } }' }),
})
.then(res => res.json())
.then(res => console.log(res.data));

alisman · 2024-01-29T19:12:44Z

packages/cbioportal-utils/src/model/Civic.ts

-    level: EvidenceLevel;
-    drugs: string[];
-    disease?: string;
+export interface EvidenceCountsByType {


lets put Civic in the name since it's particular to Civic, right?

alisman · 2024-01-29T19:38:28Z

packages/cbioportal-utils/src/civic/CivicUtils.ts

-            let geneSymbol = mutation.gene.hugoGeneSymbol;
-            let geneEntry = civicGenes[geneSymbol];
-            let proteinChanges = [mutation.proteinChange];
+        const geneToProteinChangeSet: {


@leexgh just took another look at this code and i think it could use a few unit tests. it doesn't seem like the split protein changes are covered by the current spec, but i could be wrong.

Add unit tests for this

alisman · 2024-01-29T19:39:22Z

packages/cbioportal-utils/src/civic/CivicUtils.ts

+        }, {} as { [geneSymbol: string]: Set<String> });
+
+        for (const geneSymbol in geneToProteinChangeSet) {
+            if (geneSymbol in civicGenes) {


i think you are really just iterating through civicGenes, so no need for the outer for loop.

Good point, all genes in civicGenes should be in mutationSpecs, so we could change it to iterating through civicGenes

alisman · 2024-01-29T19:42:27Z

packages/cbioportal-utils/src/civic/CivicUtils.ts

+                const proteinChangeSet = geneToProteinChangeSet[geneSymbol];
+                const geneVariants = civicGenes[geneSymbol].variants;
+                for (const variantName in geneVariants) {
+                    if (proteinChangeSet.has(variantName)) {


again, if proteinChangeSet is a subjset of geneVarits, then you could just iterate through proteinChangeSet and make this code clearer.

proteinChangeSet is not a subset of geneVarits, they both contain some variants that are not in the other set. We can do something like find shared variants from the two sets and iterate from that list, but it doesn't make the logic easier, so maybe we can keep this

leexgh force-pushed the civic-graphql branch from 42dd186 to 993d9cf Compare November 10, 2023 03:32

leexgh added the external api This PR is related to handling an external API label Nov 10, 2023

leexgh requested a review from onursumer November 10, 2023 15:31

onursumer reviewed Nov 10, 2023

View reviewed changes

onursumer reviewed Nov 14, 2023

View reviewed changes

packages/cbioportal-utils/src/civic/CivicDataFetcher.ts Outdated Show resolved Hide resolved

packages/cbioportal-utils/src/civic/CivicDataFetcher.ts Outdated Show resolved Hide resolved

leexgh force-pushed the civic-graphql branch from a871a1c to 48b8b46 Compare November 15, 2023 07:45

leexgh force-pushed the civic-graphql branch from 51c82b9 to cdb1b84 Compare January 22, 2024 06:29

alisman self-requested a review January 22, 2024 15:38

onursumer reviewed Jan 22, 2024

View reviewed changes

packages/cbioportal-utils/src/civic/CivicDataFetcher.ts Outdated Show resolved Hide resolved

packages/react-mutation-mapper/src/component/civic/CivicCard.tsx Outdated Show resolved Hide resolved

leexgh requested a review from onursumer January 24, 2024 15:41

leexgh force-pushed the civic-graphql branch from cd7b7ac to 7dc15fd Compare January 24, 2024 20:55

alisman reviewed Jan 29, 2024

View reviewed changes

leexgh force-pushed the civic-graphql branch 3 times, most recently from bd8f301 to 4a1d818 Compare January 31, 2024 21:54

leexgh added 8 commits February 1, 2024 03:26

Update civic API to new graphql API

a0a4574

Add cross-fetch for tests

6b06414

Remove null responses

38e2e1b

Use new genes api from civic

66595b6

Update civic api to Variants and Genes queries

95adfe9

Update tests

53c90a0

Remove comments

23416fa

Fix comments

e54590b

leexgh added 2 commits February 1, 2024 03:26

Remove Apollo and use native fetch

4f637dd

Add unit tests for utils

ff5fb8b

leexgh force-pushed the civic-graphql branch from 4a1d818 to ff5fb8b Compare February 1, 2024 08:27

alisman approved these changes Feb 5, 2024

View reviewed changes

alisman merged commit 61a7cc0 into cBioPortal:master Feb 5, 2024
13 of 15 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update civic API #4775

Update civic API #4775

leexgh commented Nov 10, 2023 •

edited

Loading

inodb commented Nov 10, 2023

onursumer commented Nov 10, 2023

onursumer Nov 10, 2023

leexgh Nov 13, 2023

leexgh Nov 14, 2023

onursumer left a comment

netlify bot commented Jan 22, 2024 •

edited

Loading

onursumer left a comment

alisman commented Jan 26, 2024

alisman Jan 29, 2024

alisman Jan 29, 2024

leexgh Jan 31, 2024

alisman Jan 29, 2024

leexgh Jan 31, 2024

alisman Jan 29, 2024

leexgh Jan 31, 2024

		hugoSymbols.forEach(hugoSymbol =>
		promises.push(civicClient.getCivicGenesBatch(hugoSymbol))

Update civic API #4775

Update civic API #4775

Conversation

leexgh commented Nov 10, 2023 • edited Loading

inodb commented Nov 10, 2023

onursumer commented Nov 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

onursumer left a comment

Choose a reason for hiding this comment

netlify bot commented Jan 22, 2024 • edited Loading

✅ Deploy Preview for cbioportalfrontend ready!

onursumer left a comment

Choose a reason for hiding this comment

alisman commented Jan 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leexgh commented Nov 10, 2023 •

edited

Loading

netlify bot commented Jan 22, 2024 •

edited

Loading