Skip to content

Updating scripts to download data from AllianceMine#1176

Merged
dondi merged 32 commits intobetafrom
database
Mar 11, 2025
Merged

Updating scripts to download data from AllianceMine#1176
dondi merged 32 commits intobetafrom
database

Conversation

@ntran18
Copy link
Collaborator

@ntran18 ntran18 commented Feb 19, 2025

  • Rewrite the script to load data from AllianceMine, also make the code to follow OOP.
  • Change the schemas of the GRN and PPI to include timestamp for gene and protein table, and annotation type for network (GRN) and interactions (PPI) tables.

ntran18 and others added 20 commits January 21, 2025 19:35
…t == float for length, molecular wweights and pi
Add error handling when run main.py for network database
… protein interactions to have timestamp and for gene and protein tables and annotation type for physical interactions table
… protein_protein_interactions to include _new, and fixing the populate data scripts to add timestamp and source to the gene and protein tables, and adding annotation type for both GRN and PPI network and interactions table
… namespaces and addition of timestamp. need to test that queries work correctly
@ntran18
Copy link
Collaborator Author

ntran18 commented Feb 26, 2025

This PR should be ready to review. I'm sorry, this PR is large :(((. I didn't mean to .....

ceciliazaragoza and others added 9 commits February 26, 2025 01:01
…ry to only have timestamp query if timestamp is 2025 or later. need to fix query though since ZAP1 gene is not showing correct self-regulating expression
…ed constants to dbConstants, and now queries don't perform correctly
…s and protein-dals. correctly make query based on timestamp, need to check if add source to query
@ntran18 ntran18 changed the title Updating scripts to download data from AllianceMine and new schemas Updating scripts to download data from AllianceMine [MERGE THIS FIRST B4 CECILIA's PR] Mar 10, 2025
@ntran18 ntran18 changed the title Updating scripts to download data from AllianceMine [MERGE THIS FIRST B4 CECILIA's PR] Updating scripts to download data from AllianceMine Mar 10, 2025
Copy link
Owner

@dondi dondi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Phew! It’s a lot of code and documentation, but it’s actually a net 450 fewer lines 💪🏼

We should look into ways to protect against SQL injection in a future ticket, plus of course we should actually migrate our existing databases to this new approach. I’ll write up new issues for those

Comment on lines +50 to +53
return `SELECT DISTINCT protein1, protein2${annotation} FROM
${namespace}.physical_interactions WHERE
physical_interactions.time_stamp='${timestamp}' AND physical_interactions.source='${source}' AND
${buildNetworkProteinsQuery(proteins)} ORDER BY protein1 DESC;`;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noting that the approach leaves us theoretically vulnerable to SQL injection attacks. For future work, we should look into safer ways to build our SQL queries

@dondi dondi merged commit 3c605f0 into beta Mar 11, 2025
3 checks passed
@dondi dondi deleted the database branch March 11, 2025 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants