3482 update output format #5
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
remo87
commented
Nov 4, 2024
remo87
commented
- add jq because polars can't handle the size of the file use subprocess to execute the jq
- Refactor homologue transformer to accept binary input and generate temporary file for processing
use subprocess to execute the jq
…mporary file for processing
javfg
requested changes
Nov 4, 2024
| def transform(src: BinaryIO, dst: BinaryIO) -> None: | ||
| jq_command = '.genes | {"genes": map({id: .id, name: .name})}' | ||
|
|
||
| # Generate a temporary file path tp store the source file |
There was a problem hiding this comment.
Suggested change
| # Generate a temporary file path tp store the source file | |
| # Generate a temporary file path to store the source file |
Comment on lines
14
to
23
| temp_path = Path(f'/tmp/{uuid4()}.json') | ||
|
|
||
| temp_path.write_bytes(src.read()) | ||
|
|
||
| # Ejecuta jq con subprocess | ||
| result = subprocess.run( | ||
| ['jq', jq_command, temp_path], | ||
| capture_output=True, | ||
| text=True | ||
| ) |
There was a problem hiding this comment.
Suggested change
| temp_path = Path(f'/tmp/{uuid4()}.json') | |
| temp_path.write_bytes(src.read()) | |
| # Ejecuta jq con subprocess | |
| result = subprocess.run( | |
| ['jq', jq_command, temp_path], | |
| capture_output=True, | |
| text=True | |
| ) | |
| # Store the file and run jq on it | |
| with tempfile.NamedTemporaryFile() as temp_file: | |
| temp_file.write(src.read()) | |
| result = subprocess.run( | |
| ['jq', jq_command, temp_file.name], | |
| capture_output=True, | |
| text=True, | |
| ) |
It's better to use tempfile when creating temporary files for security reasons.
| ) | ||
|
|
||
| # extract genes list | ||
| genes_list = inputGenes.explode('genes').unnest('genes') | ||
| # Verifica si el comando se ejecutó correctamente |
|
|
||
| # read id and name | ||
| output = genes_list.select(["id","name"]) | ||
| # # read id and name |
There was a problem hiding this comment.
Suggested change
| # # read id and name | |
| # read id and name |
javfg
reviewed
Nov 5, 2024
|
|
||
| # read id and name | ||
| output = genes_list.select(["id","name"]) | ||
| # Ejecuta jq con subprocess |
There was a problem hiding this comment.
We should keep comments in english
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.