Enhance ApifyTool with additional methods #79
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit enhances the ApifyTool by adding the following methods: - get_dataset: Get the dataset associated with a given actor run - get_dataset_items: Get all items from a given dataset - get_key_value_store: Get the key-value store associated with a given actor run - download_file: Download a file from the key-value store of a given actor run - store_dataset: Store data in an Apify dataset These additional methods provide more functionality for interacting with Apify services, such as fetching datasets, accessing key-value stores, downloading files, and storing data in datasets. The README.md file has also been updated to reflect these changes and provide usage examples for the new methods.
|
Hi, can I get some feedback on this pull request? |
|
Pretty interesting, but I think we might want to break some of those function into individual tools so we can be specific around attributes as well |
joaomdmoura
requested changes
Aug 11, 2024
| - `store_dataset`: Store data in an Apify dataset. | ||
|
|
||
| You can use these methods to interact with various Apify services, such as running actors, waiting for actor runs to complete, fetching datasets, accessing the key-value store, downloading files, and storing data in datasets. | ||
|
|
There was a problem hiding this comment.
If you don't mind taking some inspiration from existing READMEs for other tools, it would be nice to have most following a standard
| @@ -37,3 +37,4 @@ | |||
| from .youtube_channel_search_tool.youtube_channel_search_tool import YoutubeChannelSearchTool | |||
| from .youtube_video_search_tool.youtube_video_search_tool import YoutubeVideoSearchTool | |||
| from .spider_tool.spider_tool import SpiderTool | |||
| from .apify_tool.apify_tool import ApifyTool | |||
There was a problem hiding this comment.
Nice, I think we just miss one extra import on the other init.py file 💪🏼
|
|
||
| # Execute the Apify action using the client | ||
| # For example, to run an actor: | ||
| actor_run = self.run_actor(actor_id="your_actor_id", run_input={"query": query}) |
There was a problem hiding this comment.
I think we want the agent to pass the actor id in this case together with the query?
Also might be worth having another tool to lsit the actor ids so it can decide / learn what to call?
Comment on lines
+42
to
+71
| def get_dataset(self, actor_run: ActorRun, dataset_id: str = None) -> Dataset: | ||
| """ | ||
| Get the dataset associated with the given actor run. | ||
| """ | ||
| if dataset_id: | ||
| return self.client.dataset.get(dataset_id) | ||
| else: | ||
| return actor_run.dataset | ||
|
|
||
| def get_dataset_items(self, dataset: Dataset) -> List[Dict[str, Any]]: | ||
| """ | ||
| Get all items from the given dataset. | ||
| """ | ||
| return list(dataset.iterate_items()) | ||
|
|
||
| def get_key_value_store(self, actor_run: ActorRun) -> Dict[str, Any]: | ||
| """ | ||
| Get the key-value store associated with the given actor run. | ||
| """ | ||
| return actor_run.key_value_store | ||
|
|
||
| def download_file(self, actor_run: ActorRun, key: str, file_path: str): | ||
| """ | ||
| Download a file from the key-value store of the given actor run. | ||
| """ | ||
| with open(file_path, "wb") as file: | ||
| file.write(actor_run.key_value_store.get_file(key).read()) | ||
|
|
||
| async def _arun(self, query: str) -> str: | ||
| raise NotImplementedError("This tool does not support async mode.") |
There was a problem hiding this comment.
I think these specifically cpould be broken into other tools as well so it's pretty specific, hte way it's setup now the agent won't be able to use them
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit enhances the ApifyTool by adding the following methods:
These additional methods provide more functionality for interacting with Apify services, such as fetching datasets, accessing key-value stores, downloading files, and storing data in datasets.
The README.md file has also been updated to reflect these changes and provide usage examples for the new methods.