Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"can only join an iterable" #257

Open
esnible opened this issue Jan 13, 2025 · 1 comment
Open

"can only join an iterable" #257

esnible opened this issue Jan 13, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@esnible
Copy link
Member

esnible commented Jan 13, 2025

Describe the bug
When Replicate isn't "warmed up", there are strange exceptions.

For example, examples/callback/README.md tells me to python repair_main.py, but this produces a confusing stack trace.

Perhaps the code that invokes Replicate or LiteLLM needs to retry on certain failures?

It would be easier to fix this if I could reproduce this. Once the failure happens then I can't produce the failure again.

To Reproduce
python repair_main.py when Replicate isn't warmed up.

Expected behavior
Either correct behavior or an error message that tells me to run the program again would be preferrable.

Additional context
The confusing stack trace was

Traceback (most recent call last):
  File "/Users/snible/src/prompt-declaration-language/examples/callback/repair_main.py", line 30, in <module>
    pdl_output: PDLResult = pdl.pdl.exec_file(
                            ~~~~~~~~~~~~~~~~~^
        "./repair_prompt.pdl", scope=cast(pdl.pdl.ScopeType, pdl_input)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl.py", line 151, in exec_file
    result = exec_program(program, config, scope, loc, output)
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl.py", line 73, in exec_program
    result, _, scope, trace = process_prog(state, scope, prog, loc)
                              ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 241, in process_prog
    for result, document, final_scope, trace in schedule([doc_generator]):
                                                ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_scheduler.py", line 107, in schedule
    msg = gen.send(v)
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 299, in step_block
    result, background, scope, trace = yield from step_advanced_block(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        state, scope, block, loc
        ^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 350, in step_advanced_block
    raise exc from exc
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 332, in step_advanced_block
    result, background, scope, trace = yield from step_block_body(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        state, scope, block, loc
        ^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 449, in step_block_body
    result, background, scope, trace = yield from step_blocks_of(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 833, in step_blocks_of
    raise PDLRuntimeError(
    ...<3 lines>...
    ) from exc
pdl.pdl_interpreter.PDLRuntimeError: Error during 'replicate/ibm-granite/granite-3.0-8b-instruct' model call: litellm.APIError: ReplicateException - can only join an iterable
@esnible esnible added the bug Something isn't working label Jan 13, 2025
@bjhargrave
Copy link
Member

This may be caused by BerriAI/litellm#7900.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants