Continue generating is not supported?

[zdwork]

Contact Details

No response

What happened?

Continue generating is not supported?

What browsers are you seeing the problem on?

No response

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

There is no continue regenerating feature. You can simply ask the AI to continue. I will take this as a feature request

[zdwork]

Hello, when is this function expected to be launched?

[zdwork]

It is more comfortable to regenerate and continue production or put it at the bottom, it is recommended

[zdwork]

It should not be possible to continue to generate, right? Is the official API also provided?

[SphaeroX]

the first thing i did was to send a request athttps://platform.openai.com/playground?mode=chat

{
"messages": [
{
"role": "user",
"content": "SCHREIBE MIR EINEN ARTIKEL ÜBER GEHWEGE"
}
],
"temperature": 1,
"max_tokens": 107,
"top_p": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"model": "gpt-3.5-turbo",
"stream": true
}

then came the token limit that I had set (107).

Then I sent another empty request and the playground sent the following to the chat (wich continues the respond):

{
"messages": [
{
"role": "user",
"content": "SCHREIBE MIR EINEN ARTIKEL ÜBER GEHWEGE"
},
{
"role": "assistant",
"content": "Gehwege sind ein wichtiger Bestandteil unserer Straßen und spielen eine entscheidende Rolle für die Sicherheit von Fußgängern. Sie sind oft die einzige Möglichkeit für Fußgänger, die Straße sicher zu überqueren oder an einem Ort entlang zu gehen. Aber wie wichtig sind Gehwege in unserer Gesellschaft und wie können sie verbessert werden?\n\nWarum sind Gehwege wichtig?\n\nGehwege ermöglichen es Fußgängern, sich frei und sicher zu bewegen"
}
],
"temperature": 1,
"max_tokens": 107,
"top_p": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"model": "gpt-3.5-turbo",
"stream": true
}

then again and the following was sent:

{
"messages": [
{
"role": "user",
"content": "SCHREIBE MIR EINEN ARTIKEL ÜBER GEHWEGE"
},
{
"role": "assistant",
"content": "Gehwege sind ein wichtiger Bestandteil unserer Straßen und spielen eine entscheidende Rolle für die Sicherheit von Fußgängern. Sie sind oft die einzige Möglichkeit für Fußgänger, die Straße sicher zu überqueren oder an einem Ort entlang zu gehen. Aber wie wichtig sind Gehwege in unserer Gesellschaft und wie können sie verbessert werden?\n\nWarum sind Gehwege wichtig?\n\nGehwege ermöglichen es Fußgängern, sich frei und sicher zu bewegen"
},
{
"role": "assistant",
"content": ". Sie sorgen dafür, dass Fußgänger von Fahrzeugen getrennt sind und somit ein höheres Maß an Sicherheit genießen können. Auf Gehwegen können Kinder zur Schule gehen, ältere Menschen spazieren und Jogger ihre Runden drehen. Sie sind auch ein wichtiger Bestandteil von städtischen Räumen, die das Leben und die Gemeinschaft fördern. Gehwege sollten deshalb so konzipiert werden, dass sie zu einer"
}
],
"temperature": 1,
"max_tokens": 107,
"top_p": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"model": "gpt-3.5-turbo",
"stream": true
}

this would be a possibility for a continue function

[zdwork]

This seems inaccurate. For example, if you give him a large code block and ask him to add comments to you and return it. When sending something to truncate, the subsequent code that continues to be generated at this time may not be submitted by you.

[danny-avila]

I saw an implementation of this somewhere and noticed 'finish_reason': 'max_tokens' is what triggers the continue generating button on the official version. Using this knowledge and @SphaeroX's example, it may be possible to send the current convo with assistant as the latest message and it will continue. Then it will be a matter of editing the frontend to handle a continuation.

This feature will officially be planned but I can't guarantee when. It's not hard but will be involved.

[zdwork]

Come on, looking forward to it, this function is very useful

[zdwork]

expect

[danny-avila]

@zdwork I'm finally getting around to this and will have it released today

similar to chatgpt, it will trigger when the api sends an irregular finish_reason

You can also manually trigger by using "stop" buttons during generation

Here is a stopped message:

continued:

[danny-avila]

Related #658 #639

[danny-avila]

This is finally complete and wrote about it in detail in my first medium article: https://medium.com/@daniel-avila/reverse-engineering-chatgpt-the-continue-generating-function-5cb4194b0ea6

[danny-avila]

@SphaeroX I gave you a shoutout in my article!

[peakjames]

Just to flag that I'm unsure this is working as intended in all cases.

I used the continue generating function today and it ate a lot of tokens by continually repeating the already existing information rather than simply continuing from where it left off and judging by the steep use of tokens seemed to repeat the original request each time as well.

I am wondering if it may be a quirk with it running into API minute/day limits that I'm currently throttled with (openai) and that it may operate under normal conditions.

[danny-avila]

can you give an example of what you mean?

Sometimes when you continue it repeats. it needs a bit of coercing, it helps when you are in the middle of a sentence. if you are not, it's more likely to repeat itself.

You can always cancel generation too.

[zdwork]

Continue generating, currently not supported yet.

[danny-avila]

It is supported:

[zdwork]

LibreChat sometimes has the "Continue generating" button, while other times only outputs half of the content without showing the "Continue generating" button. It's strange.

[danny-avila]

You have to stop the generation for “continue” to show