Streaming generated content¶
Streaming generated content is similar to Generating Content so check that out if you haven’t already.
OpenAIPrompt¶
We will be using the same OpenAICall
in Generating Content. Feel free to swap it out with a different provider.
Streaming¶
You can use the stream
method to stream a response. All this is doing is setting stream=True
and providing the OpenAICallResponseChunk
convenience wrappers around the response chunks.
import os
from mirascope import OpenAICall, OpenAICallParams
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
class RecipeRecommender(OpenAIPrompt):
prompt_template = "Recommend recipes that use {ingredient} as an ingredient"
ingredient: str
call_params = OpenAICallParams(model="gpt-3.5-turbo-0125")
stream = RecipeRecommender(ingredient="apples").stream()
for chunk in stream:
print(chunk.content, end="")
Async¶
If you want concurrency, you can use the stream_async
function instead.
import os
from mirascope import OpenAIPrompt, OpenAICallParams
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
class RecipeRecommender(OpenAIPrompt):
prompt_template = "Recommend recipes that use {ingredient} as an ingredient"
ingredient: str
call_params = OpenAICallParams(model="gpt-3.5-turbo-0125")
async def stream_recipe_recommendation():
"""Asynchronously streams the response for a call to the model using `OpenAICall`."""
stream = RecipeRecommender(ingredient="apples").async_stream()
async for chunk in astream:
print(chunk.content, end="")
asyncio.run(stream_recipe_recommendation())
OpenAICallResponseChunk¶
The stream
method returns an OpenAICallResponseChunk
instance, which is a convenience wrapper around the ChatCompletionChunk
class in openai