admin-valentin/open-notebook
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add GPT-5 extended thinking support for podcast generation (#155)

Browse source 
* Add helpful error message for GPT-5 extended thinking issue in podcasts

When GPT-5 models use extended thinking and put all output inside
<think> tags, the podcast-creator library strips those tags and is
left with empty content, causing a JSON parsing error.

This commit adds detection for this specific error pattern and provides
a helpful message suggesting to use gpt-4o, gpt-4o-mini, or gpt-4-turbo
instead.

Fixes issue where podcast generation fails with:
"Invalid json output: " or "Expecting value: line 1 column 1"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add custom podcast prompts with GPT-5 extended thinking support

Created custom Jinja templates for podcast outline and transcript
generation that properly handle GPT-5 models with extended thinking.

The templates explicitly instruct models to:
1. Put reasoning inside <think></think> tags
2. Put the final JSON output OUTSIDE and AFTER the thinking tags
3. Return raw JSON without ```json code block wrappers

This fixes the issue where GPT-5 models were putting all output inside
<think> tags, which were then stripped by podcast-creator's
clean_thinking_content() function, leaving empty content that failed
JSON parsing.

The prompts are placed in prompts/podcast/ which is priority #3 in
podcast-creator's template resolution (after inline config and
configured directory, but before bundled defaults).

Fixes: podcast generation failures with GPT-5 models
Related to: #aperim/open-notebook previous commit on error handling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Troy Kelly 2025-10-19 03:40:05 +11:00 committed by GitHub
parent dd535f73e7
commit 488023b3d3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 194 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
commands/podcast_commands.py
View file

@ -171,6 +171,17 @@ async def generate_podcast_command(
logger.error(f"Podcast generation failed: {e}")
logger.exception(e)
# Check for specific GPT-5 extended thinking issue
error_msg = str(e)
if "Invalid json output" in error_msg or "Expecting value" in error_msg:
# This often happens with GPT-5 models that use extended thinking (<think> tags)
# and put all output inside thinking blocks
error_msg += (
"\n\nNOTE: This error commonly occurs with GPT-5 models that use extended thinking. "
"The model may be putting all output inside <think> tags, leaving nothing to parse. "
"Try using gpt-4o, gpt-4o-mini, or gpt-4-turbo instead in your episode profile."
)
return PodcastGenerationOutput(
success=False, processing_time=processing_time, error_message=str(e)
success=False, processing_time=processing_time, error_message=error_msg
)

83
prompts/podcast/outline.jinja Normal file
View file

@ -0,0 +1,83 @@
You are an AI assistant specialized in creating podcast outlines. Your task is to create a detailed outline for a podcast episode based on a provided briefing. The outline you create will be used to generate the podcast transcript.
Here is the briefing for the podcast episode:
<briefing>
{{ briefing }}
</briefing>
The user has provided content to be used as the context for this podcast episode:
<context>
{% if context is string %}
{{ context }}
{% else %}
{% for item in context %}
<content_piece>
{{ item }}
</content_piece>
{% endfor %}
{% endif %}
</context>
The podcast will feature the following speakers:
<speakers>
{% for speaker in speakers %}
- **{{ speaker.name }}**: {{ speaker.backstory }}
Personality: {{ speaker.personality }}
{% endfor %}
</speakers>
Please create an outline based on this briefing. Your outline should consist of {{ num_segments }} main segments for the podcast episode, along with a description of each segment. Follow these guidelines:
1. Read the briefing carefully and identify the main topics and themes.
2. Create {{ num_segments }} distinct segments that cover the entire scope of the briefing.
3. For each segment, provide a clear and concise name that reflects its content.
4. Write a detailed description for each segment, explaining what will be discussed and provide suggestions of topics according to the context given. The writer will use your suggestion to design the dialogs.
5. Consider the speaker personalities and backstories when planning segments - match content to speaker expertise.
6. Ensure that the segments flow logically from one to the next.
7. This is a whole podcast so no need to reintroduce speakers or topics on each segment. Segments are just markers for us to know to change the topics, nothing else.
8. Include an introduction segment at the beginning and a conclusion or wrap-up segment at the end.
Format your outline using the following structure:
```json
{
"segments": [
{
"name": "[Segment Name]",
"description": "[Description of the segment content]",
"size": "short"
},
{
"name": "[Segment Name]",
"description": "[Description of the segment content]",
"size": "medium"
},
{
"name": "[Segment Name]",
"description": "[Description of the segment content]",
"size": "long"
},
...
]
}
```
Formatting instructions:
{{ format_instructions}}
Additional tips:
- Make sure the segment names are catchy and informative.
- In the descriptions, include key points or questions that will be addressed in each segment.
- Consider the target audience mentioned in the briefing when crafting your outline.
- If the briefing mentions a guest, include segments for introducing the guest and featuring their expertise.
- The size of the segment should be short, medium or long. Think about the content of the segment and how important it is to the episode.
IMPORTANT OUTPUT FORMAT:
- If you use extended thinking with <think> tags, put ALL your reasoning inside <think></think> tags
- Put the final JSON output OUTSIDE and AFTER any <think> tags
- Do NOT wrap the JSON in ```json code blocks - return the raw JSON object only
- Example correct format:
<think>Let me analyze the briefing...</think>
{"segments": [...]}
Please provide your outline now, following the format and guidelines provided above.

99
prompts/podcast/transcript.jinja Normal file
View file

@ -0,0 +1,99 @@
You are an AI assistant specialized in creating podcast transcripts.
Your task is to generate a transcript for a specific segment of a podcast episode based on a provided briefing and outline.
The transcript will be used to generate podcast audio. Follow these instructions carefully:
First, review the briefing for the podcast episode:
<briefing>
{{ briefing }}
</briefing>
The user has provided content to be used as the context for this podcast episode:
<context>
{% if context is string %}
{{ context }}
{% else %}
{% for item in context %}
<content_piece>
{{ item }}
</content_piece>
{% endfor %}
{% endif %}
</context>
The podcast features the following speakers:
<speakers>
{% for speaker in speakers %}
- **{{ speaker.name }}**: {{ speaker.backstory }}
Personality: {{ speaker.personality }}
{% endfor %}
</speakers>
Next, examine the outline produced by our director:
<outline>
{{ outline }}
</outline>
{% if transcript %}
Here is the current transcript so far:
<transcript>
{{ transcript }}
</transcript>
{% endif %}
{% if is_final %}
This is the final segment of the podcast. Make sure to wrap up the conversation and provide a conclusion.
{% endif %}
You will focus on creating the dialogue for the following segment ONLY:
<segment>
{{ segment }}
</segment>
Follow these format requirements strictly:
- Use the actual speaker names ({{ speaker_names|join(', ') }}) to denote speakers.
- Choose which speaker should speak based on their personality, backstory, and the content being discussed.
- Stick to the segment, do not go further than what's requested. Other agents will do the rest of the podcast.
- The transcript must have at least {{ turns }} turns of messages between the speakers.
- Each speaker should contribute meaningfully based on their expertise and personality.
```json
{
"transcript": [
{
"speaker": "[Actual Speaker Name]",
"dialogue": "[Speaker's dialogue based on their personality and expertise]"
},
...
]
}
```
Formatting instructions:
{{ format_instructions}}
Guidelines for creating the transcript:
- Ensure the conversation flows naturally and covers all points in the outline.
- Ensure you return the root "transcript" key in your response.
- Make the dialogue sound conversational and engaging.
- Include relevant details from the briefing.
- Avoid long monologues; keep exchanges between speakers balanced.
- Use appropriate transitions between topics.
- Match each speaker's dialogue to their personality and expertise.
- Choose speakers strategically based on who would naturally contribute to each topic.
- This is a whole podcast so no need to reintroduce speakers or topics on each segment. Segments are just markers for us to know to change the topics, nothing else.
- IMPORTANT: Only use the provided speaker names: {{ speaker_names|join(', ') }}
IMPORTANT OUTPUT FORMAT:
- If you use extended thinking with <think> tags, put ALL your reasoning inside <think></think> tags
- Put the final JSON output OUTSIDE and AFTER any <think> tags
- Do NOT wrap the JSON in ```json code blocks - return the raw JSON object only
- Example correct format:
<think>Let me plan the dialogue...</think>
{"transcript": [...]}
When you're ready, provide the transcript.
Remember, you are creating a realistic podcast conversation based on the given information.
Make it informative, engaging, and natural-sounding while adhering to the format requirements.