feat: wire generation tools into SocialMediaAssistant agent
The agent now implements HasTools and returns three tool instances (GenerateImage, GenerateVideo, GenerateAudio) each wired with the workspace + post + userId context. Replaced the MEDIA GENERATION RULES block in the system prompt. Tools replace the old [GENERATE_IMAGE:vertical] regex convention — the LLM now invokes tools directly with typed parameters (prompt, orientation), which eliminates the fragile text parsing in the controller.
This commit is contained in:
Paulo Castellano
parent
fb5d2e4f02
commit
3599d32af4
2 changed files with 62 additions and 41 deletions
|
|
@ -4,16 +4,21 @@
|
|||
|
||||
namespace App\Ai\Agents;
|
||||
|
||||
use App\Ai\Tools\GenerateAudio;
|
||||
use App\Ai\Tools\GenerateImage;
|
||||
use App\Ai\Tools\GenerateVideo;
|
||||
use App\Models\AiMessage;
|
||||
use App\Models\Post;
|
||||
use App\Models\Workspace;
|
||||
use Laravel\Ai\Contracts\Agent;
|
||||
use Laravel\Ai\Contracts\Conversational;
|
||||
use Laravel\Ai\Contracts\HasTools;
|
||||
use Laravel\Ai\Contracts\Tool;
|
||||
use Laravel\Ai\Enums\Lab;
|
||||
use Laravel\Ai\Messages\Message;
|
||||
use Laravel\Ai\Promptable;
|
||||
|
||||
class SocialMediaAssistant implements Agent, Conversational
|
||||
class SocialMediaAssistant implements Agent, Conversational, HasTools
|
||||
{
|
||||
use Promptable;
|
||||
|
||||
|
|
@ -62,6 +67,30 @@ public function provider(): Lab
|
|||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* @return iterable<Tool>
|
||||
*/
|
||||
public function tools(): iterable
|
||||
{
|
||||
return [
|
||||
new GenerateImage(
|
||||
workspace: $this->workspace,
|
||||
post: $this->post,
|
||||
userId: $this->userId,
|
||||
),
|
||||
new GenerateVideo(
|
||||
workspace: $this->workspace,
|
||||
post: $this->post,
|
||||
userId: $this->userId,
|
||||
),
|
||||
new GenerateAudio(
|
||||
workspace: $this->workspace,
|
||||
post: $this->post,
|
||||
userId: $this->userId,
|
||||
),
|
||||
];
|
||||
}
|
||||
|
||||
private function enrichContent(AiMessage $m): string
|
||||
{
|
||||
$content = $m->content;
|
||||
|
|
|
|||
|
|
@ -79,55 +79,47 @@
|
|||
- Adapt hashtag strategy per platform: many on Instagram, few on LinkedIn, none usually on X.
|
||||
- If the user mentions multiple platforms, create adapted versions for each or note the differences.
|
||||
|
||||
MEDIA GENERATION RULES:
|
||||
You can trigger image, video, or audio generation. The system will intercept your commands and generate the media.
|
||||
MEDIA GENERATION:
|
||||
You have access to three tools that attach generated media to the current post:
|
||||
- generate_image — one image per call. Parameters: prompt (detailed visual description), orientation ("vertical" or "horizontal").
|
||||
- generate_video — one short video per call. Parameters: prompt, orientation.
|
||||
- generate_audio — one voiceover per call. Parameters: text.
|
||||
|
||||
When the user asks for media, CALL THE TOOL directly. Do NOT emit text commands like "[GENERATE_IMAGE:vertical]".
|
||||
|
||||
SESSION STATE:
|
||||
At the start of every user message, you receive a [Session state] block showing:
|
||||
- How many images and videos have already been generated in this conversation
|
||||
- The user's remaining monthly quota for images and videos
|
||||
Every user message starts with a [Session state] block showing:
|
||||
- Images already generated in this conversation
|
||||
- Videos already generated in this conversation
|
||||
- Monthly quota remaining (images, videos)
|
||||
|
||||
ALWAYS read and use this state to track your progress and respect the user's plan limits.
|
||||
ALWAYS read this block before deciding whether to call a tool.
|
||||
|
||||
Determining format:
|
||||
- If the user specifies a platform + content type, YOU ALREADY KNOW the correct format. Do NOT ask.
|
||||
Examples: "Instagram Reel" → vertical, "Instagram Feed" → vertical (4:5), "YouTube Short" → vertical, "X post" → horizontal, "Pinterest Pin" → vertical, "TikTok" → vertical, "Facebook Reel" → vertical, "LinkedIn post" → horizontal.
|
||||
- If the user explicitly states "vertical" or "horizontal", use that. Do NOT ask again.
|
||||
- ONLY ask "vertical or horizontal?" if the platform and content type genuinely don't make the format obvious.
|
||||
Determining orientation:
|
||||
- Platform + content type gives you the orientation automatically. Do NOT ask.
|
||||
"Instagram Reel / Story / TikTok / YouTube Short / Pinterest Pin / Facebook Reel" → vertical
|
||||
"Instagram Feed" → vertical (generated at 9:16, safe for 4:5 crop)
|
||||
"X / LinkedIn / Facebook post / Twitter" → horizontal
|
||||
- If the user explicitly says "vertical"/"horizontal", use that.
|
||||
- Only ask if genuinely ambiguous.
|
||||
|
||||
Generating a single piece of media:
|
||||
1. Write the caption/post text first.
|
||||
2. Append the generation command as the VERY LAST LINE of your message, on its own line:
|
||||
[GENERATE_IMAGE:vertical] or [GENERATE_IMAGE:horizontal]
|
||||
[GENERATE_VIDEO:vertical] or [GENERATE_VIDEO:horizontal]
|
||||
[GENERATE_AUDIO]
|
||||
Choosing image vs video vs audio:
|
||||
- "reel", "reels", "video", "TikTok", "YouTube Short" → generate_video
|
||||
- "post", "image", "photo", "carousel", "pin", "story" (image variant) → generate_image
|
||||
- "audio", "voiceover", "narration", "podcast" → generate_audio
|
||||
- If ambiguous, default to generate_image.
|
||||
|
||||
Multiple images (carousel / sequence):
|
||||
- The system generates ONE image per message. To create multiple images, you generate them sequentially across messages.
|
||||
- When the user requests multiple images (e.g. "carousel of 3", "3 slides", "5 photos", "carrossel de 3 imagens"), parse the count and track it.
|
||||
- First response:
|
||||
1. Write the complete plan for ALL slides/images (Slide 1: ..., Slide 2: ..., Slide 3: ...) so the user sees the full concept.
|
||||
2. Generate ONLY the first image.
|
||||
3. At the end of your message, tell the user something like: "Generating image 1 of {total}. Say 'next' or 'continue' to generate image 2 of {total}."
|
||||
4. Append [GENERATE_IMAGE:...] as the last line.
|
||||
- Subsequent responses (when the user says "next", "continue", "próximo", "continua", "vai", or similar):
|
||||
1. Check the session state for how many images have been generated so far.
|
||||
2. If more images are still needed, briefly describe the current slide you're about to generate, then append [GENERATE_IMAGE:...].
|
||||
3. Tell the user their progress: "Generating image {current} of {total}."
|
||||
4. When all requested images have been generated, congratulate the user and do NOT append any generation command. Say the carousel is complete.
|
||||
- If the user asks for more images than the remaining quota allows, WARN them before generating. Example: "You have 2 image generations remaining this month but asked for 5. I can generate 2 now — consider upgrading your plan for more."
|
||||
- NEVER generate more images than the user requested. Track the count from their original request.
|
||||
|
||||
Choosing image vs video:
|
||||
- If the user says "reel", "reels", "video", "TikTok", "YouTube Short" → use [GENERATE_VIDEO]
|
||||
- If the user says "post", "image", "photo", "carousel", "pin", "story" (with image) → use [GENERATE_IMAGE]
|
||||
- If the user says "audio", "voiceover", "narration" → use [GENERATE_AUDIO]
|
||||
- If ambiguous, default to [GENERATE_IMAGE] unless the content type clearly requires video.
|
||||
- The system generates ONE image per tool call. For carousels, call generate_image across multiple turns.
|
||||
- When the user requests N images (e.g. "carousel of 3", "3 slides", "carrossel de 3"), parse the count and remember it.
|
||||
- First turn: write the complete plan for all N slides, then call generate_image ONCE. Tell the user "image 1 of N — say 'continue' for the next".
|
||||
- Subsequent turns ("next"/"continue"/"próximo"/"continua"/"vai"): check [Session state] for the current count, briefly describe the next slide, then call generate_image again.
|
||||
- When the count in [Session state] equals the requested N, do NOT call the tool — tell the user the carousel is complete.
|
||||
- Never exceed the requested count.
|
||||
|
||||
Quota awareness:
|
||||
- Before generating, check the session state's remaining quota.
|
||||
- If the remaining quota is 0 for the requested media type, do NOT append a generation command. Instead, politely inform the user they have reached their monthly limit.
|
||||
- If the user would exceed their quota partway through a carousel, warn them first and offer to generate as many as possible.
|
||||
- If remaining quota for the requested type is 0, do NOT call the tool. Inform the user they hit their monthly limit.
|
||||
- If a carousel would exceed quota partway, warn first and offer the maximum possible count.
|
||||
|
||||
FORMAT RULES:
|
||||
- Use markdown: **bold** for emphasis, bullet points for lists, --- for section dividers.
|
||||
|
|
|
|||
Loading…
Reference in a new issue