In our one-shot tests in particular, it isn't clear if we're expecting the assistant to ask the user clarifying questions, make reasonable guesses about missing information, or add in placeholder code. Whichever we pick, it will be the wrong choice in many situations.
Benefits:
- If there was more back-and-forth interaction with the user, the assistant could adapt to the level of detailed expected in each scenario. The assistant could generate forms and buttons, for filling in details or click "decide later".
- We wouldn't have to teach users best practices on using the assistant (e.g. "tell AI the entire workflow all the context in one prompt"). It would guide the user if they don't give enough details, but allow them to skip quickly if they're just drafting an outline.
- This might also make the assistant feel faster when generating more complicated workflows, as the assistant wouldn't go away for such a long time preparing the details in the background.
The implementation might be too slow if we treated each as a conversation turn, and so this should be implemented as tool use responses ideally.
In our one-shot tests in particular, it isn't clear if we're expecting the assistant to ask the user clarifying questions, make reasonable guesses about missing information, or add in placeholder code. Whichever we pick, it will be the wrong choice in many situations.
Benefits:
The implementation might be too slow if we treated each as a conversation turn, and so this should be implemented as tool use responses ideally.