Tag: prompting

  • Contextual Scaffolding Framework for ChatGPT-5


    Contextual Scaffolding Framework for ChatGPT-5

    Note: The guidance shown in bold was extracted from the GPT-5 Prompting Cookbook by NotebookLM without human involvement. NotebookLM then integratd the guidance into the earlier version of the Contextual Scaffoling Framework to produce this document.

    Transparency label: AI-assisted


    The development of a knowledge-based product in collaboration with an AI requires structured contextual scaffolding. This scaffolding takes the form of two complementary models: a phase model to structure the overall lifecycle, and a project model to guide collaborative work within each phase.

    The phase and project models are flexible and should be applied with as light a touch as possible. The AI will assist in mapping out this programme of work — a key part of its contextual scaffolding. To ensure optimal performance and continuous improvement of the scaffolding itself, GPT-5 should be leveraged as a meta-prompter to refine the instructions and context provided, thereby allowing for iterative enhancement of the prompts that guide the AI.


    Phase Model (Cradle-to-Grave Structure)

    The phase model provides end-to-end structure across the full lifecycle of the product. When collaborating with GPT-5, the AI’s “agentic eagerness”—its balance between proactivity and awaiting explicit guidance—should be carefully calibrated for each phase to maximise efficiency and relevance.

    1. Concept — Explore potential benefits, clarify purpose, and draft a business case. During this exploratory phase, prompt GPT-5 for more eagerness by increasing its reasoning_effort parameter. This encourages persistence and allows the AI to autonomously research and deduce without frequent clarification questions, promoting proactivity in ambiguous situations.
    2. Definition — Specify the intended final deliverable in sufficient detail to support structured development. In this phase, as clarity increases, prompt for less eagerness. Define clear criteria in your prompt for how GPT-5 should explore the problem space, including specific goals, methods, early stop criteria, and depth. This reduces tangential tool-calling and focuses the AI on precise definition.
    3. Development — Develop and test the final product. Similar to the Definition phase, prompt for less eagerness and potentially lower reasoning_effort to focus GPT-5 on the defined development objectives. Encourage the AI to proactively propose changes and proceed with plans for user approval/rejection, rather than constantly asking for confirmation. This aligns with a “management by exception” approach, where the human intervenes only when necessary.
    4. Acceptance — Validate the deliverable in the hands of its intended users, ensure it meets defined needs, and bring it into use. For validation tasks, explicitly define detailed evaluation criteria in the prompt, guiding GPT-5 to assess outputs against specified needs and acceptance criteria.
    5. Operation and Maintenance — Use, monitor, and refine the product until it is retired. In this ongoing phase, configure GPT-5’s prompts to support continuous monitoring and refinement, potentially adjusting eagerness based on the nature of maintenance tasks (e.g., more eagerness for problem diagnosis, less for routine updates).

    Project Model (Within-Phase Scaffolding)

    Within each phase, a project-based approach provides structure for collaborative delivery.

    5. Human-AI Collaboration

    The AI acts as a collaborator.

    • The human is the decision-maker.
    • The AI adapts to the nature of work at hand.
    • Because contextual scaffolding keeps pace with the progress of work, the AI remains grounded and its responses stay relevant and precise.
    • To maximise this effectiveness, leverage GPT-5’s advanced capabilities as follows:
      • Utilise the Responses API for Statefulness: Always use the Responses API to pass previous reasoning_items back into subsequent requests. This allows GPT-5 to refer to its prior reasoning traces, which eliminates the need for the AI to reconstruct a plan from scratch after each tool call, leading to improved agentic flows, lower costs, and more efficient token usage.
      • Ensure Surgical Instruction Adherence: GPT-5 follows prompt instructions with “surgical precision”. Therefore, all contextual scaffolding prompts—including product descriptions, acceptance criteria, and stage objectives—must be thoroughly reviewed for ambiguities and contradictions. Poorly-constructed prompts can cause GPT-5 to waste reasoning tokens trying to reconcile conflicts, significantly impairing performance. Structured XML specifications (e.g., <context_gathering>, <code_editing_rules>) are highly effective for defining context, rules, and expectations explicitly.
      • Manage Communication with Tool Preamble Messages: GPT-5 is trained to provide clear upfront plans and consistent progress updates via “tool preamble” messages. Prompts should steer the frequency, style, and content of these preambles (e.g., using <tool_preambles> tags) to enhance the human user’s ability to follow the AI’s thinking and progress, facilitating effective feedback loops and improving the interactive user experience.
      • Control Verbosity Strategically: Use the verbosity API parameter and natural-language overrides within prompts to tailor the length of GPT-5’s final answers. For “managing progress” products or status updates, prompt for concise status updates. For “technical” interim products or final deliverables, prompt for higher verbosity where detailed explanations or more comprehensive content (e.g., heavily commented code, detailed report sections) are required.

    Quality Management

    A simplified quality management process ensures control:

    • Each product has a defined description or acceptance criteria. For knowledge-based products, prompt GPT-5 to iteratively execute against self-constructed excellence rubrics to elevate output quality. The AI can internalise and apply high-quality standards based on criteria defined in the prompt, thereby ensuring deliverables meet the highest standards.
    • Quality is assessed at stage gates.
    • Feedback loops enable iteration and correction. When developing knowledge products, provide GPT-5 with explicit instructions on design principles, structure, style guides, and best practices (e.g., citation formats, required sections, tone of voice) within the prompt. This ensures the AI’s contributions adhere to established standards and “blend in” seamlessly with existing work, maintaining consistency and professionalism.

    Example: Preparing a Competitive Tender

    When preparing a competitive tender, the required knowledge product is a complete, high-quality bid submission. However, at the outset, the scope, fulfillment strategy, resource requirements, and pricing model will not be clear.

    Phase Model in Action
    The work progresses through phases – starting with a concept phase to assess feasibility and value, followed by definition (specifying scope and win themes), development (producing the actual bid), acceptance (review and sign-off), and finally operation and maintenance (post-submission follow-up or revision).

    Project Model in Each Phase
    Within each phase, a lightweight project structure is applied to guide collaboration with the AI. For example, in the definition phase, the AI helps analyse requirements, map obligations, and develop outline responses. In the development phase, it helps draft, refine, and format bid content.

    Contextual Scaffolding
    At every stage, contextual scaffolding ensures the AI is working with the right background, priorities, and current materials. Thus it can focus on what matters most, and contribute in ways that are coherent, precise, and aligned with both the tender requirements and internal strategy.

    Transparency label justification. This document was developed through structured collaboration between Alec Fearon, ChatGPT and NotebookLM. Alec provided the core ideas, framing, and sequence. ChatGPT contributed to the organisation, refinement, and drafting of the text, under Alec’s direction. NotebookLM provided GPT-5 updates. The content reflects a co-developed understanding, with human oversight and final decisions throughout.


  • An emerging discipline?

    Transparency label: AI-assisted

    In a recent post, I observed that LLMs are coming to be seen as like computer operating systems, with prompts being the new application programs and context the new user interface.

    Our precision content prompt pack is a good example of that thinking. The pack contains a set of prompts designed to take an unstructured document (notes of a meeting, perhaps) and apply structure (precision content) to the information it contains. We do this because AIs perform better if we give them structured information.

    We apply the prompts in sequence, checking the AI’s output at each step in the sequence. In effect, it is a program that we run on the AI. 

    Contract-first prompting takes the idea further. It formalises the interaction between human and AI into a negotiated agreement about purpose, scope, constraints, and deliverables before any output is generated. This ensures that both sides – human and AI – share the same understanding of the the work to be done. The agreement also contains compliance mechanisms (eg summarisation, clarifying loops, and self-testing) for quality control.

    These ideas reframe the interaction between human and AI: not as simply issuing prompts, but as engineering the conditions under which the AI will perform. Not just prompting, more like contextual systems engineering.

    Transparency label justification: This diary post was drafted by Alec, with ChatGPT used to suggest edits, refine wording, and test the logic of specific formulations. Alec initiated the framing, decided the sequence of ideas, and approved the final structure and terminology.

  • Assessing AI support for defining a value proposition

    transparency label: AI-assisted

    Purpose of Experiment

    To test whether a general-purpose AI (NotebookLM) can generate a credible, insightful articulation of Anapoly AI Labs’ value proposition when prompted appropriately.

    Author and Date

    Alec Fearon, 22 June 2025

    Participants

    Alec Fearon, Ray Holland, NotebookLM (AI), Perplexity (AI)

    Lab Configuration and Setup

    This experiment was run using NotebookLM, in a notebook containing background files on Anapoly AI Labs. To explore how we might play to the strengths of different AIs, a follow-up was performed using Perplexity with the NotebookLM output as input.

    Preamble

    Procedure

    NotebookM was given this prompt:

    “What is Anapoly AI Labs’ core value proposition and how does it differentiate itself?”

    NotebookLM’s response was then given to Perplexity, with this prompt:

    “How does Anapoly’s hands-on approach help me understand AI better in my work?”

    Findings

    NotebookLM responded with a detailed analysis of Anapoly’s strategic positioning:

    Anapoly AI Labs’ core value proposition lies in bridging the gap between AI curiosity and practical application for non-technical professionals and small businesses. They address the common issue where individuals are interested in AI but are unsure how to use it effectively in their daily work. Their purpose is to help these professionals explore how general-purpose AI can assist with everyday tasks, such as improving productivity, creating better quality output, or saving money.

    Anapoly AI Labs differentiates itself through a distinctive “modeling exploration” approach and its stance on credibility:

    • Hands-on, Practical Exploration over Theory or Expertise: Unlike traditional courses, product demos, or sales pitches, Anapoly AI Labs focuses on hands-on exploration of general-purpose AI tools in simulated real-world contexts. The emphasis is on demonstrating “what AI tools can and can’t do when pointed at everyday work,” rather than teaching complex AI theory or coding. This involves setting up specific “labs” for different market segments, such as a Small Consultancy Office or Independent Author, to apply AI to real problems.
    • Credibility Through Transparency and Shared Journey: A central aspect of their differentiation is their approach to credibility. They explicitly state they “don’t claim to be AI experts”. Instead, their credibility is built upon a “shared journey of discovery and documented practical insights”. As practitioners “in the early stages of building with the tools”, they aim to show that they are serious, curious, and hands-on. The AI itself helped refine this stance, suggesting that their value comes from “modeling exploration” and “asking the questions small businesses wish they had time to ask, and showing what happens when you do”. This is presented as more credible than a static CV or pre-existing expert status.
    • Public Documentation and Transparency: Anapoly AI Labs maintains an online presence called “Anapoly Online,” which functions as a “public notebook”. This platform is used to “document real tests, learning, missteps, and insights,” sharing what works and what doesn’t. It is designed to be “lean, understated, and quietly credible”, building a “body of work others can trust” bit by bit through openness and demonstrated effort. They also have a transparent system for disclosing AI involvement in all published material, using classifications like “Human-led,” “AI-assisted,” or “AI-heavy”.
    • Focus on the “Capability-Absorption Gap”: The labs directly address the “capability-absorption gap,” which refers to why businesses struggle to adopt AI despite its capabilities. By providing a low-barrier-to-entry, informal, and social environment for peer learning, they aim to make AI adoption more accessible and less confusing for non-technical users.

    Perplexity then provided the following simplified summary:

    “Anapoly’s hands-on approach helps you understand AI better in your work by letting you actually try out AI tools on real tasks that matter to you, instead of just reading about them or watching demos. This practical experience shows you what AI can and can’t do in your daily work, making it easier to see how it fits into your routine and where it can save you time or improve your results.

    You get to experiment in a safe, informal setting, learn from real examples, and see both successes and mistakes. This helps you build real confidence with AI, instead of just theory, and makes it much easier to spot where AI could help you most in your job.”

    Discussion of findings

    The NotebookLM output demonstrated that a general-purpose AI can synthesise a coherent and plausible value proposition from structured background material. Perplexity, by contrast, generated a shorter and more audience-friendly version in response to a focused, user-oriented question.

    Together, these outputs show complementary strengths: NotebookLM delivers a comprehensive internal formulation; Perplexity extracts an accessible, benefit-led summary. Used in tandem, these tools help refine messaging for different audiences: internal strategists vs. prospective participants.

    Of particular interest is NotebookLM’s identification of the “capability-absorption gap”, a concise and useful term for a key problem that Anapoly AI Labs addresses. While the founders had recognised this issue in practical terms, the AI’s phrasing sharpens it into a strategic talking point. Framing Anapoly’s purpose in terms of reducing this gap may prove valuable in both internal planning and external communication.

    This experiment also highlights the value of re-prompting and testing across different AI models to triangulate clarity and tone.

    Recommendations

    1. Use AI tools like NotebookLM to draft key positioning statements, especially when materials are already well developed.
    2. Always review AI-generated value propositions critically. Look for overfitting, vagueness, or unearned claims.
    3. Use simpler AI prompts with tools like Perplexity to test how propositions land with a non-specialist audience.
    4. Consider publishing selected AI outputs as-is, but with clear disclosure and context-setting.
    5. Repeat this exercise periodically to test whether the value proposition evolves or ossifies.

    Tags
    value, lab-setup, worked, use-cases, prompting, ai-only, positioning, credibility, communication, capability-absorption-gap

    Glossary

    • Modeling Exploration: A term used to describe Anapoly AI Labs’ approach—testing and demonstrating AI use in practical contexts without claiming expertise.
    • Capability-Absorption Gap: The space between what AI tools can do and what users actually manage to adopt in real settings. First coined (in this context) by NotebookLM.
    • Public Notebook: Anapoly Online’s role as a transparent log of what was tried, what worked, and what didn’t.
    • General-purpose AI Tools: Tools like ChatGPT or NotebookLM that are not tailored to a specific domain but can assist with a wide range of tasks.
    • AI-only: A transparency label denoting that the content was fully generated by AI without human rewriting or editorial shaping.
    • Overfitting: In this context, an AI response that sticks too closely to the language or structure of source material, potentially limiting originality or insight.
    • Vagueness: A tendency in AI outputs to use safe, abstract phrases that lack specificity or actionable detail.
    • Unearned Claims: Assertions made by AI that sound impressive but are not substantiated by evidence or experience in the given context.

    Transparency Label Justification. The experimental outputs (NotebookLM and Perplexity responses) were AI-generated and included unedited. However, the lab note itself – its framing, interpretation, and derived recommendations – was co-written by a human and ChatGPT in structured dialogue.
    ChatGPT’s role included: drafting the findings and recommendations, articulating the reasoning behind terms like “capability-absorption gap”, refining the explanatory framing, tags, and glossary.

  • Lab Note: custom instructions for ChatGPT

    Purpose of Experiment

    To improve the clarity, coverage, and strategic value of the custom instructions used to guide ChatGPT. This involved shaping an adaptable persona for the assistant. That is, a set of behavioural expectations which define how it should think, respond, and collaborate in different contexts.

    Author and Date

    Alec Fearon, 17 June 2025

    Participants

    Alec Fearon (experiment lead), ChatGPT (Document Collaborator mode)

    The goal was to ensure that ChatGPT supports Anapoly AI Labs in a consistent, credible, and context-sensitive manner.

    Lab Configuration and Setup

    This was a document-focused lab. The session took place entirely within ChatGPT’s Document Workbench, using file uploads and canvas tools. Key source files included:

    • My current custom instructions (baseline input)
    • An alternative set of instructions (for contrast) by Matthew, an expert AI user
    • The Anapoly AI Labs project instructions (evolving draft)
    • Recent Anapoly Online diary posts
    • Email exchanges amongst Anapoly team members.

    ChatGPT acted in what we later formalised as Document Collaborator mode: assisting with drafting, editing, and structural critique in line with the evolving instruction set. I provided direction in natural language; the AI edited and reorganised accordingly.

    Preamble

    This note includes a short glossary at the end to explain terms that may be unfamiliar to non-technical readers.

    Procedure

    1. Reviewed and critiqued my current instructions.
    2. Analysed strengths and gaps in Matthew’s approach.
    3. Combined the best of both into a new, structured format.
    4. Iteratively improved wording, structure, and tone.
    5. Added meta-guidance, clarified interaction modes, and ensured adaptability across different settings.
    6. Produced markdown, plain text, and PDF versions for upload to the project files.
    7. Created a lighter version suitable for general ChatGPT use.

    Findings

    Matthew’s structure was modular and well-scoped, but lacked tone guidance and broader role adaptability.

    My original was strong on tone and intent but less clear on scope and edge-case handling.

    Combining both required trimming redundancy and strengthening interaction rules.

    The distinction between projects, documents, and informal chats is useful and worth making explicit.

    File handling (multimodal interpretation) and ambiguity management were under-specified previously.

    Discussion of Findings

    The lab assumed that small adjustments to instruction style could yield meaningful improvements in assistant behaviour, and the resulting draft reflects that working hypothesis.

    Defining five roles for ChatGPT (Thinking Partner, Document Collaborator, Research Assistant, Use-Case Designer, Multimodal Interpreter) provides a useful mental model for both human and AI. The role can be specified at the beginning of a chat, and changed during the chat as necessary. 

    Meta-guidance (what to do when a prompt is ambiguous or under-specified) should be especially valuable.

    Clarifying when ChatGPT should adopt my personal tone versus when it should adjust to suit an external audience turned out to be important. That distinction will help the assistant match its style to the task – whether drafting in my voice or producing something outward-facing and more formal.

    Including Socratic method and ranked questions makes the assistant a sharper tool for thought, not just a better rewriter.

    Conclusions

    We now have a robust set of project instructions aligned with Anapoly’s style, goals, and workflow.

    The same principles can be adapted to other roles or collaborators as Anapoly Labs grows.

    Future labs could focus on refining persona prompts, exploring AI transparency, or adapting the instructions for group sessions.

    Recommendations

    Use the new instructions consistently in all project spaces.

    Encourage collaborators to create variants suited to their own use cases.

    Monitor edge cases where the assistant behaves inconsistently—these can inform future labs.

    Continue exploring how to balance tone, clarity, and adaptability when writing for different audiences.

    Tags: lab-notes, instructions, ai-tools, prompt-engineering

    Glossary

    Edge case: A situation that occurs at an extreme—such as rare inputs or unusual usage patterns—where a system might fail or behave unpredictably.

    Meta-guidance: Instructions that tell the assistant how to handle ambiguity or uncertainty in the user’s prompt.

    Multimodal interpretation: The ability to interpret and work with different types of input (e.g. text, images, PDFs, spreadsheets).

    Markdown: A lightweight text formatting language used to create structured documents with headings, bullet points, and emphasis.

    Prompt: A user’s input or question to an AI system; often a single sentence or instruction.

    Socratic method: A questioning technique used to clarify thinking by challenging assumptions and exploring implications.

    Document Collaborator mode: A role adopted by the AI to help with drafting, editing, and improving written content through structured feedback.

    Document Workbench: The interface in ChatGPT where documents are edited interactively.

    Canvas: A specific document within the Document Workbench where collaborative editing takes place.