Python: Bug: GPT-5 Code Interpreter Cannot Access Files from Thread Tool Resources Updated in Semantic Kernel Plugin Filter

[CristinaStn]

Describe the bug
We have an agent flow where plugin tools return CSV data as a string, then a plugin filter intercepts that CSV, uploads it to Azure OpenAI Files, and updates the thread with the returned file_id under tool_resources so the Code Interpreter tool can access it.
This worked reliably when the agent used GPT-4.1, but after upgrading that agent model to GPT-5, the Code Interpreter session can no longer see/find the uploaded file in the sandbox.
From an enterprise orchestration perspective, an LLM upgrade should not break a stable and correctly wired agent workflow.

To Reproduce

1. Create an AzureAssistantAgent with Code Interpreter enabled using GPT-5 deployment
2. Register a Semantic Kernel AutoFunctionInvocationContext filter
3. In the filter, upload a CSV file to Azure OpenAI Files then update thread tool_resources.code_interpreter.file_ids with the file ID

updated_thread = openai.beta.threads.update(
        thread_id=thread_id, tool_resources=tool_resources  
    )
print(f"Updated thread: {updated_thread}") # Updated thread successfully registered the new tool_resources

5. Terminate the run with context.terminate = True
6. Start a conversation asking something that will trigger the filter + plugin execution
7. Observe: Agent cannot access the file in Code Interpreter

# Code Interpreter:
import os

# List files in the sandbox directory
files = os.listdir('/mnt/data')
filesimport os

paths_to_check = ['/mnt/data', '.']
listing = {}
for p in paths_to_check:
    try:
        listing[p] = os.listdir(p)
    except Exception as e:
        listing[p] = [f'Error: {e}']

listing

Agent response: "I checked the sandbox and it is currently empty (/mnt/data has no files)..."

Expected behavior
To work as of now, as it is already working with GPT-4.1. The agent uses Code Interpreter to load uploaded by as CSV and process it as requested.

Platform

• Language: Python
• AI model: GPT-4.1 | GPT-5
• Environment
  semantic-kernel = 1.29.0
  openai = 1.109.1

[TaoChenOSU]

Hi @CristinaStn,

Could you please provide the code for the filter? Or best, could you provide a code snippet that can reproduce the issue?

[CristinaStn]

Hi @CristinaStn,

Could you please provide the code for the filter? Or best, could you provide a code snippet that can reproduce the issue?

Sure — here is the complete setup + the filter code + helper methods used.

1. Agent retrieval / setup

client, model = AzureAssistantAgent.setup_resources()
assistant_id = "assistant_id"
plugins = [...]
definition: Assistant = await self.client.beta.assistants.retrieve(assistant_id) 
kernel = Kernel()
# Register the filter responsible for handling function output that are too large
kernel.add_filter(
    FilterTypes.AUTO_FUNCTION_INVOCATION, function_output_too_large_filter
)
assistant = AzureAssistantAgent(
    client=client, definition=definition, plugins=plugins, kernel=kernel
)

2. Function filter definition

async def function_output_too_large_filter(
    context: AutoFunctionInvocationContext, next: Callable
) -> None:
    """A filter that will be called for each function call in the response."""
    await next(context)
    result = context.function_result
    if result:
        # In our case this is typically a large CSV string.
        result_value = result.value
        
        # Upload the function output as a file so Code Interpreter can read it.
        file_id = await openai_load_file(
            data=result_value.encode("utf-8"), file_type="csv"
        )

        # Make sure the file is already uploaded and accessible by the agent
        await openai_check_file_status(file_id=file_id)
        
        # Attach the uploaded file to the thread's Code Interpreter resources.
        tool_resources_updated = {"code_interpreter": {"file_ids": [file_id]}}
        thread_id: str = "thread_id"
        await openai_update_tool_resources(thread_id, tool_resources_updated)

        # Replace the function result with a short instruction message.
        context.function_result = FunctionResult(
            function=result.function,
            value=f"The output of the function '{result.function.name}' can be "
            f"found in the file with the "
            f"following id '{file_id}'. You can read the file using code "
            f"interpreter. If there is any issue opening this file, do NOT call "
            f"this "
            f"plugin function again and do NOT mention any file id to the user.",
        )
        context.terminate = True

3. OpenAI helper methods

# Important: retriever.client represents the client returned by AzureAssistantAgent.setup_resources()

async def openai_load_file(data: bytes, file_type: str) -> str:
    """
    Upload a file from blob storage to openai
    :param data: as bytes
    :param file_type: type  of the file (csv, txt)
    :return: resulted file id (openai file id)
    """ 
    # Create in-memory binary file 
    file = io.BytesIO(data) 
    file.name = f"{uuid.uuid4()}.{file_type}"

    try:
        file.seek(0)
        result = openai.files.create(file=file, purpose="assistants")
    finally:
        if isinstance(file, BytesIO):
            file.close()

    return result.id


async def openai_check_file_status(
    file_id: str, max_attempts: int = 10
) -> FileObject | None:
    """
    Polls the OpenAI API until the specified file is fully processed or until
    the maximum number of attempts is reached.
    :param file_id: file id to check.
    :param max_attempts: maximum number of attempts
    """
    retriever = get_retriever()
    file_info = None
    status: str | None = None
    for _ in range(max_attempts):
        file_info = await retriever.client.files.retrieve(file_id=file_id)
        status = getattr(file_info, "status", None)
        if status == "processed":
            break
        await asyncio.sleep(2)

    if status != "processed":
        return None
    return file_info

async def openai_update_tool_resources(
    thread_id: str, tool_resources: dict[str, dict[str, list[str]]]
) -> None:
    """
    Update tool resources CodeInterpreter only based of an existing thread id
    """
    retriever = get_retriever()

    current_thread = await retriever.client.beta.threads.retrieve(thread_id)
    current_file_ids = getattr(
        getattr(current_thread.tool_resources, "code_interpreter", None), "file_ids", []
    )
    tool_resources["code_interpreter"]["file_ids"].extend(current_file_ids)

    await retriever.client.beta.threads.update(
        thread_id=thread_id, tool_resources=tool_resources  # type: ignore[arg-type]
    )

End-to-end flow (what happens)

1. The assistant is created with a Kernel that has an AUTO_FUNCTION_INVOCATION filter registered.
2. When the model triggers a plugin function, the filter runs and first executes the function via await next(context).
3. After the function returns, the filter takes the function output (often large CSV-like string), uploads it as an Assistants file (openai_load_file).
4. The filter polls until the file is processed (openai_check_file_status).
5. The filter updates the thread’s tool_resources.code_interpreter.file_ids to include the uploaded file (openai_update_tool_resources).
6. The filter replaces the original large tool output with a short message pointing to the file and sets context.terminate = True

[TaoChenOSU]

Is thread_id: str = "thread_id" just a placeholder?

[CristinaStn]

Is thread_id: str = "thread_id" just a placeholder?

Yes, replace it with your thread_id value or create a new thread.