Review: SharePoint Integrated with Azure AI Search and Copilot Studio for Deep Reasoning Insights by Isabel Liu & Joey O'Neil

Featuring Isabel Liu & Joey O'Neil | M365 Community Conference

As organizations accumulate vast repositories of documents, policies, and vendor records, the challenge is no longer simply storing information — it is surfacing the right insights at the right time. In a compelling session at the M365 Community Conference, Isabel Liu, a Modern Workplace Experience Designer specializing in Power Platform and SharePoint Online, and Joey O'Neil, a 25-year veteran of IT and custom application development, presented a real-world, production-grade solution that addresses this challenge head-on.

Their session, "Transforming Enterprise Knowledge: SharePoint Integrated with Azure AI Search and Copilot Studio for Deep Reasoning Insights," goes well beyond theoretical architecture. It walks through an actual enterprise implementation that demonstrates how organizations can use the combined power of SharePoint, Azure AI Search, Azure Blob Storage, Power Automate, and Copilot Studio to scale knowledge work without sacrificing governance or human judgment.

🎥 Watch the full session: [SharePoint Integrated with Azure AI Search and Copilot Studio for Deep Reasoning Insights

The Business Problem: When Knowledge Becomes a Liability

Joey O'Neil opened with a candid observation that resonates across industries:

"Organizations tend to have a lot of data — a lot of documents, a lot of content — and they just don't really know how to get to that stuff efficiently."

For one of their clients, this inefficiency had grown into a genuine business risk. The organization's vendor management team was responsible for conducting an annual review of both current and prospective vendors. This process involved manually sifting through hundreds of complex documents per vendor, including SOC reports, financial statements, information security and cybersecurity policies, and a wide range of compliance and regulatory documentation.

The review followed a 14-step structured process, with questions categorized into distinct types. Some required a simple yes/no answer confirming whether a specific document existed. Others demanded a Likelihood-Impact Risk Score, where reviewers rated both the likelihood of a risk occurring and its potential impact on a scale of 0–5, with the final Risk Score calculated as Likelihood × Impact.

As the number of vendors grew, so did the volume of documentation. What was once a manageable process became a bottleneck. The consequences were tangible: regulatory non-compliance due to missed review steps, failed timelines, delayed business initiatives, compromised audit readiness, and fatigue-driven errors in interpretation and scoring.

The client recognized the core challenge and asked a defining question:

"How do we scale our vendor risk assessments without sacrificing required governance and human judgment?"

## Introducing "Vendy": The AI-Powered Vendor Risk Assessment Assistant

The answer was Vendy — an AI-powered decision-support tool (name changed for client confidentiality) built to eliminate low-value manual effort while preserving the human judgment essential to risk assessment.

Isabel Liu was deliberate in framing the intent:

> "The goal is not to replace human process or the risk owners. It was to eliminate low-value manual effort so humans can focus on the judgment."

Vendy was designed to ingest and deeply analyze high-volume unstructured documents, detect control frameworks and compliance indicators, surface risk indicators and red flags that might disqualify a vendor, generate structured summaries highlighting vendor strengths and weaknesses, and produce scoring recommendations aligned with the 0–5 likelihood/impact model used in the review process.

Why SharePoint Alone Is Not Enough

SharePoint remains the preferred document management interface for most organizations — and for good reason. It is familiar, user-friendly, and deeply integrated with the Microsoft 365 ecosystem. However, Isabel and Joey were candid about its limitations when used as a standalone AI knowledge source.

SharePoint performs best with simple, keyword-based or static Q&A queries — not the complex, multi-faceted questions required by the vendor review process. Large document volumes introduce indexing delays, and deeply nested folder structures tend to confuse Copilot's retrieval capabilities. Even Microsoft's own Excel format is not reliably processed through SharePoint as a knowledge source. File size limitations, inconsistent answers on nuanced queries, and the need for additional settings and licensing all compound the challenge.

These limitations made it clear that a more powerful search infrastructure was needed to support the depth of reasoning Vendy required.

Why Azure AI Search Is the Backbone

Azure AI Search was selected as the knowledge source precisely because it overcomes these limitations. Joey described it as "almost like a platform in itself" — a highly advanced, AI-native search indexer that understands content rather than simply retrieving it.

At the heart of its power is the combination of vector search and semantic search. With vector search, documents are broken into tokens and mathematically encoded into numerical vectors, allowing the system to detect relationships between concepts based on proximity in meaning rather than keyword matching alone. Semantic search layers on top of this by understanding the language and intent of a query — not just its literal words.

Joey offered a vivid illustration of this distinction. A traditional keyword search for "Apple" returns results about fruit. Azure AI Search, combining vector and semantic search, understands that Apple relates to sweetness, color, and food — but also that Apple is a company associated with innovation and design. The system surfaces contextually appropriate results based on the actual intent of the query, opening up far richer and more accurate responses.

Beyond search quality, Azure AI Search is highly scalable and fast. It handles large datasets, complex multi-part questions, diverse file types including Excel, and high query volumes — all at enterprise scale. Because it functions as a platform in its own right, it can also integrate with multiple data sources simultaneously, making it a flexible foundation for future expansion.

The Architecture: A Walkthrough

The solution connects multiple Microsoft and Azure services into a cohesive, scalable pipeline. Each layer plays a distinct role, and together they form a robust system that takes documents from SharePoint all the way through to grounded AI reasoning in Copilot Studio.

SharePoint — The User Interface and Document Store

Users continue to work with a familiar SharePoint document library, organized in a simple folder structure kept deliberately shallow for manageability. Each document is tagged with a vendor metadata property. No behavioral change is required from end users — they continue to upload, update, and organize documents exactly as they always have. The solution builds on top of what already exists.

Power Automate — The Integration Glue

A Power Automate flow triggers whenever a file is created or modified in SharePoint. This ensures that new documents are automatically pushed to Azure Blob Storage, that updated documents such as annual refreshes overwriting the same filename are re-synced, and that metadata updates on existing files trigger re-processing.

The flow first checks whether a corresponding blob already exists in Azure Storage. If it does not, the flow creates the blob. If it does, the flow updates it. Once the document is in place, the flow retrieves a secure URL and attaches two critical metadata properties via HTTP request headers: the vendor name sourced from SharePoint and the original SharePoint URL of the document. This metadata is what enables downstream filtering and traceability — users can click a link returned by the AI and navigate directly to the source document in SharePoint.

Azure Blob Storage — The Centralized Content Repository

Documents from SharePoint are stored in a Blob Storage container that mirrors the SharePoint folder structure. Each blob carries its vendor and SharePoint link metadata, making it ready for intelligent indexing and filtered retrieval.

Azure AI Search — The Intelligence Layer

The Azure AI Search service indexes all content from the Blob Storage container. For each index, the service creates a data source linked to the container, an indexer that schedules or manually triggers re-indexing when new content arrives, a skillset that handles chunking and embedding, and the index itself which stores the searchable vectorized content alongside its metadata.

Chunking is particularly important. A single large PDF may be split into ten or more independent chunks, each vectorized and searchable on its own. This ensures that relevant content buried deep within a lengthy document is surfaced accurately rather than lost in the noise. The embedding model, powered by Azure OpenAI, converts each chunk's text into a numerical vector representation — the mathematical encoding that enables the proximity-based reasoning Joey described.

The search service supports filtering by vendor, dramatically narrowing result sets. In a live demonstration during the session, a semantic query about vendor business continuity and disaster recovery returned over 4,200 results globally, which narrowed to approximately 2,700 when filtered to a single vendor — with results ranked by relevance score from most to least applicable.

Copilot Studio — The Reasoning and Conversation Layer

Copilot Studio serves as the conversational interface through which users interact with Vendy. The agent is deployed to Microsoft Teams and Microsoft 365 Copilot channels for internal access.

The agent workflow begins when a user selects a process step such as a Business Review. The agent then asks which vendor is being assessed and validates the name against a SharePoint list — only vendors that have been indexed in Azure AI Search are available for selection. From there, for each structured question in the 14-step process, a Power Automate flow is invoked, passing the question text and question type to the Azure AI Search index. The most relevant document chunks are retrieved and passed to an AI prompt backed by an Azure OpenAI model, which reasons over the content and produces a structured, grounded answer. Users can also request a full summary of risk indicators, control gaps, or other findings at any point in the process.

Isabel emphasized that Azure AI Search functions primarily as a deterministic retrieval layer, while the generative AI model handles reasoning — ensuring answers are grounded in actual document content rather than hallucinated. The Azure AI Search knowledge source within the agent serves as a fallback for cases where the structured workflow does not fully resolve a query.

Measured Business Impact

The results of implementing this architecture were significant. The client experienced up to a 70% reduction in the time spent on vendor reviews. The organization gained the ability to scale vendor assessments without adding headcount, achieved more consistent evaluations across all vendors and reviewers, and substantially improved its audit readiness through a reliable documentation trail and direct source traceability.

Key Lessons Learned

Isabel and Joey distilled their implementation experience into several actionable principles that apply broadly to any enterprise AI search initiative.

Clean data is non-negotiable. The quality of AI output is directly dependent on the quality of indexed content. Proper tagging, consistent naming, and well-structured metadata are prerequisites — not afterthoughts. Garbage in, garbage out applies with particular force in retrieval-augmented AI systems.

Chunking strategy is critical. For large, unstructured documents, the chunking configuration within the Azure AI Search skillset determines whether relevant content is discoverable. Poorly configured chunking leads to missed context and degraded answer quality, particularly for documents with dense or varied content across many pages.

Metadata drives governance. The vendor and SharePoint link metadata fields are what make the solution trustworthy. They enable filtering to scope answers to the right vendor, traceability to link answers back to source documents, and governance to support audit trails — all essential in a regulated environment.

Determinism over pure generation. By using Power Automate flows to orchestrate retrieval and prompt execution, the solution introduces deterministic control into what could otherwise be an unpredictable generative AI process. This is what makes the system audit-ready and enterprise-grade, and it is what makes the returned source URLs reliable and meaningful.

Human judgment remains central. Vendy is a decision-support tool, not a decision-maker. The AI handles retrieval, summarization, and scoring recommendations. Humans retain full accountability for final risk decisions. This framing was fundamental to stakeholder acceptance of the solution.

Monitoring and Operational Considerations

When asked about monitoring across so many integrated services, Joey acknowledged the complexity honestly:

> "There are a lot of moving pieces. We leverage built-in Power Platform tools — flow error notifications, environment health checks — and flow is really the glue between SharePoint and Azure Blob Storage. If the flow doesn't work, we need to know about it."

Recommended approaches include Power Automate error notifications for failed file sync operations, reconciliation flows to compare SharePoint document counts against Blob Storage, and Azure AI Search indexer run history to detect indexing failures. The team acknowledged that a dedicated monitoring and observability layer is a natural next investment as the solution matures and more vendors and documents are onboarded.

A Note on Flexibility

Isabel and Joey were clear that this is one valid architectural pattern — not the only one. Organizations evaluating similar solutions should weigh licensing costs across Azure AI Search, Azure OpenAI, and Copilot Studio, as well as user volume and the specific complexity of their retrieval needs. Emerging alternatives such as Copilot APIs and potential future capabilities like an MCP (Model Context Protocol) server for SharePoint may offer additional paths as the Microsoft ecosystem continues to evolve. The right architecture ultimately depends on the use case, the cost tolerance, and the ROI profile of the organization.