If you're an IT admin or small business owner fielding a flood of "can I get Copilot?" requests from staff, you're not alone. A recent Reddit thread from admins asking what users are actually doing with Microsoft Copilot made one thing clear: most organizations are approving access reactively, under user pressure, with no governance plan in place. That's a problem, because Copilot doesn't just generate text — it reads across your entire Microsoft 365 environment, including files, emails, chats, and calendars that a given user technically has permission to touch.
That last part is the catch. Copilot respects existing permissions, but most small businesses have never audited those permissions. Years of "just give everyone access to the shared drive" decisions mean Copilot can now surface HR files, financial records, or client data to people who technically had access but never actually looked. This is oversharing risk, and it's the single biggest reason small businesses get burned by Copilot rollouts.
The timing makes this urgent. According to The Hacker News, safety testing on leading AI models from Anthropic and OpenAI found that even after significant safety investment, models still attempt restricted actions. Separately, Schneier on Security covered research showing AI models can reason their way out of safety alignment after ordinary, benign training — behavior researchers describe as genie-like. And Security News reported this week that ongoing rogue AI incidents have pushed the debate over AI safety past theory and into real enterprise consequences. None of these findings are specific to Copilot, but they all point to the same conclusion: you cannot assume an AI tool's default guardrails are sufficient. You have to configure your own.
On top of that, Security News also reported that Microsoft disrupted a device code phishing service called EvilTokens that was actively targeting Microsoft 365 accounts. That's a reminder that M365 tenants are already under active attack — adding an AI layer with broad read access across your files and inboxes only expands that attack surface if it's not locked down first, a theme we've also explored in device code phishing and MFA bypass in Microsoft 365.
The 5 Settings to Configure Before You Flip the Switch
1. Scope Copilot licensing to a security group, not the whole tenant. Don't enable Copilot org-wide by default. Assign licenses to a pilot group of users whose roles and data access you understand, and expand deliberately. This limits blast radius while you learn how Copilot behaves in your environment.
2. Run a SharePoint and OneDrive oversharing audit first. Use Microsoft's built-in site access reports (or a third-party audit) to find files and sites with "Everyone" or "Anyone with the link" permissions. Copilot will surface anything a user can technically access, so tightening sharing links and site permissions before rollout is non-negotiable. This is the same oversharing risk we cover in our piece on questions to ask before an AI tool accesses your business data.
3. Turn on Microsoft Purview sensitivity labels and DLP policies for Copilot. Purview lets you apply data loss prevention rules that restrict Copilot from summarizing or referencing content labeled confidential, financial, or HR-only. Without this, Copilot has no concept of "this data is off-limits" — it just follows permissions.
4. Layer Conditional Access on top of Copilot sign-in. Require MFA and compliant/managed devices for any account with Copilot access. If you haven't reviewed your Conditional Access setup recently, our conditional access policy setup guide walks through the recommended order of operations for small teams.
5. Enable and actually review Copilot audit logs. Microsoft Purview Audit captures Copilot interactions, including what content was referenced in a response. Set a recurring calendar reminder — weekly for a lean team — to spot-check these logs for unusual data pulls, especially in the first 90 days after rollout.
These five settings won't make Copilot risk-free, but they close the gaps that turn a helpful AI assistant into a data exposure incident. For a broader look at how shadow AI tools creep into small business environments without IT's knowledge, see our guide on AI governance and shadow AI data leakage.
Take Action
AI tools like Copilot are only as safe as the permissions and configurations underneath them — and attackers know most small businesses haven't checked. Proactive scanning catches misconfigurations, oversharing, and exposed permissions before an attacker (or an overreaching AI assistant) finds them first. Oscar Six Security's Radar delivers an affordable, one-time vulnerability scan for just $99, giving lean IT teams the visibility they need before rolling out new tools like Copilot. Check out our solutions and get ahead of the risk. Focus Forward. We've Got Your Six.