Why I Built the Blackbird M365 Assessment Tool - and Why You Need It Right Now.
I've spent nearly 30 years in IT and cybersecurity. I've inherited environments from other MSPs that were supposed to be "fully managed." I've seen the gap between what clients were told and what they actually had. And I've watched the threat landscape shift in ways that make that gap more dangerous every year.
The Blackbird M365 Assessment tool — Scout and Raptor — exists because of that gap.
Why I Built It
When I started Blackbird IT Solutions, I committed to one thing above everything else: no client of mine would ever discover they weren't protected the hard way.
That means I need to know the real state of every client's Microsoft 365 environment — not what the sales sheet says, not what the previous MSP told them, but what's actually configured. What's actually exposed.
The problem is that manual M365 security assessments are slow, expensive, and inconsistent. Doing them properly takes hours. Doing them for every prospective client — before they're even paying you — is impractical. And most of the time, small businesses can't afford a full security engagement just to find out how bad their situation is.
So I built the tool I needed.
Scout and Raptor are automated Microsoft 365 security assessment tools that analyze your M365 tenant for real security gaps — the kind of gaps attackers are actively exploiting right now.
I built them for my clients. Then I realized every small business in Louisville and Southern Indiana could benefit from them, whether they're a Blackbird client or not.
Scout vs. Raptor: Health Check vs. Full Audit
The two tools are different in scope and purpose. Understanding which one you need starts with understanding what each one is designed to do.
Scout is a health check — a fast, focused look at the 10 most critical M365 security controls. Think of it as a vital signs check. It won't tell you everything, but it will tell you immediately if something is seriously wrong. Ideal as a first look, a quick sanity check, or an annual baseline.
Get Scout — $250Raptor is a full security audit — 65 checks across 13 security domains. It produces a complete picture of your M365 security posture with prioritized remediation guidance. This is what you run when you want to know everything, not just the highlights. Includes Microsoft Secure Score benchmarking and coverage of Defender, Intune, SharePoint, Purview, and more.
Get Raptor — $500Both tools run in under 10 minutes from setup to report. Both deliver a secure PDF with findings and next steps. And both include 48-hour support — more on that in a moment.
The Setup Takes 10 Minutes. Your Credentials Are Never Stored.
The question I hear most often: "Do you need my password to run this?"
No.
The assessment works through a Microsoft feature called App Registration — a standard, official mechanism for granting read-only access to your M365 tenant. We guide you through a 10-minute setup process that creates this access. Once the assessment completes, you get instructions to delete the App Registration immediately. The access window closes as soon as your report is generated.
Your credentials are never stored. Our API calls are read-only — we cannot modify, create, or delete anything in your environment. Your report arrives via a time-limited, signed download link. It never travels through your email system. The App Registration we use to run the assessment is deleted the moment we're done.
This isn't a trust-me situation. It's how Microsoft designed third-party access to work. We just built a tool that uses it properly.
About the 48-Hour Support
Every Scout and Raptor assessment includes 48-hour email support — and I want to be clear about what that means, because it matters.
This support is for your assessment findings, not for your Microsoft 365 environment. It's not managed IT support. It's not a help desk. It's not troubleshooting your M365 tenant.
What it is: if you receive your report and don't understand what "CAE not enforced" means, or why "refresh token lifetime unlimited" is flagged as a risk, you can ask us. A real person will explain what we found and what it means for your business. We'll tell you what needs to be fixed and why it matters — in plain language, not security jargon.
If the findings reveal issues you want help addressing, that's a separate conversation. But the 48-hour support ensures you never walk away from a report with a list of findings you don't understand.
How We Stay Current: Built for Emerging Threats
Here's something most security tools don't talk about openly: the threat landscape changes faster than most assessment tools are updated.
I built Scout and Raptor as a maintained, updatable platform — not a one-time script — specifically because of this problem. When new attack techniques emerge, we update the checks to match. That distinction matters more than most people realize.
A perfect example is the device code authentication flow attack that's been active since early 2026.
Security researchers at Huntress documented a campaign that hit more than 340 organizations across the U.S., Canada, Australia, New Zealand, and Germany. The attack targets Microsoft 365 identities — and what makes it particularly dangerous is that the access tokens it generates remain valid even after an account's password is reset. Healthcare, legal, financial services, and manufacturing were among the industries targeted.
Here's how the attack works in plain terms:
- 01 An attacker requests a device code from Microsoft's authentication system — this is a legitimate Microsoft feature designed for devices that can't open a browser.
- 02 The attacker sends a convincing phishing email urging the victim to visit Microsoft's legitimate sign-in page (microsoft.com/devicelogin) and enter the device code.
- 03 The victim — seeing a real Microsoft page — enters the code along with their credentials and MFA. Everything looks legitimate because it is.
- 04 The attacker retrieves a set of valid access tokens. From that point forward, they have persistent access that survives a password reset. MFA doesn't help either — the victim already completed it.
The campaign used construction bid lures, DocuSign impersonation, voicemail notifications, and fake Microsoft Forms pages to trick victims. It hit every vertical I serve.
Our Raptor assessment checks for the specific configurations that either prevent this attack or detect that it has already occurred — device code flow blocking, token lifetime policies, Continuous Access Evaluation settings, and legacy authentication controls. When new attack techniques emerge, we update the checks to match.
Most businesses find out about threats like this from the news, weeks after attackers have already moved to the next target. Our tool is designed to find the gaps before the attackers do.
The Bottom Line
You don't know what's misconfigured in your M365 environment until you look. Most businesses haven't looked. Most attackers have.
Scout gives you a health check — fast, focused, and clear. Raptor gives you a full audit — comprehensive, prioritized, and actionable. Either one will tell you more about your actual security posture than most businesses have ever known.
The device code phishing campaign hitting hundreds of organizations right now is one example of what happens when the controls aren't in place. There will be another campaign next month. And the month after that. The question is whether your environment is ready.
Start with Scout at $250 or go straight to the full Raptor audit at $500. One-time payment, no subscription, report in under 10 minutes.