On July 1st, Cloudflare – whose infrastructure handles 20% of global web traffic –  automatically blocked all AI bots from accessing customer sites, unless publishers explicitly opt in.

At the same time, the company launched “Pay-Per-Crawl,“ a private marketplace where site owners can charge AI firms like OpenAI and Anthropic per-crawl fees to scrape their content.

Why are we covering this story?

Because Cloudflare is taking a direct shot at Big AI’s “scrape now, ask later” model – and, in doing so, inserting itself as the gatekeeper between AI companies and web data.

The Bigger Story

For years, the deal was simple: publishers let search engines like Google crawl their sites in exchange for traffic. But AI crawlers have changed the math.

According to Cloudflare’s June Radar data, GPTBot – used by ChatGPT to scan public websites – makes about 750 requests for every visitor it sends back. ClaudeBot, from Anthropic, is even more extreme: 70,000 crawls per referral.

Legacy search bots? They hover around 15:1. 

That imbalance is what Cloudflare is targeting. By blocking AI bots by default, it puts the burden on crawlers (not creators) to get consent. And with Pay-Per-Crawl, Cloudflare is monetizing a layer of the stack that’s never been priced before: web-wide training data access.

Major publishers like Condé Nast, Time, and the Associated Press are already in the beta. Now, AI firms face a choice: pay to crawl, negotiate custom terms, or lose access to critical training inputs. Either way, the days of limitless free data are fading fast.

Why You Should Care

This move positions Cloudflare competitively – not just as a CDN, but as a gatekeeper between content owners and the LLM economy.

If you’re sitting on proprietary content, it points to a future where distribution becomes your moat, and training data shows up as a line item on someone else’s P&L.

The takeaway?👇 

I’ve said it before in this newsletter, and I’ll say it again:

In the AI economy, competitive advantage won’t just go to those who build the models – it’ll flow to those who control access to the data that makes them valuable.

You weren’t hired to build battlecards.

You were hired to help your company win.

But tactical busywork is stealing your time and your influence.

That’s why we built Compete Agent – your new competitive intel operative.

Compete Agent automatically:

• Monitors every sales call for competitive threats
• Instantly alerts sellers with deal-specific responses
• Generates fresh competitor profiles in seconds
• Keeps all intel updated daily (not quarterly)

The result? Your team stops getting blindsided by competitors and starts winning more deals.

📌 Teams like Gainsight are already saving 250+ hours on research while hitting 72% seller adoption! Don’t let your competitors get there first.

See Compete Agent in action 👇 

A few weeks back, Kimberly Bauer (Director of CI, SentinelOne) jumped on a Klue webinar to show us how the best teams set buyer decision criteria instead of reacting to them.

We kept the conversation rolling after the event and turned her playbook into a full-length guide, which we’ll be dropping next week. 

In the meantime, here’s the quick-sip version👇

The Value Wedge Matrix: Your RFP Game-Changer

Kim maps every differentiator on two axes:

• How much buyers currently care (Y-axis)

• How uniquely you deliver it (X-axis)

Most teams focus on the top-right—what buyers already value and you already do well. But the real opportunity is in the bottom-right: areas where you’re uniquely strong but buyers don’t realize they should care yet.

That’s where you bring in education, storytelling, and reframing. As Kim puts it: “That’s where the magic happens.”

Kim’s 4-Step Process to Shape Criteria (Not React to It)

1. Mine Your Wins for Real Criteria
Skip the internal assumptions. Look at actual win-loss data to understand what made buyers choose you. Kim once discovered a technical capability her team considered basic was actually winning deals – so they built an entire sales play around making it critical in RFPs.

2. Craft Your Point of View
Your POV shouldn’t be a feature list. It’s a take on what actually drives success.
If your competitors win on breadth, but that breadth creates bloat, your angle might be simplicity. Frame it conversationally:

“A lot of teams pick tools based on feature count. What we’ve found is that time-to-value is the better predictor of success.”

3. Build Competitive Sales Plays
Kim’s bar for clarity? “Can you explain the differentiator to your nine-year-old?” She tried this with her daughter – “Let me tell you, it’s a humbling experience.”

4. Enable Reps to Reframe Early
Arm sellers with natural reframes like: “Before we dive into these requirements, can we step back and talk about what success looks like for your team?”

❤️ Kim insight we loved

Kim reframes RFPs as “reverse engineered positioning documents.” This completely changes how you should approach them. Instead of seeing RFPs as neutral evaluation tools, you’ll start recognizing them as competitive weapons – which means you need to get upstream and shape the criteria before that document ever gets written. The key takeaway: whoever educates the buyer first sets the rules everyone else has to follow.

Start Tomorrow

Pick one deal where you suspect a competitor influenced the criteria. Work with that rep to:

• Identify which requirements favor the competitor
• Map them to your value wedge matrix
• Develop one reframe that shifts focus to your strength
• Test it in the next buyer conversation

Because here’s the truth: someone will shape how your buyer defines value. Might as well be you.

📌 Go Deeper: Watch Kim’s full breakdown of the Value Wedge Matrix and RFP strategy in her recent webinar, or check out our new guide dropping next week on klue.com/blog