Opus 4.7 vs Opus 4.6: Should You Switch?

4. Ignores Direct Instructions

In among the examples shared on the Web, customers have reported that the newest Claude mannequin utterly ignores explicitly written directions inside a immediate. Reddit person @drivetheory, as an illustration, shares their expertise with the Opus 4.7. Having written extremely particular directions on how they need their response to be structured, the brand new Opus mannequin utterly ignored most of the instructions inside the immediate. This included the configuration necessities, in addition to quotation wants for the actual reply.

Apart from these main ones, there are numerous complaints in opposition to the brand new Opus 4.7, most of which have been shared by the prevailing Claude customers who cherished Opus 4.6. So, to check out these claims, we ran our personal checks on the mannequin.

Let’s Evaluate Opus 4.7 vs Opus 4.6 on Various Duties

Right here is how the brand new Opus 4.7 carried out throughout duties.

Right here is the duty I assigned to Opus 4.7 for this:

“Undergo this report by the IMF for India’s Monetary System Stability Evaluation, and analyse the dangers that India’s monetary sector faces. Price these dangers primarily based on the almost definitely ones to influence the sector within the coming years, and provides one-line options to avert every of those dangers utterly.”

Opus 4.7 Output:

  

Opus 4.6 Output:

  

Commentary:

Each fashions got here out with correct outputs detailing precisely what was requested. But, when you look intently, there’s a huge distinction in how they got here to the conclusion and the way they each offered it.

Opus 4.7 lays out a whole, detailed plan of seven steps, executing completely different steps within the workflow, earlier than it even begins to write down the ultimate output. That is precisely what many customers are complaining about, as this prolonged reasoning can be a serious cause for the heightened token use throughout every output. Whereas the mannequin is attempting to be as correct as doable, it breaks down the steps a lot that value effectivity goes out of the window.

And in spite of everything this computing, the ultimate output is in a easy textual content format with one paragraph laid out after one other. Correct, sure, however presentable – no approach.

In distinction, Opus 4.6 hardly took 3 steps of execution earlier than it began delivering the ultimate output. What’s extra, its output can clearly be seen in a far more presentable format than what Opus 4.7 gave out. Although we didn’t particularly ask it to, it created a brand new dashboard to current its findings in a extra interesting approach. You may deal with it as deviation, or as further marks. Your selection.

With nearly comparable content material but much more visible enchantment, Opus 4.6 would clearly be my most popular mannequin right here.

2. Reasoning

To check its reasoning capabilities, right here is the immediate I used:

“You’re being evaluated for precision, brevity, and instruction-following.

Activity:
An organization has 4 venture proposals and might fund solely 2 of them. Select one of the best pair.

Tasks:
A. Price: $4M | Anticipated 3-year return: $8M | Danger of failure: 35% | Strategic worth: Excessive | Requires 20 engineers
B. Price: $3M | Anticipated 3-year return: $5M | Danger of failure: 15% | Strategic worth: Medium | Requires 10 engineers
C. Price: $5M | Anticipated 3-year return: $11M | Danger of failure: 45% | Strategic worth: Very Excessive | Requires 25 engineers
D. Price: $2M | Anticipated 3-year return: $3.5M | Danger of failure: 10% | Strategic worth: Low | Requires 6 engineers

Constraints:
– Whole price range can’t exceed $7M
– Whole accessible engineers = 30
– The corporate desires no less than one “Excessive” or “Very Excessive” strategic worth venture
– Keep away from selecting a pair if each tasks have failure danger above 30%

Output guidelines:
1. First line: write solely the chosen pair, like “A + B”
2. Second line: write just one sentence of most 25 phrases explaining why
3. Third line: write solely “Rejected pairs:” adopted by the rejected pairs separated by commas
4. Don’t present calculations
5. Don’t clarify your reasoning
6. Don’t add headings, bullet factors, or disclaimers

Necessary:
In case you violate any output rule, your reply is wrong.”

Opus 4.7 Output:

Opus 4.6 Output:

Commentary:

Within the reasoning check, each Opus 4.6 and Opus 4.7 arrived on the similar appropriate reply, adopted the required format, and prevented bloated justification. That’s necessary as a result of this immediate was designed particularly to catch two alleged weaknesses: losing tokens on reasoning and ignoring direct directions. Neither mannequin actually slipped right here. Opus 4.7 stayed inside the construction and stored its rationalization compact, which is sweet information for Anthropic. But, we are able to word right here that there is no such thing as a dramatic separation seen from Opus 4.6. In different phrases, Opus 4.7 doesn’t fail this check, however it additionally doesn’t show a transparent leap over its predecessor from this outcome alone.

3. Coding

To check the coding capabilities of the Opus 4.7, right here is the immediate I used:

You’re being examined for coding precision, instruction-following, and avoiding pointless output.

Activity:
Repair the Python operate under so it returns the size of the longest substring with out repeating characters.

Buggy code:
def longest_unique_substring(s):
seen = {}
left = 0
finest = 0

for proper in vary(len(s)):
if s[right] in seen:
left = seen[s[right]] + 1
seen[s[right]] = proper
finest = max(finest, proper – left + 1)

return finest

Necessities:
1. Return solely corrected code
2. Don’t clarify something earlier than or after the code
3. Hold the operate identify unchanged
4. Use the sliding window strategy
5. Time complexity should stay O(n)
6. Add precisely 3 check circumstances as Python assert statements
7. Don’t use feedback
8. Don’t redefine the issue
9. Don’t present different options

Your reply is fallacious if:
– you embrace any rationalization
– you alter the operate identify
– you present greater than 3 asserts
– the code fails on repeated characters that happen earlier than the present window

Opus 4.7 Output:

Opus 4.6 Output:

Commentary:

On the coding check, each Opus 4.6 and Opus 4.7 did the necessary factor proper: they fastened the bug, returned solely the corrected code, stored the identical operate identify, and resisted the temptation to dump further rationalization. That issues as a result of one of many largest complaints round Opus 4.7 has been wasted tokens and pointless commentary. Right here, that drawback didn’t actually present up. If something, each fashions had been disciplined. The distinction is that Opus 4.7 doesn’t clearly outperform 4.6 on this case. It’s appropriate, sure, however so is 4.6. So this outcome doesn’t assist the declare of a serious coding improve. It solely reveals that Opus 4.7 can nonetheless behave properly on tightly constrained coding duties.

Remaining Take: Opus 4.7 vs Opus 4.6

Effectively, up till now, we now have seen what Anthropic says about its all-new Opus 4.7. We’ve got had a take a look at all the brand new options it brings to the desk, after which some ways by which it’s supposedly higher than the outgoing mannequin, i.e., the Opus 4.6.

On the flip facet, we now have additionally seen the assorted person experiences that counter these claims. The experiences shared by these customers present that the Opus 4.7 is clearly missing the wow issue {that a} common improve to such a revered mannequin brings.

After which we put all that to the check in a hands-on experiment of our personal, the place we put each fashions facet by facet for a complete of three use circumstances throughout content material extraction and era, reasoning, and coding. Here’s what is evident after an in depth breakthrough to date.

1. Sure, Opus 4.7 makes use of far more tokens: Effectively, that is evident from Anthropic’s personal accounts in addition to from the outcry that has adopted the launch of the brand new mannequin. The very design of the Opus 4.7 makes it eat up tokens extra ferociously than ever earlier than.

So, in case you are planning to make use of the mannequin for advanced, agentic duties, my suggestion could be – don’t. At the very least in case you are acutely aware of your day by day restrict or API price range. In case the price range is not any problem, then be happy to attempt your hand on the new Opus 4.7 and what it’s able to.

2. Sure, Opus 4.7 performs numerous iterations unnecessarily: As many customers have identified, and from what I might determine from my very own use, Opus 4.7 performs far more iterations in its pondering course of than obligatory, particularly so when you evaluate it to Opus 4.6.

After which when the output just isn’t at par with that of different fashions, you have a tendency to consider all that compute as a whole waste of time, efforts, and most significantly, tokens.

3. No, Opus 4.7 just isn’t inaccurate: At the very least in our use with it, the Opus 4.7 didn’t falter even as soon as, and managed to stay to the directions fairly fantastically, churning out tremendous correct outputs with all types of prompts. So full marks to the mannequin on that entrance.

Conclusion

Backside line – undoubtedly give Opus 4.7 a attempt. However to shift your complete workflow to it, particularly when it entails in depth steps and power calling could be a waste of your tokens I imagine. As there is no such thing as a apparent distinction within the high quality of outputs it comes up with, vis-a-vis what Opus 4.6 was able to.