Esato

Forum > Manufacturer Discussion > Samsung > Samsung uses software tricks to inflate Note 3 benchmark scores

Previous  123  Next
Author Samsung uses software tricks to inflate Note 3 benchmark scores
Wishmaster89
Sony Xperia Z
Joined: Jun 02, 2006
Posts: 333
From: Warsaw, Poland
PM
Posted: 2013-10-01 22:17
Reply with quoteEdit/Delete This PostPrint this post
Educated? About what?
I was taking part in that discussion and met with the same kind of response as this time - "it's no problem that they made it perform better in benchmarks than in other applications".

And here we are again with Samsungs newest and best device yet and the situation repeats itself but hey it's me that's being childish and with confidence problems.

Heck, even USB3.0 something that was praised as first on market and very useful etc. was proved by anandtech to be no different from USB2.0 at this point.

@Rikken
And you think I'm not pissed about this?! Problem is that you might get a great screen as it was proven with Z Ultra or Z1 or get lower quality one, this lottery is what really pisses me off. That is why I'm not going to buy another Sony device until they sort it out.
About the camera it's more complicated, things like decentered lenses that can cause part of the image to be out of focus is something that happens all the time with lenses that are more expensive than Z1 and Note 3 combined yet nobody judges it quality until he gets good copy, the same should apply to camera phones. Guys responsible for letting out Z1's with decentered lenses should get fired.

That's not a good example, it's not about getting the lowest time from 0-60 but achieving the highest possible speed. And as you said they all do their tests under ideal conditions with the help of best test drivers available.

If all OEM's would do the same thing as samsung and use 100% of the power just for the sake of getting the highest score than it wouldn't be a problem but if it's exclusive for samsung than it distorts the whole picture, don't you understand that?
rikken
Xperia Arc Black
Joined: Feb 23, 2007
Posts: > 500
From: Norway
PM
Posted: 2013-10-01 22:26
Reply with quoteEdit/Delete This PostPrint this post
As I said before, I don't care about benchmarks coz it's real-life performance that matters and the Note 3 is outstanding. If Sammy has cheated it is not my problem. I just want the best overall package
Wishmaster89
Sony Xperia Z
Joined: Jun 02, 2006
Posts: 333
From: Warsaw, Poland
PM
Posted: 2013-10-01 22:37
Reply with quoteEdit/Delete This PostPrint this post

On 2013-10-01 22:26:52, rikken wrote:
As I said before, I don't care about benchmarks coz it's real-life performance that matters and the Note 3 is outstanding. If Sammy has cheated it is not my problem. I just want the best overall package



And that is what I wanted to hear
If you're all OK with the fact that Samsung cheats but their devices either way perform as they should then I have nothing more to add in this discussion
rikken
Xperia Arc Black
Joined: Feb 23, 2007
Posts: > 500
From: Norway
PM
Posted: 2013-10-01 22:49
Reply with quoteEdit/Delete This PostPrint this post
Where did I say I think it is ok with cheating (I am not sure it is cheating) ? I said I don't care about these things, it is not my problem nor my responsibility

Review by anandtech : http://www.anandtech.com/show/7376/samsung-galaxy-note-3-review/4

"CPU performance is honestly excellent. The Galaxy Note 3 is more or less the fastest Android smartphone we've tested up to this point. In the situations where we can do cross platform (OS/browser) comparisons, it isn't quite as fast as the iPhone 5s but in some cases it comes close. I should mention that the Note 3 (like many other Android devices - SGS4, HTC One) detects certain benchmarks and ensures CPU frequencies are running at max while running them, rather than relying on the benchmark workload to organically drive DVFS to those frequencies. Max supported CPU frequency is never exceeded in this process, the platform simply primes itself for running those tests as soon as they're detected."

It's also interesting to note that the Galaxy Note 3 appears to outperform all other Snapdragon 800 smartphones we've tested thus far. There's a couple of potential explanations here. First, the Galaxy Note 3 is using newer drivers than any of the other S800 platforms we've tested:

Note 3: 04.03.00.125.077
Padfone: 04.02.02.050.116
G2: 4.02.02.050.141
[ This Message was edited by: rikken on 2013-10-01 22:12 ]
Gitaroo
Model not set
Joined: Aug 17, 2013
Posts: > 500
PM
Posted: 2013-10-01 23:10
Reply with quoteEdit/Delete This PostPrint this post
This is no difference from nVidia cheating with their drivers when they release their shitty Geforce FX line up back then. Their DX9 performance was complete crap that it was down right embarrassing so they got to cheat to keep up with ATI Radeon 970 pro. Sure at the time in DX9 wasn't as important and their DX8 performance was still acceptable and comparable but still It shows how much confidence they got with their products to begin with if they got to do this sort of thing.
Tsepz_GP
Apple iPhone 6 Plus
Joined: Dec 27, 2006
Posts: > 500
From: Johannesburg, South Africa
PM
Posted: 2013-10-01 23:23
Reply with quoteEdit/Delete This PostPrint this post

On 2013-10-01 22:49:37, rikken wrote:
Where did I say I think it is ok with cheating (I am not sure it is cheating) ? I said I don't care about these things, it is not my problem nor my responsibility

Review by anandtech : http://www.anandtech.com/show/7376/samsung-galaxy-note-3-review/4

"CPU performance is honestly excellent. The Galaxy Note 3 is more or less the fastest Android smartphone we've tested up to this point. In the situations where we can do cross platform (OS/browser) comparisons, it isn't quite as fast as the iPhone 5s but in some cases it comes close. I should mention that the Note 3 (like many other Android devices - SGS4, HTC One) detects certain benchmarks and ensures CPU frequencies are running at max while running them, rather than relying on the benchmark workload to organically drive DVFS to those frequencies. Max supported CPU frequency is never exceeded in this process, the platform simply primes itself for running those tests as soon as they're detected."

It's also interesting to note that the Galaxy Note 3 appears to outperform all other Snapdragon 800 smartphones we've tested thus far. There's a couple of potential explanations here. First, the Galaxy Note 3 is using newer drivers than any of the other S800 platforms we've tested:

Note 3: 04.03.00.125.077
Padfone: 04.02.02.050.116
G2: 4.02.02.050.141
[ This Message was edited by: rikken on 2013-10-01 22:12 ]



Excellent find Rikken
So...its not only Samsung Androids who are doing this, and 2nd, Samsung have ensured the Note 3 has the very latest drivers in the same way then ensured the GS4 had the very best S600 CPU available Iit's no wonder they are so successful and got some around here feeling so bitter towards them, as clearly shown in this thread.
I expect a repeat of all this when the Galaxy S5 arrives. It's a whole lot of fuss over nothing
[ This Message was edited by: Tsepz_GP on 2013-10-01 22:26 ]
Phone: iPhone 15 Pro Max Black Ti 512GB
Tablet: iPad Pro 11” 2020 Space Gray 256GB
Watch: Series 3 Nike Edition Space Gray
Droid: Huawei Mate 40 Pro 256GB
Tsepz_GP
Apple iPhone 6 Plus
Joined: Dec 27, 2006
Posts: > 500
From: Johannesburg, South Africa
PM
Posted: 2013-10-02 10:51
Reply with quoteEdit/Delete This PostPrint this post
Galaxy Note 3 not the only one free of “benchmark shenanigans” accusations



....As the Samsung Galaxy Note 3 appears on US-shores we’ve seen talk of not quite on-the-level benchmark boosting by the likes of what, at first, would seem to be the manufacturer of the smartphone. This report originates with Ars Technica where they suggest that the smoking gun is the fact that this smartphone brings up four full-powered CPU cores whenever it’s asked to open any one of several big-name benchmarking apps. What we’ve found is that this situation is not – by any means – limited to Samsung’s hardware.


We’ve run the same System Monitor app by Christian Gollner on several devices – with more on the way. What we’ve found is that the Samsung Galaxy Note 3, the HTC One, and the LG G2 all work with similar boosts in benchmarking apps. While outside of these apps, each of these devices works with one CPU core or – at most – two CPU cores without apps running, it would seem that nearly without fail the four-core fire-up appears in most major benchmark tests......

SlashGear
Phone: iPhone 15 Pro Max Black Ti 512GB
Tablet: iPad Pro 11” 2020 Space Gray 256GB
Watch: Series 3 Nike Edition Space Gray
Droid: Huawei Mate 40 Pro 256GB
reeflotz
Hazel
Joined: Jun 15, 2010
Posts: > 500
From: Philippines
PM
Posted: 2013-10-02 12:21
Reply with quoteEdit/Delete This PostPrint this post

On 2013-10-01 22:03:45, rikken wrote:
I would be far more pissed of at Sony for always promising great displays and the best of cameras and not delievering

[ This Message was edited by: rikken on 2013-10-01 21:09 ]



Sorry but I think you are misinformed, the display I might agree with that is if only you do care about viewing angles, but Sony did deliver the best display in the ZU, as for Z1 it's only the viewing angles on some not all, still try to view all screens head on and I'm pretty sure the Sony screens beats the competition in color accuracy and sharpness, hence why it's equipped with Triluminous.

As for the camera, Z1 is currently the best in Android smartphones no doubt especially in lowlight and high ISO performance. Just look at the samples by esato members on the Z1 photo thread . Were you expecting it to beat 1020 which has a hump and a larger sensor?, If there's anything that's a flop, it's the S4 Zoom. Z1 comes very close to 1020, now if only GsmArena used a retail unit rather than a preproduction unit, then the review would have been fair.

The problem is if Samsung already provides the fastest phone, then why do they need to do this? Surely you wouldn't need to boost scores if you're already leading. It does not make sense.
Hazel J20i [Bricked], Sony Xperia Go
reeflotz.blogspot.com
reeflotz@deviantart
MyP910
Sony Xperia Z
Joined: Feb 16, 2005
Posts: 499
PM
Posted: 2013-10-02 13:34
Reply with quoteEdit/Delete This PostPrint this post
Personally, I don't really care about Samsung and to some extent, Korean products. In my eyes, they are cheap and low quality brand and too far from premium. This is just me.

With this report, they going very low. Tweaking the device so that it will show a high score in a benchmark is so bad. I can't imagine the reason behind that. Why must they do it ? What's wrong with letting the app evaluate it true practical potential ? Showing high score in benchmark will not solve your laggy occasionally stuttering UI (touchwiz).

Surprise to read someone try to bring Sony into this. Sony promise good display they deliver Z1 and Z ultra with a good display (IMO the brightest display with great outdoor capability and great colour accuracy) minus at the extreme viewing angle. Sony promise great camera and they deliver with the best camera on Android smartphone and come very close to Lumia 1020.
badassmam
Sony Xperia Z1
Joined: Nov 07, 2007
Posts: > 500
PM
Posted: 2013-10-02 15:33
Reply with quoteEdit/Delete This PostPrint this post
The ZU has a better screen than the the Note 3. The Z1 has a better camera than any Samsung. Cheating is wrong and that applies to Sony, Samsung, LG, humans etc. I had a S4 for a week before I disposed of it because of its lag, guess I was LIED TO ABOUT THE BENCHMARK SCORES. For all those who say they're happy with the Note 3 despite the inaccurate benchmark results, good for you but you know you got it because of the benchmarks and a million average consumers did the same. If they're making a massive killing because of that, Samsung should be ashamed.
randomuser
Apple iPhone 5S
Joined: Sep 13, 2011
Posts: > 500
PM
Posted: 2013-10-02 15:44
Reply with quoteEdit/Delete This PostPrint this post
The point really isn't about cheating on benchmarks. The fact is, Samsung devices are so heavily bloated with crapware that the user experience is far below what these "cheated" benchmark scores show. When you see these numbers you expect the phone to be crazy fast in actual day to day usage and have NO lag whatsoever.

But in reality Samsung only cheats benchmark apps and provides arguably the least smooth android experience among flagship devices. The same applies to the S4 as well which had the highest benchmark scores but in reality was much less smooth compared to the HTC One, LG G Pro, Nexus 4 and probably on par with the Xperia Z which has an inferior S4 Pro chipset.

And then there's the UI design. No change from Android 2.3 Gingerbread as the Verge review mentioned. I personally cannot stand the ugly cartoonish UI in Samsung and LG devices. Whether you are running 2.3, 4.0, 4.1 or 4.3 just doesn't matter as they all look and feel the same.



Samsung should really concentrate on improving the software/user experience and build quality of their devices rather than "cheating" with benchmarks to create a false image in the minds of consumers
[ This Message was edited by: randomuser on 2013-10-02 14:48 ]
Tsepz_GP
Apple iPhone 6 Plus
Joined: Dec 27, 2006
Posts: > 500
From: Johannesburg, South Africa
PM
Posted: 2013-10-02 16:08
Reply with quoteEdit/Delete This PostPrint this post

On 2013-10-01 20:40:10, rikken wrote:
...Another storm in a teacup, people are dying to punish Samsung coz' they can't tolerate their success.



You were seriously on point with this one.
This thread is getting very entertaining, so far my favourite:
...the least smooth android experience among flagship devices.


Said about the ONE Android OEM who brings the smoothest, fastest and most functional UI so sour.
The very reason why Samsung is the only manufacturer making Android look good by having Apple on their toes is due to them bringing the best functionality, the openess of Android all in a package that is just as fast and smooth as iOS.

Love how Htc, LG and other OEMs are ignored for doing the same, only goes to prove your point even further Rikken
Gotta love Esato these days.
[ This Message was edited by: Tsepz_GP on 2013-10-02 15:17 ]
Phone: iPhone 15 Pro Max Black Ti 512GB
Tablet: iPad Pro 11” 2020 Space Gray 256GB
Watch: Series 3 Nike Edition Space Gray
Droid: Huawei Mate 40 Pro 256GB
Gitaroo
Model not set
Joined: Aug 17, 2013
Posts: > 500
PM
Posted: 2013-10-02 16:12
Reply with quoteEdit/Delete This PostPrint this post
@randomuser

agree and basically how I feel about my S3 and note 2 and I can't wait to get rid of those and stay the hell away from anything Samsung. I used to buy almost everything samsung before, from tv down to dvd burner, all turn out garbage and boycott them for a long time. When S3 came out with all the "amazing" reviews, I figure maybe I should give it another chance, but I was definitely wrong.
MyP910
Sony Xperia Z
Joined: Feb 16, 2005
Posts: 499
PM
Posted: 2013-10-02 16:43
Reply with quoteEdit/Delete This PostPrint this post
Samsung brings the smoothest, fastest UI ???? What ? Maybe some people never use other device therefore unable to realize that Samsung UI is laggy.
randomuser
Apple iPhone 5S
Joined: Sep 13, 2011
Posts: > 500
PM
Posted: 2013-10-02 16:49
Reply with quoteEdit/Delete This PostPrint this post
From ArsTechnica review

In terms of performance, the Note 3 is not what I would call "blazing fast." We've run into all sorts of stutters in animation and even outright pauses while testing it. While the hardware is top of the line, Samsung appears to have pushed the software envelope a little too far this time, and it doesn't feel as responsive as other modern Android phones. We've even run into weird error messages like "low memory" warnings when opening GFXBench, despite the 3GB of ram and Android's automatic memory management.
[ This Message was edited by: randomuser on 2013-10-02 15:50 ]
Access the forum with a mobile phone via esato.mobi