| Author |
SE ranks poorly in reliability/quality |
mib1800
Joined: Mar 18, 2004
Posts: > 500
PM |
Quote:
|
On 2005-08-31 14:01:07, max_wedge wrote:
In the Which survey, 20% of SE users reported faults, and 10% of Nokia users reported faults. The survey sample was 50 users, hardly enough to make any sort of realistic differentation between a 10-20% result.
Seriously, this "survey" would not be considered a reliable sample by any professional statistician.....
Everybody read this report and decide for yourself if the method was accurate or not. I myself am appalled it even made the news. It's sensationalism as far as I am concerned....
http://news.bbc.co.uk/2/hi/technology/4745205.stm
[ This Message was edited by: max_wedge on 2005-08-31 13:04 ]
|
|
Pls dont jump the gun. I think sample size was 5000+ not 50. The 50 were related to phones specifically used in 3 network. I think BBC (or anyone in their right mind) would not pick up the report if it were 50 sample size. 
[ This Message was edited by: mib1800 on 2005-08-31 13:38 ]
[ This Message was edited by: mib1800 on 2005-08-31 13:49 ] | |
|
max_wedge
Joined: Aug 29, 2004
Posts: > 500
From: Australia
PM, WWW
|
oops! 
Okay fair nuff. That's not so bad. However, I still don't think 10% difference is that big. In two months time the result could be the other way around. The mobile phone market is very fickle.
Besides the survey does say that there was no differentation between network and handset faults, so I still don't think the survey is truly representative of manufacturers. After allowing for which are hadnset, and which are operator, it might turn out that SE have been unfairly branded (no pun intended  ) by a poor operator performance.
I still think the survey needs to be better designed to really see the true trend.
|
goldenface
Joined: Dec 17, 2003
Posts: > 500
From: Liverpool City Centre
PM |
I think alot of damage was caused by the recent K750i launch / freezing issue, which I think has been repaired now. |
mib1800
Joined: Mar 18, 2004
Posts: > 500
PM |
@max_wedge:
if the 5000+ is statistically representative, the 10% difference can mean hundred of thousands (or even millions).
Like I said before, it doesnt matter whether it is network or phone faults. I think the survey is more phone-users oriented than manufacturer. What can be concluded from the survey is that those who bought SE/Moto are twice as likely to be inconvenienced due to the need to take the phone in for repair/service.
[ This Message was edited by: mib1800 on 2005-08-31 14:23 ] |
scotsboyuk
Joined: Jun 02, 2003
Posts: > 500
From: UK
PM, WWW
|
Quote:
|
On 2005-08-31 03:46:50, mib1800 wrote:
You can't expect others that carry the news feed to print the full article, do you? You cannot say it is meaningless just because you dont know the full details. It just like saying the quarter sales figure issue by SE is meaningless because the news feed did not carry the full breakdown details (running into tens of pages).
|
|
I can expect that, whether they do it is another matter.
The sales figures you mention would be somewhat meaningless because they only give one piece of information without further details. As I said before, such reports can suggest trends or display a certain reaction, but without aproper analysis and breakdown they aren't really all that useful in guaging the full situation. One can take something from them.
Quote:
|
Most of the time we are just interested in the final conclusion.
|
|
Which is a problem.
Quote:
|
Of course those who were given the bad limelight would try to rubbish it. btw 3 didnt provide alternative figures to disprove the findings except only saying sample size is not big enough.
|
|
I couldn't care less whether 3 provided figures or not, their point still stands. The sample size of their customers was very small, they don't need alternative figures to prove that.
[ This Message was edited by: mib1800 on 2005-08-31 03:14 ]
[ This Message was edited by: mib1800 on 2005-08-31 03:17 ]
[/quote]
"I may be drunk my dear woman, but in the morning I will be sober, and you will still be ugly." WSC |
mib1800
Joined: Mar 18, 2004
Posts: > 500
PM |
scotsboyuk:
Quote:
| The sales figures you mention would be somewhat meaningless because they only give one piece of information without further details. As I said before, such reports can suggest trends or display a certain reaction, but without aproper analysis and breakdown they aren't really all that useful in guaging the full situation. One can take something from them. |
|
You are supposed to digest the INDICATIVE trends information and not do a micro analysis which will bring out another hundreds of other micro-factors. You are trying to disprove the big picture just because some of these micro-factors show contrary information. Have you ever thought when all these these micro-factors when brought together will give you the exact general conclusion/trends.
Whether you like it or not statistics are never exact science. It is just an exercise in futility if you try to make exact science out of it. All is does is give us some indication that something may be going right/wrong. At least this is much much better than "gut feel".
So can this survey be wrong? Yes, there can always be this possibility. And so does all the other surveys which accorded SE shining accolades. .
Is the survey meaningless. Absolutely No. By reading this hopefully the person is wiser (at least he/she will pause and think before making silly rhetoric like "xxx makes the greatest phone in the world" or "yyy makes the worst phone in the world")
Quote:
| I couldn't care less whether 3 provided figures or not, their point still stands. The sample size of their customers was very small, they don't need alternative figures to prove that.
|
|
OK. So what do you think is the sample size of SE phones within the 5000+ respondents? Is it big enough for you? If you say yes, then I would tell you the same general conclusion from the survey as I've told max (in previous post). If you say no, then see the paragraph below.
Dilemma. Say someone wants to do a comparative survey/analysis of different brand of phones. Lets take a hypothetical figure of 5000 as a good sample size (i think you would agree with me on this number). But since SE market share is only 6%, in order to get minimum of 5000 for SE, you need to have about 80,000 respondents. You tell me which survey can get this kind of respondents short of a population census? So you are saying we can all dismissed all these comparative surveys as being meaningless. imo, you are opening the pandora box whereby anyone can rubbish any survey that awards accolades to a small player like SE.
[ This Message was edited by: mib1800 on 2005-09-01 03:19 ]
[ This Message was edited by: mib1800 on 2005-09-01 03:46 ] |
Gigs
Joined: Jan 19, 2004
Posts: > 500
From: The planet Snibertron!
PM, WWW
|
One SMALL teency sticking point..
Like I said before, it doesnt matter whether it is network or phone faults.
If you're going to call into question a manufacturers reliability then the above matters a great deal.
Lets say for arguments sake that everyone of the  respondants just happened to be an orange (it was orange wasn't it?) user with a K700i, the txting issue that was at fault within their network would have made an absolute joke of any statistical result.
After all, by that token CD/DVD burners would have horrible reputations.. Given that many are based on the cheapest shoddiest media people can find to use in them. It must however be the units fault, not the media used in it.
|
scotsboyuk
Joined: Jun 02, 2003
Posts: > 500
From: UK
PM, WWW
|
Quote:
|
On 2005-09-01 04:16:55, mib1800 wrote:
You are supposed to digest the INDICATIVE trends information and not do a micro analysis which will bring out another hundreds of other micro-factors. You are trying to disprove the big picture just because some of these micro-factors show contrary information. Have you ever thought when all these these micro-factors when brought together will give you the exact general conclusion/trends.
|
|
Generalisations are not something I often appreciate. I prefer to consider all the facts and base my opinion upon them. A conclusion is only one's opinions based upon the underlying information, which may be subjective.
As I said before, this report indicates a trend, without the full list of parameters used and an indepth analysis of the methods employed I don't consider it to be of any significant value. I was trained as a scientist and as such I was taught to question and look for the underlying causes. Trends are one thing, what they are based upon is another.
Quote:
|
Whether you like it or not statistics are never exact science. It is just an exercise in futility if you try to make exact science out of it. All is does is give us some indication that something may be going right/wrong. At least this is much much better than "gut feel".
|
|
First of all you know very well that I am not a fan of employing statistics (which is why I wasn't best pleased to have write about them below). Secondly I am not trying to make an exact science out of it. Thirdly you agree with my point about trend indication.
Quote:
|
So can this survey be wrong? Yes, there can always be this possibility. And so does all the other surveys which accorded SE shining accolades. .
|
|
Hence why people shouldn't put much stock in such surverys.
Quote:
|
Is the survey meaningless. Absolutely No. By reading this hopefully the person is wiser (at least he/she will pause and think before making silly rhetoric like "xxx makes the greatest phone in the world" or "yyy makes the worst phone in the world")
|
|
How will someone be wiser for reading this survey? By following a generalisation, which offers no real insight into the conclusion, which it presents? Hardly.
As for silly rhetoric, it would seem more silly to suggest that one base their buying decision upon a survey, which offers very little in the way of factual insight into the nature of its research; no comparative analaytical model to draw parallels from; an isolated market sample with little or no consideration for cultural or social factors; a lack of distinction between the end results used to form a conclusion, and which has been sesationalised thus possibly rendering the objectivity of those reading it, with a view to basing a buying decision on it, suspect.
Quote:
|
OK. So what do you think is the sample size of SE phones within the 5000+ respondents? Is it big enough for you? If you say yes, then I would tell you the same general conclusion from the survey as I've told max (in previous post). If you say no, then see the paragraph below.
Dilemma. Say someone wants to do a comparative survey/analysis of different brand of phones. Lets take a hypothetical figure of 5000 as a good sample size (i think you would agree with me on this number). But since SE market share is only 6%, in order to get minimum of 5000 for SE, you need to have about 80,000 respondents. You tell me which survey can get this kind of respondents short of a population census? So you are saying we can all dismissed all these comparative surveys as being meaningless. imo, you are opening the pandora box whereby anyone can rubbish any survey that awards accolades to a small player like SE.
|
|
I'll just rebut this in one I think, rather than leaving them as seperate points.
First of all the small survey sample was in reference to 3, but let's go with the SE point shall we?
We don't know what size the SE sample was in this survey, that's one of the problems!
What if the SE sample was too big? That would then suggest that more faults should be found with SE products because of it's larger contribution to the overall survey. So if it was too big then that skews the results. Unfortunately the survey doesn't inform us of the sample composition.
What if the SE sample was just right? That would seem to indicate that the conclusion is accurate. Well that may be the case assuming that all the other mnaufacturers tested were also at their optimum sample sizes in this survey. Were they? Again, we don't know! It would also assume several other criteria e.g. standardisation of fault reporting; was this undertaken? Was possible bias against or for particular manufacturers taken into consideration? How were faults defined? There are so many questions.
What if the SE sample was too small? If one has a survey sample of 5,000 then one could include 300 SE users, which is 6% of 5,000. This then reflects SE's market share within that 5,000 whilst constituting a sufficiently large number of samples. What you are talking about is conducting a survey of 80,000 people, not 5,000. The target of your survey is SE, you wish to interview 5,000 SE users and you wish to have those 5,000 represent 6% of an overall survey group. Well first of all 6% of 80,000 is 4,800, not 5,000. Secondly the same results can be achieved by using a smaller group. There should be no extreme change in the overall figures. Let me illustrate this with a wee example:
We are interviewing 5,000 people on mobile phone faults. Now if 300 of them use SE handsets and the fault rate of SE handsets is, for example, 20%, then approximately 60 of those SE customers will report a fault. If we are interviewing 80,000 people and 4,800 of them use SE handsets and the fault rate is still 20%, then approximately 960 will report a fault. More faults may be reported, but the ratio is approximately the same.
The underlying fact does not change. If you ask 1 million people a question with a yes or no answer e.g. has your handset had a fault? and the fault rate coming out of the factory is 10%, then approximately 100,000 will answer affirmative. If you ask the same question to 100 people and the fault rate is still the same then the results will remain roughly the same with around 10 people confirming faults. Unless you reduce the sample size to an extreme level e.g. 2 people, the results will be approximately the same assuming that the conditions remain constant.
The basic point about this survey is that whatever the sample size was, we don't know how it was composed. Was there even any thought to reflecting market share in the survey? Was the sample size divided up equally amongst the manufacturers? Furthermore we are also not considering bias on the part of the respondents, were they prejudiced towards certain brands or favourable? The whole survey lacks credibility as anything more than a trend indicator in my opinion.
You are correct, I am rubbishing these surveys, because they are meaningless. They only indicate trends and do not offer enough information to form a factual opinion. Even if everyone in the world was included in this survey it would still be meaningless on a factual level because it would still only indicate a trend. Without providing the fundamental information behind the survey results it is worthless save as an indication of a trend amongst a particular group of people. It tells us nothing of that group of people nor of the details involved.
"I may be drunk my dear woman, but in the morning I will be sober, and you will still be ugly." WSC |
mib1800
Joined: Mar 18, 2004
Posts: > 500
PM |
Quote:
|
On 2005-09-01 05:19:11, Gigs wrote:
One SMALL teency sticking point..
Like I said before, it doesnt matter whether it is network or phone faults.
If you're going to call into question a manufacturers reliability then the above matters a great deal.
Lets say for arguments sake that everyone of the respondants just happened to be an orange (it was orange wasn't it?) user with a K700i, the txting issue that was at fault within their network would have made an absolute joke of any statistical result.
After all, by that token CD/DVD burners would have horrible reputations.. Given that many are based on the cheapest shoddiest media people can find to use in them. It must however be the units fault, not the media used in it.
|
|
well, u have a good point. I'm dont know about the orange problem, but it wont change a thing because phones from other manufacturers in would also experience the same problem and this would even out this anamoly statistically. However, if you say the problem only happens with K700 then the fault lies with SE phones, isnt it.
ditto for your DVD analogy (i.e. other brands dvd player owner would also blame the dvd player).
|
mib1800
Joined: Mar 18, 2004
Posts: > 500
PM |
@scotsboyuk
that was a long reply.
Quote:
|
Well first of all 6% of 80,000 is 4,800, not 5,000. |
|
You are really meticulous arent you?. .
Quote:
| What if the SE sample was just right? That would seem to indicate that the conclusion is accurate. Well that may be the case assuming that all the other mnaufacturers tested were also at their optimum sample sizes in this survey. Were they? Again, we don't know! |
|
Do you think the people in Which? who did the survey (and analysed it) is SO STUPID to print something that is statistically undefendable??? I thought there is still libel law in UK. If only someone has the magazine!!.
Quote:
| Furthermore we are also not considering bias on the part of the respondents, were they prejudiced towards certain brands or favourable? |
|
I thought this is about phone-users feedback survey and not about QC/engineering survey (of which you need to go to factory production line to get). What if the survey just ask a simple question? "Did you buy a new phone in the previous 12 months. If yes, what brand is it and on which network and did you encounter any fault?".
Any survey of this type would inherently contains the variability that you mentioned. There are thousands of such surveys/polls (from politician approval ratings to tv programme polls) happening everyday. All these may be meaningless to you but some way or others these surveys would have some effect on something.
Quote:
| The whole survey lacks credibility as anything more than a trend indicator in my opinion. |
|
The survey is saying nothing more than a trend. So I dont know why you want to question credibility. If you agree that the survey gave a trend indicator, then the survey data should be credible enough to derive this trend, nothing more and nothing less.
Quote:
| Without providing the fundamental information behind the survey results it is worthless save as an indication of a trend amongst a particular group of people. It tells us nothing of that group of people nor of the details involved. |
|
It seems to me you are trying to question the independence of the survey. I would agree with you on this if and only if you can ascertain that readers of Which? are bias towards certain technology/brand/practice (i.e. something similar to SE-biased esato). Maybe you tell me since you r from UK, you should know better.
|
max_wedge
Joined: Aug 29, 2004
Posts: > 500
From: Australia
PM, WWW
|
Quote:
|
On 2005-09-01 07:11:34, mib1800 wrote:
Quote:
|
On 2005-09-01 05:19:11, Gigs wrote:
One SMALL teency sticking point..
Like I said before, it doesnt matter whether it is network or phone faults.
If you're going to call into question a manufacturers reliability then the above matters a great deal.
Lets say for arguments sake that everyone of the respondants just happened to be an orange (it was orange wasn't it?) user with a K700i, the txting issue that was at fault within their network would have made an absolute joke of any statistical result.
After all, by that token CD/DVD burners would have horrible reputations.. Given that many are based on the cheapest shoddiest media people can find to use in them. It must however be the units fault, not the media used in it.
|
|
well, u have a good point. I'm dont know about the orange problem, but it wont change a thing because phones from other manufacturers in would also experience the same problem and this would even out this anamoly statistically. However, if you say the problem only happens with K700 then the fault lies with SE phones, isnt it.
ditto for your DVD analogy (i.e. other brands dvd player owner would also blame the dvd player).
|
|
Actually mib, he said the texting problem might have been NETWORK related, but that the respondents USED K700's. So that means the K700's would be fine on another network, hence not the phone's fault.
With 5000 respondents then many anomolies related the respondent demographics will even out, true. However there may only be 20 or 50 phone models, and 20 operators included in the survey, so it's quite possible that the results for the reporting of faults is not accurate with regard to handsets. Just because user A had problems with phone B on network C, doesn't mean that phone B is faulty. That's a long stretch, and as I've said I don't know of any statistician who would support such a conclusion.
The truth is this survey is designed to gauge general customer dissatisfaction, and is not designed to highlight specific manufacturers as being more reliable than another. The results for handsets used by respondents reporting faults would be like surveying road crashes and saying more crashes involved sedans, therefore sedans are more dangerous, when in fact there are simply more sedans on the road. Without knowing how many handsets from each manufacturer were the cause of the reported faults, how can you say that this or that manufacturer has more reliable handsets? |
scotsboyuk
Joined: Jun 02, 2003
Posts: > 500
From: UK
PM, WWW
|
Quote:
|
On 2005-09-01 10:10:12, mib1800 wrote:
@scotsboyuk
that was a long reply.
|
|
I like long replies ... 
Quote:
|
You are really meticulous arent you?. .
|
|
Well it's not is it?
Quote:
|
Do you think the people in Which? who did the survey (and analysed it) is SO STUPID to print something that is statistically undefendable??? I thought there is still libel law in UK. If only someone has the magazine!!.
|
|
I am quite sure they could defend it on all manner of fronts, but as I said we do not know! If they say that they composed their sample in a certain way then fine, we can then take the report's findings in that light. However, they haven't and probably won't. Giving us the details of how the survey was conducted would have been most helpful indeed.
Quote:
|
I thought this is about phone-users feedback survey and not about QC/engineering survey (of which you need to go to factory production line to get). What if the survey just ask a simple question? "Did you buy a new phone in the previous 12 months. If yes, what brand is it and on which network and did you encounter any fault?".
|
|
Again we don't know, perhaps someone can enlighten us to what it was people were asked? The trouble is that in asking such a simplistic question with no further details one is inherently seeking a vague answer. That question itself has problems e.g. why handsets from the lasy 12 months, why not 6 months, or 18 months? This could affect the result. What is a 'new phone', a brand new model from the factory or a new phone for that particualr person even though the phone may be older? Why ask about network if one isn't going to offer any breakdown of faults?
Without some sort of differntiation of the results obtained it is a trend indicator.
Quote:
|
Any survey of this type would inherently contains the variability that you mentioned. There are thousands of such surveys/polls (from politician approval ratings to tv programme polls) happening everyday. All these may be meaningless to you but some way or others these surveys would have some effect on something.
|
|
This is just it though, they are meaningless as anything but trend indicators. Political polls indicate trends, they don't actually break down reults and present a detailed picture of the rationale behind people's choices. One can use such polls to note a possible trend, but fundamentally they do not provide one with any sort of meaningful adat, from which to bas eone's own decision on. An opinion poll is unlikely to affect someone's voting decision, they will be mor elikely to abse that decision on other factors. These sort of polls don't incorporate such fcators into a comprehensive model, which could be used to make an informed decision.
A political poll is aksing an opinion, that is it is aksing what someone thinks. This survey was allegedly asking afactual question i.e. has your handset had a fault? A political poll deliberatly seeks to encompass as wide a group of people as possible. This Which? survey should have attempted to incoporate a balanced mixture of samples representing the manufacturers since it aimed to differentiate its reults by brand. We don't know if they did though.
Hence we really don't know what the variability involved in this survey was. Was it reflective of the market? Was it just a random sample of people?
Quote:
|
The survey is saying nothing more than a trend. So I dont know why you want to question credibility. If you agree that the survey gave a trend indicator, then the survey data should be credible enough to derive this trend, nothing more and nothing less.
|
|
Look back over this thread, I have been saying it is nothing more than trend indicator some a while now. It is nice to see you agreeing with me.
The survey indicates a possible trend, but without knowing the criteria and methodology use done cannot even be sure of that. My point is that this survey is next to useless in making a buying decision because it give sno hard facts or analytical view of the data. At best it give sone an idea of a possible trend, which may or may not be true depending upon the survey was carried out.
This survey lacks credibility because it is supposed to help consumers looking to buy a mobile phone. In that respect it fails in my opinion.
Quote:
|
It seems to me you are trying to question the independence of the survey. I would agree with you on this if and only if you can ascertain that readers of Which? are bias towards certain technology/brand/practice (i.e. something similar to SE-biased esato). Maybe you tell me since you r from UK, you should know better.
|
|
This is just the point, we don't know! There are so many occassions when one can say those words in relation to this survey.
The readers themselve sdon't have to be biased, the poll sample could have been or even the people conducting the poll.
Now I am sure that the good people at Which? may very well have taken steps to ensure objectivity, but once more I return to the phrase 'we don't know'.
What we have is a survey, which shows a posssible trend. I can take something from this survey, anmely that SE handsets have had more faults reported amongst a certain group of people than most other manufacturers. What I can't take from this survey is what the nature of those faults were; whether the results were skewed in any way; how the faults correspond between manufacturer and network or whether the sample itself was inherently flawed.
"I may be drunk my dear woman, but in the morning I will be sober, and you will still be ugly." WSC |
mib1800
Joined: Mar 18, 2004
Posts: > 500
PM |
@scotsboyuk:
i'll answer your last paragraph first. This I believe is the crux of the matter.
Quote:
| What I can't take from this survey is what the nature of those faults were; whether the results were skewed in any way; how the faults correspond between manufacturer and network or whether the sample itself was inherently flawed. |
|
imo, i think what you are asking is non-existence because the good people at Which? would most probably not interested in this kind of survey in the first place. What you're seeking is only of interest to engineers and QC persons. I doubt the general public can comprehend such nitty-gritty details of whether fault is due firmware/hardware/network etc. Many don't even know what the heck is firmware.
Quote:
| My point is that this survey is next to useless in making a buying decision because it give sno hard facts or analytical view of the data. At best it give sone an idea of a possible trend, which may or may not be true depending upon the survey was carried out.
|
|
From my point above, this is precisely the kind of survey that would affect buying decision. Which? obviously structured their survey this way so that results can be understood by the general public.
On the matter whether survey is fair, I agree with you that only the people at Which? can shed light on this. All I can say is that if they can put it to print, they would have a reasonable confidence of defending it. However, I disagree with you that without knowing the survey parameters would dilute the usefulness or impact of the results.
I agree with you we should raise questions on the results presented by these type of survey. But my opinion is that we should only raise questions within the scope of the matter (i.e. not deviate too much from the central premise of the survey, otherwise there is no end to it).
Even if you know the survey results down to the categorisation of "faults" or "group of users", what you achieve is no more than just knowing. In the end, we still end up with the general indicative trend as was given. |
augustborn9917
Joined: Jun 03, 2003
Posts: > 500
From: Uranus
PM |
to the long winded debaters:
*will you be arguing every single minutiae of the survey had the results been reversed? SE reliable, and nokia unreliable?
my point is how much of this discussion spurred by blind devotion? surely you are not shocked that SE is labelled as unreliable/buggy. the fact of the matter is historically SE is unreliable and buggy compared to other brands.
IMO discussing the innacuracies of the survey is moot, because it has been published for everyone to see. the damage is done. whether the survey was properly conducted or not, it just quantifies the rep of SE for being unreliable. truthfully, did the results really surprise you?
i am a fan of SE, and will continue to be a fan because i like the SE vision and technology.
i find myself constantly making excuses to myself why i get so many dropped calls/ freezing/rebooting screens, subpar RF: "i just need a FW update"; while i look to my wife's nokia: rock solid from day1
bottomline: SE is unreliable
|
Gigs
Joined: Jan 19, 2004
Posts: > 500
From: The planet Snibertron!
PM, WWW
|
Yet august some of us have no problems. My k700i, said to be a plague of death phone by some, performed flawlessly and still does.
Taking age into account and features also is something. For instance if you were talking to say 3310 and t100 users, you probably wouldn't find much at all wrong with them. But then there's hardly anything there to go wrong.
Sure some of its devotion but its also about taking results on a level playing field, as I stated if you're going to blanket any manufacturer with being unreliable in product then you can't just take it on a survey that doesn't take into account differences between network and phone faults.
I think if anything it IS fair to say is inconsistant.
@mib:
The fault was only inherent to 1 network, vodafone and other users had no issue at all so it was nothing to do with the phone. But if you were hit with this you'd blame the phone and thus the survey would label your phone as unreliable. Hardly right in any case
| |
|
Access the forum with a mobile phone via esato.mobi
|
Posted: 2005-08-31 14:36
