Announcing: Slashdot Deals - Explore geek apps, games, gadgets and more. (what is this?)

Thank you!

We are sorry to see you leave - Beta is different and we value the time you took to try it out. Before you decide to go, please take a look at some value-adds for Beta and learn more about it. Thank you for reading Slashdot, and for making the site better!

Truth in Advertising?

Cliff posted more than 10 years ago | from the devils-in-the-details dept.

Technology 393

PerformanceEng wonders: "I work as an engineer for a large technology company in the U.S., and have been privy to what I find a interesting practice. It's well known that marketing data sheets often paint the best picture of a product while leaving the devil in the details. I've come to expect this, and when I am evaluating technology, I always have a skeptic's eye for claims made by the sales and marketing folks. However, I've also witnessed our product go into test labs (usually for the purposes of running a series of tests for a 'bake off' in a trade publication). Not uncommon is the attempt to 'tune' the configuration of the device under test to perform in the best light (not unlike tuning your car to pass emissions tests). I have seen it go as far as exploiting weaknesses in the test that, if the test operator discovered, would be considered bad faith. To the other engineers: Are you aware of this kind of practice at your company? To the IT professionals: How much faith do you put in these sorts of publications and their 'bake offs'? To everyone: When does spin doctoring cross the line and become false advertising?"

Sorry! There are no comments related to the filter you selected.

Video drivers (4, Informative)

Anonymous Coward | more than 10 years ago | (#11056790)

It wasn't an uncommon practice for video card makers to tweak their drivers to perform better on benchmarks.

ATI's 'Quake' optimization. (3, Informative)

oneiros27 (46144) | more than 10 years ago | (#11056886)

I assume you're refering to the discovery that ATI did some cheating if it saw that Quake was being run:

Re:ATI's 'Quake' optimization. (1)

lightknight (213164) | more than 10 years ago | (#11056937)

Yeah, that was a while back. Nvidia has been picking up the slack (on cheating) ever since.

Re:Video drivers (1)

Gentlewhisper (759800) | more than 10 years ago | (#11056982)

This is a fact of life man...

Don't guys try to dress up nicely before going for a date too? That way the girl will think he is some decent guy, when instead he is just another dork who always sits in front of his PC reading /.

Life man. Advertising is about generating sales, telling the truth only happens when it is illegal to do otherwise!

Peer review (3, Insightful)

BWJones (18351) | more than 10 years ago | (#11056791)

I've also witnessed our product go into test labs (usually for the purposes of running a series of tests for a 'bake off' in a trade publication). Not uncommon is the attempt to 'tune' the configuration of the device under test to perform in the best light (not unlike tuning your car to pass emissions tests). I have seen it go as far as exploiting weaknesses in the test that, if the test operator discovered, would be considered bad faith.

Oh, you work for Intel then. :-) Seriously though, this has been the whole problem with "benchmarks" like SPEC and others that ultimately results in pissing matches between manufacturers saying "my product is faster than yours" which for 99% of the users out there means nothing. In fact, even for that 1% of us where it does make a difference, specific optimizations to ones code or algorithms typically will get you more performance. So, what it really comes down to is how productive is the product + environment + task that you are assigning to the platform.

To answer your question of false advertising, I would say keep to the standard that most of us scientists do: Specifically, peer review and ensure that your results can be duplicated by said peers. If results cannot be duplicated, then it is false advertising.

Re:Peer review (4, Insightful)

krautcanman (609042) | more than 10 years ago | (#11056825)

To answer your question of false advertising, I would say keep to the standard that most of us scientists do: Specifically, peer review and ensure that your results can be duplicated by said peers. If results cannot be duplicated, then it is false advertising.

Even science has a problem of touting the best data and "leaving the devil in the details." Research is driven by money just as much as industry. If you're not producing good results, you won't get funding.

Re:Peer review (1)

BWJones (18351) | more than 10 years ago | (#11056875)

Even science has a problem of touting the best data and "leaving the devil in the details." Research is driven by money just as much as industry. If you're not producing good results, you won't get funding.

And if you are caught falsifying data then you will never get funding again. At least from traditional sources this is true and you will have major problems finding a position in academia. There have been a few cases where folks even spent time in jail for scientific fraud. On the whole, most scientists are reputable.

Re:Peer review (1)

Incongruity (70416) | more than 10 years ago | (#11056889)

Even science has a problem of touting the best data and "leaving the devil in the details." Research is driven by money just as much as industry. If you're not producing good results, you won't get funding.

If you do produce results that are consistently not reproducible by your peers, then you'll quickly no longer have a career, much less funding. It may not happen right away but it will happen in reasonable course, especially the more impressive your claimed results are and the more clear it becomes that you were not trying to practice "good science" but rather were just looking to make a name for yourself.

False Advertising (2, Funny)

Anonymous Coward | more than 10 years ago | (#11056793)

What about all those people on Slashdot who claim First Post but fail it? Isn't that false advertising?

Re:False Advertising (0)

Anonymous Coward | more than 10 years ago | (#11056915)

First post!

Re:False Advertising (1)

GhostseTroll (582659) | more than 10 years ago | (#11056994)

Your Bowels Cleansed

Let me ask you this...which is worse:

A. The engine on your Lexus freezes up at 160,000 miles instead of 300,000. You take a financial hit and you are forced to buy a Camry this time.

B. You start bleeding during bowel movements. You go to the doctor and get poked, prodded, X-ray'd, biopsied, etc. 3 days later you get a call for a consultation. The doctor informs you that you have advanced colon cancer at 45 years old. You have anywhere from 6 months to 5 years left to live. He tells you it's time to get your house in order because you'll be checking out soon. Chemotherapy starts today.

A friend of mine who was a science and health researcher at the University of Chicago, just died this past year of colon cancer at 42. In the midst of the prime of his life, he said goodbye, and left his wife and child behind, wondering what just hit them.

Why do you brush your teeth? Are your teeth falling out right now? For most of us, we do it so we won't need false teeth and Fixodent down the road...right? We want to be able to eat apples. Hey, I agree with that. Natural teeth are great.

But have you ever seen someone who was forced to endure a colonectomy? Someone who now will be spending the rest of their life carrying a bag around?

Incredibly, this is an area where even the staunchest MD's AGREE with us!! Can you believe it? If they knew you had the greatest colon cleanse in the world, I bet they might even refer people to you. NO, I'm not kidding...

This subject is not even up for debate. It's a proven fact. The problem is, most people are not doing anything about it. Please don't be one of them.

****WARNING***** The next section of this email contains graphic
material which may not be suitable for squeamish individuals.

Let's talk stools.

The stool tells you a lot about your colon health. If it's dark brown in color, and it sinks, and it stinks, that's not good. And don't feel bad, that's the way most people are. What you want to see is light brown color, which means it's full of fresh bile from the liver, very mild odor, and a stool that floats. We're talking low-density here folks. The more compaction you have the darker the color and the faster it sinks. Compaction is not good. Also, moving bowels should be SIMPLE. If the veins are popping out of your neck and you feel like your doing the bench press, you NEED to cleanse your colon.

When you do the cleanse, for the first few days....things are a little weird. But you know you're cleansed when you see the above good stuff happening, and you are eliminating at least 2-3 times per day.

Cleansing your colon is a 30-day process. Its also very economical at under $50. You may be very surprised at some of the benefits you will receive besides just losing 1-5 lbs of cr*p from your body and brightening your future health.

People have reported more energy, less allergies, clearing of acne, cessation of migraines, and many other results, not to mention restored regularity. When your body is void of old, poisonous toxins that are constantly being reabsorbed through the colon walls, it can begin to heal again. And when the colon walls are clean, the good nutrients from your food and supplements can be absorbed again. You will be thrilled with the results.

At this point you are either nauseated thinking about what is inside your own colon, or you're ready to do something about cleaning it out.

Want more info? Click here and I'll send it to you, including instructions on how to take it. It is private, all natural, totally safe, inexpensive, and very effective. And yes, I have taken it myself.

Currently available only in the U.S. and Canada. Seeking Distributors to meet high demand.

More Infor [goat.cx]

cmdrtaco@slashdot.org?subject=SendColonCleanseIn fo


Please do not 'reply' to this email as we might not see your message. Please use the links above.

Truth in advertising? (1)

Neil Blender (555885) | more than 10 years ago | (#11056797)

Is that a rhetorical question?

Re:Truth in advertising? (-1, Troll)

Anonymous Coward | more than 10 years ago | (#11056832)

Case Title: Breast feeding and suction can induce orgasm. Reader: 10/04/2004> I have a question pertaining to Breast Feeding, during the time of breast feeding my babies I would frequently have orgasms, is this normal.Now I also have orgasms occassionally when my husband suckles my breasts, and sometimes milk will flow from my nipples, when i am sexually excited or when I have orgasms,Is this also normal. Dr. Lin: 10/04/2004>It is normal to have orgasm by breast sucking when your brain's dopamine nervous system is powerful to stimulate your hypothalamus/pituitary glands for sufficient oxytocin release in conjunction with a flooding of prolactin. Prolactin is the milking hormone and a constriction inducer while oxytocin is the emotion-binding hormone and a dilation inducer to allow milk letdown from the breast (lactiferous) ducts. Both hormones alternate the binding of norepinephrine and epinephrine into the sympathetic nervous alpha-2 and beta receptors for a periodic contraction of the breast ducts and the uterus. Prolactin forces norepinephrine and epinephrine into alpha-2 for constriction while oxytocin deflects norepinephrine and epinephrine into the beta receptors for dilation. Breast sucking sends the nervous reflexion through the vagus sensory nerve to the hypothalamus where the nervous interaction occurs. The vagus nerve (a parasympathetic nerve outflowing from CN-X above the neck to all the internal organs between your neck and pelvic cavity ) also spread down the clitoris where the L1 and L2 sympathetic nervous circuits and S2-S4 parasympathetic nervous circuits are met.

Re:Truth in advertising? (1, Insightful)

Anonymous Coward | more than 10 years ago | (#11056936)

Yeah it is rather Oxymoronic, like military intelligence or Stable Windows.

Multiple Sources is best. (0)

eeg3 (785382) | more than 10 years ago | (#11056805)

If you're naive enough to trust companies, rather than getting multiple reviews of products or services from unaffiliated sources... then you need to get your head checked.

You have to do your own bake off (2, Interesting)

wheelbarrow (811145) | more than 10 years ago | (#11056807)

I work in developing web applications. When choosing server technologies I have learned to conduct my own bake off using the application that will be deployed. Each application is unique. Comparing your custom app to a published bake off is usually an apples to oranges comparison.

This is a different situation. (3, Informative)

oneiros27 (46144) | more than 10 years ago | (#11056978)

You're comparing how products perform under a specific test that you have devised. (which ideally, is similar to your production environment).

Tuning can have a dramatic difference in performance, and unless you're familiar with all of the products involved, it's impossible to get the best performance out of each one.

The original poster is talking about where one of the systems has been modified so it is not a default install, and specifically customized before being sent to the testor, so that they will perform better. (like with ATI's Quake 'optimization' [tech-report.com] ).

As another example, there were some folks trying to get higher rankings in SETI@home [zdnet.com.au] , who would return bogus results -- as that was faster than actually performing the calculations. If someone knows that the results won't be checked for accuracy (or can't), and only for time, they can boost their rankings dramatically.

Well known truth (2, Insightful)

freedom_india (780002) | more than 10 years ago | (#11056810)

It's NOT a surprise at all. Product Reviews by companies about their products are like asking hens to protect their own coop.

The product brochure may lie or hide facts, but the product's technical details book (like the manual for Kyocera VMSE47 Phone) HAS to tell details and truth.

I always make it a practice to read the technical manual of any product i buy over the web. if the company can't provide the manual, then it isn't worth buying.

Well known truth-Rambo-chicken. (0)

Anonymous Coward | more than 10 years ago | (#11056910)

"It's NOT a surprise at all. Product Reviews by companies about their products are like asking hens to protect their own coop."

These hens did pretty good. [imdb.com]

Political Spin. (1, Insightful)

Anonymous Coward | more than 10 years ago | (#11056811)

"To everyone: When does spin doctoring cross the line and become false advertising?"

Been reviewing the previous election, have you?

Consumer Reports pays cash (4, Interesting)

LostCluster (625375) | more than 10 years ago | (#11056814)

Consumer Reports [consumerreports.org] is a such a respected publication because they have strict standards for the products they test. They don't accept items from the product makers, they go out into the marketplace and buy their test subjects using cash whenever possible. (Up until a while ago they even bought cars with cash, until they realized that car dealers began recognizing them as the only people who paid cash for cars, and the IRS requirement of reporting large cash transactions got in their way too.) As a result, their tests are immune to any tweaking...

It'd be nice if the tech publications could afford to do this, because at times they start to resemble the video game websites set up by kids who do it only to get prerelease copies of games for free under the guise of reviewing them. Such kids always have to write glowing reviews of everything they get because as soon as they post a negative review their stream of free stuff grinds to a halt.

Bottom line is that there's a foolproof way of preventing tampering in any review, but it costs money. Any review that involves accepting free stuff compromises the integrity from the start.

Re:Consumer Reports pays cash (1, Insightful)

klausner (92204) | more than 10 years ago | (#11056861)

CU may buy their products on the market to avoid tweaking by the manufacturers, but they then apply a set of biases and prejudices that render their test results problematic.

Re:Consumer Reports pays cash (2, Interesting)

Moofie (22272) | more than 10 years ago | (#11056950)

While I don't disagree with you on the face of it, do you have any substantiation for your argument?

I see how poor CU's testing of bicycles and computers is (two subjects I know rather a lot about), but I've always hoped they were better about washing machines and cars.

Re:Consumer Reports pays cash (0)

Anonymous Coward | more than 10 years ago | (#11056960)

You wouldn't care to elaborate with specific examples? What are they biased for? or against?

What flaws in their methods create the results you say are problematic?

astroturf alert (2, Interesting)

funny-jack (741994) | more than 10 years ago | (#11056990)

What are you, a Sharper Image [sfgate.com] salesman or something?

Re:Consumer Reports pays cash (1)

OAB_X (818333) | more than 10 years ago | (#11056898)

Best of all, Consumer Reports is available in your local library for free.

I am always skeptical of a review where the reviewer at the end says "thanks thermaltake for providing this product", and just before that was a great review explaining how great the product was, overlooking how the thing weighs 800 grams and if you move your case with that hsf on will break your mobo.

I've seen review sites post pictures of sealed boxes just to show that their chip was not specially selected for a cpu review. Anandtech is usually a reliable site for reviews, as with any user done review on tech sites. Tomshardwareguide is biased towards Intel and nVidia (at least historiclaly). Even 3dmark03 benchmarks are "optimized" for nVidia cards, though this is less of a concern now with product patches available.

Re:Consumer Reports pays cash (1)

gcaseye6677 (694805) | more than 10 years ago | (#11056944)

I've noticed Consumer Reports has given glowing reviews to Sears products (Craftsman and Kenmore) in the past and I have purchased some of them based on a CR recommendation, only to be very disappointed in their performance. A vaccum cleaner, drill, refrigerator and leaf blower were all highly recommended and then gave me nothing but problems until a short time later when I replaced them all with products of a different brand. Perhaps Sears somehow sponsors or supports CR? Either that or they just have no realistic way of testing reliability over a period of time.

Re:Consumer Reports pays cash (2, Insightful)

eweu (213081) | more than 10 years ago | (#11057032)

Perhaps Sears somehow sponsors or supports CR? Either that or they just have no realistic way of testing reliability over a period of time.

You can't look at the top rated model and decide that it is the best one long term. The ratings in a CR review represent how the products performed during the test. The ratings do not necessarily represent the best products.

Nearly every CR review has another section that details the reliability of the brands represented in the test over a period of time that they've been testing the products. Also, poorly performing brands will generally be noted as such in the performance ratings if a product scores well.

Re:Consumer Reports pays cash (1)

rackhamh (217889) | more than 10 years ago | (#11056972)

Just because CR doesn't get the product straight from the manufacturer doesn't mean they're honest.

I'm not saying they're *not*, but it's worth noting that there are many ways for manufacturers to curry favor that aren't immediately apparent.

dating (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#11056815)

The best analogy is dating - you put your best foot forward as you meet and greet the opposite sex.

Oh wait, slashdot, what am I thinking...

Blogger LinkMod has some thoughs (0)

Anonymous Coward | more than 10 years ago | (#11056818)

on his Hackers In The Free Market [h4xx.com] entry on his blog. I think it's pretty insightful, but he tends to be a little too libertarian for my tastes to. YMMV.

Truth in general...? Dishonest companies? (0)

Anonymous Coward | more than 10 years ago | (#11056821)

...what can one do if your company is dishonest? This has been happening a lot more lately in my company - billing is highly overflated (seven hours billed for two and a half done), things promised to clients that there is no intention of delivering, etc.

What can be done to show those in charge the error of their ways? As long as the money keeps flowing, it seems the problem will just get worse.

In my company (0, Redundant)

Anonymous Crowhead (577505) | more than 10 years ago | (#11056824)

The marketting people are a pack of liars. In their work and in their life. They have been spouting bs for so long that it has permeated their very being. I don't trust word one out of any of their mouths.

Re:In my company (2, Funny)

vettemph (540399) | more than 10 years ago | (#11057066)

Your an Engineer?

Truth - Advertising? (4, Insightful)

IamGarageGuy 2 (687655) | more than 10 years ago | (#11056826)

As a person who worked in the advertising business side, I can say wholeheartedly that truth in advertising is a complete misnomer. The whole concept of advertising rejects the idea of truth. I don't sound bitter do I?

Well... realistically.... (1)

flappinbooger (574405) | more than 10 years ago | (#11056827)

When does spin doctoring cross the line and become false advertising?

When you get sued or someone dies or both.

Re:Well... realistically.... (1)

RealityMogul (663835) | more than 10 years ago | (#11057070)

Somebody could die the safest car in the world, and
it would still be the safest car in the world as long as more people died in other cars.

Simple Answer (1)

nicotinix (648645) | more than 10 years ago | (#11056830)

When does spin doctoring cross the line and become false advertising?

When you get caught!

Truth is never in advertising. (2, Interesting)

Telastyn (206146) | more than 10 years ago | (#11056834)

Similar to the game journalism post earlier today, if you want honest impressions, you're going to get them from your buddy saying the stuff rocks/sucks than from any sort of review/preview/ad.

Require a loaner to test... (1)

Saeed al-Sahaf (665390) | more than 10 years ago | (#11056835)

When we are in the market for hardware of a certain value, we require a loan of the actual device to test ourselves.

Re:Require a loaner to test... (0)

Anonymous Coward | more than 10 years ago | (#11057048)

I wouldn't lend anything to a paki twat like you, I'd never get it back. Or if I did it would smell.

Bill Hicks (RIP) said it all.. (5, Funny)

little_fluffy_clouds (441841) | more than 10 years ago | (#11056838)

"By the way if anyone here is in advertising or marketing... kill yourself. No, no, no it's just a little thought. I'm just trying to plant seeds. Maybe one day, they'll take root - I don't know. You try, you do what you can. Kill yourself.

Seriously though, if you are, do. Aaah, no really, there's no rationalisation for what you do and you are Satan's little helpers, Okay - kill yourself - seriously. You are the ruiner of all things good, seriously. No this is not a joke, you're going, "there's going to be a joke coming," there's no fucking joke coming. You are Satan's spawn filling the world with bile and garbage. You are fucked and you are fucking us. Kill yourself. It's the only way to save your fucking soul, kill yourself. Planting seeds.

I know all the marketing people are going, "he's doing a joke... there's no joke here whatsoever. Suck a tail-pipe, fucking hang yourself, borrow a gun from a Yank friend - I don't care how you do it. Rid the world of your evil fucking machinations. I know what all the marketing people are thinking right now too, "Oh, you know what Bill's doing, he's going for that anti-marketing dollar. That's a good market, he's very smart." Oh man, I am not doing that. You fucking evil scumbags! "Ooh, you know what Bill's doing now, he's going for the righteous indignation dollar. That's a big dollar. A lot of people are feeling that indignation. We've done research - huge market. He's doing a good thing." Godammit, I'm not doing that, you scum-bags!

Quit putting a godamm dollar sign on every fucking thing on this planet!

"Ooh, the anger dollar. Huge. Huge in times of recession. Giant market, Bill's very bright to do that." God, I'm just caught in a fucking web! "Ooh the trapped dollar, big dollar, huge dollar. Good market - look at our research. We see that many people feel trapped. If we play to that and then separate them into the trapped dollar..." How do you live like that? And I bet you sleep like fucking babies at night, don't you?"

[We miss you, Bill.]

Re:Bill Hicks (RIP) said it all.. (1)

Martin Blank (154261) | more than 10 years ago | (#11057055)

Bill Hicks sometimes reminded me of a lower-volume version of Sam Kinison. Go back and re-read that, but put Kinison's voice (and about two dozen more expletives) in your mind instead.

We miss Sam, too.

Analyze, analyze, analyze (5, Insightful)

rackhamh (217889) | more than 10 years ago | (#11056839)

This isn't just a phenomenon in the IT arena. Have a look at medical journals some time... You have to be VERY careful when putting stock in the findings of studies -- the first thing to check is who *funded* the study.

I think it's just a fact of life: everybody wants their product to be seen in the best light, and to sell well (in the case of commodities or services).

That's why Amazon.com has reader reviews, sites like epinions.com exist, and Slashdot has moderator points. It's also why there are hardware review sites -- we can't just trust the manufacturer's PR now, can we?

So, people may be inherentely biased and often untruthful, but with proper monitoring (read: community involvement), the truth will out.

Re:Analyze, analyze, analyze (1, Insightful)

Anonymous Coward | more than 10 years ago | (#11056916)

yeah, and remember when Amazon had the glitch which showed how many anonymous reviewers were the book authors?

Astroturfing is even more common on epinions (where it might not just be the first review you can't trust).

Oh, and nobody has submitted a story to Slashdot pointing to a poorly written essay they wrote themselves. No, that would never fucking happen.

Re:Analyze, analyze, analyze (2, Insightful)

FiReaNGeL (312636) | more than 10 years ago | (#11057025)

Totally agree, and it even extend to scientific journals, not just medical ones. I've even seen it happen... 'ignoring' stuff that doesn't back up your hypothesis, keeping only the 'good' results to do statistical analysis, choosing a very specific concentration on a dose-response analysis because it's the only one where its 'working'... its sad, but not generalized. I guess everywhere humans are involved and under pressure (for money or papers in the case of science), cheating WILL happen, no matter what kind of sanction you put against it.

My Sig... (1)

TrollBridge (550878) | more than 10 years ago | (#11056840)

...sums up this article perfectly.


muntjac (805565) | more than 10 years ago | (#11056842)

companies are (nearly) ALWAYS unethical if it will make more money.

When does spin doctoring cross the line (1)

SoSueMe (263478) | more than 10 years ago | (#11056855)

"When does spin doctoring cross the line and become false advertising?"

When your product fails to perform, as advertised, in the environment you specify.
Most of us call this lying to your customers.

Regulations are for the weak! (1)

rastin (727137) | more than 10 years ago | (#11056858)

OK, I really don't mean that but its the attitude you see in a lot of places. My last job was for a mega retailer, we had a strict policy about transmitting and storing any data with customer's credit cards and such, it has to be encrypted. But out auditors deliberatly ignored the fact that if you went online and bought a gift card the data between the client browser and web server were secure but a trickle feed between 2 internal systems sent the same data back and forth across the internal network as clear text. The attitude was that if you could nod your head and say "We use encryption" then you pass the test.

Commendable (1)

InternationalCow (681980) | more than 10 years ago | (#11056860)

... but naive.. Come on, what were you smoking? Of course the benchmarks/testing/what have you will be done in such a way as to out the product to be sold in the best possible position. Your question is naive. Even us scientists do this when providing paper plans for our bosses. We paint the best possible picture, do serious window-dressing and interpret our results in the most optimistic manner compatible with science. If you think that an advertising campaign will feature objective (if such a thing exists in benchmarking) performance comparison, you really need to get a reality check. Or, if your conscience is giving you problems, find a profession that doesn't require a conscience. I hear that law is rather profitable these days.

Common phenomenon (1)

Almond Paste (838493) | more than 10 years ago | (#11056862)

Companies' own figures and test results are always generous and I believe that customers are aware of this. It just like with "60 minutes"..

Consumer audio (4, Interesting)

theLOUDroom (556455) | more than 10 years ago | (#11056863)

To everyone: When does spin doctoring cross the line and become false advertising?"

In the consumer audio market, that's when.

From over-unity speakers (200W watts output from a 10W wall-wart), to "better-sounding" fiber optic cable, no claim seems too outrageous or fraudulent for a great many consumer audio manufacturers.

As an engineer who loves audio, it drives me nuts to see the bullshit that is constantly perpetrated in that market.

I'm sure there are tons of slashdotters who can post examples of incredibly unprofessional and possibly fraudulent specmanship in this arena.

Re:Consumer audio (1)

KD5YPT (714783) | more than 10 years ago | (#11057017)

200W of power from 10W... DANG! They can solve the world's energy crisis!

Reminds me of marketing... (1)

Lead Butthead (321013) | more than 10 years ago | (#11056865)

Some years ago when I was working for a certain SCSI RAID HBA company (that shall be referred to as company M) we were shock to find out a certain OTHER company (that shall be referred to company D) is advertising that their SCSI RAID HBA out performs ours by substantial margine in LARGE BOLD letters.

When we took a closer look at the disclosure (in fine print) it states: Company M HBA tested in single threaded mode (READ: Tag Queuing Disable.)

That's when I lost all respect for marketing people...

Re:Reminds me of marketing... (1)

PitaBred (632671) | more than 10 years ago | (#11056907)

You think marketing people knew how to disable Tag Queueing? They had some help from a geek of some sort... let's lynch him!

Re:Reminds me of marketing... (1)

rduke15 (721841) | more than 10 years ago | (#11057002)

So how could you counter this very misleading advertizing?

With more advertizing, I would guess.

So you see, misleading advertizing is not only good for the product (and it may even end up backfiring on the product) but it is certainly good for the advertizing business. That is what counts for advertizers, after all.

goddaman managers (1)

ShakuniMama (785662) | more than 10 years ago | (#11056867)

Well its funny because I was having this discussion with my co-worker 10 minutes ago about how many people publish data in scientific journals that is at best massaged to an exten that it can't be reproduced. The problem, sadly isn't just in industries but has perpretaded academia too and probably has been there for many decades. I wouldn't know because I'm still young and naive:). But seriuosly, I think most of the corporate problems exist due to managers who get paid insanely high salaries and need to justify and maintain them by making engineers work like slaves and give them crap about TPS reports and fucking morale and what not. C'mon say it with me... get rid of managers, lawyers, insurance companies. What do you know, insane amounts of coffee while listening to Rage against the machine does make you hate the man. Aaaaaah

Oh, _I_ put no faith in the results (2, Interesting)

lakeland (218447) | more than 10 years ago | (#11056869)

But the fancy numbers aren't for me, they're for PHBs that like to see lots of impressive numbers -- after all, the other product has them so if this one doesn't...

Looking at computer specs lately I'm beginning to think the principal point of them is to bulk out the specs -- make it look like it has lots of features, and the actual content of the specs is irrelevant.

i work (0)

Anonymous Coward | more than 10 years ago | (#11056872)

for a major worldwide advertising company that happens to do ads for a major hardware manufacturer. As the "IT" person at this location, i am often asked to come in and help to "explain" what such and such a piece of hardware does.

It never ceases to amaze me that these people (paid quite well to "understand" us (IT folk) have absolutely, completely no clue about any of the products they are trying to sell, let alone about more complicated then "this is a laptop, that is a desktop".

Perhaps a better question (1)

CokeJunky (51666) | more than 10 years ago | (#11056887)

is to ask who isn't doing this. Is there anyone reading this that works for a company that doesn't use this kind of practise and instead relies on the quality of the product to stand on it's own?

Thats what I thought. If there was, that company can't expect to last long in competition. This world is plainly insane, but so are all of us willing to buy into it, so I suppose it doesn't matter.

Scares Me (1)

mugnyte (203225) | more than 10 years ago | (#11056892)

Unlike many popular forms of advertising, I don't trust testimonials. When a piece of equipment is reviewed, I judge the review by it's source, since perhaps as a tech I'm a bit happier with a "clumsy UI" than with sheer abilities of, say, hardware.

So I look to Toms, [H], Ars for reviews by people who seem to have similar knowledge as myself. Then, when tests are formed, I don't trust just one benchmark, nor just one test or review.

If a company is going to game the testing, I'm disappointed. This lowers the confidence I have in these tests. Since blatent ads and testimonials turn me off, where else do it look? I'll have to just rely on the repeatability of any review's scores. This usually uncovers companies that try to dupe the reviewer.

Scares Me-Ebert gives Slashdot, two thumbs up. (0)

Anonymous Coward | more than 10 years ago | (#11057053)

"Unlike many popular forms of advertising, I don't trust testimonials. "

Hi. I'm AC, and I give Slashdot two thumbs up. The staff is knowledgable and friendly. The moderation is always on the mark, and the community is insightful, and often funny. And with Slashdot's preview feature I can see upcoming stories before most of the audiance. A tech site that is technically above, viewable on all standard browsers. I highly recommend Slashdot, for all your information needs.

Sun cheated on Java benchmarks (2, Interesting)

SimHacker (180785) | more than 10 years ago | (#11056893)

Sun responds to Java accusations [com.com]

Sun Microsystems (SUNW) shrugged off accusations today of unfairly reporting test scores for the beta version of one of its Java compilers.

Pendragon Software yesterday said that Sun, using Pendragon's CaffeineMark benchmarking tool, inaccurately inflated the test results of the Solaris 2.6 just-in-time Java compiler by optimizing the compiler specifically for that test. Solaris is Sun's version of the Unix operating system.

Sun responded by calling such optimization standard practice.

"The idea is that you want people to optimize for the benchmark," said Brian Croll, director of marketing for Sun's Solaris products. "We'll do everything in our power to do really well on all the benchmarks we get our hands on."

A benchmark is a battery of tests that gauges the speed and performance of software running in various configurations. Several developers have created Java benchmarks; CaffeineMark, which Croll called "the best benchmark we've got," is available free off the Web.

But how much optimization is fair play? Pendragon president Ivan Phillips contended Sun inflated the test results of the Solaris 2.6 just-in-time compiler by lifting code from CaffeineMark and inserting it into the compiler.

"The logic test is contained in the 'logicatom.class' file, and almost 50 percent of that file appeared in the compiler," he said. "The probability that this code made its way there accidentally is infinitesimal."

Reusing such a large chunk of specific code risks diverting too much of the compiler's resources, resulting in lower performance once the compiler is deployed in the real world, Phillips added.

Croll denied that Sun used CaffeineMark code but said the company "optimized around it." It will be difficult to determine who is correct, given that the beta compiler in question is no longer available. Croll stressed that the compiler is designed to perform well on a benchmark because that's what determines good real-world performance.

"If certain things happen frequently in a benchmark, you want to make sure you handle them well," he said. "If it turns out the benchmark doesn't truly represent true application performance, you need to evolve the benchmark."

The charges come at a time when Sun and Microsoft are entangled in tit-for-tat lawsuits over Microsoft's use of Java in its Internet Explorer 4.0 browser.

In an October 20 press release, Sun bragged that Solaris had the "world's fastest Java performance" and ran Java applications 50 percent faster than rival operating system Windows NT. After taking issue with Sun's test results, Pendragon said it asked Sun to retract its claims and remove the compiler from its Web site.

Sun removed the entire JDK 1.1.4 for Solaris on October 29 because the beta evaluation period ended, according to Croll. The company didn't take down the press release or rescind its claims, however, and Phillips responded yesterday by publishing his accusations.

Pendragon doesn't usually double-check testers' CaffeineMark scores. But when it saw Sun's results--the Solaris compiler hit a score of 1.4 million on the "logic" test, while the previous high for that test was 22,000--the software firm decided to investigate, fearing that CaffeineMark contained a bug.

If Sun indeed took deliberate steps to skew its results, Phillips was surprised at the lack of subtlety.

"If a company really wanted to conceal what they were doing, they could do a better job," he said.

Red-Handed, Red-Faced, Red Alert (2, Informative)

SimHacker (180785) | more than 10 years ago | (#11056946)

Red-Handed, Red-Faced, Red Alert [informationweek.com]

Developer Quote Of The Week: "What we do is, given a benchmark, we try to do as well as we can on it, and make sure that our system is the fastest benchmark -- I mean, fastest system -- in the world." -- Brian Croll, Sun Microsystems' director of marketing for Solaris

Two weeks ago, Sun Microsystems got caught with its hand in the benchmarking cookie jar. Or did it? Depending on your point of view, Sun either grossly misrepresented the performance of its Solaris Java just-in-time compiler by fooling Pendragon Software's CaffeineMark performance test, o r Sun proved the CaffeineMark is not an acceptable measure of Java compiler performance.

For those who may have missed it, here's the background: In a Nov. 4 press release, Ivan Phillips, president of Pendragon Software, in Libertyville, Ill., a developer of software for personal digital assistants, accused Sun of engineering its new Java compiler to trick the CaffeineMark into reporting higher performance results.

When Sun's compiler detected a block of 600 bytecodes unique to the CaffeineMark (a technique known as pattern matching), the compiler bypassed data processing, and instead returned a value expected by the benchmark. This fooled the test into reporting performance results 300 times faster than the compiler would deliver in real-world use. Third-party developers subsequently validated Phillips' assertion. Interestingly, when Pendragon's engineers altered the test to appear different to Sun's compiler, the compiler's branching was short-circuited, and its performance plummeted. Java compilers under Windows 95, Windows NT, and the Mac OS delivered uniform results under both the original and altered tests.

Sun officials initially admitted no wrongdoing, and were quick to point out that optimizing software to improve benchmark scores is an accepted practice among computer technology vendors. "People are optimizing against the benchmark," says Brian Croll, Sun's director of marketing for Solaris.

Further, Croll maintained that the aberrant results indicate a fundamental flaw in Pendragon's benchmark suite, and do not represent any impropriety by Sun. "I don't know how valid the [CaffeineMark] is," Croll said. Then last week, during a day-long media briefing at Sun's Mountain View, Calif., headquarters, Sun officials updated their explanation of events. SunSoft president Janpieter Scheerder said the company was not trying "to do anything malicious;" rather, Sun engineers simply "optimized too much."

A Sun spokesperson at the event blamed the incident on human error, and said an engineering prototype somehow found its way through Sun's rigorous (you would think) development and quality assurance processes, and onto the Web, with documentation, and overblown press release in tow.

What if Pendragon officials had not discovered Sun's alleged trickery? What if Sun engineers tweaked their compiler to only improve its score 10-fold, instead of the eye-popping 300-fold increase that flagged Pendragon officials?

Sun's PR machine had already posted a press release, in which they touted their "new Web-enhanced Solaris operating environment" as delivering "the world's fastest Java technology performance." The release also claimed Solaris' compiler was 50% faster than the best Windows NT score, and cited the CaffeineMark as proof.

If Pendragon officials had not discovered the ruse, Sun's formidable sales and marketing machine would now be steam-rolling press and IT decision-makers alike, trumpeting Solaris' performance advantage over Microsoft's Window s NT, waving Sun's illicitly obtained CaffeineMark results as evidence in hand.

"Any benchmark, no matter what its original purpose, is subject to use as 'benchmarketing,'" says Larry Gray, board member of the Standard Performance Evaluation Corp. (SPEC), in Manassas, Va., a consortium that administers many well- known benchmarks. "I'd guess maybe 20% to 30% of the tricks vendors pull to make the benchmarks run faster are never seen by real-world applications." (Gray is also a product manager with Hewlett-Packard's server marketing group in Cupertino, Calif.)

Sun not only spoofed the CaffeineMark; it attempted to spoof developers and IT organizations around the world. Worse, in its effort to discredit Pendragon's CaffeineMark algorithms, the company demonstrated it was, initially at least, willing to cast a shadow on virtually all CaffeineMark results published to date.

It's common practice for technology vendors to optimize their hardware or software to perform better under benchma rk testing. In fact, benchmark suite vendors and consortiums encourage the practice, believing legitimate optimizations lead to improved real-world performance for IT users. "If the benchmarks are any good, and a vendor optimizes their product to improve their scores, they've also found a way to make their product run better," explains SPEC's Gray.

But there are times-and this is one of them-when an optimization is more correctly defined as cheating. "If the thing recognizes cases that look like the particular benchmark, the word 'cheating' is perfectly justified," says Thomas Plum, president of Plum Hall Inc., in Kamuela, Hawaii, a developer of C/C++ testing and conformance tools, and the chairman of the International Standards Organization WG21 C++ standards committee. "Most [experts] consider [pattern matching] as going too far."

Such cheats have no purpose in the never-ending benchmarking and optimization cycle, Plum says, and serve only to imply a particular technology will delive r performance it cannot conceivably provide in real-world systems.

If the evidence overwhelmingly indicates Sun cheated this time, it's safe to assume it may do it again. It's also safe to assume that, next time, the company might not get caught. For their part, company officials continue to deny any intent to swindle the public, but the events of the last two weeks should, at a minimum, concern every consumer of benchmarking data.

Developers, be forewarned: Just as a virus can slip through the best antivirus defenses, the best benchmarks can and are being spoofed with alarming regularity. Because most popular benchmarks are managed by industry consortiums or private corporations, companies with their hands in the cookie jar are rarely flogged in public, and the practices remain benchmarking's dirty little secret.

Sun Called on Java Claims (2, Informative)

SimHacker (180785) | more than 10 years ago | (#11056984)

Sun Called on Java Claims [wired.com]

Tweaking Java test?: Sun Microsystems has been accused of manipulating Java benchmark software and using the results to state that its Solaris "runs Java applications 50 percent faster than Windows NT." Pendragon Software, maker of the benchmark software CaffeineMark, has put out a press release that claims Sun found a way to cheat on the benchmark tests, and then advertised the bogus scores. Sun has since removed the Java compiler from its download page, Pendragon says, but the original press release remains on the Sun site.

Sun admits Java testing error [com.com]

Sun Microsystems (SUNW) today conceded errors in the results of recent tests involving its Java programming language.

The company erred in not admitting that it matched code from a Java benchmark tool for one of its own Java compilers, Sun Software president Janpieter Scheerder said today. A benchmark is a battery of tests that measures the speed and performance of software running in various configurations.

Kicking off the "Inside Sun Software Day," Scheerder began his remarks with a mea culpa for Sun's actions, revealed last week in a report by CNET's NEWS.COM. At that time, Pendragon Software, makers of the CaffeineMark Java benchmark test, accused Sun of taking code from the CaffeineMark software and adding it to a beta version of the Solaris 2.6 Java just-in-time compiler. CaffeineMark is one of several developers that have created Java benchmarks.

Last week, Brian Croll--director of product marketing for Solaris, Sun's flavor of the Unix operating system--denied that Sun lifted the code. Today, however, Scheerder made it clear that Sun had made a big mistake.

"Nobody was trying to do anything malicious," Scheerder said. "We just optimized [the Solaris Java compiler] too much."

A Sun public relations manager called the episode a "big-time organizational breakdown" in which an engineering prototype that was never meant to go public was posted on the Web with all attendant documentation, along with a press release that touted the software's performance. Sun has also posted an explanation on its Web site.

"Sun committed an unintentional error when we published Java performance numbers for an engineering prototype that included code that specifically looked for a piece of code in the Caffeinemark 3.0 benchmark," according to a company statement.

In a release dated October 20, Sun bragged that, according to the CaffeineMark 3.0 test, Solaris 2.6 ran Java applications 50 percent faster than Windows NT. But it neglected to say that it had set the compiler to look specifically for a chunk of code from CaffeineMark. Reusing such a large chunk of specific code risks diverting too much of the compiler's resources, resulting in lower performance once the compiler is deployed in the real world, said Ivan Phillips, president of Pendragon.

After taking issue with Sun's test results, Phillips said he asked Sun to retract its claims and remove the compiler from its Web site. As of last week, Sun had not retracted its claims, so Phillips went public with his accusations.

Scheerder stressed today that the compiler, which was part of the Solaris 2.6 Java Development Kit 1.1.4 beta, was not shipping product. The company pulled it from its Web site soon after Phillips contacted them last month.

The news comes four days before the International Organization for Standardization (ISO) decides if Sun is qualified to be the official submitter of Java technology if and when Java becomes an international standard.

The official submitter has the responsibility to gather industry consensus and present it to the ISO's technical committee for consideration. There is some concern that Sun, which owns Java, might not be a neutral submitter. So far, 11 countries have voted yes on Sun's bid and one country--the United States--has voted no. A total of 27 countries are scheduled to vote by Friday.

when does it cross the line? (0, Redundant)

the_ambient_one (415217) | more than 10 years ago | (#11056909)

To everyone: When does spin doctoring cross the line and become false advertising?
When you get caught.

When? Instantly. There is no gray area. (2, Insightful)

mooncaine (778422) | more than 10 years ago | (#11056919)

"To everyone: When does spin doctoring cross the line and become false advertising?"

Instantly. There is no gray area between honesty and dishonesty. You either tell the truth, or you tell a lie. Your company either attempts to subvert tests [i.e., lies], or it doesn't [i.e., does not attempt to lie]. No ambiguity exists in this case.

Your question reminds me of a question posed on the cover of a national "news" magazine in the wake of revelations that the New York Times had published falsified news reports. Their question was, to paraphrase: "Does this signal a new standard in journalism?". Of course it signaled no such thing; it only signaled that some publications, or at least reporters writing for them, were willing to be dishonest and to print lies. I wonder if the author of that question was perhaps a bit hopeful that, yes, this event did signal a relaxing of ethics?

Plenty of gray (1)

slashing1 (818431) | more than 10 years ago | (#11057022)

I wish the world were so simple. Is it lying if you don't advertise that your product does poorly on a particular benchmark? Is it lying if you don't advertise that your product will fail under a condition that occurs, on average, once every 20 years?

I worked in a test lab... (4, Interesting)

aiken_d (127097) | more than 10 years ago | (#11056925)

...for about 5 years in the mid to late 90's. I started doing the testing on basic network equipment and graduated over time to oversee the testing methodology for every product comparison we ran.

I can tell you that, if the testers themselves are competent, it's a moot point. For instance, when testing server hardware by using a database application, I always insisted that the databases be identical and configured as identically as possible. Normal stumbling blocks were issues with stock disk sizes, but we always ensured that RAID configurations were as similar as possible within the realm of reason.

Testing is an art form. It requires a thorough and repeatable plan as well as a good bit of knowledge about real world usage of equipment and software (would it be realistic to enable a non-battery backed write cache on a raid controller in a database application?)

I can say that many, many vendors attempted to put one over on us. And it's entirely possible that I missed some of them, and they benefitted because of it. However, in general, professional test procedures should expose and nullify any sort of vendor tweakage of equipment or software.

Key principles for good testing:
- Set any basic configuration to manufacturer's public recommendations

- Don't let vendor representatives touch anything. If they need to send someone into the lab, allow them to recommend changes, and document all of those for later review / revocation

- If third party hardware/software is involved in a test, use the third party as a sounding board. If you're testing a layer 3 switch using streaming media, talk to the streaming media provider about realistic stream rates and usage patterns.

- If at all possible, wipe and reload vendor equipment and software. You should be looking at the setup process anyway, so that helps the test as well as helping to prevent shenanigans.

In short, good test procedures prevent, or at least mitigate, the kind of abuse in question. And, as consimers of reviews and tests, it's in all of our best interests to get educated and develop opinions about the competence, thoroughness, and honesty of any souce.


When does it cross the line? (0)

Anonymous Coward | more than 10 years ago | (#11056927)

I tell you when - when you reach down that hooker's pants and, "OH MY GOD YOU'RE A MAN!" - that's crossing the line.

Depends. (0)

Anonymous Coward | more than 10 years ago | (#11057069)

If she looks like this [shemaleandtranny.com] , I just might hit it.
Or how about her? [shemalez.net]

would you believe aircraft and spacecraft, even? (0)

Anonymous Coward | more than 10 years ago | (#11056928)

I was an engineer doing testing of composite materials and structures for aircraft and spacecraft for ten years, and the clients were all very clear about wanting the testing done and the data reported in the most optimistic light.

One REALLY BIG COMPANY ate shit over this and lost a billion dollar contract for jet aircraft engines, resulting in massive layoffs that were, remarkably, not reported as the result of management ineptitude.

Surprise! Surprise!

Dilbert (3, Funny)

behindthewall (231520) | more than 10 years ago | (#11056929)

Reminds me of the strips where Wally had to impersonate the demo.

When the image got fuzzy, they tried a razor.

Not in IT, but... (2, Insightful)

laughing rabbit (216615) | more than 10 years ago | (#11056934)

... when I worked for a German owned plumbing fixture manufaturer's US subsidary, we had to have all faucets certified for lead contanimation (leaching from the solder and brass compounds). As it turned out, a lot of what we were already selling in the US market would not come close to passing. The Fatherland offered to send faucets that were garanteed to pass. All we had to do was tell them what levels that they needed to meet for a particular model (has a lot to do with the length of the flow chamber).

They seemed quite upset that the units had to be pulled at random from stock. Maybe they were just to use to cheating.

Fairly Common (1)

lesv (258710) | more than 10 years ago | (#11056945)

I can't remember a time where I haven't been asked to tweak things a bit for a better showing in a review, performance test, or bake off. As an engineer, I like things objective, and let the chips fall where they lie. But in business good results can be life or death for the company. Ethics are a very complex subject. Especially when it involves your livelihood.

Opine (2, Interesting)

T3kno (51315) | more than 10 years ago | (#11056952)

To the IT professionals: How much faith do you put in these sorts of publications and their 'bake offs'?

Absolutely none, I rely solely on product packaging.

Seriously though, I hold the belief that all sales and marketing folk are born liars and will never change. I purchase solely on word of mouth (from people I trust) and my past experience with a particular brand/manufacturer. I am the person that advertisers hate because I sit in front of the TV and explain to my wife exactly which mind fucts the advertiser is utilizing. Sales and Marketing (S&M how ironic) folk are beneath lawyers, politicians and criminals in my book.

Re:Opine (0)

Anonymous Coward | more than 10 years ago | (#11057062)

I am the person that advertisers hate because I sit in front of the TV and explain to my wife exactly which mind fucts the advertiser is utilizing.

Advertisers? No offense, but your wife's probably not too crazy about you, either...

Sad But True (1)

RAMMS+EIN (578166) | more than 10 years ago | (#11056961)

You are talking about cheating here, and representing things better than they are, but it's not uncommon to find flat out lies in advertisement.

I've seen advertisement claiming that the product was the cheapest or only, where I clearly knew different.

It happens outside advertizing as well. Steve Ballmer has made some very untrue statements, and so have certain people in the previous administration.

The worst thing is that people often believe the lies rather than the truth. It's like in Zen and the Art of Motorcycle Maintenance: "The truth knocks on the door and you say, 'Go away, I'm looking for the truth,' and so it goes away.

The Scientific Method (1)

Rheagar (556811) | more than 10 years ago | (#11056964)

Here is how you can get good results from Bake Offs:
Outline what you are testing, how the tests were set up, and let the results point you to a conclusion.
Use the conclusion to make other predictions. Test the predictions with another experiment.
If a user sets the same problem up on their own, they should see the same results. If not, then something was cooked funny.

Personally, I look to independent sources for reviews. I would not trust claims made by a manufacturer.

Are you kidding me? (2, Informative)

ip_freely_2000 (577249) | more than 10 years ago | (#11056965)

You must be new. I've been working for in high-tech for about 20 years now for various companies, and I would not want my products to be evaluated on a level playing field. I will put in any tweak necessary to win a comparison. This is not kindergarten...fair is nice, but I know my competitors are doing the same thing. And the old college try does not pay a very good Christmas bonus.

Truth in advertising works. (3, Interesting)

centipetalforce (793178) | more than 10 years ago | (#11056974)

In the pay per click world of google adwords (those text ads you see when you search) I advertise a free service. But since this free service is bundled with other nonfree services I put the prices on the ad itself.
So although they may be looking for something free, I don't pay for the click unless they know they're going to pay *something*, the visitor is better informed, and I get a higher conversion rate from the qualified traffic.
So although this may not be on the exact topic of yours, I submit that honesty in advertising works, especially when you pay for performance.

Read carefully (3, Insightful)

westendgirl (680185) | more than 10 years ago | (#11057004)

As long as the company explains the conditions under which its product achieved certain standards, the company is not lying. In most cases, the marketing materials explain the test scenario or the environment of the customer who achieved the results.

Marketing materials do not set out the faults of the product. This is not the role of marketing. Marketing aims to connect buyers to sellers. Providing information about faults does not help to make that connection. Also, many of the "tests" cited by marketers are labeled with titles such as, "Customer Success Story". This should be a clue that the material will not detail unsuccessful characteristics of the product.

Finally, marketers in most companies are not technical experts. They have to rely on the information provided by engineers and programmers. Many companies avoid ever telling the marketing department anything negative. As a result, in many cases, marketers aren't lying when they make claims -- they're explaining what they were told. Many of these marketers, especially the ones writing up collateral, are junior, new to the company, or even working on contract, so they don't have the depth of knowledge to tell that they've been given misleading information. Other people in the company sometimes lie to the marketers. It's not always black and white. (Not that all marketers tell the truth, of course.)

Benchmarking Tuning: Just as bad as Karma Whoring (4, Funny)

Nova Express (100383) | more than 10 years ago | (#11057006)

The sort of people who would tune their software for a specific benchmark are the same sort of people who would karma whore here on Slashdot by throwing off-topic lines with guaranteed Slashdot appeal.

And you know who else hates that type of benchmarking whoring? Linus Torvalds, that's who! Linus would never stoop to such a thing, because Linus is a great guy!

And you know who else would never do it? Apple Computer, the people who make the greatest computers in the world! They would never stoop to rigging benchmarks!

Or karma whoring.

Its an (2, Funny)

northcat (827059) | more than 10 years ago | (#11057008)

Truthful Advetising is an oxymoron.

I worked for HP.... (3, Informative)

Anonymous Coward | more than 10 years ago | (#11057010)

we had a bake-off on one of our products...i was called in when the results didnt meet what management were expecting. after adding cpus(!) to the product under test (without the knowledge of the tester) we finally got a result which beat our competition. this was in 2001. i was later first on the chopping block in 2002 after i noted at a meeting that we should not try to publicize the results too much since it might backfire. the VP who canned me noted that if we got results we should publicize them as much as possible and i was an "impediment to future marketing capaigns"). i got an above average severance package tho so i guess they paid me off to leave quietly. ironically HP's results got beaten by IBM which simply threw money at the problem 4 months later and won.

Doesn't stop there... (2, Interesting)

DaFallus (805248) | more than 10 years ago | (#11057015)

In my experience, the lies don't stop with the advertising. My bosses are both salesmen. The only thing they do at the small company that I work at is sell our software, and they'll tell anyone whatever it is they want to hear so that they'll buy it. But I've noticed that this is definately not the end of their dishonesty. They treat their staff, me included, just like a buyer. They promise us stuff like compensation for working weekends, etc, but then just like our software, they fail to deliver.

I have a huge problem with salespeople and advertisements specifically because of my bosses. IMHO everyone who works in sales is nothing but a glorified 419 scam artist. Politics really aren't any different either.

That is why I like open source so much. Almost everything is free, so there's no reason to lie.

Car industry.... (0)

Anonymous Coward | more than 10 years ago | (#11057020)

I'm not going to say who, but I know that some cars that we are getting ready for the Detroit Auto show right now have been given multiple coats of paint as well as a double clear coat. They paint the brake discs too...

Devil's in the details (1)

Dark Coder (66759) | more than 10 years ago | (#11057021)

Or, for a careless end-user, is that the devil sold his/her soul and make a killing?

Any which way you dice or slice it, it boils down to the "trial" run to overcome the buyer's skepticisms. Be that it may, a trial balloon, trial-by-fire, trial-by-jury, ... whatever.

In the case of Internet-based products, it takes a true network engineer to understand the fine subtleties between UDP throughput and TCP throughput (as well as any other application/presentation/session layered throughput combinations) and to procure an actual Internet traffic composition when placing the DUT (device under test) into operational mode (and under duress, no puns intended).

For an average I&T guy, the best way to evaluate an Internet product is to ask for a 30-day trial period and dedicate a portion of your corporate network. I'd say, sic it to the development group (hey, I'm one of them too!) as they should be focusing on their coding/design effort, not reading Slashdot :-P

When such a DUT chokes under nominal traffic scenario despite publicized (and ominously rosy) one-sided benchmark, it usually a strong indicator that the DUT is a poor design and "SCREAMS" stay away.

30-day trial is your best friend. But the enemy of your enemy (the Devil's advocate) is also your friend.

'derated' AC to DC switching power supplies (1)

vettemph (540399) | more than 10 years ago | (#11057026)

Power supply makers always seem to derate their stuff. for instance, a 75 watt supply rated for use up to 60 Deg C may lose up to 40% of its output power from 50c to 60c. Thats 47 watts at 59c. Funny thing is, most manufactures will tell you this out right and provide the derating curve on their web site. others don't mention it.

It's a pretty well known practice (1)

Remillard (67835) | more than 10 years ago | (#11057029)

It might be a problem if it were not so well known. However, it's very common to test a part under optimal conditions because numbers sell. A 1.2 GHz capable part sounds better than a 1 GHz capable part.

That being said, anyone who takes the numbers completely at face value gets what they deserve. We have an entire facility that acquires parts we are considering using, and "qualifies" them, and then publishes their findings in the company's part database. So somewhere between the vendor data sheet and the homespun one, the truth lies.

Advertising Claims (2, Informative)

Smiffa2001 (823436) | more than 10 years ago | (#11057034)

I work in the forklift truck industry and understandably the advertising there revolves around saving storage space, increasing efficiency per handling transaction which both in turn saves money.

The upshot of all of this is that when it comes to it, a prospective customer will usually say "prove it" and you well, have to. I for one took great pride in being part of the tech/development/demonstration team in that I had a say on what went into the sales literature as I'd often be the one proving it...

Needless to say, as it was MY arse on the line, I managed to complete demonstrations without any screw-ups.

I completely disregard benchmarks (1)

necro2607 (771790) | more than 10 years ago | (#11057035)

I've always completely disregarded benchmarks, etc. other than those I've run myself.

It's kind of like Microsoft's BS-filled "Linux TCO vs. Windows TCO" ads here on slashdot. Sure, maybe Windows Server 2k3 is cheaper to operate than linux (What a bloody joke) in Microsoft's excessively convoluted idea of how servers/whatever might be run, but chances are extremely high that Microsoft has no damned clue about how my servers are run, what content they serve, etc. etc., not to mention the fact that there's rarely a way for individuals to verify the accuracy of the benchmarks in the first place.

Like the article already expresses, benchmarks and "tests" essentially always treat the competing products very differently, placing their own product(s) in far more favorable environments to skew results to their advantage. It's plainly obvious, and personally I would be amazed to have anyone disagree with what I'm saying.

As a fairly well educated and "aware consumer" (or something), I can assure you I don't really care what your company tries to tell me, I'll go by my friends' experiences with the company's products, and entirely ignore any sort of "factual studies" (which are 99% of the time done by some company that is paid to do them).

Of course I unfortunately speak for a rather small percentage of consumers, as far as I can tell. It's pretty depressing actually.

Speaking of which... (1)

Malevolyn (776946) | more than 10 years ago | (#11057036)

Does this sound familiar to Verizon's "It's $14.95 extra a month to add computers to your cable connection. Not so with DSL!" commercials? How about Microsoft's "Get the Facts" campaign?

Engineering Ethics (0)

Anonymous Coward | more than 10 years ago | (#11057049)

the existence of such practices is not all that suprising, in and of itself. but if this kind of thing is being done by professional engineers, then i am a little dissmayed. as an engineering student in college, ethics have been an important part of our education, and i would be quite disapointed to learn that this kind of unethical behavior is widespread among engineers in industry.

It has been going on for decades (2, Interesting)

MavenW (839198) | more than 10 years ago | (#11057060)

To the other engineers: Are you aware of this kind of practice at your company?

More often we become aware of it when the competitor does it.

About 20 years ago there were a series of "shootouts" between Novell, Microsoft, and 3COM, to see which network OS was faster. That was when I was literated to the fact that tweaking parameters can make a HUGE difference in test results. If you have even more control, you can even tweak the tests. We used to have to supply "debunking" documents that explained how the competing companies got the results they published. Sometimes it was hard to reproduce their numbers, even tweaking our own sofware in the worst ways.

These days a lot of journalists try to maintain a neutral position. They go to great lengths to be fair, and document even tiny things that might give one product a slight edge over another. It gives them more credibility to those of us that have been through these product wars.

Whats new? (2, Insightful)

Realistic_Dragon (655151) | more than 10 years ago | (#11057061)

If companies can get away with spouting total bollocks (first 64 bit desktop anyone - my Mesh Alpha (from a consumer desktop computing company) is obviously now very valuable since it never existed?) and not get fined, what incentive do they have for telling the truth?

Lies sell, since most people are stupid and believe whatever they are told.

The thruth about advertising (1)

NeedleSurfer (768029) | more than 10 years ago | (#11057063)

Is that there aren't any truth in advertising. Long gone are the days of the devil in the detail, even though capitalism prepared us to what is happening now we still have to confront the harsh truth:
The single most unifying caracteristic of capitalist entities is to, at any cost, give you less than what they sold you in such way that you believe you are the bad guy, the one who overevaluated the product. Try hard, when was the last time you bought something and it worked as advertised, as implied, was as sturdy and functionnal as told?

We are so used to it we even consider people who are pissed at recieving a burger that looks like a quarter of what is found on the pictures morons or naive. they are not, we are, the more we let capitalism win the more we lose. Don't get me wrong, capitalism is potentialy a great thing but only if kept on leash, real tight leash and if it's teeth are removed. Right now any occidental citizen who believe he's free should take a serious look at his lifestyle, we are servant, nothing more. You work your entire life so that a few people live happily and rich, on our sweat, with our money.

Bake off in mags: pure bullshit, period. the best way to buy products, and this is what I personnaly do with for my company, is to order test versions of said product from the company for evaluation and test the shit out of those, only then you will KNOW what is right for you. Any salesman, any publicity, bake off, review, comparison chart is a load of bullshit, whatever the site whatever the source. Every tech review website (anandtech, hardocp, tom's hardware), are all bullshit, they are held by people with agendas. the review and advices on Slashdot seen in the forums, bullshit, again, written by someone with an agenda even if this agenda is to make him look knowledgeable, an expert in some way.

the only thing I trust is my experience and it works very well, you should try.

Truth in advertising (2, Funny)

ShatteredDream (636520) | more than 10 years ago | (#11057065)

Isn't that like.....

Sexual freedom in Saudi Arabia
Fiscal accountability in corporate America
Bug-free programming in Microsoft products...
Intelligence and integrity in GWB?

A bribe works well, too (0)

Anonymous Coward | more than 10 years ago | (#11057068)

I worked on a product where the rumor was that a few thou slipped under the apartment door of an editor's girlfriend got it the "editor's choice" award in a well-known computer publication. The product failed, anyway.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?