Great write-up, CountVohn! There’s simply no substitute for real-world testing data.
Very good post. Dynos aren’t standardized and not really comparable between each other.
I linked this thread to the one on AZ. Hopefully the mods don’t delete my post.
Great post, thanks for the info
great writeup, should be good background for newer guys who missed all the going-ons years ago.
Yeah beem I think that’s the problem here. The new guys have no idea of what went on in B67. The old and current B5 guys haven’t figured it out yet either but that’s another story.
Most of the RS4 guys didn’t come from the B67. I really don’t know where they came from to be honest.
For someone like me, moving to the rs4 in 2011 meant a couple of things were obvious
- Ignore dyno sheet marketing
- Demand performance testing from companies selling parts
- Buy JHM stuff because no other companies seem to have figured out the NA RS4, much the way when I bought my b7 s4 in 08, nobody else had figured out the NA S4.
I am a Stg 3+, constructed by JHM, so they are my choice of company, I was wondering if you might shine some light for me on the B5 saga as I have only been into this platform since early 2012 (if its too long of a story, maybe the cliff note version or a keyword that I can do some searching myself). Cheers and thank you.
Eli
B5 guys have always been a group of dyno stats.
My 770 setup put out this much whp on xx-dyno, what’s your RS6-K04 hybrid set up put out? Or my healthy stage II+ put out xxx-whp on the dyno and will out run any EVOIX…
Basically, majority of the B5 scene when I had my B5 was about the bigger turbo with a modified tune by VARIOUS self-tuners. Members posted these gigantic 500+ whp gains and it was all the rave but for some odd reason, a lot of the owners could never make it to the track due to maintenance or other reasons.
I think that’s what Saki was implying.
OT:
[quote]I know there are going to be those guys who say. “I just want more” To that I say, you don’t get much more then the end result. You don’t get much more the physical proof. You don’t just get a “sheet” you get recorded data showing acceleration and speed at several points, a video to show you what it looks like. Not only that but you can validate the time, location, temperature and elevation at the time the test was done.
[/quote]
I was once one of these guys until Euro got a hold of my neck and wrangled some sense into me. Thanks for the post Vohn, great stuff.
Introduction
CV invited me to post here, and I will attempt to provide a perspective based on my industry. When I develop a new product, I follow a process called “Quality Function Deployment.” This is a structured process that translates a customer need into engineering characteristics. In the following paragraphs, I will provide a highly simplified example of pharmaceutical development, and relate it to a new aftermarket exhaust being developed.
Product Performance
Let’s say that a the standard treatment for a bacterial infection currently takes 2 weeks, and has a high adverse event rate. The customer’s need is to eliminate the infection in less time, with fewer adverse events. The product performance is then based on how fast the drug kills the bacteria in the patient, and rate of adverse events. Clinical studies provide means to measure this performance against a standard treatment (and/or no drug at all).
Ok, so let’s start an active comparison to the AR example:
https://farm4.staticflickr.com/3733/13914195861_f74407e2a2_z.jpg
Product Testing
Unfortunately, it usually isn’t quite so simple to jump straight into human clinical testing. Before we start clinical tests on humans (in-vivo testing), we develop ways to quantify the product’s attributes outside of humans (in-vitro testing). An example of in-vitro testing would be how well this new drug can kill bacteria in a test-tube. These tests that are used to develop the drug, and often become specifications for how we verify that we are making the drug correctly.
But even that isn’t easy. These in-vitro tests must be “validated”. To make a validated measurement, we develop a testing procedure that is precise and accurate within a pre-defined range. The primary purpose of this test method “validation” is to control variation. Without validation, it can be difficult to impossible to isolate a source of variation. In problem solving, the source of variation is often referred to as the “6 Ms”: man, machine, method, measurement, materials, and mother-nature.
https://farm8.staticflickr.com/7417/13937320065_1a7c26b580_z.jpg
Transfer Functions
Now comes the MOST important part: verifying that the performance (clinical results) can be predicted with the in-vitro testing (lab results). In the very simple example, the test-tube potency must relate to the bactericidal capability in clinical studies. Without this relationship, the in-vitro test is meaningless. We call this relationship a transfer function.
https://farm4.staticflickr.com/3712/13937762344_d982f145bd_z.jpg
Discussion
Defining and meeting the customer need is everything.
In the AR aftermarket exhaust example, neglecting to translate a dyno improvement to a measureable acceleration improvement means that the product has not proven its ability to meet the customer need. In the Pharma example, people go to jail if the company markets a product without clinical verification. It is a heavily regulated industry, because the price of failure is usually tragic.
The auto aftermarket industry is NOT heavily regulated by government agencies. It is primarily consumer regulated. That is where a website like AR comes in, demanding excellence from vendors, and rewarding those that deliver.
For fun, let’s play out the role of the unscrupulous and/or incompetent aftermarket vendor.
This vendor has a great idea (Z-pipe, anyone?), and puts every last cent into an exhaust design. They take glamour shots of their new exhaust prototype and begin the marketing campaign. They know that the product is awesome, but want to enhance their ability to sell by showing a dyno.
They don’t actually own a car, but they have access to one. That car is owned by an enthusiast who doesn’t mind helping out to get some free dyno runs. They install the prototype exhaust, and plan to use a baseline dyno plot available on the internet as a point of comparison.
They know a competitor is claiming +15 hp from stock, so they figure their product must be at least equivalent, and most likely superior. They rent a dyno, and generate some numbers that are much lower than expected. WTF? They assume that their competitor must have used a more “friendly” dyno, or perhaps the car they have needs TLC. So they adjust the correction factor and other items to bump up the numbers. Why not? It would be unfair to their product to use artificially low dyno numbers.
Once they have a dyno graph that exceeds their competitor’s dyno numbers, the marketing machine touts the improvements. They indirectly offer a comparison of their dyno improvements to the competitor’s data straight off the competitor’s website (public domain). Now add in customer testimonials “best ever!”, a sense of urgency “group buy!”, slick pictures of the 1000 hr welding job on the prototype unit… and the Medicine Wagon has rolled into town.
Discussion
- The hypothetical vendor never related the customer need (go faster) to a performance measure (1/4 mile).
- The hypothetical vendor never established a transfer function between the performance measure (1/4 mile) and the test method (hp/tq dyno).
- The hypothetical vendor never generated a baseline (control condition) to demonstrate that their product offered any measurable improvement.
- The hypothetical vendor manipulated non-standardized dyno conditions and public domain information to force a pre-determined conclusion.
- The hypothetical vendor leveraged a public bias that a dyno graph = proof, and that there are underlying controls and standards that were utilized to develop this data.
- The hypothetical vendor performed no robustness testing on the unit.
- The hypothetical vendor did not perform a root-causal analysis to determine why the unit originally performed below expectations on the dyno.
- The hypothetical vendor attempted to generate an emotional response in their marketing to sway consumer judgement.
While it may be bad science, bad engineering, bad business practices, bad ethics, bad behavior, etc., etc… did this hypothetical vendor break any laws?
This thread is excellent. Thank you to those who put in the time to put this in writing. Plus karma will be provided once I’m in from of my pc. Well done.
Thanks for the info everyone, esp CV, very good read.
The medical analogy is something that we see a lot. For example, say I come up with with a new drug aimed at reducing stomach acid with the goal of preventing ulcers. We’ll call it bangomanozole. It increases pH in the stomach by 25% (less acidic) more than another drug, lansoprazole. It must be MOAR BETTER right?! Even if it costs 20% more!
Well, reducing acid by 25% is the answer to the wrong question being asked. We should be asking other questions too:
- does it actually reduce the incidence of ulcers?
- 25% of what?
The vast majority of this site’s membership would be most concerned about the rate of reduction of ulcers.
The bottom line question is how do we go faster, not how do we make more hp.
^ bangoman - is there a better job in the world than pharma sales & marketing? You don’t create anything or cure anything… yet you get to be the middleman for the people that do both.
I dunno it seems to be a pretty shitty job these days. Gone are the days of all expenses paid golf trips, cruises etc… When I first came to Canada in 2010, the drug reps used to buy us textbooks. Now, they can’t. I think much of their job is dealing with frustrating administrators.
I can think of a few
My comment was not meant to be taken seriously. Next time I’ll be sure to include the “roll eyes” emoticon. 
Indeed. 
Here’s a great example of the dyno being an issue
Guy dynos B8 S4 stock 280 whp
Guy then dynos GIAC stage 2 - 337 whp
Guy is disappointed and says something is wrong.
B8 forum agrees ‘somethings wrong’…’ seems really low’.
Guy says he is going to buy the GIAC DSG tune to see if it helps.
This is dyno marketing 101. If you’ll recall GIAC put out a completely bogus 402 whp dyno sheet in their marketing material 2 years ago. You may also remember APR put out stock numbers of around 320 whp on their hub dyno (which reads nothing like a typical mustang dyno like this guy went on).
As a result of results like those, the general population’s reference points for a B8 S4 get reestablished. Now the guy and the forum for the most part, thinks he is down 30-50 whp and needs to buy more stuff to see it.
The guy gained 57 whp. That’s 20%. That’s massive delta. That’s what Apr gained. That’s wheat everyone gains. The peak ‘after’ number is so low because he is testing on a tough reading mustang dyno. All that matters is delta, and his delta is excellent.
He also complains about the shape of his curve being too flat so he must be losing hp. Or… The multiplying effect of a calculated hp number when you scale down the numbers also scales down the slope of the curve. When you scale it up, the slope steepens.
Had he been on a happy dyno for before after, he and the forum would think his numbers were fine. Had he been on a happy dyno, he and the forum would think the slope of his curve is fine.
But he wasn’t so now he feels the need to buy a transmission tune in order to get more delta…which is frigging impossible since you don’t change gear on the dyno last time I checked.
All around a great reason for this JHM thread to exist, showing why the dyno was never meant to be used for this stuff and is dangerous in the hands of people who don’t understand it.
And here’s a good video by AMS on the perils of the dyno for comparisons or before after testing to prove the validity of a modification. As we know they are another shop who track tests everything to show standardised performance test gains.
That video was funny
Great writeup!