JHM Dyno sheet.

Everyone seems to be in a big uproar about testing and dyno sheets. The good news is JHM tests on the dyno but they take it one step further and do everyone the favor of actually doing what the community deserves to get and that’s results. Proof in the real world. You don’t drive to work on a dyno you drive to work on the street and people want to know how these parts effect them in the real world. There are going to be people that think one way or another. This post is just intended to give some insight on why JHM does it a different way.

Where is my dyno sheet? A good rule of thumb nowadays, if you want to see the shops that can’t compete, look for the ones talking about a dyno sheet. Anyone smart enough to use or test on a dyno should also be smart enough to know the limits of the tool. Not just in the testing sense but in the relationship of how it should relate to real world and others.

The dynamometer or ‘dyno’ is a tool of calculated measurement. Sounds great right? It can be, but what one are we going to use? One from Dynapack, Dynojet, Mustang, SuperFlow, A land and sea, an inertia, load-bearing, hydraulic or electric break dyno. Is it going to be drum,. Are we talking single drum, full rollers, top and bottom rollers for each wheel or hubcentric. There are more we can choose from, but there is more we need to know, what are the formula settings set at for calculation? Are we using smoothing, what is that going to be set at? What about alt correction, or temperature offset correction? What is the weight offset set at? Where is the estimated wheel slip setting adjusted to. Are we looking at WHP or recalculated CHP? Are we doing speed or RPM? This standard question can go on for pages. No dyno will read the same, even if it’s the same dyno make and model.

The dyno is a great tool if used properly, the issue is when it went from a tool of measurement to a tool of publicity. It became a sales tool, other than a testing tool. This is where it all went wrong. With a testing tool that has no standardization why were so many looking for the data from this tool? Frankly, it’s been from good marketing. It required less work on the part of developers to prove their product worked. The public started to get excited about graphs and figures so easily available, thinking that this was a true representation of performance. There are three parts design, testing and proof in the real world, to back up your product works. The standards for proof dropped from real world results to paper results, paper results with nothing to back them up and no standards to by what the results were calculated, other than to just say a dyno. The industry and performance for the customers was impacted in a negative light, because the standards dropped. People got fooled into just accepting the testing results. The testing is different with every company, you can’t validate the testing conditions or variables. The testing is done in controlled conditions that nobody outside of the testing knows what they are. The community started seeing rubber tubes increase power by 10hp. It’s easy to make a dyno look good, even without real gains from a product. it’s not so easy to improve 0-60 and 1/4 times without real gains. Not only that but you can validate the conditions the testing was done in.

One of the examples of how the dyno impacted the Audi scene was with the B5. In the beginning the B5 community was driven by real performance testing. 0-60 and ¼ mile. After the dyno games started you saw the cars going slower but the dyno sheets started to read more and more. Before you knew it, you saw companies putting out dyno sheets for ko4 cars reading 500whp (an impossible number on PJ ko4s the comp wheel isn’t capable of pushing that much CFM) even after the 500whp dyno sheets came out, the cars were slower than ko3 cars. Then for the 4.2 guys there was the M90 dyno sheets. Come get your 400whp M90 tested on the dyno. Sad part was, these cars couldn’t even break out of the 13s’. The RS4 M90 dyno read over 440whp but was slower then a stock RS4 with just a JHM catted exhaust. The last thing people wanted was more dyno sheets. The consumer needed real proof.

Without real proof and real results, the community can fall apart and there can be an influx of parts that don’t actually work. When we entered the B6/7 S4 market, nobody would buy a tune. There were plenty of tunes plenty of dyno sheets and just as many disappointed customers. The community was fed up with dyno sheets and no actual results that meant anything. Every part was represented by nothing more than a dyno sheet. The dyno sheets showed what would have been impressive hp numbers, but when it came down to it, there were no gains seen and the membership would tell new members don’t waste your money on a tune or exhaust, they don’t do anything. The dyno marketing campaign was proven to be a bunch of false claims. Members started to speak out, the dyno sheets were doing the same thing in the B6/7 as they did in the B5. They were just clever marketing aimed at sheet shoppers. Guys who didn’t know much about cars but would hop from company to company to see what hp number was the highest. The only thing the dyno sheets were proving was that, ink sticks to paper and if you make your dyno sheet read 5 more hp then the next guy (regardless of what the product really makes) you don’t have to do any work to back it up. Sheet shoppers will eat it up and put their dyno sheet results under their pillows at night to make themselves feel better after the money is spend and there is no power change.

The results of dyno sheet fallout was a massive distrust. Sure your stock B6 S4 exhaust was 2.23’ and the 2.25’ milltek exhaust was basically the same size. Even being almost the same size milltek put out a dyno sheet showing great gains vs stock. Somehow the great looking gains on the milltek exhaust vs stock were never there on the milltek when tested again. Not only that but, it turns out the best exhaust for the B6/7 S4 was the 2.5’ with the X. This got tested on both the track and the dyno. The issue was, the dyno testing results for the 2.5" looked much like the milltek exhaust results, but on the street the gains were shown to be a very large difference between the two. Two cars tested side by side showed there was a noticeable difference. Were the milltek results on the dyno sheet inflated or incorrect? Later when more extensive testing was done on the 2.25" Vs 2.5" it continued to back up that 2.5" was not only needed for performance but much much better. But there was a dyno sheet, right? As mentioned above this had been going on for years. The stack of dyno sheets and empty results in the real world was out of control. In the dyno sheet heyday for the b6 you could grab a milltek 2.25 exhaust a rubber tube and a hot air intake. That if we went by the dyno sheets would have picked you up an easy 45hp. Well, the actual results were a bit different. You actually lost hp and acceleration with those mods. The results were actually worse, even though the dyno sheet said otherwise.

After the standards and requirements dropped and so did the results. It was easier and easier for companies to just get a dyno sheet and completely bypass actual testing on the street. How do you solve this? You raise the bar, you demand better of yourself and your products. You don’t just test, you prove. Take the testing way past just printing out an sheet. After all, most people didn’t have a reference on what XX hp actually meant outside of numbers on the dyno sheet. You make a stand and say and realize that the dyno sheet is testing and you’re testing to see how you impacted the acceleration of the car. You only know how you effected the acceleration of the car, by testing the acceleration. More hp and tq should effect acceleration right? The dyno sheet results don’t tell you how that actually effects you or the acceleration of your car. The conversations shifted from results to testing because that beited the dyno sheet shops. Today people have convinced themselves that testing equals results and we have proven that isn’t the case. When you test and test you realize that the tests are not the big picture but, if you want you can paint a picture appealing to everyone willing to accept less then actual real world results.

How do you raise the bar? How do you demand better of yourself and your products?

Simple you prove the car works in the environment where everyone drives their cars. The mass public drive on the street. The JHM testing is extensive and goes beyond what people have been given in the past, it went from showing test results to ‘Proof’. JHM offers end user road ready results, taking testing beyond what most other companies even think about doing. Going that extra step to show the product works in the ‘real world’. JHM took testing in a controlled environment that nobody drives in or on and said, “lets take this to the real world” Lets prove these parts work and lets show them in the environment where they drive, the street. To do that you need to have a place to do that safely and consistently. There is such a place that people can go test the acceleration of their cars in a safe controlled environment. Its the drag strip. The big part is JHM is doing the testing for you. Nobody has to go to the track, JHM will do it for you and show you what your parts can do.

Every publication from day one has posted acceleration data. People buy performance parts to see performance. Showing performance needs to be set on a standardized system. A dyno is one gear in a short range of RPM. The ¼ mile not only measures acceleration but will do it from 0- 100+ mph and it will cover every gear you would use. NHRA ¼ mile tracks are everywhere. They are standardized for grade, distance and acceleration points . It has the 60’ showing take off. The 330’ showing low speed acceleration and then the 1/8th mile for mid-range acceleration. After the 1/8th you have the 1000’ mark and then the ¼ mile. These all show a range of proof of acceleration in all aspects. Not only will it show acceleration in time but in speed at each point.

I know there are going to be those guys who say. “I just want more” To that I say, you don’t get much more then the end result. You don’t get much more the physical proof. You don’t just get a “sheet” you get recorded data showing acceleration and speed at several points, a video to show you what it looks like. Not only that but you can validate the time, location, temperature and elevation at the time the test was done.

To those guys who just want to see a dyno sheet. Sure, there are tons of companies that love people like that and I’m sure they would love to take your money. You don’t have to demand proof in the real world if you don’t want to. Further more JHM almost always tells you what HP and Tq you can expect. Still there are those that just want to see a dyno sheet. Once again, sure I understand some people don’t like to accept that their method of belief of testing has been abused. We know there is always going to be some resistance to change and demanding a higher standard. Its about providing a standard that shuts the door on giving fraudulent companies an opportunity to take advantage of consumers. Or even community members that will never be our customers.

I have also seen lots of talk from guys on the road race about how the 1/4 doesn’t pertain to them.

The ¼ mile is a measuring point of acceleration. The ¼ mile should be one of the biggest tools in helping understand how the acceleration translates to you on the road. Each track has low speed turns that opens up to a big back stretch. That low speed turn onto the back stretch that’s where you can make up a lot of time. The ¼ mile time slip right there shows you how much time you can make up with the different mods. There are also sections of the track where you’re acceleration is from low speeds to WOT for a distance only a few hundred feet away. The 60’ to the 330’ mark on the ¼ mile slip helps show that gain.

one more for good measure.

Sub’d for good info!

Tremendously well put. Incredibly informative.

Great post!

Great great stuff! Thanks for sharing.

Tremendous information!

Good little history lesson and an explanation.

Great write-up, CountVohn! There’s simply no substitute for real-world testing data.

Very good post. Dynos aren’t standardized and not really comparable between each other.

I linked this thread to the one on AZ. Hopefully the mods don’t delete my post.

Great post, thanks for the info

great writeup, should be good background for newer guys who missed all the going-ons years ago.

Yeah beem I think that’s the problem here. The new guys have no idea of what went on in B67. The old and current B5 guys haven’t figured it out yet either but that’s another story.

Most of the RS4 guys didn’t come from the B67. I really don’t know where they came from to be honest.

For someone like me, moving to the rs4 in 2011 meant a couple of things were obvious

  1. Ignore dyno sheet marketing
  2. Demand performance testing from companies selling parts
  3. Buy JHM stuff because no other companies seem to have figured out the NA RS4, much the way when I bought my b7 s4 in 08, nobody else had figured out the NA S4.

I am a Stg 3+, constructed by JHM, so they are my choice of company, I was wondering if you might shine some light for me on the B5 saga as I have only been into this platform since early 2012 (if its too long of a story, maybe the cliff note version or a keyword that I can do some searching myself). Cheers and thank you.

Eli

B5 guys have always been a group of dyno stats.

My 770 setup put out this much whp on xx-dyno, what’s your RS6-K04 hybrid set up put out? Or my healthy stage II+ put out xxx-whp on the dyno and will out run any EVOIX…

Basically, majority of the B5 scene when I had my B5 was about the bigger turbo with a modified tune by VARIOUS self-tuners. Members posted these gigantic 500+ whp gains and it was all the rave but for some odd reason, a lot of the owners could never make it to the track due to maintenance or other reasons.

I think that’s what Saki was implying.

OT:

[quote]I know there are going to be those guys who say. “I just want more” To that I say, you don’t get much more then the end result. You don’t get much more the physical proof. You don’t just get a “sheet” you get recorded data showing acceleration and speed at several points, a video to show you what it looks like. Not only that but you can validate the time, location, temperature and elevation at the time the test was done.
[/quote]
I was once one of these guys until Euro got a hold of my neck and wrangled some sense into me. Thanks for the post Vohn, great stuff.

Introduction

CV invited me to post here, and I will attempt to provide a perspective based on my industry. When I develop a new product, I follow a process called “Quality Function Deployment.” This is a structured process that translates a customer need into engineering characteristics. In the following paragraphs, I will provide a highly simplified example of pharmaceutical development, and relate it to a new aftermarket exhaust being developed.

Product Performance

Let’s say that a the standard treatment for a bacterial infection currently takes 2 weeks, and has a high adverse event rate. The customer’s need is to eliminate the infection in less time, with fewer adverse events. The product performance is then based on how fast the drug kills the bacteria in the patient, and rate of adverse events. Clinical studies provide means to measure this performance against a standard treatment (and/or no drug at all).

Ok, so let’s start an active comparison to the AR example:

https://farm4.staticflickr.com/3733/13914195861_f74407e2a2_z.jpg

Product Testing

Unfortunately, it usually isn’t quite so simple to jump straight into human clinical testing. Before we start clinical tests on humans (in-vivo testing), we develop ways to quantify the product’s attributes outside of humans (in-vitro testing). An example of in-vitro testing would be how well this new drug can kill bacteria in a test-tube. These tests that are used to develop the drug, and often become specifications for how we verify that we are making the drug correctly.

But even that isn’t easy. These in-vitro tests must be “validated”. To make a validated measurement, we develop a testing procedure that is precise and accurate within a pre-defined range. The primary purpose of this test method “validation” is to control variation. Without validation, it can be difficult to impossible to isolate a source of variation. In problem solving, the source of variation is often referred to as the “6 Ms”: man, machine, method, measurement, materials, and mother-nature.

https://farm8.staticflickr.com/7417/13937320065_1a7c26b580_z.jpg

Transfer Functions

Now comes the MOST important part: verifying that the performance (clinical results) can be predicted with the in-vitro testing (lab results). In the very simple example, the test-tube potency must relate to the bactericidal capability in clinical studies. Without this relationship, the in-vitro test is meaningless. We call this relationship a transfer function.

https://farm4.staticflickr.com/3712/13937762344_d982f145bd_z.jpg

Discussion

Defining and meeting the customer need is everything.

In the AR aftermarket exhaust example, neglecting to translate a dyno improvement to a measureable acceleration improvement means that the product has not proven its ability to meet the customer need. In the Pharma example, people go to jail if the company markets a product without clinical verification. It is a heavily regulated industry, because the price of failure is usually tragic.

The auto aftermarket industry is NOT heavily regulated by government agencies. It is primarily consumer regulated. That is where a website like AR comes in, demanding excellence from vendors, and rewarding those that deliver.

For fun, let’s play out the role of the unscrupulous and/or incompetent aftermarket vendor.

This vendor has a great idea (Z-pipe, anyone?), and puts every last cent into an exhaust design. They take glamour shots of their new exhaust prototype and begin the marketing campaign. They know that the product is awesome, but want to enhance their ability to sell by showing a dyno.

They don’t actually own a car, but they have access to one. That car is owned by an enthusiast who doesn’t mind helping out to get some free dyno runs. They install the prototype exhaust, and plan to use a baseline dyno plot available on the internet as a point of comparison.

They know a competitor is claiming +15 hp from stock, so they figure their product must be at least equivalent, and most likely superior. They rent a dyno, and generate some numbers that are much lower than expected. WTF? They assume that their competitor must have used a more “friendly” dyno, or perhaps the car they have needs TLC. So they adjust the correction factor and other items to bump up the numbers. Why not? It would be unfair to their product to use artificially low dyno numbers.

Once they have a dyno graph that exceeds their competitor’s dyno numbers, the marketing machine touts the improvements. They indirectly offer a comparison of their dyno improvements to the competitor’s data straight off the competitor’s website (public domain). Now add in customer testimonials “best ever!”, a sense of urgency “group buy!”, slick pictures of the 1000 hr welding job on the prototype unit… and the Medicine Wagon has rolled into town.

Discussion

  1. The hypothetical vendor never related the customer need (go faster) to a performance measure (1/4 mile).
  2. The hypothetical vendor never established a transfer function between the performance measure (1/4 mile) and the test method (hp/tq dyno).
  3. The hypothetical vendor never generated a baseline (control condition) to demonstrate that their product offered any measurable improvement.
  4. The hypothetical vendor manipulated non-standardized dyno conditions and public domain information to force a pre-determined conclusion.
  5. The hypothetical vendor leveraged a public bias that a dyno graph = proof, and that there are underlying controls and standards that were utilized to develop this data.
  6. The hypothetical vendor performed no robustness testing on the unit.
  7. The hypothetical vendor did not perform a root-causal analysis to determine why the unit originally performed below expectations on the dyno.
  8. The hypothetical vendor attempted to generate an emotional response in their marketing to sway consumer judgement.

While it may be bad science, bad engineering, bad business practices, bad ethics, bad behavior, etc., etc… did this hypothetical vendor break any laws?

This thread is excellent. Thank you to those who put in the time to put this in writing. Plus karma will be provided once I’m in from of my pc. Well done.