Radar Gun Fun, Part 3
Demystifying the Radar Gun: Comparing Radar Systems
"Any fool can know. The point is to understand". This quote by Albert Einstein fuels my belief as a coach. Just because we know something, doesn't mean we truly understand it. That is when you turn into the guy that just repeats things that have been heard over time. To truly understand things we must investigate, question, and research.
I brought this philosophy to radar guns, which sparked this series of articles. The first logical step was to understand the components and how they operate. Then I moved on to how errors and interference can affect otherwise accurate readings. With that base information, I can now move on to the fun part: comparing radar guns and in which situations each can be useful. This is the article in the Radar Gun Fun series you have been waiting for. It’s a long one, so settle in, and let's get down to business.
The Radar Guns
The Stalker radar guns are well known. Scouts love them and law enforcement trusts them. They are often considered the gold standard of radar guns, boasting their accuracy, and when they are calibrated right, they are hard to beat. The price of the gun reflects that reputation too. The two Stalker guns used in this experiment were the Pro II and Sport 2 coming in at $1,250 and $499, respectively. The Pro II is the model most scouts use, while the Sport 2 is the more portable and cost-friendly consumer product.
The Pocket Radar app allows you to overlay velocity on a video
The next radar gun is the Pocket Radar. This unit is also used by some scouts and many pro trainers and coaches. If you pay attention to MLB games or baseball twitter, you might have heard of Rob Friedman AKA @PitchingNinja. His GIFs of pitchers throwing filth have become widespread. So widespread in fact that he uses his publicity to promote amateur pitchers with a new Twitter handle, @Flatgroundapp. On most of the videos pitchers post, there is an overlay of a velocity reading. This is done with the app that connects to the Pocket Radar Smart Coach model. Along with its small, compact size and not needing to be calibrated, the app is just another feature that makes this radar gun appealing. The Smart Coach model used in this comparison comes in at $399. There is also another model, the Ball Coach Radar at $299 that has the same accuracy but is not compatible with the Pocket Radar app.
Then we have the first radar gun I ever bought: a five year old, $99 Bushnell. I didn't have a lot of money then, but I wanted to have a radar gun. There is not a lot of information on it because of its age, but it can be assumed to be your run of the mill, average, cheap radar gun. If you google "Bushnell Radar Gun" you most likely will see the model come up first.
The last "radar" is the Diamond Kinetics PitchTracker baseball. This one, like the Bushnell, comes in at $99, however the app associated with it is only compatible with an iPad/iPhone. This smart ball also tracks spin rate, spin axis, and a few other metrics that your standard radar gun does not. However, the cost starts to go up significantly because you need to pay a monthly subscription fee to get access to these extra metrics beyond velocity. This ball does not operate like the previous radar guns as it is not actually a radar gun. There is a smart chip inside the ball that allows for the calculation of the different metrics. Since this affordable piece of technology is becoming more common, testing its reliability against traditional radar guns is important.
Methods, Procedures, & Data Collection
The set up was simple but done in two different sets. The first set involved the two Stalker radar guns (Sport 2 and Pro II) and the Pocket Radar Smart Coach (PR). The PR and Pro II were each set up on tripods, about 10 feet behind the pop-up netting. The Sport 2 was handheld at roughly the same position as the other two. The pitcher then set up about 40 feet away on the other side of the net and threw about 35-40 pitches. The second set up was similar, but this time with the Sport 2, Diamond Kinetics PitchTracker Baseball (DK), and the Bushnell. The Sport 2 was on the tripod while the Bushnell gun was handheld. The DK was the ball thrown by the pitcher, so an iPhone was set up close to the other two radars to easily see all three velocities. Once again, 35-40 pitches were thrown. After collecting the data, I calculated different statistics to compare all the radar guns back to the Stalker Sport 2. I decided on the Stalker Sport 2 for two reasons: Stalker's reputation of being accurate, and the Sport 2 operates with the same frequency as the Pocket Radar Smart Coach and the Bushnell.
Data Analysis
Time to dig into what the numbers really mean. Here is the raw data in Google sheets. The statistical analysis was completed in Excel.
Stalker Pro II vs Stalker Sport 2
This first comparison is between the two Stalker guns. The Pro II operates at a frequency of 37 GHz, compared to the Sport 2's 24 GHz, but since they are made by the same company, their accuracy should be similar. The accuracy claimed by Stalker on both of these models is +/- 3% of the reading, so at 70 mph there could be a discrepancy of +/- 2 mph. I used the data from this comparison to have a control for natural pitch to pitch variability. This will also show the reason to never count on any single radar reading.
In the first graph, we have a regression chart. For this type of study, we are looking for a regression line with a slope as close to 1 as possible (that would mean that the radar guns were obtaining the same readings). Here we see it comes in at about 1.01. Not bad at all. The R^2 for this comparison is 0.957. This number describes how well the regression line fits the data points. The closer R^2 is to 1, the more of a correlation. Once again, not bad.
So, the math shows there is a good correlation between the two Stalker radar guns' readings. This is more data for the long run of a set of pitches. In order to really see pitch to pitch variability, we need another graph.
A Bland-Altman plot takes the averages of two variables and plots them against the differences between those same two variables. This will show more of the variability between the two radar guns. Without making any standard deviation (SD) calculation, we can see a trend that the majority of the data is within +/- 1 mph. Only 4 data points are outside of that, and if you include the +/- 3% that Stalker claims, then it becomes only 1 data point. This trend shows that they don't vary much, and therefore can be considered reliable. It should be noted here that even the most expensive radar guns vary. In fact, the two Stalker guns were different more times than they were the same. If you talk to pro scouts they will tell you that you can easily see a 2 mph difference between multiple radar guns on the same pitch. This also shows why you are kidding yourself if you think you can read any pitch with an accuracy down to 0.1 mph resolution.
Pocket Radar vs Stalker Sport 2
Now that we have an idea of what can be expected for normal variance on the most expensive radars, we can look at the other radar guns. Up next we have the Pocket Radar Smart Coach model. This small, mobile radar gun operates on the same frequency band as the Sport 2 at about 24 GHz. Pocket Radar claims an accuracy of +/- 1 mph which is more precise than the +/- 3% that Stalker claims on their guns. Let’s see how they match up.
The regression chart looks eerily similar. When rounded in the same fashion, the numbers are the same as the Stalker test with a slope of 1.01 and R^2 of 0.957. The reason we round these numbers is because we are getting down to some very small numbers, and therefore these values offer less significance. Using three significant figures in this situation is enough to determine what we want to know.
Looking at the per pitch variability. Here we see that basically, all data points lie within the +/- 1 mph that Pocket Radar claims. The one data point outside of that range is only at -2 mph which could be the result of the +/- 3% specification of the Stalker. Once again, we didn't take SD into account as we have a range we would like the data to fall within, which it obviously did. This is also a reliable radar gun without much variance, it was actually less variation than between the two Stalker guns.
Bushnell vs Stalker Sport 2
As we get into the lower-cost equipment, we start to see more variance. The Bushnell radar gun also operates at 24 GHz, boasting the same accuracy of the Pocket Radar of +/- 1 mph. If this is true, then this gun would prove beyond valuable. Let's see if the numbers stack up.
The regression chart looks somewhat promising. With a slope of 0.984 and R^2 of 0.952, we see a little less adherence to the straight line. If we look at the average of the velocity readings, we see the bigger picture. The average velocity comes in at roughly 68 mph, while the Sport 2 measured the same pitches with an average of 70.2 mph. This is significantly worse than the Sport 2-Pro II and Sport 2-Pocket Radar comparisons as their averages ran within 0.7 and 0.4, respectively.
The Bland-Altman plot tells more of the story that the average velocity started telling. The middle blue line represents the average difference between pitches, sitting at about -1.14 mph. Once again, consistently lower than the Sport 2 over the long run. When we add Standard Deviation in this plot, it helps determine what to expect pitch to pitch. With that, a 2SD interval gives us 95% confidence that the Bushnell will read between roughly +1.2 mph and -3.4 mph compared to the Sport 2. The upper and lower lines represent that 2SD interval from the mean. Basically, you can assume that the Bushnell gun will naturally vary in this range of about 5 mph pitch to pitch, without other interference. This is where we start to see the accuracy decrease significantly Also, you should consider that these measurements were all made at a distance of about 50 feet and the maximum range of the Bushnell radar is specified at 90 feet. So if you get back behind a backstop at over 100 feet away you will see even more inaccuracy as described in the previous article: how errors and interference can affect otherwise accurate readings.
Diamond Kinetics PitchTracker vs Stalker Sport 2
This ball is pretty interesting. It is attempting to measure certain metrics for pitchers that would be impossible for the high school or low-budget coach to know. The easiest metric to compare is the velocity reading, as there are radars with known accuracy such as Stalker and Pocket Radar. It is interesting to note that Diamond Kinetics does not provide an accuracy specification for this ball.
So how does the smart ball measure up against the traditional Doppler radar guns? The regression line shows that it is significantly different with a slope of 1.04, R^2 comes in at 0.881. This shows that the data doesn't quite fit the line as well as the others fit theirs. But that's not enough to draw any sort of drastic conclusions, as that is a decent correlation.
In this comparison, the DK ball comes in with an average difference in velocity of +2.7 mph (middle line). Not too far off, but we do see quite a range of per pitch readings. Using the 95% confidence interval, we can expect the DK ball to read anywhere from 7.3 mph (upper line) and -1.9 mph (lower line). That range of 9.2 mph is quite large, and leans towards the faster end of the spectrum, which is opposite of the Bushnell gun and double the error range. This means that it is quite a bit less accurate pitch to pitch than all of the other radar guns.
In simple terms, this means is that for 95 out of 100 pitches you can expect the DK ball to be somewhere within a range of about +7 to – 2 mph compared to what you would get on an accurate radar gun like a Stalker or Pocket Radar. Of course, that also means that for 5 out of 100 pitches it will be different by more than that 9 mph range.
Conclusions
Getting past all of the numbers, each of these tools is useful as long as we understand their limits. The data is what allows us to define those limits and draw conclusions.
Stalker Pro II & Sport 2
These high-powered radar guns provided us with some good data. They showed that even within similar radar guns there can be a slight variance of +/-1 mph on every pitch. This further proves the rule to never trust anyone reading, and don’t expect an accuracy down to 0.1 mph resolution. That being said, the Pro II and Sport 2 are great guns for scouts. Their range of 500 ft (Pro II) and 300 ft (Sport 2) make them great for long-range stadium guns. That, combined with their accuracy and limited pitch to pitch variance, means a scout can't go wrong with either of these guns.
Pocket Radar
The Pocket Radar Smart Coach proved itself in this experiment. Matching or exceeding the statistics we saw from the Stalker guns, it showed it can keep up with the most expensive radar guns. This small, compact gun doesn't resemble its counterparts, but it is just as good when it comes to accuracy. The only difference we see is its maximum range of 120 ft as compared to Stalker's 300+ ft range. With that, Pocket Radar is perfect for training and scouting on almost all amateur fields. It can be used at a close distance right behind the backstop in a game or for bullpen work, measure pull-downs, and even be used to measure exit velocity. The app for the Smart Coach version calls out velocity and makes those overlay videos seen all over Twitter. Even Tom Brady uses the Ball Coach Radar to prove he isn't over the hill yet. This radar is a great asset for any serious coach or athlete to have in their training tool belt.
Bushnell
With a price of $99, we expect a little drawback here. The Bushnell radar gun does have its place though. Youth sports and those on a limited budget are more likely to be able to afford it, and having some data is better than none. By using it for average velocity over a bullpen or game, the larger sample size would serve its purpose if you take into account the assumption that the average velocity readout is about 2 mph slower than the actual average velocity. Pitch to pitch difference for a bullpen or game shouldn't be relied on, as that data showed it can vary quite a bit. Knowing the data is what allows us to make this gun effective for the less competitive environment. The 90 ft range limitation may be an issue in a game situation.
Diamond Kinetics PitchTracker
Here is the fun one. Also at $99, like the Bushnell, and using a "smart chip" in the ball, I had some skepticism when first using it. I had that memory of being a kid and having a ball that had a digital radar reader implanted in it. It looked like a digital clock on the ball. We had no problem getting that to light up to 95 mph. Trust me, I wasn't throwing 95, I was 10. This ball developed by Diamond Kinetics isn't anything like that. Like the Bushnell, it proved that over the long run, it could come within a few mph of the average velocity. The PitchTracker Ball tended to run higher than the actual velocity. The results show that using this for anyone's personal record is a problem. It did give a general sense of velocity though and can be used in bullpens for those that are curious about a general average. However, since the pitch to pitch variation was seen to be over 9 MPH this can be a problem if you are trying to use it as a feedback device for tuning in your pitch speeds. It is also curious that they provide the speeds with a resolution down to 0.1 mph when the accuracy clearly shows that it cannot support that.
Remember, this smart baseball requires an iPhone/iPad to run the app. With that app, you can get other measurements beyond velocity, including spin rate, spin axis, time to plate, extension, delivery, and reach back to release. However, these extra measurements require you to pay a monthly subscription fee. Driveline Baseball compared those metrics' accuracy in their lab. The data shows it is reliable in giving a general sense of the average of those seven metrics. Using it as a bullpen ball (the app also allows for video and pitch charting) and checking long-run averages would be the best use of this product.
End Game
This last part was a long, grueling, data-fueled article. Within it and the rest of the series, I hope you found actionable data and information that will help you to make better use of your radar gun.
So, what does all this detailed testing show us?
With even the most expensive radar guns you can expect them to vary by about +/- 1 mph on any given pitch. The Stalker radars and the Pocket Radar met their accuracy specifications and were very reliable when used as directed. The less expensive products showed significantly more variation from pitch to pitch, but could still be useful for getting a feel for the general average, provided you understand their limitations and measure a large number of pitches. You also should not expect to use one to set your personal best or do any pitch-by-pitch development work.
Only by questioning, testing, exploring, and researching can we truly understand. "Any fool can know. The point is to understand".
