R&D Tools: Player Testing

In my last article, I wrote about PING Man and the role that robotic testing plays in club research and development. The perfect complement to robotic testing, where we can precisely control clubhead delivery and impact location, is player testing. Understanding how the end user performs and experiences a prototype serves as the ultimate validation of a concept or design. In many instances, player-testing results match up quite well to what we see in robotic testing, but this is not always the case. Turf interaction, player perception, and the way a club can influence a player’s kinematics are all insights unique to player testing. Within our testing department, player tests are by far the most requested of tests. On average, we conduct 200 different player tests a year, compared to about 100 Ping Man tests. This equates to about 55,000 shots hit a year during player testing.

Generally, player tests are setup using a launch monitor and some bespoke player-testing software.   Groups of test subjects (players) are chosen based on the target demographic of the clubs to be tested, and usually, consist of 20 to 40 players. The majority of our player tests utilize PING employees, but there are circumstances where we use subject groups from outside PING.  Following the test, data is uploaded to the player-test database. Reports are then created that query the database of test results and produce datasets containing launch-monitor output, club info, and visualizations of the results. The full dataset, along with filtered subsets of data, are reviewed and statistically analyzed to gain insights into the performance of the clubs tested. Another very important element of this type of test is the interview. Players are asked a number of qualitative questions about what is important to them and how satisfied they were with each club with respect to distance, control, sound, look, etc. Connecting a player’s perception to performance is a critical element provided by player testing.

There are different flavors of player testing conducted at PING. Here are examples of the variations and the results each produces.

Figure 2: Player test trajectory comparison of G Crossover vs. G400 Crossover

• Standard Club Comparison: A “standard” player test setup is straightforward. Test subjects hit multiple shots at a target with two to three different clubs, rotating clubs periodically. Variations of this test may include different types of lie (fairway vs. rough) or various types of golf ball. Validation testing of the new G400 crossover design is a good example of this type of test. A number of changes were made with the G400 design (face material/geometry, weighting, sole geometry) to gain the performance improvements we were targeting.
  We conducted PING Man and Motion Capture testing of this design during development, but comparing the new design to the previous generation on real turf served as a holistic measure of expected performance. This provided both a measure of trajectory and dispersion improvements, along with a qualitative understanding of turf interaction and player feedback (see figures 2 and 3).
  
  

  Figure 3: Crossover sole comparison following a player test

• Gapping Test: When designing a set of irons, fairways, or hybrids, we want to ensure the yardage gaps between clubs is optimal for the target customer. To validate this element of a set, we conduct a gapping test. We select a group of test subjects and have each player hit every club in the set. The player rates the quality of strike, and we register three high-quality shots from each club. We then look at the average differences between the clubs to evaluate yardage gaps. Figure 4 is an example of a hybrid gapping test result from our library, looking at the gapping between a 3, 4, and 5 hybrids.
  

  Figure 4: Player test comparison of a 3, 4, and 5 hybrids.

• On-Course Simulation: In some instances, we like to understand performance when a player is forced to hit different clubs, one shot at a time, to simulate a round of golf. This helps mitigate the tendency of players to adjust to the behavior of a particular club when hitting shots repeatedly, one after the other. In a test like this, a player is presented with fairway boundaries and target pins on the test range and is asked to hit one of the drivers followed by another club. A good example of this is a test we conducted to demonstrate the influence of swingweight on clubhead delivery and trajectory. The resulting Offline vs. Carry Distance plot is shown in figure 5. You can see for this test the players hit a hybrid, 7-iron, or pitching wedge in between drives.
  

  Figure 5: On-course simulation test result

Player testing is a critical experimental tool for us at PING as we conduct research and develop high-performing products. As a result, it is one of the most requested test types from our Analysis and Testing group. Serving as a test subject is also one of the fun benefits of working at PING headquarters (I just participated in a player test halfway through writing this article). Additionally, it serves as a great opportunity to test the latest equipment, giving engineers the vital real-world data that translates to true innovation and game-changing technology.