Design and Analysis of Catapult Full Factorial Experiment

1. Click SigmaXL > Design of Experiments > 2-Level Factorial/Screening > 2-Level Factorial/Screening Designs.
2. The Number of X Factors can be 2 to 19. Using process knowledge, we will limit ourselves to 3 factors: Pull Back Angle, Stop Pin and Pin Height. Pull Back will be varied from 160 to 180 degrees, Stop Pin will be positions 2 and 3 (count from the back), and Pin Height will be positions 2 and 3 (count from the bottom).
3. Select Number of Factors = 3.
4. The available designs are then given as: 4-Run, 2**(3-1), 1/2 Fraction, Res III and 8-Run, 2**3, Full-Factorial. If we had more than 5 factors, a Resolution III or Plackett-Burman Screening design would typically be used. Here we will choose the 8-Run, 2**3, Full-Factorial design.




5. Next we will review the Power Information to help us determine the number of replicates. This design currently shows the following:
   
   These power calculations assume 3 center points:
   Very Low Power to detect Effect = 1*StDev (1-Beta < 0.5);
   Very Low Power to detect Effect = 2*StDev (1-Beta < 0.5);
   Medium Power to detect Effect = 3*StDev (0.8 <= 1-Beta < 0.95).
   
   Note: the power calculations require an estimate of experimental error, hence an assumption of 3 center points is used.
   
   We would like to have a reasonable chance (medium power) to detect an Effect = 2*StDev.
   Change the Number of Replicates to 2. The Power Information is now:
   Very Low Power to detect Effect = 1*StDev (1-Beta < 0.5);
   Medium Power to detect Effect = 2*StDev (0.8 <= 1-Beta < 0.95);
   Very High Power to detect Effect = 3*StDev (1-Beta >= 0.99).
   We will therefore choose two replicates. The number of replicates will always be a tradeoff between the desired power and the cost of the experimental runs.
   
   Specify 2 or more blocks if there are constraints such as the number of runs per day or some other known external “nuisance” variable (like 2 different catapults or 2 operators). Here we will keep Blocks = 1 (i.e. no Blocking).
   
   Center Points are useful to provide an estimate of experimental error with unreplicated designs, and allow detection of curvature. Typically 3 to 5 center points are used. Here we will not use center points because we have replicated the design twice and do not expect significant curvature in the distance response. Furthermore, center points could not be set for Pin Height and Stop Pin (without drilling additional holes!).
   
   
6. Complete the Factor Names, Level Settings and Response Name as shown:
   
   
   
   
7. Click OK. The following worksheet is produced:




8. You can enter information about the experiment in the fields provided. If you have access to a catapult, perform the experimental runs in the given randomized sequence, and enter the distance values in the Distance column.
9. If you are not able to perform the catapult experiment, open the file Catapult DOE V6.xls.
10. Click SigmaXL > Design of Experiments > 2-Level Factorial/Screening > Analyze 2-Level Factorial/Screening Design.




11. We will use the default analyze settings (all terms in the model) to start. Click OK. The resulting Analysis report is shown:
    
    
    
    
    
    
12. The model looks very good with an R-Square value of 99.9%! The standard deviation (experimental error) is only 1.03 inches. Clearly Pull Back Angle is the most important predictor (X factor), but all the main effects and two way interaction are significant. However, the three-way interaction is not significant, so it should be removed from the model.
13. Click Recall Last Dialog (or press F3).
14. Remove the ABC interaction term as shown:
    
    
    
    
15. Click OK. The revised report is shown below:






16. All the terms in the model are now significant, and there is no evidence of lack of fit (P-value for lack-of-fit is 0.128 which is > .05).
17. The Residual Plots also look very good:




18. Scroll up to the Predicted Response Calculator. Enter the predicted values shown. These initial settings were determined by trial and error.





19. Excel’s Solver may also be used to get a more exact solution:
    
    
    
    
    
    
20. The model prediction must then be confirmed with actual experimental runs at the given settings of Pull Back Angle = 179.5, Stop Pin = 2, and Pin Height = 2.
21. Alternative settings to achieve the target distance may be obtained with Contour/Surface Plots. Click SigmaXL > Design of Experiments > 2-Level Factorial/Screening > Contour/Surface Plots. Set the Pin Height to 2 as shown (after clicking OK, you can use Recall SigmaXL Dialog to create another Contour/Surface plot with Pin Height set to 3) :
    
    
    
    
22. Click OK. The following Contour and Surface Plots are displayed (with Pin Height = 2). Note the contour line with Catapult target distance = 100 inches. Although pin settings are discrete, they appear as continuous, so this will be a constraint in our selection of alternative settings. In addition to Pull Back Angle = 179.5, Stop Pin = 2, Pin Height = 2, we see that Pull Back Angle = 171.4, Stop Pin = 3, Pin Height = 2 is also a valid setting. Alternative setting options are valuable in a designed experiment because they allow you to select lowest cost optimum settings, or settings that are easier to control.



Tip: Use the contour/surface in conjunction with the predicted response calculator to determine optimal settings.