Essay Sample: Threats to Secondary Data Validity

There are many threats to secondary data validity that are found in the Week 5 Discussion Scenario, especially those associated with the research methodology used. One of these threats is the sampling approach employed in selecting the participants for the study. Based on the description of the sample used, "online survey to 3,000 male firefighters and 500 female firefighters working in a large city", it can be concluded that a purposive sample was used to choose participants the study. According to Bryman (2016), purposive sampling is a non-probability sampling technique that does not involve random selection of potential participants.

Is your time best spent reading someone else’s essay? Get a 100% original essay FROM A CERTIFIED WRITER!

Order now

In purposive sampling approach, the researcher samples participants with particular characteristics capable of addressing his or her research questions or hypotheses. The primary limitation of purposive sampling is that the sample being studied is non-representative of the population (Baran, 2016). It is worth noting that the kind of sampling design used in a study to gather data is very crucial because it affects the validity of the statistical analysis results and external validity of the findings (Boo & Froelicher, 2013). In the present Week 5 Discussion Scenario, the use of non-probability sampling technique limits the external validity or the generalizability of the study findings. Consequently, the results cannot be extended to other populations other than the sample used (Barratt, Ferris, & Lenton, 2015).

Benefits of the Reported Data

One of the benefits of the Week 5 Discussion Scenario data is that it enables other researchers and readers to know how working as a firefighter affects worker's stress level without necessarily conducting a primary study. According to Smith et al. (2011), many studies cannot be carried out within a reasonable timeframe and cost by employing primary methods of data collection because of high cost and the length of time it takes to complete such studies. This is especially true if the research is longitudinal and if the researchers are junior and do not have access to adequate financial and time resources (Smith et al., 2011). In such instances, secondary data provides junior researchers with access large sample sizes, appropriate measures, and longitudinal data, thus enabling them to come up with a generalizable answer to a research question without the need to conduct a primary study.

Another benefit associated with the use of secondary data is that for researchers keen to conduct primary data collection, starting with secondary data analysis is useful in providing a general overview of epidemiologic trends that future primary research studies can investigate in details. In the current Week 5 Discussion Scenario data, knowledge of the statistically significant relationship between working as a firefighter and stress can be used as a basis for determining whether firefighter's socio-demographic variables mediate the association between firefighter job and workers' high-stress levels.

Limitation

One of the limitations of Week 5 Discussion Scenario secondary data is a failure by the researchers to specify the reliability and validity of the survey used to gather participants' stress level data. Failure to report the reliability and validity of the survey puts the validity of the collected data in jeopardy. This is because it is not possible to judge or evaluate whether or not the data collected was relevant to the study. Also, the authors' failure to provide detailed information regarding the instrument used to collect participants' data limits the validity of the collected data. Cheng and Phillips (2014) noted that this limitation could be addressed through concise documentation of vital information regarding the validity of the data.

The second limitation of the Week 5 Discussion Scenario is that the researchers reported the findings as a causal relationship between two variables, yet the original study design was quantitative, cross-sectional, and correlational. This a threat to the validity of the study because cross-sectional and correlational studies cannot determine the cause-effect relationship (Polio & Friedman, 2016). Lastly, even though the authors reported the statistical significance of the association between the variables, the direction and the strength of the relationship was not mentioned. This is a threat to data validity because it is impossible to ascertain whether or not the association was strong and whether it was positive or negative.

Strategies to Overcome Threats to Validity and Limitations

One of the threats to the validity of the study was the use of purposive sampling to select participants for the study. This limited the external validity or the generalizability of the research findings. This threat can be overcome by using probability sampling techniques in future studies. This is because probability sampling approaches ensure that the results have high external validity or are generalizable (Trochim, Donnelly, & Arora, 2015; Taylor, 2017). Secondly, the limitation associated with a failure by the researchers to specify the reliability and validity of the survey instrument used to gather participants' stress data can be overcome by using reliable and validity data collection instrument. Lastly, the limitation linked to the inability of establishing a cause-effect relationship in cross-sectional and correlational studies can be addressed by employing experimental research designs.

References

Baran, M. L. (2016). Mixed methods research for improved scientific study. IGI Global.

Barratt, M. J., Ferris, J. A., & Lenton, S. (2015). Hidden Populations, Online Purposive Sampling, and External Validity: Taking off the Blindfold. Field Methods, 27(1), 3-21.

Boo, S., & Froelicher, E. S. (2013). Secondary analysis of national survey datasets. Japan Journal of Nursing Science, 10(1), 130-135.

Bryman, A. (2016). Social research methods. Oxford University Press.

Cheng, H. G., & Phillips, M. R. (2014). Secondary analysis of existing data: opportunities and implementation. Shanghai Archives of Psychiatry, 26(6), 371-375.

Polio, C., & Friedman, D. A. (2016). Understanding, evaluating, and conducting second language writing research. Taylor & Francis.

Smith, A. K., Ayanian, J. Z., Covinsky, K. E., Landon, B. E., McCarthy, E. P., Wee, C. C., & Steinman, M. A. (2011). Conducting high-value secondary dataset analysis: an introductory guide and resources. Journal of General Internal Medicine, 26(8), 920-929.

Taylor, R. R. (2017). Kielhofner's research in occupational therapy: methods of inquiry for enhancing practice. F.A. Davis.

Trochim, W., Donnelly, J. P., & Arora, K. (2015). Research methods: the essential knowledge base. Cengage Learning.