DATA COLLECTION/BUILDING UP OF THE CORPUS/CONTENT ANALYSIS
As per the submitted project proposal, a literature search was conducted for retrieving quality guidelines from scientific papers, journals, or books. Scientific repositories and databases, such as PubMed, Web of Science, Web of Knowledge, Springer Link, Scopus, ScienceDirect, PsycINFO, Ingenta, Mendeley, and the Scholar interface of Google were utilised in searching for such, published during any year between 1960 to 2012, utilising the following keywords appended to “qualitative research” and “health”: quality, criteria, guidelines, evaluation, review process, standards, appraise/appraising/appraisal, and assess. Additionally, we also referred to the bibliography of the retrieved literature and identified other relevant published literature. Eventually, 133 guidelines on the quality of qualitative research were identified and the texts containing the criteria were extracted (Appendix 1. List of the 133 guidelines).
Descriptive analysis of 133 guidelines & quantitative analysis of the structure of the information
The corpus was classified according to the year of publication and disciplines. The articles were observed to have been published between 1981 and 2011. The guidelines were then classified on the basis of the discipline that was primarily determined by referring to the aims and scope of the publishing journal or book or institution. Accordingly, there was a predominance of articles published in medical (45) and nursing (39) journals/books and, to a lesser extent, purely methodological (20), public health (19), and psychiatry and/or psychology (10) journals/books. The great diversity of the guidelines, both in terms of form and content, necessitated additional analyses as follows: the number of categories and subcategories constituting each guideline were counted with the aim of capturing the structure of the information contained in each guideline, as well as determining the depth of description, the degree of explicitness of the criteria proposed. This analysis highlighted an extremely variable number of criteria, ranging from a minimum of 1 criterion for a very condensed guideline to a maximum of 28 criteria for the longest guideline, with an average of 7.9 criteria for the entire corpus. (Appendix 2. Descriptive and quantitative analysis of the 133 guidelines in the corpus).
Content analysis of the guidelines
A content analysis of the criteria and their foci was conducted on the text comprising of the criteria extracted from each guideline. There existed a great variation in terms of criteria and types of foci adopted by the authors. The criteria were: general or specific; adapted or not for qualitative research; focused on methodological aspects; focused on theories; focused on content of publications; enumeration of the qualities expected from the researcher; or, focused on the description of the conduct of research (Appendix 3. Content analysis and identification of categories based on the principal criteria in the corpus). The primary and general result from theses analyses indicated how most guidelines have a variability of criteria, with their authors having different levels of objectives, resulting in making these incomparable. Therefore, the identification of the use of pertinent quality criteria for qualitative research becomes very difficult. Furthermore, the different guidelines revealed a very personalised and subjective lexicon, without necessarily providing precise definitions of the terms. These preliminary results elucidate why it is so difficult to reach a consensus on quality criteria for qualitative research since the guidelines may correspond to different objectives and evaluate various levels.
Content analysis for the identification of epistemological underpinnings
Each guideline was reviewed in its entirety and excerpts stating epistemology or epistemological beliefs and/or those sentences that could be useful in determining the epistemologies were extracted. The stated beliefs were accordingly noted. To further ground the findings, a second relevant paper of the first author of each guideline was located (failing which, a relevant paper by the second author), with the relevant texts being extracted. These were subjected to a thematic analysis for facilitating the identification of different paradigms underpinned by the criteria. This prolonged process of identification enabled associating the criteria proposed for assessing the quality of qualitative research with the different theoretical approaches and paradigms, which were explicitly or implicitly stated in the fields of health concerned. As a result, a table of a continuum of paradigms was created, highlighting the complexity of the epistemological and paradigmatic differences present in the analysed guidelines.
Given the confusion and profusion of terms used by the authors, the guidelines were reviewed for instances and occurrences of keywords extracted from the continuum of paradigms. These excerpts were collated into a glossary of terms used in the corpus, i.e. on paradigms, methods/approaches, as well as the explanations as given by authors of these guidelines. (Appendix 4. Glossary)