But before using an Interrater reliability method, qualitative researchers should understand what they are and how they work. The first question should be: what do you want to test? Have several researchers created the same code in a theoretical grounding approach? How many times have the codes been used? How many highlights/quotes were coded in the same way? Each of these situations can be tested for the IRR, but the latter is the most common. Viera, A., Garrett, J., 2005. Understanding the Interobserver Agreement: Kappa`s Statistics. Family Medicine, Vol37:5. www1.cs.columbia.edu/~julia/courses/CS6998/Interrater_agreement.Kappa_statistic.pdf Armstrong, D., Gosling, A., Weinman J, Martaeu, T., 1997. The place of Interrater`s reliability in qualitative research: an empirical study. Sociology, V31:3, 597-607 www.researchgate.net/publication/240729253_The_Place_of_Inter_Rater_Reliability_in_Qualitative_Research_An_Empirical_Study So what do we do? I would say that we need to identify, accept and understand these differences. The parts of interpretation on which we disagree are probably the most interesting, as are the disagreements of our respondents. Why did people use different codes there? Why didn`t people code it in the same way? Are there some random runaways, or does it show a systematic difference in interpretation? Perhaps we want to accept these differences and understand that networked life and differences in the experience or background of researchers are an important tool to better understand the complex lived experience in qualitative research.
We must recognize and recognize our subjectivity, and accept that, for the questions posed by qualitative research and how we interpret them, there is not always a single way of programming – no gold standard that can be targeted. If you want to download Quirkos and try software developed by quality researchers for a purely qualitative approach, you can test the full version for free. It aims to make qualitative analysis software visual and attractive and also facilitates cooperation and project sharing.https://wikifama.com/inter-rater-agreement-for-qualitative-items/